Retries turn a temporary failure into a business risk when the first request may have succeeded. The test question is not simply whether a client tries again, but whether it can do so without duplicating a side effect.
Why this Skill is needed
The Retry Testing Skill reviews retry triggers, backoff, idempotency, retry limits, and duplicate side effects as one controlled behavior. It separates a policy description from an executed retry result.
What the Skill does
The critical boundary is the interval where the client cannot tell whether the first attempt took effect.
Use this Skill for payment, notification, job, and API flows where timeouts or transient errors can cause repeated requests. It helps prepare safe, isolated validation without calling production systems.
A good review names the retriable condition, backoff rule, idempotency key, attempt limit, side-effect expectation, evidence source, stop condition, and Human boundary.
Use it when a service is adding retries, an incident shows duplicate writes, or a timeout path has no clear rule for uncertain success.
What it does not do
The Retry Testing Skill can organize material, evidence, and next actions for Trigger judgment. It does not replace:
- Confirmation of rules, scope, and risk by the accountable domain owner.
- A real environment, account, dataset, log, or run record; static analysis does not become runtime evidence by itself.
- Authorization for release, compliance, production actions, or residual-risk acceptance.
- A Human decision when supplied sources conflict.
What it checks
The value of this Skill is not another keyword list. It connects each focus area to an observable input, a judgment, and a way to close the loop. Start with a small matrix based on the source Skill’s output contract:
| Focus | Question before analysis | Handoff output |
|---|---|---|
| Trigger judgment | Which failures should or should not retry | Policy and evidence |
| Timing behavior | Backoff, jitter, ordering, and attempt limit | Expected timeline |
| Side-effect safety | How duplicate writes or notifications are prevented | Idempotency and data check |
| Validation boundary | What can run safely and when to stop | Owner and rollback |
If a row has only a conventional expectation and no source or validation method, keep it open instead of turning it into a pass.
Audit inputs before you start
Before analyzing Trigger judgment, classify the input into six evidence states. A gap is not automatically a failure, but it must not disappear inside the conclusion.
| State | Meaning | How this Skill should handle it |
|---|---|---|
| known | Directly supported by the supplied material | Keep the source, version, and time with the judgment |
| missing | Needed for this pass but not supplied | Name the smallest evidence action and limit the conclusion |
| conflicting | Sources disagree | Show both sources and route the conflict to an owner |
| stale | Present but outside the relevant version or time window | Mark freshness; old evidence is not current proof |
| out_of_scope | Related but excluded from this pass | Keep the boundary explicit |
| assumptions | Temporarily adopted to continue analysis | State how and when the assumption will be checked |
Keep the input version, scope, environment, evidence locations, and accountable owner together. Without a run record, deliver analysis, design, or a validation plan—not an execution pass.
From problem to structured Finding
Connect the source, scope, evidence state, analysis, owner, action, close condition, and validation before writing the conclusion. The case below keeps this Skill’s identifier and domain context.
Keep the decision layers separate
For Trigger judgment, do not compress four different kinds of language into “recommended to pass”:
| Layer | How to write it | Application here |
|---|---|---|
| Fact | What the supplied material directly shows | Cite the source, version, input, or run record for the focus |
| Evidence-backed Inference | What several facts support together | Show the inference chain and retain uncertainty |
| Recommendation | The smallest next action | Name the evidence, review, execution, or regression path |
| Human Decision | What an accountable person must decide | Leave scope, risk acceptance, resources, and release meaning to the owner |
A complete case
This case follows Input, Analysis, Finding, Decision, and Validation. When material is incomplete, keep missing, conflicting, or assumptions visible instead of turning them into a pass.
Input
| Material | What to provide | What to do when it is missing |
|---|---|---|
| Retry policy | Error classes, timeout handling, backoff, jitter, and limit | Keep unspecified behavior pending |
| Operation contract | Read or write semantics, idempotency key, and side effect | Do not infer safety from HTTP method |
| Failure scenario | Timeout, connection reset, 429, 5xx, or worker restart | Model one controlled trigger |
| Evidence and environment | Logs, request IDs, mock dependency, data checks, and rollback | Do not use production credentials |
Use a request like this:
Use the retry-testing Skill.
Task: Prepare validation for a payment-capture client after a connection reset during the first request.
Inputs: [retry policy, API contract, idempotency semantics, request logs, mock provider, data assertions]
Scope: [isolated staging flow]
Constraints: [no real charge, bounded attempts, explicit cleanup]
Cover triggers, backoff, idempotency, limits, and duplicate side effects. Keep execution and pass claims separate from the plan.Analysis
Use the input, matrix, and evidence state to form the judgment before writing the Finding; keep missing material as a gap.
Finding
Focused example: Retry a payment capture without charging twice
A connection reset occurs after the provider may have captured the payment. The Skill should require an idempotency key, a bounded retry, and a query or provider mock that distinguishes one capture from two. A successful second HTTP response alone is not evidence that duplicate side effects were prevented.
The analysis prepares a controlled retry validation. It cannot prove idempotency, billing correctness, or absence of duplicate effects until an isolated run produces request and data evidence.
Example finding: turn one problem into a handoff
The field example below shows the recording pattern; it is not an execution result.
If the supplied material cannot prove that Trigger judgment meets its contract, write the finding like this. It does not invent the missing rule or turn missing evidence into a failure.
| Field | Example wording |
|---|---|
| Source and scope | Record the requirement, version, environment, and the concrete object for Trigger judgment |
| Finding | The condition or result for Trigger judgment is not yet traceable to evidence |
| Evidence state | missing / assumptions; use conflicting when sources disagree |
| Impact and priority | Name the affected user, journey, or delivery decision without inflating severity |
| Owner and Human decision | Ask the product, engineering, security, or test owner to confirm the rule and trade-off |
| Action and close condition | Add the smallest missing evidence; close only when source, judgment, and owner can be reviewed |
| Validation | Name one repeatable check, query, or run and retain the raw artifact |
The point is to let the next person walk from the finding back to the source and run an action that can change the decision.
Decision
The accountable owner confirms the decision question and risk trade-off; the Skill does not make that choice.
Validation
Before closing the finding, run the stated validation and retain the raw artifact. Without an execution record, the status remains unverified.
How a Finding enters the next stage
Handoff output
| Output field | Why it exists | Example status |
|---|---|---|
| Trigger judgment | Which failures should or should not retry | Policy and evidence |
| Timing behavior | Backoff, jitter, ordering, and attempt limit | Expected timeline |
| Side-effect safety | How duplicate writes or notifications are prevented | Idempotency and data check |
| Validation boundary | What can run safely and when to stop | Owner and rollback |
Every conclusion should point to a source, evidence state, and next action. If evidence is missing, use pending, blocked, unassessed, or NOT_SCORED instead of filling the gap with confidence.
Next-stage route
At minimum, hand off the source, evidence state, owner, close condition, and validation action; the next-stage conclusion remains bounded by the evidence state.
How to prepare better input
Provide error classification, timeout semantics, API contract, idempotency behavior, attempt limit, backoff rules, request IDs, mock or sandbox, data assertions, and cleanup. If any is missing, keep the corresponding conclusion pending.
A useful handoff includes input versions, scope, time window, evidence index, assumptions, Human decision boundary, and the smallest validation action.
Working with other Skills
- API Idempotency Testing:checks repeated requests against the API contract.
- Timeout Testing:isolates uncertain completion and timeout behavior.
- Circuit Breaker Testing:checks when repeated failures should stop calls.
Common traps
- Retrying every 4xx and 5xx response without checking whether the operation is safe.
- Checking the final response but not the number of writes, messages, or charges.
- Leaving backoff and maximum attempts implicit, so an outage becomes a request storm.
Install and invoke
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/retry-testing -gUse the retry-testing Skill.
Include objective, scope, versions, evidence paths, constraints, and decision boundary.
Audit inputs first, preserve evidence states, and finish with owner, close condition, and validation method.FAQ
Is an idempotency key always enough?
No. The key, server behavior, retention, response semantics, and side-effect evidence all need to match the operation contract.
Can I test retries against production?
Not by default. Use a mock, sandbox, or isolated environment with bounded attempts and cleanup; production testing requires explicit authorization.
The Skill is most useful when attached to one real project artifact and kept with its source evidence. Start narrow, validate the uncertain part, and expand only when evidence supports it.
References
Source Skill and execution contract
The complete execution contract lives in Retry Testing prompt. The source directory may also contain evaluation cases and supporting material.
Static plans, file presence, and dry runs keep their evidence state. They do not become runtime proof, an all-passed claim, or release approval.
Reference links
- Retry Testing prompt:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/retry-testing/prompts/retry-testing.md
- Retry Testing Skill source:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/retry-testing
- Retry Testing details:https://inaodeng.com/en/qaskills/retry-testing/
- Awesome QA Skills on GitHub:https://github.com/naodeng/awesome-qa-skills