An error message can be technically accurate and still leave the user with no safe next action. A stack trace can help an operator and expose a secret at the same time.

Why this Skill is needed

Error Handling Design Review checks whether each designed failure path has an evidence-backed behavior before implementation: how it is classified, propagated, exposed to callers, observed, and recovered.

What the Skill does

Use this Skill before implementation to review failure taxonomy, caller action, retryability, fallback, observability, privacy, data consistency, and recovery across distinct failure modes. It prepares findings and validation actions; it does not run fault injection or prove runtime behavior. It is especially useful when inputs are incomplete or conflicting but a bounded review is still needed, or a handoff must identify what remains unverified.

Awesome QA Skills: https://github.com/naodeng/awesome-qa-skills organizes Skills by language and testing stage. The series overview: https://inaodeng.com/en/blog/ai-testing/introduction_of_awesome_qa_skills/ explains the shared installation model; this guide stays with Error Handling Design Review.

A useful review follows one failure across three contracts: the caller gets a safe and stable result, the operator gets enough privacy-safe evidence to diagnose it, and the system preserves data consistency or names the required recovery decision. Apply it to paths such as validation rejection, authorization denial, timeout, dependency outage, rate limiting, duplicate requests, transaction conflict, partial success, or corrupted data.

What it does not do

The Error Handling Design Review Skill can organize failure-mode evidence and next actions. It does not replace:

  • Confirmation of rules, scope, and risk by the accountable domain owner.
  • A real environment, account, dataset, log, or run record; static analysis does not become runtime evidence by itself.
  • Authorization for release, compliance, production actions, or residual-risk acceptance.
  • A Human decision when supplied sources conflict.

What it checks

Connect each failure path to an observable input, a judgment, and a closeable next action. Start with a small matrix based on the source Skill’s output contract:

Failure pathReview questionsFinding evidence
Validation, authorization, and rate limitWhat triggers the path, what boundary applies, and what can the caller do?Contracted rejection and caller-visible result
Timeout, dependency outage, and transaction conflictIs retry safe, and can the original action have committed?Retry/backoff or human-intervention rule, propagation, and data consistency
Duplicate request, partial success, and corruptionWhat state may already have changed?Idempotency, compensation, fallback, or safe-stop decision
Breaker, degradation, and handoffWhat stops, degrades, or moves to an operator?Owner, close condition, and recovery evidence
Telemetry and privacyWhat can diagnose the failure without exposing secrets or payloads?Required log/metric/trace fields and privacy boundary

If a row has only a conventional expectation and no source or validation method, keep it open instead of turning it into a pass.

Audit inputs before you start

Before analyzing a failure path, classify the input into six evidence states. A gap is not automatically a failure, but it must not disappear inside the conclusion.

StateMeaningHow this Skill should handle it
knownDirectly supported by the supplied materialKeep the source, version, and time with the judgment
missingNeeded for this pass but not suppliedName the smallest evidence action and limit the conclusion
conflictingSources disagreeShow both sources and route the conflict to an owner
stalePresent but outside the relevant version or time windowMark freshness; old evidence is not current proof
out_of_scopeRelated but excluded from this passKeep the boundary explicit
assumptionsTemporarily adopted to continue analysisState how and when the assumption will be checked

Keep the input version, scope, environment, evidence locations, and accountable owner together. Without a run record, deliver analysis, design, or a validation plan—not an execution pass.

From problem to structured Finding

Connect the source, scope, evidence state, analysis, owner, action, close condition, and validation before writing the conclusion. The case below keeps this Skill’s identifier and domain context.

Keep the decision layers separate

For each failure mode, separate facts, evidence-backed inferences, recommendations, and Human decisions:

LayerHow to write itApplication here
FactWhat the supplied material directly showsCite the source, version, input, or run record for the focus
Evidence-backed InferenceWhat several facts support togetherShow the inference chain and retain uncertainty
RecommendationThe smallest next actionName the evidence, review, execution, or regression path
Human DecisionWhat an accountable person must decideLeave scope, risk acceptance, resources, and release meaning to the owner

A complete case

This case follows Input, Analysis, Finding, Decision, and Validation. When material is incomplete, keep missing, conflicting, or assumptions visible instead of turning them into a pass.

Input

MaterialWhat to provideWhat to do when it is missing
Failure scopeValidation, queue, worker, storage, download, and affected user stateName included stages and their owner
Behavior contractTrigger/boundary, retryability, propagation, caller result, and data effectKeep undocumented behavior as a question
Evidence and recoveryTelemetry/privacy rules, retry/backoff, fallback, compensation, and handoffIdentify the smallest source or validation action

Use a request like this:

Use the error-handling-design-review Skill.

Task: review the designed failure paths for a report export that can fail validation, queueing, generation, storage, or download before implementation
Inputs: [requirements, versions, links, logs, reports, or data paths]
Scope: [included and excluded objects]
Unknowns: [missing environment, accounts, data, or permissions]
Expected output: [risk-ordered findings, evidence status, and next actions]

Audit the inputs first. Separate facts, assumptions, and open questions. Do not claim execution without a run record.

Analysis

Start with the export stages and build one row per failure mode. For a storage write failure after generation, first establish the trigger and boundary, then classify the expected path as retryable, non-retryable, human intervention, or safe rejection from the supplied contract. Trace whether the report job may already exist, what the caller is told, which fallback or handoff applies, and which privacy-safe telemetry fields are required. If idempotency, commit state, retry policy, or caller wording is not supplied, leave it open; do not prescribe a retry or claim a fault was reproduced.

The handoff should preserve the input version, time window, evidence index, owner, and next validation action. The Skill can organize uncertainty; it cannot manufacture the missing artifact.

Finding

Example finding: turn one problem into a handoff

The field example below shows the recording pattern; it is not an execution result.

Use an EH-## record for one failure mode at one boundary. The example below leaves behavior unresolved where the source contract is absent; it is a design finding, not an incident or execution result.

EH-01 fieldStorage write fails after report generation
Trigger and boundaryStorage rejects or times out after generation; exact commit boundary is not supplied
Expected behavior typeunknown until the contract identifies retryable, non-retryable, human-intervention, or safe-rejection behavior
Propagation and fallbackCaller result, fallback, and handoff are missing; do not infer a retry
Data consistencyDetermine whether the generated artifact/job already exists and whether repeating the action can duplicate work
Observable evidenceConfirm the required job/correlation identifier and error class; exclude file contents and secrets per the privacy contract
Source and evidence stateRequirement/API version not supplied: missing; no fault injection performed
Impact, priority, and ownerDescribe user/job impact from evidence; export/API owner confirms priority and behavior
Validation and close conditionReview the source contract, then define a repeatable authorized check; close the design gap only when expected behavior, caller result, owner, and evidence are reviewable

This handoff lets the next reviewer trace the failure path to its source and see which behavior or evidence remains unresolved.

Decision

The accountable owner confirms the decision question and risk trade-off; the Skill does not make that choice.

Validation

Before closing the finding, run the stated validation and retain the raw artifact. Without an execution record, the status remains unverified.

How a Finding enters the next stage

Handoff output

OutputRequired fields for each EH-##Example status
Failure boundaryMode, trigger, boundary, expected behavior type, and sourcemissing where contract is absent
Caller/system behaviorPropagation, fallback, retry rule, visible result, and data consistencyunassessed until source is reviewed
Evidence and ownershipPrivacy-safe telemetry, impact/priority, owner, validation, and close conditionunverified until the stated check runs

Do not write “passed” without the evidence required for that claim. A useful review asks the same failure to make sense at three levels: the caller sees a stable result, the operator sees a diagnosable privacy-safe event, and the system has an evidence-backed consistency or recovery path.

Next-stage route

At minimum, hand off the source, evidence state, owner, close condition, and validation action; the next-stage conclusion remains bounded by the evidence state.

How to prepare better input

If the first request contains only a one-line goal, keep the output limited. Add the source version, affected objects, environment, known defects, and decision owner to move from a plausible checklist to a useful review.

A richer input changes the answer here because failure taxonomy, user action, retryability, fallback, observability, privacy, and recovery must be tied to evidence rather than inferred from a familiar pattern.

Best practices

  • Review one trigger and boundary at a time, then follow expected behavior through propagation, retries or fallback, state changes, and telemetry.
  • Do not recommend a retry until the source establishes whether the original action committed and whether repeating it is safe.
  • Prioritize high-impact gaps and request the smallest evidence or isolated check that can resolve them.

Working with other Skills

Common traps

  1. One generic error for unrelated failure classes.
  2. Logging sensitive payloads for diagnosis.
  3. Retrying before deciding whether the original operation committed.

Install and invoke

Install the English Skill from its language directory. The series overview carries the shared installation instructions.

npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en --skill error-handling-design-review

After installation, invoke it with the error-handling-design-review Skill and attach the real project material.

FAQ

Should users see technical error codes?

Expose a safe support reference when useful; keep internal details and secrets out of the message.

What if no fallback exists?

Say so explicitly, define the stop or escalation action, and avoid pretending the error is recoverable.

The example above is a design and review pattern, not an execution result. Keep static analysis, runtime evidence, human approval, and release acceptance separate.

References

Source Skill and execution contract

The complete execution contract lives in the Error Handling Design Review prompt: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/error-handling-design-review/prompts/error-handling-design-review.md. Read it before invoking the Skill; the prompt defines the detailed workflow and output contract. The source directory: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/error-handling-design-review contains the entry point and supporting assets where they exist.

The entry point centers failure taxonomy, user action, retryability, fallback, observability, privacy, and recovery. Keep its decision boundary visible and do not turn a static design into an execution claim.

Further reading

More Resources

Share