Source note: This article is based on the Awesome QA Skills AI Agent Testing prompt and Skill. The repeated-trial example, quality loop, diagram, and FAQ are editorial expansions to help readers apply the source requirements; reliability claims still depend on the project’s repetition plan and actual trace evidence.
AI Agent Testing evaluates whether an agent completed its task and how it got there. Each test needs a task-success oracle (a predefined criterion for deciding whether the goal was met) and a run trace of the steps and tool interactions. A plausible answer can still hide a skipped step, unauthorized tool call, contaminated memory, or unsafe stop decision.
Getting started: choose one bounded task
- Choose one multi-step task with clear scope. State its success criteria, allowed tools and resources, prohibited side effects, and stopping conditions.
- Audit input completeness, credibility, recency, and comparability: record what is confirmed, missing, conflicting, stale, out of scope, or assumed; include sources, versions, and evidence state, plus the environment, time budget, and boundaries that must not be crossed.
- Fix a sample ID and record model, Prompt, tool, data, and environment versions. Redact sensitive data or use an isolated environment.
- During a run, retain run/step IDs, tool arguments, authorization decisions, responses, and state changes. Do not infer execution from a missing trace.
- Judge the final outcome, process constraints, and evidence completeness separately. One run describes one observation, not reliability.
Define the task and its permitted path
For each sample, record the task ID, success oracle, allowed tools and resources, prohibited side effects, and stopping conditions. Then compare the final result with the run and step trace.
| Dimension | What to observe | Failure that can be hidden by the final answer |
|---|---|---|
| Task outcome | Whether the predefined success oracle is met | A complete-looking answer that misses the actual goal |
| Planning | Step order, dependencies, and response to new information | A skipped prerequisite or unsupported plan change |
| Multi-step consistency | Whether constraints, state, and user goals persist across steps | A later step loses an earlier constraint or uses stale state |
| Tool use | Selection, arguments, permissions, and result interpretation | A correct-looking answer after an unsafe or irrelevant call |
| Memory | What state persists, resets, or leaks between tasks | One user’s data or an earlier task influencing the next run |
| Recovery | Retry, fallback, and human handoff after a failed step | A retry repeats side effects (an idempotency failure), or the agent continues after a blocker |
| Termination | Success, goal failure, budget exhaustion, cancellation, safety block, or environment error | Missing terminal evidence being mistaken for completion |
| Cost and latency | Actual consumption against project-defined budgets or targets | Good-looking output that hides excessive cost or timeout |
Check tool permissions as part of the safety boundary. If the project has not defined cost, latency, or pass thresholds, list them as open questions instead of inventing values.
Repeat trials when behavior can vary
Use the same sample IDs and record model, prompt, tool, data, and environment versions. Follow the project’s repetition plan; do not invent a sample threshold. Report per-sample pass/fail counts and keep intermittent failures visible. A single successful run shows one observed result, not reliability.
For example, a multi-step support task may pass on its final answer but fail its process check if the trace shows an unapproved data lookup. Judge outcome correctness and process compliance separately; either failure can prevent an overall pass. Preserve run and step IDs so reviewers can reconstruct the decision.
Record a multi-step task under one sample ID: define the success oracle, allowed tools and resources, prohibited side effects, and stopping conditions before the run. Then capture model, prompt, tool, data, and environment versions, and link each step’s input, arguments, authorization decision, tool response, and state change to a run ID. The table below illustrates how to interpret one run:
| Observation | Interpretation |
|---|---|
| The final answer meets the oracle and every step meets the process assertions defined for the case | This run meets the case’s outcome and process conditions |
| The final answer is correct, but the trace includes an unauthorized data lookup | Process failure; the answer does not offset it |
| The final answer looks correct, but a key step or terminal record is missing | Insufficient evidence to judge the full path |
The table describes one run. Repeat trials under the project’s plan and report each result before judging whether behavior is stable.
Produce cases with an oracle and evidence
Build a risk or failure model from the supplied input, then turn high-priority concerns into cases. Each proposed case needs preconditions, an action or stimulus, expected behavior, and required evidence. Separate static analysis or a designed fixture from a real run. If the project has not supplied a success rule, permission, or stop threshold, list it as an open question instead of creating one.
Use the project’s risk scheme, such as P0/P1/P2/P3, or an equivalent scale, and explain the basis for each priority. Link conclusions to a source, evidence state, and validation method; give high-risk items an owner or next action. Prioritize high-impact, likely, or hard-to-detect paths without presenting priority as verified severity.
The output can include a capability matrix, task suite, trace assertions, recovery and safety cases, scoring rubric, observability measures, blockers, and next actions. Keep confirmed facts, assumptions, and unknowns visible. The Skill helps structure the work; it does not certify the agent or a release.
Use Agent Tool Call Testing: https://inaodeng.com/en/qaskills/agent-tool-testing/ for detailed tool contracts and side effects. For an AI feature embedded in a product, see AI Feature Testing: https://inaodeng.com/en/qaskills/ai-feature-testing/; for untrusted instructions and attack paths, see Prompt Injection Testing: https://inaodeng.com/en/qaskills/prompt-injection-testing/.
Focus: put Agent testing into a quality loop
Keep the same samples and raw traces each round, then use evidence to judge whether a change to the test method or system helped:
- Freeze the baseline: retain the task, success oracle, allowed tools, prohibited side effects, stopping conditions, model, and environment versions.
- Run under the plan: follow the project’s repetition plan. If none exists, mark it open instead of inventing a sample count.
- Review result and trace together: link the final answer to each step’s input, call, authorization, response, and state change.
- Classify failures and gaps: distinguish task rules, Prompt, tool contract, memory, recovery, environment variation, and missing evidence.
- Revise one evidence-backed factor: record the version and reason, and retain the original failing sample.
- Regress on the same samples and add a new one: report each result and intermittent failure; keep unrun checks marked as not run.
Best practices: make outcomes and process reviewable
- Use a sourced success oracle for the task outcome; assess process compliance, permissions, and side effects separately.
- Start with high-impact, likely, or hard-to-detect risks, then expand to more capabilities and environments.
- Preserve every run/step trace and version; do not use one successful run as evidence of reliability.
- For production, safety, or privacy work, use least privilege, redacted data, mocks, or an isolated environment.
- Put stop, escalation, rollback, and human-handoff conditions in each case; keep missing conditions open.
Install and invoke
Install the Skill:
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ai-agent-testingProvide the task goal, success oracle, allowed tools and resources, prohibited side effects, stopping conditions, and a redacted run/step trace. Start with one bounded sample:
Review this multi-step task with ai-agent-testing.
Task goal and success oracle: […]
Allowed tools and resources: […]
Prohibited side effects and stopping conditions: […]
Available redacted run/step trace: […]
Judge the outcome against the oracle and the process against its constraints separately. List missing evidence and open questions. Do not invent a pass threshold or describe a designed case as an executed test.FAQ
Does a correct final answer make the case a pass? Not by itself. Check whether the trace meets the case’s permission, tool, step, and stopping assertions. If a key part of the trace is missing, report insufficient evidence.
How many times should a task be repeated? Follow the repetition plan defined by the project. If the plan or stability threshold is missing, list it as an open question instead of inventing a count or pass threshold.
Can process compliance be judged without a run/step trace? No. You can review the test design or available output, but a process judgment should remain insufficient evidence until the minimum trace is supplied.
References
- AI Agent Testing prompt: https://github.com/naodeng/awesome-qa-skills/blob/main/skills/en/testing-types/ai-agent-testing/prompts/ai-agent-testing.md
- AI Agent Testing Skill: https://github.com/naodeng/awesome-qa-skills/blob/main/skills/en/testing-types/ai-agent-testing/SKILL.md
- AI Agent Testing source directory: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ai-agent-testing
- AI Agent Testing directory on this site: https://inaodeng.com/en/qaskills/ai-agent-testing/