Some systems are too large or too probabilistic to tell you the exact correct answer for every input. You can still ask whether a controlled input change should preserve or change a relationship.

Why this Skill is needed

Metamorphic Testing uses transformations and relations between executions to test systems whose exact oracle is expensive, incomplete, or unavailable.

What the Skill does

The Metamorphic Testing Skill is useful when a team needs a structured review of input transformation, expected relation, invariant, oracle limitation, randomized coverage, and violation evidence.

Awesome QA Skills organizes Skills by language and testing stage. The series overview explains the shared installation model; this guide stays with Metamorphic Testing.

A practical standard for this Skill is simple: A good metamorphic design states the relation before generating cases. “The result should be similar” is not an oracle until similar means a set overlap, monotonic score, or preserved invariant.

Use it when the team needs a repeatable review of input transformation, expected relation, invariant, oracle limitation, randomized coverage, and violation evidence—especially when several evidence sources disagree or a handoff must explain what remains unverified.

What it does not do

The Metamorphic Testing Skill can organize material, evidence, and next actions for Metamorphic relation. It does not replace:

  • Confirmation of rules, scope, and risk by the accountable domain owner.
  • A real environment, account, dataset, log, or run record; static analysis does not become runtime evidence by itself.
  • Authorization for release, compliance, production actions, or residual-risk acceptance.
  • A Human decision when supplied sources conflict.

What it checks

The value of this Skill is not another keyword list. It connects each focus area to an observable input, a judgment, and a way to close the loop. Start with a small matrix based on the source Skill’s output contract:

FocusQuestion before analysisHandoff output
Metamorphic relationTransformation and expected relationSource and confidence
ViolationWhat changed beyond the relationInput pair and artifact
Follow-upOracle, model, data, or implementation checkOwner and next run

If a row has only a conventional expectation and no source or validation method, keep it open instead of turning it into a pass.

Audit inputs before you start

Before analyzing Metamorphic relation, classify the input into six evidence states. A gap is not automatically a failure, but it must not disappear inside the conclusion.

StateMeaningHow this Skill should handle it
knownDirectly supported by the supplied materialKeep the source, version, and time with the judgment
missingNeeded for this pass but not suppliedName the smallest evidence action and limit the conclusion
conflictingSources disagreeShow both sources and route the conflict to an owner
stalePresent but outside the relevant version or time windowMark freshness; old evidence is not current proof
out_of_scopeRelated but excluded from this passKeep the boundary explicit
assumptionsTemporarily adopted to continue analysisState how and when the assumption will be checked

Keep the input version, scope, environment, evidence locations, and accountable owner together. Without a run record, deliver analysis, design, or a validation plan—not an execution pass.

From problem to structured Finding

Connect the source, scope, evidence state, analysis, owner, action, close condition, and validation before writing the conclusion. The case below keeps this Skill’s identifier and domain context.

Keep the decision layers separate

For Metamorphic relation, do not compress four different kinds of language into “recommended to pass”:

LayerHow to write itApplication here
FactWhat the supplied material directly showsCite the source, version, input, or run record for the focus
Evidence-backed InferenceWhat several facts support togetherShow the inference chain and retain uncertainty
RecommendationThe smallest next actionName the evidence, review, execution, or regression path
Human DecisionWhat an accountable person must decideLeave scope, risk acceptance, resources, and release meaning to the owner

A complete case

This case follows Input, Analysis, Finding, Decision, and Validation. When material is incomplete, keep missing, conflicting, or assumptions visible instead of turning them into a pass.

Input

MaterialWhat to provideWhat to do when it is missing
System behaviorInput, output, known invariant, and acceptable variationState what cannot be asserted exactly
TransformationSynonym, permutation, scaling, duplication, or irrelevant-field changeKeep the transformation isolated
EvidenceOriginal and transformed outputs, relation check, model version, and seedRecord the first violation

Use a request like this:

Use the metamorphic-testing Skill.

Task: test a product-search ranker when adding a synonym should preserve the relevant product set even if exact scores change
Inputs: [requirements, versions, links, logs, reports, or data paths]
Scope: [included and excluded objects]
Unknowns: [missing environment, accounts, data, or permissions]
Expected output: [risk-ordered findings, evidence status, and next actions]

Audit the inputs first. Separate facts, assumptions, and open questions. Do not claim execution without a run record.

Analysis

Start with a bounded pass—test a product-search ranker when adding a synonym should preserve the relevant product set even if exact scores change. Add a synonym to the search query and compare the top-20 product IDs. Allow ranking movement, but require a minimum relevant-set overlap and no appearance of products excluded by the category filter.

The handoff should preserve the input version, time window, evidence index, owner, and next validation action. The Skill can organize uncertainty; it cannot manufacture the missing artifact.

Finding

Example finding: turn one problem into a handoff

The field example below shows the recording pattern; it is not an execution result.

If the supplied material cannot prove that Metamorphic relation meets its contract, write the finding like this. It does not invent the missing rule or turn missing evidence into a failure.

FieldExample wording
Source and scopeRecord the requirement, version, environment, and the concrete object for Metamorphic relation
FindingThe condition or result for Metamorphic relation is not yet traceable to evidence
Evidence statemissing / assumptions; use conflicting when sources disagree
Impact and priorityName the affected user, journey, or delivery decision without inflating severity
Owner and Human decisionAsk the product, engineering, security, or test owner to confirm the rule and trade-off
Action and close conditionAdd the smallest missing evidence; close only when source, judgment, and owner can be reviewed
ValidationName one repeatable check, query, or run and retain the raw artifact

The point is to let the next person walk from the finding back to the source and run an action that can change the decision.

Decision

The accountable owner confirms the decision question and risk trade-off; the Skill does not make that choice.

Validation

Before closing the finding, run the stated validation and retain the raw artifact. Without an execution record, the status remains unverified.

How a Finding enters the next stage

Handoff output

Output fieldWhy it existsExample status
Metamorphic relationTransformation and expected relationSource and confidence
ViolationWhat changed beyond the relationInput pair and artifact
Follow-upOracle, model, data, or implementation checkOwner and next run

Do not write “passed” without a run record, query result, or source artifact. A good metamorphic design states the relation before generating cases. “The result should be similar” is not an oracle until similar means a set overlap, monotonic score, or preserved invariant.

Next-stage route

At minimum, hand off the source, evidence state, owner, close condition, and validation action; the next-stage conclusion remains bounded by the evidence state.

How to prepare better input

If the first request contains only a one-line goal, keep the output limited. Add the source version, affected objects, environment, known defects, and decision owner to move from a plausible checklist to a useful review.

A richer input changes the answer here because input transformation, expected relation, invariant, oracle limitation, randomized coverage, and violation evidence must be tied to evidence rather than inferred from a familiar pattern.

Working with other Skills

Common traps

  1. Choosing a transformation without a domain relation.
  2. Using vague similarity thresholds.
  3. Treating a relation violation as a product defect before checking data and model drift.

Install and invoke

Install the individual Skill. The series overview carries the longer installation explanation.

npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/metamorphic-testing -g

After installation, invoke it with the metamorphic-testing Skill and attach the real project material.

FAQ

Is metamorphic testing only for ML?

No. Parsers, compilers, APIs, calculations, and workflows can all have useful relations.

What if the relation is uncertain?

Mark it as a hypothesis and validate the domain expectation before using it as a failure oracle.

The example above is a design and review pattern, not an execution result. Keep static analysis, runtime evidence, human approval, and release acceptance separate.

References

Source Skill and execution contract

The complete execution contract lives in the Metamorphic Testing prompt. Read it before invoking the Skill; the prompt defines the detailed workflow and output contract. The source directory contains the entry point and supporting assets where they exist.

The entry point centers input transformation, expected relation, invariant, oracle limitation, randomized coverage, and violation evidence. Keep its decision boundary visible and do not turn a static design into an execution claim.

ACP (Agent Client Protocol)Active MemoryAgent EvalsAgentic WorkflowAI Agent
Share