Combinatorial Testing Skill: Cover interacting factors without multiplying every possibility
A checkout flow can vary by payment method, currency, locale, device, and network. Testing every combination is expensive; testing none of the interactions is worse.
Why this Skill is needed
Combinatorial Testing selects a small set of factor combinations that covers important interactions while preserving constraints and risk priorities.
What the Skill does
The Combinatorial Testing Skill is useful when a team needs a structured review of factors, levels, constraints, pair or t-way coverage, risk weighting, and explainable selection.
Awesome QA Skills organizes Skills by language and testing stage. The series overview explains the shared installation model; this guide stays with Combinatorial Testing.
A practical standard for this Skill is simple: A good combination design starts with the risk model and treats constraints as first-class input. It can explain why one rare payment and locale combination was kept while dozens of low-risk pairs were omitted.
Use it when the team needs a repeatable review of factors, levels, constraints, pair or t-way coverage, risk weighting, and explainable selection—especially when several evidence sources disagree or a handoff must explain what remains unverified.
What it does not do
The Combinatorial Testing Skill can organize material, evidence, and next actions for Combination set. It does not replace:
- Confirmation of rules, scope, and risk by the accountable domain owner.
- A real environment, account, dataset, log, or run record; static analysis does not become runtime evidence by itself.
- Authorization for release, compliance, production actions, or residual-risk acceptance.
- A Human decision when supplied sources conflict.
What it checks
The value of this Skill is not another keyword list. It connects each focus area to an observable input, a judgment, and a way to close the loop. Start with a small matrix based on the source Skill’s output contract:
| Focus | Question before analysis | Handoff output |
|---|---|---|
| Combination set | Selected cases and factor values | Constraint-aware |
| Coverage claim | Pairs or t-way interactions covered | Scope and exceptions |
| Risk gap | High-risk combination not represented | Owner and next case |
If a row has only a conventional expectation and no source or validation method, keep it open instead of turning it into a pass.
Audit inputs before you start
Before analyzing Combination set, classify the input into six evidence states. A gap is not automatically a failure, but it must not disappear inside the conclusion.
| State | Meaning | How this Skill should handle it |
|---|---|---|
| known | Directly supported by the supplied material | Keep the source, version, and time with the judgment |
| missing | Needed for this pass but not supplied | Name the smallest evidence action and limit the conclusion |
| conflicting | Sources disagree | Show both sources and route the conflict to an owner |
| stale | Present but outside the relevant version or time window | Mark freshness; old evidence is not current proof |
| out_of_scope | Related but excluded from this pass | Keep the boundary explicit |
| assumptions | Temporarily adopted to continue analysis | State how and when the assumption will be checked |
Keep the input version, scope, environment, evidence locations, and accountable owner together. Without a run record, deliver analysis, design, or a validation plan—not an execution pass.
From problem to structured Finding
Connect the source, scope, evidence state, analysis, owner, action, close condition, and validation before writing the conclusion. The case below keeps this Skill’s identifier and domain context.
Keep the decision layers separate
For Combination set, do not compress four different kinds of language into “recommended to pass”:
| Layer | How to write it | Application here |
|---|---|---|
| Fact | What the supplied material directly shows | Cite the source, version, input, or run record for the focus |
| Evidence-backed Inference | What several facts support together | Show the inference chain and retain uncertainty |
| Recommendation | The smallest next action | Name the evidence, review, execution, or regression path |
| Human Decision | What an accountable person must decide | Leave scope, risk acceptance, resources, and release meaning to the owner |
A complete case
This case follows Input, Analysis, Finding, Decision, and Validation. When material is incomplete, keep missing, conflicting, or assumptions visible instead of turning them into a pass.
Input
| Material | What to provide | What to do when it is missing |
|---|---|---|
| Factor model | Factors, values, pair or t-way target, and business risk | Do not treat every value as equally important |
| Constraints | Impossible or conditional combinations, account rules, and environment limits | Record constraints before generation |
| Expected result | Coverage report, selected cases, and uncovered high-risk interactions | Make selection explainable |
Use a request like this:
Use the combinatorial-testing Skill.
Task: design a checkout matrix across payment type, currency, locale, device, and network without testing impossible combinations
Inputs: [requirements, versions, links, logs, reports, or data paths]
Scope: [included and excluded objects]
Unknowns: [missing environment, accounts, data, or permissions]
Expected output: [risk-ordered findings, evidence status, and next actions]
Audit the inputs first. Separate facts, assumptions, and open questions. Do not claim execution without a run record.Analysis
Start with a bounded pass—design a checkout matrix across payment type, currency, locale, device, and network without testing impossible combinations. Keep card, wallet, and bank transfer across two currencies and two network conditions, but exclude wallet payments in unsupported countries. Weight currency conversion and offline recovery pairs higher than cosmetic locale differences.
The handoff should preserve the input version, time window, evidence index, owner, and next validation action. The Skill can organize uncertainty; it cannot manufacture the missing artifact.
Finding
Example finding: turn one problem into a handoff
The field example below shows the recording pattern; it is not an execution result.
If the supplied material cannot prove that Combination set meets its contract, write the finding like this. It does not invent the missing rule or turn missing evidence into a failure.
| Field | Example wording |
|---|---|
| Source and scope | Record the requirement, version, environment, and the concrete object for Combination set |
| Finding | The condition or result for Combination set is not yet traceable to evidence |
| Evidence state | missing / assumptions; use conflicting when sources disagree |
| Impact and priority | Name the affected user, journey, or delivery decision without inflating severity |
| Owner and Human decision | Ask the product, engineering, security, or test owner to confirm the rule and trade-off |
| Action and close condition | Add the smallest missing evidence; close only when source, judgment, and owner can be reviewed |
| Validation | Name one repeatable check, query, or run and retain the raw artifact |
The point is to let the next person walk from the finding back to the source and run an action that can change the decision.
Decision
The accountable owner confirms the decision question and risk trade-off; the Skill does not make that choice.
Validation
Before closing the finding, run the stated validation and retain the raw artifact. Without an execution record, the status remains unverified.
How a Finding enters the next stage
Handoff output
| Output field | Why it exists | Example status |
|---|---|---|
| Combination set | Selected cases and factor values | Constraint-aware |
| Coverage claim | Pairs or t-way interactions covered | Scope and exceptions |
| Risk gap | High-risk combination not represented | Owner and next case |
Do not write “passed” without a run record, query result, or source artifact. A good combination design starts with the risk model and treats constraints as first-class input. It can explain why one rare payment and locale combination was kept while dozens of low-risk pairs were omitted.
Next-stage route
At minimum, hand off the source, evidence state, owner, close condition, and validation action; the next-stage conclusion remains bounded by the evidence state.
How to prepare better input
If the first request contains only a one-line goal, keep the output limited. Add the source version, affected objects, environment, known defects, and decision owner to move from a plausible checklist to a useful review.
A richer input changes the answer here because factors, levels, constraints, pair or t-way coverage, risk weighting, and explainable selection must be tied to evidence rather than inferred from a familiar pattern.
Working with other Skills
- Pairwise Testing:provides focused two-factor coverage.
- Equivalence Partitioning:reduces factor levels before combination.
- Decision Table Testing:handles conditional combinations.
Common traps
- Generating combinations before writing constraints.
- Reporting pair coverage without naming excluded pairs.
- Using the smallest suite without preserving high-risk interactions.
Install and invoke
Install the individual Skill. The series overview carries the longer installation explanation.
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/combinatorial-testing -gAfter installation, invoke it with the combinatorial-testing Skill and attach the real project material.
FAQ
Is pairwise coverage always enough?
No. Payment, security, and state interactions may require three or more factors; choose t-way coverage from risk.
Can impossible combinations stay in the matrix?
They can be documented as constraints, but they should not consume executable test slots.
The example above is a design and review pattern, not an execution result. Keep static analysis, runtime evidence, human approval, and release acceptance separate.
References
Source Skill and execution contract
The complete execution contract lives in the Combinatorial Testing prompt. Read it before invoking the Skill; the prompt defines the detailed workflow and output contract. The source directory contains the entry point and supporting assets where they exist.
The entry point centers factors, levels, constraints, pair or t-way coverage, risk weighting, and explainable selection. Keep its decision boundary visible and do not turn a static design into an execution claim.
Reference links
-
Combinatorial Testing prompt:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/combinatorial-testing/prompts/combinatorial-testing.md
-
Awesome QA Skills: Combinatorial Testing source:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/combinatorial-testing
-
Awesome QA Skills GitHub:https://github.com/naodeng/awesome-qa-skills
-
Combinatorial Testing details:https://inaodeng.com/en/qaskills/combinatorial-testing/