Example-based tests show that a few inputs work. Property-based tests ask whether a rule survives a much wider input space—and give you the smallest counterexample when it does not.

Why this Skill is needed

Property-Based Testing defines invariants and generates many inputs to explore them, while preserving seeds and shrinking failures into useful cases.

What the Skill does

The Property-Based Testing Skill is useful when a team needs a structured review of property, generator, domain, seed, shrinking, invariant, and counterexample.

Awesome QA Skills organizes Skills by language and testing stage. The series overview explains the shared installation model; this guide stays with Property-Based Testing.

A practical standard for this Skill is simple: A strong property test has a domain property, not merely a generic assertion like “the function does not crash.” The property should matter to the user or system state.

Use it when the team needs a repeatable review of property, generator, domain, seed, shrinking, invariant, and counterexample—especially when several evidence sources disagree or a handoff must explain what remains unverified.

What it does not do

The Property-Based Testing Skill can organize material, evidence, and next actions for Property result. It does not replace:

  • Confirmation of rules, scope, and risk by the accountable domain owner.
  • A real environment, account, dataset, log, or run record; static analysis does not become runtime evidence by itself.
  • Authorization for release, compliance, production actions, or residual-risk acceptance.
  • A Human decision when supplied sources conflict.

What it checks

The value of this Skill is not another keyword list. It connects each focus area to an observable input, a judgment, and a way to close the loop. Start with a small matrix based on the source Skill’s output contract:

FocusQuestion before analysisHandoff output
Property resultInvariant held, violated, or untestableRun and seed
CounterexampleMinimal input that breaks the propertyReproducible fixture
Domain gapGenerator bias or missing meaningful classGenerator update

If a row has only a conventional expectation and no source or validation method, keep it open instead of turning it into a pass.

Audit inputs before you start

Before analyzing Property result, classify the input into six evidence states. A gap is not automatically a failure, but it must not disappear inside the conclusion.

StateMeaningHow this Skill should handle it
knownDirectly supported by the supplied materialKeep the source, version, and time with the judgment
missingNeeded for this pass but not suppliedName the smallest evidence action and limit the conclusion
conflictingSources disagreeShow both sources and route the conflict to an owner
stalePresent but outside the relevant version or time windowMark freshness; old evidence is not current proof
out_of_scopeRelated but excluded from this passKeep the boundary explicit
assumptionsTemporarily adopted to continue analysisState how and when the assumption will be checked

Keep the input version, scope, environment, evidence locations, and accountable owner together. Without a run record, deliver analysis, design, or a validation plan—not an execution pass.

From problem to structured Finding

Connect the source, scope, evidence state, analysis, owner, action, close condition, and validation before writing the conclusion. The case below keeps this Skill’s identifier and domain context.

Keep the decision layers separate

For Property result, do not compress four different kinds of language into “recommended to pass”:

LayerHow to write itApplication here
FactWhat the supplied material directly showsCite the source, version, input, or run record for the focus
Evidence-backed InferenceWhat several facts support togetherShow the inference chain and retain uncertainty
RecommendationThe smallest next actionName the evidence, review, execution, or regression path
Human DecisionWhat an accountable person must decideLeave scope, risk acceptance, resources, and release meaning to the owner

A complete case

This case follows Input, Analysis, Finding, Decision, and Validation. When material is incomplete, keep missing, conflicting, or assumptions visible instead of turning them into a pass.

Input

MaterialWhat to provideWhat to do when it is missing
PropertyInvariant, relation, or safety ruleWrite the oracle before the generator
Generator domainValid, invalid, boundary, and dependent dataConstrain unrealistic values
Failure evidenceSeed, shrunk case, versions, and intermediate valuesKeep the minimal counterexample

Use a request like this:

Use the property-based-testing Skill.

Task: test a cart total calculation across item quantities, discounts, currencies, and rounding
Inputs: [requirements, versions, links, logs, reports, or data paths]
Scope: [included and excluded objects]
Unknowns: [missing environment, accounts, data, or permissions]
Expected output: [risk-ordered findings, evidence status, and next actions]

Audit the inputs first. Separate facts, assumptions, and open questions. Do not claim execution without a run record.

Analysis

Start with a bounded pass—test a cart total calculation across item quantities, discounts, currencies, and rounding. Generate carts with quantities and decimal prices, then assert that adding an item cannot lower the total unless a documented promotion applies. Preserve the seed and shrink a failing cart to the smallest discount-and-rounding case.

The handoff should preserve the input version, time window, evidence index, owner, and next validation action. The Skill can organize uncertainty; it cannot manufacture the missing artifact.

Finding

Example finding: turn one problem into a handoff

The field example below shows the recording pattern; it is not an execution result.

If the supplied material cannot prove that Property result meets its contract, write the finding like this. It does not invent the missing rule or turn missing evidence into a failure.

FieldExample wording
Source and scopeRecord the requirement, version, environment, and the concrete object for Property result
FindingThe condition or result for Property result is not yet traceable to evidence
Evidence statemissing / assumptions; use conflicting when sources disagree
Impact and priorityName the affected user, journey, or delivery decision without inflating severity
Owner and Human decisionAsk the product, engineering, security, or test owner to confirm the rule and trade-off
Action and close conditionAdd the smallest missing evidence; close only when source, judgment, and owner can be reviewed
ValidationName one repeatable check, query, or run and retain the raw artifact

The point is to let the next person walk from the finding back to the source and run an action that can change the decision.

Decision

The accountable owner confirms the decision question and risk trade-off; the Skill does not make that choice.

Validation

Before closing the finding, run the stated validation and retain the raw artifact. Without an execution record, the status remains unverified.

How a Finding enters the next stage

Handoff output

Output fieldWhy it existsExample status
Property resultInvariant held, violated, or untestableRun and seed
CounterexampleMinimal input that breaks the propertyReproducible fixture
Domain gapGenerator bias or missing meaningful classGenerator update

Do not write “passed” without a run record, query result, or source artifact. A strong property test has a domain property, not merely a generic assertion like “the function does not crash.” The property should matter to the user or system state.

Next-stage route

At minimum, hand off the source, evidence state, owner, close condition, and validation action; the next-stage conclusion remains bounded by the evidence state.

How to prepare better input

If the first request contains only a one-line goal, keep the output limited. Add the source version, affected objects, environment, known defects, and decision owner to move from a plausible checklist to a useful review.

A richer input changes the answer here because property, generator, domain, seed, shrinking, invariant, and counterexample must be tied to evidence rather than inferred from a familiar pattern.

Working with other Skills

Common traps

  1. Generating unrealistic data that hides real behavior.
  2. Using a property weaker than the business invariant.
  3. Losing the random seed or shrunk example after a failure.

Install and invoke

Install the individual Skill. The series overview carries the longer installation explanation.

npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/property-based-testing -g

After installation, invoke it with the property-based-testing Skill and attach the real project material.

FAQ

Can property-based tests replace examples?

They complement examples; representative examples still explain known rules and support debugging.

What if shrinking removes the meaningful context?

Keep the shrunk counterexample and the original generated case when context affects the interpretation.

The example above is a design and review pattern, not an execution result. Keep static analysis, runtime evidence, human approval, and release acceptance separate.

References

Source Skill and execution contract

The complete execution contract lives in the Property-Based Testing prompt. Read it before invoking the Skill; the prompt defines the detailed workflow and output contract. The source directory contains the entry point and supporting assets where they exist.

The entry point centers property, generator, domain, seed, shrinking, invariant, and counterexample. Keep its decision boundary visible and do not turn a static design into an execution claim.

ACP (Agent Client Protocol)Active MemoryAgent EvalsAgentic WorkflowAI Agent
Share