Playwright UI Automation Testing: Build reliable regression with stable locators and controlled data
Intermittent Playwright failures rarely originate in the assertion alone. Brittle locators, implicit timing, and shared data can make a green run look more trustworthy than it is.
The Playwright UI Testing Skill builds checks around observable states, durable locators, and isolated data so a failure explains what broke.
This guide uses reproducible scenarios to show how to apply it: define the risk and evidence first, then organize the checks and results into a defensible next step.
Awesome QA Skills organizes Skills by language and testing stage. The series overview covers repository structure and shared installation options; this guide stays with Playwright UI Testing.
Read the source Skill first
The main prompt covers Quality Bar, Output Format Options, How to Use, Reference Files, Common Pitfalls. Those headings are navigation; the project artifacts still provide the facts.
The source directory contains 1 example files, 2 references, 1 script entries. Start with Test scenario context example, Framework guide, Test execution script.
From artifacts to a runnable entry point
The task is concrete: Automate the checkout journey with Playwright, including stable locators, waits, isolated data, and failure evidence
The input can stay short, but it needs facts.
Journey: sign in → create order → pay → read result
Environment: staging
Available artifacts: interface definition, test account, CI command
Deliverable: tests/e2e/checkout.spec.ts, plus the local command and failure evidenceThe Skill should confirm versions, authentication, and data cleanup before generating files. This output fragment demonstrates structure; it does not claim a run occurred.
tool: Playwright
entry: tests/e2e/checkout.spec.ts
checks: user-facing locators, isolated data, trace or screenshot evidence
run_evidence: pendingKeep run_evidence pending until a command, report, or trace exists. The source prompt also calls out Quality Bar, Output Format Options, How to Use.
Turn the fragment into a project skeleton
A code fragment becomes useful when its path, command, and artifacts are explicit. Start with one journey.
tests/e2e/checkout.spec.ts
├── scenario and assertions
├── data or feeder
├── environment configuration
└── failure artifacts written to artifacts/Use one reproducible local and CI command.
npx playwright test tests/e2e/checkout.spec.ts --trace on-first-retryPrefer role, label, and text locators; use storage state only when the test is not about sign-in.
Definition of integrated
| Check | Minimum bar | If it fails |
|---|---|---|
| Repeatability | A run does not depend on leftover data | Rework setup and cleanup |
| Diagnosis | HTML report, trace.zip, screenshots, and video when enabled identifies the same run | Add a run ID and build ID |
| CI decision | Process exit status matches the quality gate | Fix reporter or threshold configuration |
| Maintenance | Shared authentication and setup have one edit point | Extract a fixture, specification, or user action |
Expand into errors, boundaries, and concurrency only after this journey behaves the same locally and in CI.
A prompt you can adapt
Replace the bracketed fields with project facts. Specific material leaves less room for guessing.
Use the ui-test-playwright Skill.
Task: Automate the checkout journey with Playwright, including stable locators, waits, isolated data, and failure evidence
Version and environment: [requirement / build / environment]
Inputs: [file paths or links]
Scope: [included and excluded journeys]
Constraints: [accounts, data, time, compliance]
Check framework version, paths, and authentication first. Generate the smallest runnable entry, command, and artifact list. Mark unexecuted code as not verified.
Finish with open questions. Do not invent missing facts.Use the first pass to inspect structure and gaps. Supply missing material before asking for the handoff-ready artifact.
Advanced use, from one call to a maintained flow
Use projects for browser and environment differences and shards for CI duration. Enable retries only in CI and keep the first-retry trace to classify product failures and flaky tests.
Keep a baseline for duration, pass rate, flaky cases, failure classes, and evidence completeness. Pass rate alone hides too much.
A three-Skill chain
requirements-analysis → ui-test-playwright → test-reporting
| Handoff | Payload | Receiver check |
|---|---|---|
| Upstream to ui-test-playwright | Source versions, scope, risks, open questions | Playwright UI Testing staleness and conflicts |
| ui-test-playwright to downstream | Primary artifact, evidence index, unfinished work | Playwright UI Testing executability and owners |
| Feedback to ui-test-playwright | Runs, defects, new risks | Playwright UI Testing baseline and regression update |
Do not paste three complete outputs into one large prompt. Give Playwright UI Testing a structured summary and accessible source artifacts. It saves context and makes defects traceable.
Team gates
| Gate | Check | Failure action |
|---|---|---|
| ui-test-playwright input | Version, environment, owner, accessible sources | Stop Playwright UI Testing and list gaps |
| ui-test-playwright artifact | Material claims carry basis and status | Return Playwright UI Testing for evidence |
| ui-test-playwright execution | Command, exit status, report are reproducible | Classify infrastructure or test failure |
| ui-test-playwright decision | Residual risks have accepter and date | Do not enter the next stage |
Review Playwright UI Testing adoption, human edit rate, unsupported claims, and failure-to-diagnosis time each sprint. Record a baseline for several cycles before setting targets.
Common failure modes for this tool family
- Code is generated without a run command, leaving the next person unable to verify it.
- Versions and dependencies are omitted even though Playwright configuration and reporters change.
- Tests share dirty data. API, UI, and performance suites all suffer from leftovers.
- One green run is described as long-term stability. Keep reports, logs, and retry evidence.
Install and invoke
Install the individual Skill. The series overview carries the longer installation explanation.
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ui-test-playwright -g -a codex -yInvoke it with “Use the ui-test-playwright Skill,” then attach the real artifacts.
Two practical questions
Will Playwright UI Testing hand me a runnable project?
With complete definitions, versions, paths, and dependencies, it can generate a strong starting point. You still need to install dependencies, run it in your repository, and fix environment differences.
When should generation stop?
Stop when authentication, test data, or the target version is unknown. More generation would only produce a polished guess.
What should be checked first after generation?
Confirm that the entry command discovers the target file and writes failure artifacts to the agreed path. Expand coverage after that works.
Can it enter a release gate immediately?
Wait until local and CI runs use the same command, data resets cleanly, and evidence is traceable.
Run Playwright UI Testing against one real artifact and keep the input, output, and review notes. The fragments here establish structure; project evidence must still come from the project.
References
- Awesome QA Skills project:https://github.com/naodeng/awesome-qa-skills
- Awesome QA Skills series overview:https://inaodeng.com/en/blog/ai-testing/introduction_of_awesome_qa_skills/
- Test scenario context example:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ui-test-playwright/examples/sample-context.md
- Playwright UI Testing (EN) supporting references:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ui-test-playwright/references/framework-spec.md
- Playwright UI Testing (EN) helper script: run-tests.sh:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ui-test-playwright/scripts/run-tests.sh
- Awesome QA Skills: Playwright UI Testing (EN) Skill source:https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/ui-test-playwright
- Playwright UI Testing Skill details:https://inaodeng.com/en/qaskills/ui-test-playwright/