Observability starts with a question an operator must answer during a failure: which user flow is affected, where did it break, and what evidence can narrow the cause? This Skill reviews whether the telemetry design can answer that question before implementation.
When this Skill helps
Use Observability Design Review for logging, metrics, traces, context propagation, SLO and SLI definitions, alerts, sampling, retention, privacy, and cost. It reviews supplied designs and produces OBS-## findings. It does not query production signals, run probes, assess an incident, or declare system health.
Review workflow
Start by naming the service boundary, business flow, environment, version, and time window. Classify each input as known, missing, conflicting, stale, out_of_scope, or assumptions. Then trace a failure path across the signals that should explain it.
| Signal area | Questions to answer |
|---|---|
| Logs | Which event and fields identify the object and outcome? Are sensitive values masked? |
| Metrics | Are units, dimensions, cardinality, and aggregation semantics explicit? |
| Traces | Do spans preserve context across service and asynchronous boundaries? |
| SLOs and SLIs | Are service objectives, indicators, windows, and measurement rules defined by a supplied source? |
| Dashboards | Does each panel support a named operator or product decision, with units and context visible? |
| Alerts | What condition triggers the alert, who responds, and what action follows? |
| Operations | Are sampling, retention, access, and cost constraints stated? |
Do not fill in missing SLOs, thresholds, retention periods, or sample rates from platform defaults. Record the source and evidence state for each conclusion.
Worked example
The following are illustrative design-review examples, not claims about a live service:
| Supplied material | What the review can conclude | What remains unproven |
|---|---|---|
| The checkout design lists a gateway request ID, payment request ID, and order event, but no join rule | The design lacks a cross-step correlation contract and needs evidence | This does not show that runtime traces are necessarily broken |
| An alert rule has a trigger condition but no response owner or runbook action | The response handoff is incomplete; ask for the owner role and first diagnostic action | This does not prove that the alert fired or that on-call can complete the response |
For follow-up, a masked synthetic request can check the planned trace path. Retain the request identity, environment, time window, and raw signals. A document-only result remains a design-review conclusion.
What to hand off
Each OBS-## should identify the signal and covered object, source and evidence status, expected fields or semantics, gap and impact, detection action, owner role, close condition, and validation method. Keep design facts, evidence-backed inferences, recommendations, and Human decisions separate. If runtime identity, time, environment, or raw signals are missing, mark runtime conclusions unverified or unassessed.
Best practices
- Prioritize high-impact gaps and ask for the smallest evidence or safe check that can resolve them.
- Keep facts, evidence-backed inferences, recommendations, and Human decisions distinct; do not turn assumptions into conclusions.
Boundaries
A dashboard proves that a view was designed; it does not prove that an alert fires or that a service is healthy. Do not choose SLOs, incident severity, sample rates, retention, or cost budgets for the owner. Keep production data out of a design-only review.
Related Skills
- Log Analysis: https://inaodeng.com/en/qaskills/log-analysis/ examines supplied log evidence.
- Distributed Trace Analysis: https://inaodeng.com/en/qaskills/distributed-trace-analysis/ follows trace data across services.
- Quality Dashboard Design: https://inaodeng.com/en/qaskills/quality-dashboard-design/ organizes decision-facing panels after signal contracts are clear.
Install and invoke
Install the individual Skill from the Awesome QA Skills repository:
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en --skill observability-design-review
Then invoke observability-design-review with the telemetry design, scope, environment, available evidence, and the decision the review should support.
Use the observability-design-review Skill.
Service and flow: [boundary, version, and environment].
Review material: [log, metric, trace, alert, and SLI/SLO designs].
Decision to support: [the operator or product question].
Constraints: [privacy, retention, sampling, and cost rules].
Identify missing or conflicting evidence and propose the smallest safe validation. Do not query production or infer service health from design documents.
FAQ
Does a dashboard design prove that the service is observable?
No. It shows that a view is planned. Runtime observability needs actual signals with identity, time, environment, and enough linked evidence to answer the diagnostic question.
What if the SLI or alert threshold is missing?
Mark it as missing and name the owner who must define it. Do not borrow a platform default or invent a target.
Source
Read the Observability Design Review prompt: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/observability-design-review/prompts/observability-design-review.md before using the Skill. The source directory: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/observability-design-review contains its entry point and supporting evaluation material.
More Resources
- Personal website: https://inaodeng.com/en/
- AI-generated test detection project: https://github.com/AI-Native-QA-Lab/ai-test-auditor
- DeepSeek QA Workbench: https://github.com/naodeng/dsh-qa
- More AI testing skills: https://inaodeng.com/en/qaskills/
- More testing wiki: https://inaodeng.com/en/wiki/
- More AI Wiki: https://inaodeng.com/en/AIWiki/