Observability starts with a question an operator must answer during a failure: which user flow is affected, where did it break, and what evidence can narrow the cause? This Skill reviews whether the telemetry design can answer that question before implementation.

When this Skill helps

Use Observability Design Review for logging, metrics, traces, context propagation, SLO and SLI definitions, alerts, sampling, retention, privacy, and cost. It reviews supplied designs and produces OBS-## findings. It does not query production signals, run probes, assess an incident, or declare system health.

Review workflow

Start by naming the service boundary, business flow, environment, version, and time window. Classify each input as known, missing, conflicting, stale, out_of_scope, or assumptions. Then trace a failure path across the signals that should explain it.

Signal areaQuestions to answer
LogsWhich event and fields identify the object and outcome? Are sensitive values masked?
MetricsAre units, dimensions, cardinality, and aggregation semantics explicit?
TracesDo spans preserve context across service and asynchronous boundaries?
SLOs and SLIsAre service objectives, indicators, windows, and measurement rules defined by a supplied source?
DashboardsDoes each panel support a named operator or product decision, with units and context visible?
AlertsWhat condition triggers the alert, who responds, and what action follows?
OperationsAre sampling, retention, access, and cost constraints stated?

Do not fill in missing SLOs, thresholds, retention periods, or sample rates from platform defaults. Record the source and evidence state for each conclusion.

Worked example

The following are illustrative design-review examples, not claims about a live service:

Supplied materialWhat the review can concludeWhat remains unproven
The checkout design lists a gateway request ID, payment request ID, and order event, but no join ruleThe design lacks a cross-step correlation contract and needs evidenceThis does not show that runtime traces are necessarily broken
An alert rule has a trigger condition but no response owner or runbook actionThe response handoff is incomplete; ask for the owner role and first diagnostic actionThis does not prove that the alert fired or that on-call can complete the response

For follow-up, a masked synthetic request can check the planned trace path. Retain the request identity, environment, time window, and raw signals. A document-only result remains a design-review conclusion.

What to hand off

Each OBS-## should identify the signal and covered object, source and evidence status, expected fields or semantics, gap and impact, detection action, owner role, close condition, and validation method. Keep design facts, evidence-backed inferences, recommendations, and Human decisions separate. If runtime identity, time, environment, or raw signals are missing, mark runtime conclusions unverified or unassessed.

Best practices

  • Prioritize high-impact gaps and ask for the smallest evidence or safe check that can resolve them.
  • Keep facts, evidence-backed inferences, recommendations, and Human decisions distinct; do not turn assumptions into conclusions.

Boundaries

A dashboard proves that a view was designed; it does not prove that an alert fires or that a service is healthy. Do not choose SLOs, incident severity, sample rates, retention, or cost budgets for the owner. Keep production data out of a design-only review.

Install and invoke

Install the individual Skill from the Awesome QA Skills repository:

npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en --skill observability-design-review

Then invoke observability-design-review with the telemetry design, scope, environment, available evidence, and the decision the review should support.

Use the observability-design-review Skill.
Service and flow: [boundary, version, and environment].
Review material: [log, metric, trace, alert, and SLI/SLO designs].
Decision to support: [the operator or product question].
Constraints: [privacy, retention, sampling, and cost rules].
Identify missing or conflicting evidence and propose the smallest safe validation. Do not query production or infer service health from design documents.

FAQ

Does a dashboard design prove that the service is observable?

No. It shows that a view is planned. Runtime observability needs actual signals with identity, time, environment, and enough linked evidence to answer the diagnostic question.

What if the SLI or alert threshold is missing?

Mark it as missing and name the owner who must define it. Do not borrow a platform default or invent a target.

Source

Read the Observability Design Review prompt: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/observability-design-review/prompts/observability-design-review.md before using the Skill. The source directory: https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/observability-design-review contains its entry point and supporting evaluation material.

More Resources

Share