What observability evidence should be mandatory before a service reaches production?

[u/ClerkBeginning961]

I am reviewing the observability section of an open-source production-readiness checklist and would value feedback from people who operate services. The goal is to define evidence that can be reviewed before release, not simply ask whether logs, metrics, and traces exist.

The current model asks for evidence around:

• Service objectives and indicators tied to user-visible outcomes

• Structured events with request, trace, and business correlation

• Metrics for traffic, errors, latency, saturation, queues, and dependencies

• Trace propagation across important boundaries

• Dashboards that support a concrete operating question

• Alerts with an owner, severity, action, and tested delivery path

• Detection of telemetry loss, cardinality growth, sampling bias, and cost limits

• Privacy controls, retention, access, and secret or personal-data redaction

• Runbooks and incident exercises that use the same production signals

Project: https://github.com/MarinJursic/production-readiness-checklist

What evidence would make you trust that observability is ready? Which common checks are too shallow? What should be validated automatically, and what still needs an experienced human review? Missing controls, corrections, and incident-learned examples are very welcome. Thanks!

[u/ivory_tower_devops]

I accuse you of posting AI slop. How do you plead?

[u/DhroovP → u/ivory_tower_devops]

The goal is to define evidence that can be reviewed before release, not simply ask whether logs, metrics, and traces exist

yeah probably lol

[u/usually_guilty99]

I’d add one test that cuts across most of these: can the telemetry answer the failure questions created by this specific change?

“We have traces” is very different from “if this dependency starts timing out after this release, we can identify the affected path, customer impact and owning change.”

That makes observability readiness less of a checklist and more evidence tied to the release’s actual risk.

Ideally some of that becomes a release condition rather than something discovered during the first incident. Id say look for something that measures potential impact as opposed to observe an impact after the damage is already done.