Reliability and Observability

Reliability and Observability · 3 min

Situation

“The app is slow” is real but underspecified.

A customer cannot open a report. Other customers appear unaffected. Engineering needs enough context to follow the failed action through the system.

Observability helps explain behavior from system outputs. Reliability is the ability to deliver the expected service consistently. Measurement supports that goal but does not guarantee it.

Mental model

Logs, metrics, and traces answer different questions.

Logs record events and details. Metrics summarize quantities over time, such as error rates and request duration. Traces connect steps of a request across components.

An alert draws attention to a condition that may need action. Uptime measures availability under a stated definition; a reachable homepage can coexist with a broken checkout.

Example

Follow the report request.

A request identifier links a customer report attempt to logs. A trace shows most time is spent waiting on a data service. A metric shows the delay affects large reports after a release.

Together, these signals are more useful than a screenshot alone. Avoid putting unnecessary personal data or credentials into diagnostic records.

Failure case

Averages hide the people having trouble.

A mean response time of 300 milliseconds can conceal a small group waiting ten seconds. Percentiles describe the distribution: p95 is a threshold at or below which about 95% of observed durations fall.

Choose measurements relevant to the workflow and segment. A global average can miss a slow region, device, or customer size.

PM question

What should a useful incident report contain?

Capture the user's intended action, time and time zone, environment, affected scope, observed and expected behavior, and request or job ID when available.

Ask whether alerts reflect user-impacting failures and have a clear owner. An alert nobody can act on becomes noise rather than protection.

Remember this

Diagnose the user's journey, not only the server.

Define reliability around the task customers need to complete. Give engineering traceable context, and make sure monitoring can detect failures along that journey.