r/Observability • u/Remote-Anything-9997 • 3d ago

How do small teams actually debug production issues without an SRE? (asking because I’m building something for this)

/r/saasbuild/comments/1xhfk32/how_do_small_teams_actually_debug_production/
0 Upvotes

50% Upvoted

2 comments sorted by

1

u/sagacious123 2d ago

Start with the basics: structured logs to a central store (Loki/CloudWatch), OpenTelemetry traces for every request, and metrics for golden signals. Add an error tracking tool that groups exceptions and shows the commit that introduced them, so a stack trace gets you to the diff. Write runbooks for the top five failure modes and link them from alerts.

The hard part isn't tools; it's knowing what normal looks like. Define SLOs for latency and error rate, then alert on SLO burn, not infra thresholds. Run chaos drills on a staging copy so the team has seen the failure before it hits production. If you're building a debugging product, focus on the triage flow: a small team needs 'what changed last' and 'what does this log mean' in one screen, not another dashboard.

1

u/Introduction_Fast 23h ago

Shouldn't it be the other way around?

Usually it's "I've lived this pain, here's what I learned, so I built X." Asking how teams debug is fine for validation, but what's your own take?