#observability
grep -l "observability" /portfolio/* · 6 entriesHolding SLOs across 116 services without paging everyone
Owning SLOs, error budgets and burn-rate alerting across 116+ production services, and why burn rate beat thresholds for deciding when to wake someone.
1 min readAnomaly detection that flagged every Monday morning
Adding machine-learning analysis to monitoring looked great until it called every Monday 9am an anomaly. The baseline didn't know what a week looked like.
1 min readA pager that cried wolf
On-call was getting paged most nights, nearly always for nothing. Counting where the pages came from showed a handful of alerts doing almost all the damage.
1 min readFour monitoring tools and no single answer
Answering 'is the customer journey healthy?' meant opening four tools. Pulling them into one view changed how fast incidents got diagnosed.
1 min readNo service goes live without monitoring
New services kept reaching production with no APM, no synthetics and no alerts. A short onboarding checklist made monitoring part of launching.
1 min readSynthetic checks caught it before users reported it
A login flow broke overnight. Real-user monitoring was quiet because nobody was using it yet. A scripted check caught it hours before the morning.