Scope
Read-only Datadog access: monitors, dashboards and log pipelines. The company is a small SaaS on AWS running about 40 services (illustrative).
Fix 1 · alert noise
What I found. About 70% of last month's pages (illustrative) came from 6 monitors (illustrative). All six flap: they fire and recover within a few minutes, mostly on CPU and queue-depth thresholds. None of them links to a runbook.
Why it matters. When most pages fix themselves, people stop reading them. The page that matters gets read the same way as the noise.
Effort: S. Add a recovery window or evaluation delay to each, move two of them to warn-only, and write a short runbook for every one that still pages.
Fix 2 · no SLOs on the revenue path
What I found. checkout-api and billing-worker have monitors on host
metrics and nothing on what customers see. No SLOs, no error budget, no
burn-rate alerts. A slow checkout would show up as a support ticket before it
showed up as a page.
Why it matters. These are the two services where a bad hour costs money directly, and they're the two with the least coverage.
Effort: M. One availability SLO and one latency SLO for each service, fast and slow burn-rate alerts on both, and the error budget on the team dashboard.
Fix 3 · log ingestion
What I found. About 60% of indexed log volume (illustrative) is DEBUG level,
all from one service, search-indexer. Nobody queried those logs in the last
30 days (illustrative).
Why it matters. You pay to ingest and index every line. Roughly $1,500 a month (illustrative) goes on logs nobody reads.
Effort: S. An exclusion filter for DEBUG on that service in the log pipeline, or drop the log level in its config. Keep a 1% sample (illustrative) if the team wants one.
What I did not look at
APM traces, RUM, synthetics, security monitoring, the AWS account itself, and the application code.
Next
The paid engagement, stage 1, takes this from three fixes to the whole map: every service, every gap, ranked by what to fix first.