The page
An alert fired in the early hours: the scripted login journey was failing at the redirect step. Real-user monitoring showed nothing wrong, because almost nobody was logging in at that time.
Triage
# status and timing, the way a user would hit it
curl -sS -o /dev/null -w "%{http_code} %{time_total}s\n" https://app.example.com/login
# follow every redirect and show where it goes
curl -sIL https://app.example.com/login | grep -iE "^(HTTP|location)"
# check the identity provider's certificate on the redirect target
openssl s_client -connect sso.example.com:443 -servername sso.example.com </dev/null 2>/dev/null \
| openssl x509 -noout -dates
The redirect went to an SSO endpoint that answered with an error.
Root cause
An overnight change on the identity side had altered the callback URL it accepted. The application still sent the old one. Nothing on the app side had changed, so nothing on the app side alerted.
The fix
Rolled the callback config back with the identity team before the morning rush. Then the synthetic script got a step to check the login actually landed on the dashboard. A 200 from the login page wasn't enough.
What I learnt
Real-user monitoring needs real users. At night, or for a feature nobody's using yet, it's blind. Synthetic checks run the journey whether anyone's there or not.
What I do now
- Script the critical journeys (login, checkout, search) end to end, every few minutes, from outside.
- Assert on content and landing page, not only status codes.
- Keep synthetics and real-user data side by side. Each covers the other's blind spot.
stage 0 shows which journeys nobody is watching → get the free audit