The problem
Services launched on the cluster and then the platform team found out what they were missing, usually during their first incident.
How I measured it
# deployments with no resource limits
kubectl get deploy -A -o json | jq -r '.items[] |
select([.spec.template.spec.containers[].resources.limits] | any(. == null)) |
"\(.metadata.namespace)/\(.metadata.name)"'
# deployments with no readiness probe
kubectl get deploy -A -o json | jq -r '.items[] |
select([.spec.template.spec.containers[].readinessProbe] | any(. == null)) |
"\(.metadata.namespace)/\(.metadata.name)"'
# single-replica deployments
kubectl get deploy -A -o json | jq -r '.items[] | select(.spec.replicas == 1) | "\(.metadata.namespace)/\(.metadata.name)"'
Most of the services launched in the previous months showed up in at least one list.
Why it kept happening
Teams knew how to write their service. They didn't know what the platform expected, because nobody had written it down in one place.
The process change
A readiness review before first production traffic. A one-page checklist and a 30-minute call:
- Requests and limits set from load testing.
- Readiness and liveness probes.
- At least two replicas, with a disruption budget.
- Monitoring onboarded, with one user-facing alert.
- A runbook: how to restart, roll back and scale.
- An owner on the on-call rota.
The checks above ran in CI against the Helm chart, so most items were green before the call. The call was for the parts a script can't check.
What I learnt
A checklist the team can run on its own chart, before the review, made the review short. The call turned into a conversation about the service instead of a list of missing YAML.
What I do now
- Write down what "ready for production" means, in one place.
- Automate every check that can be automated, in CI.
- Keep the human review short and focused on what scripts can't see.
stage 1 includes a readiness review of your services → see the engagements