When does a container without a readiness probe count as ready?

As soon as its process has started, so it receives Service traffic even if the application is still initialising.

What do maxUnavailable: 0 and maxSurge: 1 do in a rolling update?

Kubernetes adds one new pod at a time and only removes an old pod once a new one is ready, so capacity never drops below the desired replica count.

When do you need a startup probe?

For slow-starting containers, so liveness (and readiness) checks don't fail and restart them before they have finished starting.

502s During Every Deploy: Readiness Probes — Kubernetes in Production

Scenario

Example app: a Java API that takes about 20 seconds to warm up (loading caches, opening database pools) runs as a 3-replica Deployment behind a Service. The manifest has no probes.

What you see in production

Every deploy produces a burst of 502 and connection-refused errors for about 20 seconds. Scaling up during traffic spikes causes the same errors. The pods all show Running, so dashboards look green.

The naive code

containers:
  - name: api
    image: registry.example.com/api:2.3.0
    ports: [{containerPort: 8080}]
    # no readinessProbe: the pod counts as ready as soon as the container starts

Reproduce it

A slow-starting app behind a Service, without and with a readiness probe. Watch when each pod becomes an endpoint:

When does a starting pod receive traffic?

kubectl create namespace prod-ready > /dev/null
trap 'kubectl delete namespace prod-ready --wait=false > /dev/null' EXIT
make() {   # $1 = name, $2 = readiness probe yaml ("" for none)
cat <<EOF | kubectl -n prod-ready apply -f - > /dev/null
apiVersion: v1
kind: Pod
metadata: {name: $1, labels: {app: api}}
spec:
  containers:
    - name: api
      image: busybox:1.37
      command: ["sh", "-c", "sleep 8; touch /tmp/started; echo ok > /tmp/index.html; httpd -f -p 8080 -h /tmp"]
$2
EOF
}
kubectl -n prod-ready create service clusterip api --tcp=80:8080 > /dev/null
make no-probe ""
make with-probe '      readinessProbe: {exec: {command: ["test", "-f", "/tmp/started"]}, periodSeconds: 1}'
sleep 4
ready() { kubectl -n prod-ready get pod $1 -o jsonpath='{.status.containerStatuses[0].ready}'; }
echo "4 s after start (app still warming up): no-probe ready=$(ready no-probe), with-probe ready=$(ready with-probe)"
kubectl -n prod-ready wait --for=condition=Ready pod/with-probe --timeout=60s > /dev/null
echo "after warm-up:                          no-probe ready=$(ready no-probe), with-probe ready=$(ready with-probe)"

kubectl create namespace prod-ready > /dev/null
trap 'kubectl delete namespace prod-ready --wait=false > /dev/null' EXIT
make() {   # $1 = name, $2 = readiness probe yaml ("" for none)
cat <<EOF | kubectl -n prod-ready apply -f - > /dev/null
apiVersion: v1
kind: Pod
metadata: {name: $1, labels: {app: api}}
spec:
  containers:
    - name: api
      image: busybox:1.37
      command: ["sh", "-c", "sleep 8; touch /tmp/started; echo ok > /tmp/index.html; httpd -f -p 8080 -h /tmp"]
$2
EOF
}
kubectl -n prod-ready create service clusterip api --tcp=80:8080 > /dev/null
make no-probe ""
make with-probe '      readinessProbe: {exec: {command: ["test", "-f", "/tmp/started"]}, periodSeconds: 1}'
sleep 4
ready() { kubectl -n prod-ready get pod $1 -o jsonpath='{.status.containerStatuses[0].ready}'; }
echo "4 s after start (app still warming up): no-probe ready=$(ready no-probe), with-probe ready=$(ready with-probe)"
kubectl -n prod-ready wait --for=condition=Ready pod/with-probe --timeout=60s > /dev/null
echo "after warm-up:                          no-probe ready=$(ready no-probe), with-probe ready=$(ready with-probe)"

Output

4 s after start (app still warming up): no-probe ready=, with-probe ready=
after warm-up:                          no-probe ready=, with-probe ready=

Run this on your computer — see “Get Started” for setup.

Root cause

Without a readiness probe, Kubernetes considers a container ready as soon as its process starts. The pod joins the Service's endpoints and the load balancer sends it requests while the application is still starting, so they fail. During a rolling update, old pods are removed as soon as new ones are "ready", so for a moment most traffic goes to pods that can't answer.

The fix

readinessProbe:
  httpGet: {path: /ready, port: 8080}
  periodSeconds: 5
  failureThreshold: 3
startupProbe:                       # slow starters: give up to 60 s before other probes apply
  httpGet: {path: /ready, port: 8080}
  periodSeconds: 2
  failureThreshold: 30
strategy:
  rollingUpdate: {maxUnavailable: 0, maxSurge: 1}
minReadySeconds: 10

/ready returns 200 only when the app can actually serve (caches warm, pools open).
maxUnavailable: 0 keeps full capacity during rollouts; minReadySeconds waits a little before trusting a new pod.
Pair with graceful shutdown (next lessons but one) so old pods also leave cleanly.

Trade-offs

Readiness checks that include external dependencies can take every pod out of service during a dependency blip; check only what this pod needs to serve.
maxUnavailable: 0 needs spare cluster capacity for surge pods.
Probes that are too sensitive flap; use failure thresholds.

Detect it in production

5xx rate on the load balancer, correlated with deployment events.
Pods with no readiness probe (policy checks with Kyverno, Gatekeeper or CI linting).
Rollout duration and minReadySeconds versus measured warm-up time.

Key points

No readiness probe means "ready when the process starts".
Readiness gates traffic; startup probes protect slow starters.
Roll out with maxUnavailable 0 and a readiness endpoint that reflects real ability to serve.

Further reading: Configure liveness, readiness and startup probes · Probes in the tutorial