Kubernetes cookbook · Beginner

Wait for dependencies at startup without init container hacks

Pods crash-loop while a database or another service is still starting.

Create a free account to save this page, take notes and track your progress. Sign in

Approach

Make the app retry its connections with backoff, report not-ready until dependencies are reachable, and use a startup probe so slow starts aren't killed. Init containers that loop on nc only help at first start.

Solution

apiVersion: apps/v1
kind: Deployment
metadata: {name: api}
spec:
  selector: {matchLabels: {app: api}}
  template:
    metadata: {labels: {app: api}}
    spec:
      containers:
        - name: api
          image: ghcr.io/acme/api:1.4.2
          startupProbe:
            httpGet: {path: /ready, port: 8080}
            periodSeconds: 2
            failureThreshold: 60
          readinessProbe:
            httpGet: {path: /ready, port: 8080}
            periodSeconds: 5

Pitfalls

  • Liveness probes that fail while dependencies are down, restarting healthy pods.
  • Crashing on the first failed connection instead of retrying.

Variations

  • Native sidecar containers (initContainers with restartPolicy: Always) for proxies that must start first.
  • Service meshes' hold-application-until-proxy-starts options.

Further reading: Configure probes

Reference: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

More Kubernetes recipes

Was this page helpful?