Survive node drains and zone outages

Approach

Spread replicas across zones and nodes with topology spread constraints, and limit voluntary evictions with a PodDisruptionBudget that still allows drains to progress.

Solution

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata: {name: api}
spec:
  maxUnavailable: 1
  selector: {matchLabels: {app: api}}

apiVersion: apps/v1
kind: Deployment
metadata: {name: api}
spec:
  replicas: 3
  selector: {matchLabels: {app: api}}
  template:
    metadata: {labels: {app: api}}
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
          labelSelector: {matchLabels: {app: api}}
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway
          labelSelector: {matchLabels: {app: api}}
      containers:
        - name: api
          image: ghcr.io/acme/api:1.4.2

Pitfalls

• maxUnavailable: 0 (or minAvailable equal to replicas): drains block forever.
• PDBs on single-replica workloads: they either block drains or protect nothing.

Variations

• whenUnsatisfiable: DoNotSchedule for strict spreading when capacity allows.
• Pod anti-affinity for older clusters or simple "never on the same node" rules.

Further reading: Specifying a Disruption Budget