Autoscale a Deployment on CPU and memory

Approach

Set accurate CPU requests, then create an autoscaling/v2 HPA with a utilisation target and behaviour that scales up quickly and down slowly. Remove replicas from the Deployment you apply.

Solution

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: {name: api}
spec:
  scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: api}
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource: {name: cpu, target: {type: Utilization, averageUtilization: 65}}
    - type: Resource
      resource: {name: memory, target: {type: Utilization, averageUtilization: 80}}
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies: [{type: Percent, value: 100, periodSeconds: 30}]
    scaleDown:
      stabilizationWindowSeconds: 300
      policies: [{type: Pods, value: 2, periodSeconds: 60}]

Pitfalls

• No CPU requests: utilisation can't be computed.
• Requests far above real usage: the HPA never sees high utilisation.
• Memory-based scaling for apps whose memory never shrinks.

Variations

• KEDA for queue length, request rate or schedule-based scaling.
• Cluster autoscaler or Karpenter so new replicas get nodes.

Further reading: Horizontal Pod Autoscaling