Virtual Machines and Auto Scaling

Virtual machines

EC2 (Azure Virtual Machines, Google Compute Engine) gives you a server with the CPU, memory, storage and network you choose. Instance types come in families:

Treat VMs as cattle, not pets: build them from a machine image (AMI, Azure image, GCE image) and configuration code, never fix them by hand, and replace them instead of patching in place.

Auto scaling

An auto scaling group (Azure virtual machine scale set, Google managed instance group) keeps a fleet at the right size: it replaces unhealthy instances and adds or removes instances to follow a metric.

import math

TARGET_CPU = 50          # target tracking: keep average CPU near 50%
instances = 4
for load in [120, 260, 380, 300, 140, 90]:          # total load in "CPU percent units"
    current_avg = load / instances
    desired = max(2, min(20, math.ceil(load / TARGET_CPU)))
    action = "scale out" if desired > instances else "scale in" if desired < instances else "hold"
    print(f"load {load:3}: avg CPU {current_avg:5.1f}% on {instances:2} -> {action:9} to {desired}")
    instances = desired

load 120: avg CPU  30.0% on  4 -> scale in  to 3
load 260: avg CPU  86.7% on  3 -> scale out to 6
load 380: avg CPU  63.3% on  6 -> scale out to 8
load 300: avg CPU  37.5% on  8 -> scale in  to 6
load 140: avg CPU  23.3% on  6 -> scale in  to 3
load  90: avg CPU  30.0% on  3 -> scale in  to 2

Scale on a metric that tracks load (CPU, requests per target, queue depth), set sensible minimums and maximums, and spread instances across availability zones.