How does a CPU limit of 500m work under the hood?

The kernel's CFS bandwidth control gives the container 50 ms of CPU time per 100 ms period; once used up, its threads wait until the next period (throttling).

Why can a container be throttled when its average CPU usage is low?

Multi-threaded bursts can exhaust the per-period quota in milliseconds, so it is throttled within those periods even though the average over minutes stays low.

Why do many teams set CPU requests but no CPU limits?

Requests guarantee a share under contention and drive scheduling, while the absence of a limit lets services burst into idle CPU instead of being throttled; memory limits are still set.

Latency Spikes from CPU Limits — Kubernetes in Production

Scenario

Example app: a Go API with a 500m CPU limit shows average CPU usage of 30 % of its limit, yet its p99 latency is 400 ms while p50 is 20 ms.

What you see in production

The container metric container_cpu_cfs_throttled_periods_total climbs steadily; about a quarter of scheduling periods are throttled. Raising the limit fixes the latency without changing the code.

The naive code

resources:
  requests: {cpu: 250m}
  limits:   {cpu: 500m}     # 50 ms of CPU time per 100 ms period

Reproduce it

A simulation of the CFS quota: a request that needs 80 ms of CPU across 4 threads, under a 500m limit:

How a 500m limit stretches a burst of workTry it yourself

def run_burst(cpu_ms_needed, threads, limit_millicores, period_ms=100):
    quota_ms = limit_millicores / 1000 * period_ms        # CPU time allowed per period, across all threads
    elapsed, remaining, throttled_periods = 0.0, cpu_ms_needed, 0
    while remaining > 0:
        burn_rate = threads                                # ms of CPU used per ms of wall time when running
        run_for = min(period_ms, quota_ms / burn_rate, remaining / burn_rate)
        remaining -= run_for * burn_rate
        if remaining > 0:
            elapsed += period_ms                           # quota exhausted: wait for the next period
            throttled_periods += 1
        else:
            elapsed += run_for
    return elapsed, throttled_periods

for limit in (500, 1000, None):
    if limit is None:
        print(f"no CPU limit:        burst takes {80 / 4:.0f} ms (4 threads x 20 ms)")
        continue
    t, thr = run_burst(80, 4, limit)
    print(f"limit {limit:4}m:          burst takes {t:.0f} ms, throttled in {thr} period(s)")

def run_burst(cpu_ms_needed, threads, limit_millicores, period_ms=100):
    quota_ms = limit_millicores / 1000 * period_ms        # CPU time allowed per period, across all threads
    elapsed, remaining, throttled_periods = 0.0, cpu_ms_needed, 0
    while remaining > 0:
        burn_rate = threads                                # ms of CPU used per ms of wall time when running
        run_for = min(period_ms, quota_ms / burn_rate, remaining / burn_rate)
        remaining -= run_for * burn_rate
        if remaining > 0:
            elapsed += period_ms                           # quota exhausted: wait for the next period
            throttled_periods += 1
        else:
            elapsed += run_for
    return elapsed, throttled_periods

for limit in (500, 1000, None):
    if limit is None:
        print(f"no CPU limit:        burst takes {80 / 4:.0f} ms (4 threads x 20 ms)")
        continue
    t, thr = run_burst(80, 4, limit)
    print(f"limit {limit:4}m:          burst takes {t:.0f} ms, throttled in {thr} period(s)")

Output

limit  500m:          burst takes 108 ms, throttled in 1 period(s)
limit 1000m:          burst takes 20 ms, throttled in 0 period(s)
no CPU limit:        burst takes 20 ms (4 threads x 20 ms)

Root cause

A CPU limit is enforced by the kernel's CFS bandwidth control as a quota per period (100 ms by default). Multi-threaded work can consume the whole quota in a few milliseconds and then waits for the next period. Averages hide it: 30 % average usage can include bursts that hit the quota many times per second, which shows up as tail latency.

The fix

For latency-sensitive services, set CPU requests but no CPU limit: requests guarantee a share under contention, and the container can burst into idle CPU.
If limits are required (multi-tenant clusters, quotas), size them for bursts, not averages, and watch the throttling metric.
Align runtime thread pools with the CPU you actually get (Go GOMAXPROCS, which recent Go versions derive from the CPU limit; JVM -XX:ActiveProcessorCount).
Keep memory limits: memory is not compressible.

Trade-offs

No CPU limits means a misbehaving pod can use spare node CPU, so requests must be accurate and nodes monitored.
Some organisations require limits for chargeback or isolation; then accept higher limits or Guaranteed QoS.

Detect it in production

Throttled periods ratio: container_cpu_cfs_throttled_periods_total / container_cpu_cfs_periods_total.
p99 latency much higher than p50 with low average CPU.
Thread counts far above the CPU limit.

Key points

CPU limits are quotas per 100 ms period; bursts get throttled even at low average usage.
Prefer CPU requests without limits for latency-sensitive services.
Watch throttling metrics and align thread pools with available CPU.

Further reading: CPU resource units and limits · Kernel CFS bandwidth control