Hot Keys and Big Keys

Scenario

Example app: a sports app caches each match in Redis Cluster (6 shards). During a cup final, one match key gets 250,000 reads per second. Separately, a "full league standings" value has grown to 6 MB.

What you see in production

One Redis shard sits at 100% CPU while the other five idle; its network is saturated and timeouts appear for every key on that shard, not just the match. Reads of the 6 MB value take 30 ms and block the single-threaded Redis event loop for other clients.

The naive code

match = json.loads(r.get(f"match:{match_id}"))          # all 250k rps → one cluster slot → one shard
standings = json.loads(r.get("league:standings:all"))   # 6 MB value on every page view

Reproduce it

Load per shard when one key takes 40% of traffic, and after adding an in-process L1 cache and splitting the key into replicas:

import zlib
SHARDS, TOTAL = 6, 600_000
hot_share = 0.40

def shard(key): return zlib.crc32(key.encode()) % SHARDS

def load(hot_copies=1, l1_hit=0.0):
    per = [TOTAL * (1 - hot_share) / SHARDS] * SHARDS
    hot = TOTAL * hot_share * (1 - l1_hit)               # L1 absorbs most reads in-process
    for c in range(hot_copies):
        per[shard(f"match:final#{c}")] += hot / hot_copies
    return [round(x / 1000) for x in per]

print("naive              k-ops/s per shard:", load())
print("8 key copies       k-ops/s per shard:", load(hot_copies=8))
print("L1 cache (95% hit) k-ops/s per shard:", load(l1_hit=0.95))

naive              k-ops/s per shard: [300, 60, 60, 60, 60, 60]
8 key copies       k-ops/s per shard: [120, 60, 90, 150, 90, 90]
L1 cache (95% hit) k-ops/s per shard: [72, 60, 60, 60, 60, 60]

Root cause

A cache cluster spreads keys, not requests. All reads of one key go to the shard that owns its slot, so a single hot key caps at one node's throughput. Big values amplify this: network bandwidth and Redis's single-threaded command processing are consumed per byte, and large GET/DEL operations stall other clients.

The fix

• L1 in-process cache for the hottest keys with a short TTL (1–5 s): most reads never leave the app server.
• Key replication: store N copies (match:final#0..7) and read a random copy; write/invalidate all copies.
• Split big values: per-page or per-field keys (hashes), or compress; serve partial data.
• Detect hot keys early and treat them specially (events like finals are predictable).

from cachetools import TTLCache
l1 = TTLCache(maxsize=10_000, ttl=2)

def get_match(mid, copies=8):
    if (v := l1.get(mid)) is not None:
        return v
    raw = r.get(f"match:{mid}#{random.randrange(copies)}")
    value = json.loads(raw) if raw else load_and_fill(mid, copies)
    l1[mid] = value
    return value

def load_and_fill(mid, copies):
    value = db.fetch_match(mid)
    pipe = r.pipeline()
    for c in range(copies):
        pipe.set(f"match:{mid}#{c}", json.dumps(value), ex=30)
    pipe.execute()
    return value

Cache<Long, Match> l1 = Caffeine.newBuilder().expireAfterWrite(Duration.ofSeconds(2)).maximumSize(10_000).build();
Match m = l1.get(matchId, id -> redisMatches.get(id));    // only cache misses reach Redis

redis-cli --hotkeys        # needs maxmemory-policy *-lfu
redis-cli --bigkeys        # samples the keyspace for the largest keys
redis-cli --memkeys

Trade-offs

• L1 caches are per instance: invalidation needs pub/sub or very short TTLs (next module), and each instance may briefly disagree.
• Key copies multiply memory and invalidation work; use them only for the few keys that need them.
• Splitting big values means more round trips (use pipelining / MGET / HMGET).

Detect it in production

• Per-shard CPU, ops/s and network; one shard far above the others means a hot key or big key.
• --hotkeys / --bigkeys scans, slowlog entries for large values, client-side per-key metrics for top keys.

Key points

• Clusters shard keys, not requests: one hot key overloads one node.
• Use L1 caches, key replication and smaller values for the hottest data.
• Monitor per-shard load and find hot/big keys before big events.