Caching for Resilience

Caches can do more than speed things up: with stale-if-error and stale-while-revalidate, CDNs and application caches keep serving the last good response when the origin fails. Keep a longer "stale TTL" alongside the fresh TTL, coalesce refreshes, and prevent cold-cache stampedes after restarts by warming critical keys. Beware: if the system can't run without the cache, the cache is now a critical dependency that needs its own redundancy.

Resilient caching techniques

Python (soft/hard TTL)

async def get_with_stale(key: str, load, soft_ttl=60, hard_ttl=3600):
    entry = await cache.get(key)                      # {"value":..., "fetched_at":...}
    fresh = entry and time.time() - entry["fetched_at"] < soft_ttl
    if fresh:
        return entry["value"]
    try:
        value = await load()
        await cache.set(key, {"value": value, "fetched_at": time.time()}, ex=hard_ttl)
        return value
    except DependencyError:
        if entry:                                     # origin down: serve stale
            metrics.increment("cache.stale_served")
            return entry["value"]
        raise

Common pitfalls

• Systems that can't start without a warm cache.
• Serving stale data for correctness-critical values (balances, stock at checkout).
• No metric for stale responses.

Key points

• stale-if-error / soft+hard TTLs
• Coalesce and warm
• Know which data may be stale

Further reading: RFC 5861: stale-while-revalidate and stale-if-error