Pattern — Cache-Aside + Invalidation

The problem it solves

A cache sits in front of an expensive read — a heavy join, a remote call, a model score — because the reads vastly outnumber the writes. Done wrong, cache-aside fails two ways. It serves stale data: the source of truth changed but the cached copy did not, so readers see a price, balance, or config the database already retired. Or it melts the backend: a hot key expires and every concurrent reader independently reloads it — the cache stampede of Drill 73, where one price:{symbol} key expiring turned 8,000 in-flight requests into 8,000 identical 400 ms recomputes and drove the pricing database from 50 QPS to 11,000, p99 past twelve seconds. The pattern is the disciplined version of "put a cache in front of it" that avoids both.

The pattern

Cache-aside (lazy loading): the application owns the cache, and the database is the source of truth.

• Read — GET key; on a hit, return it; on a miss, load from the source, SET key value with a TTL, and return.
• Write — update the source of truth, then invalidate the key (delete it), so the next read repopulates from fresh data.

Two disciplines make it correct. Invalidate, don't update: deleting the key is idempotent and race-safe, where writing the new value into the cache from the writer can lose to a concurrent recompute and leave a stale value pinned. And a TTL is the safety net — even if an invalidation is dropped, staleness is bounded to the TTL. Then harden the miss path so a miss stays one recompute, not a herd: coalesce it, jitter the TTL, and serve stale while you revalidate.

How to apply it

Numbers below are grounded in Drill 73 — a Redis-fronted pricing service, key price:{symbol}, ~400 ms recompute.

1. Read path (lazy load). GET price:{symbol}. Hit → serve. Miss → run the recompute once, then SET price:{symbol} <value> EX 60 (plus jitter, step 4). A 60 s TTL is the drift ceiling.
2. Invalidate on write, don't update. When the source changes, DEL price:{symbol} — never SET the new value from the writer. Deletion is idempotent and can't lose a race; the next reader repopulates. The TTL is the backstop so a dropped DEL self-heals within 60 s instead of serving stale forever.
3. Coalesce the miss (single-flight). On a miss, the first request does SET lock:price:{symbol} NX EX 5 — it wins, recomputes, and writes the value; every other request blocks up to ~250 ms and reads the winner's result. This is the load-bearing change: it turns 11,000 duplicate queries into 1.
4. Jitter every TTL. Write EX 60 ± rand(10) so a cache-warm burst doesn't arm the whole key universe to expire on the same tick — expiries smear across a ~20 s band instead of firing in one instant.
5. Stale-while-revalidate. Carry a soft expiry at 60 s and a hard expiry at 90 s. Between them, serve the slightly-stale value in < 5 ms while exactly one coalesced background refresh recomputes — readers never block on the 400 ms join.
6. Cap recompute concurrency. Bound in-flight recomputes with a semaphore of ~2× the backend core count; beyond it, serve stale or shed — the cache layer can then never issue more than a safe number of concurrent expensive queries.

flowchart TD
    A["GET price:{symbol}"] --> B{"state?"}
    B -->|"fresh hit"| C["serve value"]
    B -->|"soft-expired<br/>(stale-while-revalidate)"| D["serve stale in < 5 ms<br/>+ 1 coalesced refresh"]
    B -->|"hard miss"| E{"SET NX lock<br/>acquired?"}
    E -->|"winner"| F["recompute once<br/>SET value EX 60 +/- 10"]
    E -->|"loser"| G["wait ~250 ms<br/>read winner's value"]
    H["source write"] --> I["DEL price:{symbol}<br/>(invalidate, not update)"]
    D --> C
    F --> C
    G --> C

When to use it — and when not

Reach for cache-aside when reads ≫ writes and the source is genuinely expensive (a 400 ms join read thousands of times a second) — that's where the hit rate pays for the added moving parts. When not: if a path needs read-your-writes / strong consistency, the window between the DB commit and the DEL is a real stale gap, so read that path from the primary or use write-through instead of cache-aside. Don't cache cheap computations — the cache round-trip and invalidation logic cost more than they save. Don't cache high-cardinality, one-shot keys that never get a second hit — you pay eviction churn for a ~0% hit rate. And know the sharp edge: a single hot key with a heavy recompute is exactly where naive cache-aside bites (Drill 73), so on those keys the coalescing lock is not optional.

Signals it's working

• Hit rate holds around 95%+ in steady state; a sag is your early warning, not user reports.
• Backend recompute QPS tracks the number of distinct keys expiring, not the number of readers. Alarm when recompute QPS crosses 3× the rolling baseline (e.g. > 150 QPS over a 50 QPS baseline) for more than 10 s — that's a stampede or a coalescing lock that isn't holding.
• Duplicate recomputes per miss ≈ 1, and p99 stays flat through cache-warm and market open (Drill 73: < 200 ms, versus 12 s+ before).
• Game-day proof: in staging, force-expire the hottest key under production read load and assert backend QPS rises < 2× — prove the lock holds before you need it.

Seen in the wild

• Facebook's memcache leases (2010 "thundering herd") — one client gets a lease to regenerate a hot key while others wait or serve stale; the productized single-flight lock.
• Netflix EVCache and Instagram's cache playbooks document jittered TTLs and stale-while-revalidate for exactly this failure.
• HTTP stale-while-revalidate (RFC 5861 / Cache-Control) bakes serve-stale-then-refresh into the web caching standard.
• Rails race_condition_ttl and Go's singleflight ship request coalescing as a library primitive.

See it

A ChainScene walks the read path and the coalescing lock (SET NX EX 5 → 11,000 → 1), a TilesScene/LayersScene carries the load-bearing config (invalidate-not-update, 60 s TTL ± 10 jitter, soft 60 / hard 90 SWR), and a CompareScene shows the Drill 73 outcome: DB peak 11k → 70 QPS, p99 < 200 ms.

Named concepts: cache-aside, lazy loading, cache invalidation, TTL as a safety net, request coalescing, stale-while-revalidate.