Cache Invalidation Trade-offs

A user changes the email address on their account, reloads the page, and the old address is still there for four minutes. Nothing crashed — your caches each did exactly what their TTLs said. The bug is that nobody ever decided how stale "stale" is allowed to be for this field; the TTL was inherited from a copy-pasted snippet.

Time-to-live is the one invalidation mechanism that always works: no publisher, no fan-out, no lost messages. It converts a correctness problem into a stated staleness budget, and the discipline is to set that budget per data class instead of per codebase:

Two structural facts make this harder than one table suggests.

First: worst-case staleness is the sum of every layer's TTL in the request path. A read can pass through an in-process L1 (5 s), a shared Redis L2 (300 s), and a CDN in front (60 s). A value refreshed at each layer can be up to 5 + 300 + 60 = 365 s behind, not 5 seconds. When you document a staleness SLO, walk the whole path and add the budgets — the layer you forgot is usually the one that answers the support ticket.

Second: TTL cannot give you read-your-writes. The user's own write followed by their own read is the case where staleness is most visible and least acceptable, and no TTL tuning fixes it because the read races the very cache you just wrote through. Standard answers, cheapest first: (a) bypass the cache for a short window after the user's write (sticky "fresh read" flag), (b) return the written value in the write response and render it directly, (c) attach a version or updated_at to the client and treat mismatched versions as an automatic cache bypass.

Write-through does not rescue the other layers either: your shared cache may hold the new value while the CDN in front of it serves the old one for another 60 seconds. This is the sense in which TTL bounds staleness but can never guarantee immediate correctness after a write — every guarantee beyond the TTL requires active invalidation, and active invalidation is where the trade-offs live.

Frame it this way in an interview: nobody buys "strong consistency everywhere" — the price in latency, fan-out, and availability is prohibitive — so you choose the staleness contract per data class, you make the worst case (sum of layers) explicit, and you reserve active invalidation for the small set of fields where seconds matter. Expect a follow-up: "how would you prove your staleness bound?" The answer is a synthetic check — write a sentinel value, read it through the full path on a loop, and assert the observed lag stays under budget. If you cannot measure the bound, you do not have one. That set of seconds-critical fields is exactly what the next two chapters optimize.

TTL of five minutes is too slow for a price change, so the writer publishes "price for SKU 42 changed" the moment it commits, every application node deletes its local copy, and the next read reloads fresh data. Convergence in ~50 milliseconds instead of 300 seconds — this is event-driven invalidation, and its failure modes are instructive because three separate things must go right.

1. Publish strictly after the database commit. The canonical bug: emit the invalidation, then commit (or emit outside the transaction). A reader on another node re-fetches before the commit lands, repopulates the old value, and your invalidation has now re-seeded the stale entry — which then lives out its full TTL. The robust versions:

naive:      UPDATE ... ; publish(invalidate) ; COMMIT        ← race: repopulate old value
correct:    BEGIN; UPDATE ...; INSERT INTO outbox(...); COMMIT
            → outbox relay publishes after COMMIT              (transactional outbox)
            → or publish an UPDATE-propagation event carrying the new value

The transactional outbox (a row written in the same transaction, published by a relay or CDC) closes both gaps — "committed but not published" and "published but not committed" — and is the pattern most interviewers accept as the correct answer.

2. Assume at-least-once delivery and design for duplicates. Delete is idempotent: DEL user:42 twice is harmless. That is precisely why invalidate events are safer than set events — a "here is the new value" message can arrive out of order (new value, then a delayed old value) unless it carries a monotonic version the receiver compares. And some mechanisms are best-effort by design: Redis keyspace notifications and similar pub/sub hooks may silently drop messages during failover. The TTL stays on every key as the correctness backstop; the event only shortens the typical case. Belt and braces, and the belt is cheap.

3. Close the repopulation race. The nastiest interleaving needs no lost message at all:

reader:   MISS → read DB (sees price 9.00, pre-update)      ... context switch ...
writer:   commit price 10.00 → publish invalidation → cache now clean
reader:   SET product:42 = {"price": 9.00}                   ← stale value resurrected

Mitigations, pick by stakes: always set with a TTL (bounds the damage); carry updated_at in the value and reject a SET whose timestamp is older than what is stored (a small Lua/CAS compare); or use versioned keys, which eliminate the race by construction — chapter three.

Fan-out economics decide scalability. One event → N in-process caches is fine at N = 50. Trouble appears when invalidation granularity is fine: one entity update touching price, stock, reviews, images × 3 locales × 2 device classes is 48 keys, and CDN purge APIs bill or rate-limit beyond a few thousand paths per request. Keep invalidation units coarse — invalidate the entity and let the reader rebuild its projections — and maintain tag→keys indexes (tag:sku42 → {keys}) instead of enumerating keys by hand.

The design sentence to remember: events buy you speed, TTL buys you correctness, and the outbox is what makes the fast path trustworthy. When even that leaves a delete race you cannot close, stop deleting — make old entries unreachable instead.

Deleting cache keys is a coordination problem: between your DEL and the next reader's SET, a stale value can slip in, and across three cache layers plus a CDN, "delete everywhere" is a fan-out you must get right every time. Versioned keys change the shape of the problem — instead of removing old data, you make it unreachable, and correctness becomes a property of naming.

Per-entity versions. Give each key a version that moves forward with the data:

user:17:v83:name          ← value written when user 17 was updated at version 83
config:v42:feature-flags  ← bumped when the config service publishes v42

On write, you write user:17:v84:… — nobody ever reads v83 again, so nobody can be served it. No delete, no race window, and read-your-writes falls out for free: the client that triggered update v84 reads only v84 keys. Where does the version come from? A config/feature version for whole-keyspace resets (schema changes, pricing-rule rollouts), or the entity's own updated_at/monotonic version for per-row freshness.

The costs are real and worth stating before you adopt it:

• Cold misses on bump. A global version bump makes every key miss at once — a thundering herd you scheduled. Ramp it: pre-warm the hottest keys before flipping the pointer, or publish the new version to 5% of nodes first.
• Double memory during the ramp. Old and new entries coexist until TTL expires the old ones. Size the cache for 2× during bumps, or run explicit cleanup with SCAN-based deletion throttled to a few hundred keys/second.
• Key churn. Per-entity versions on hot-update rows (counters!) rewrite the keyspace constantly and can fragment memory. Use versions for correctness-critical freshness, not for every write.

Tag-based invalidation handles "one row, many keys" without enumerating paths. Keys register under tags (sku:42 tags → price, stock, reviews, image-webp), and one invalidate tag:sku:42 deletes the whole group — Varnish's BAN/softpurge, Memcached's delete-by-group idiom, Redis with a SET tag:sku:42 member1 member2 … index cleaned up after deletion. The known race: a key added to the tag set after the tag was purged survives with a stale value — close it by having writers bump the entity version (write the new key under the new version) or by re-checking the tag generation on insert. Tags reduce fan-out from dozens of paths to one call; they do not remove the need for a TTL backstop.

Soft purge (mark-stale, not delete) is the latency-conscious variant: the entry is flagged invalid — future requests revalidate it — while concurrent requests are still served the stale copy during revalidation. Combined with stale-while-revalidate from CDN and Edge Caching, a soft purge gives you immediate freshness semantics without the origin burst a hard purge causes.

Probabilistic early expiration handles the expiry-time stampede without any event at all: on each read, recompute early with probability rand() < C × (age / ttl) (XFETCH, C ≈ 1), so entries are refreshed one at a time, jittered, before the synchronized expiry instant — the invalidation problem solved by never letting the herd arrive together.

Choose by stakes: TTL alone for data where the budget is honest; events + outbox when seconds matter and the key space is small; versioned keys when correctness must not depend on a delete succeeding; tags when one entity fans out to many keys. Whatever you choose, keep a non-zero TTL everywhere — it is the backstop that converts "we missed an invalidation" from a data-incident into a five-second delay. With caching now placed, keyed, and invalidated, the next topic turns to what happens when the systems behind the cache start failing: Timeouts and Retries.