Caching in System Design Interviews: Patterns, Invalidation and Failure Modes
"Add a cache" is the most common answer in system design interviews, and the least explained. A cache makes reads faster and cheaper, but it also creates a second copy of your data that can be wrong, empty or overloaded. Senior answers talk about all three.
When a cache is worth it
A cache earns its place when:
- Reads far outnumber writes (10:1 or more is common for feeds, profiles, product pages).
- The same data is read repeatedly: a small set of hot items gets most of the traffic.
- Some staleness is acceptable, or you can invalidate precisely.
- The source is slow or expensive: a heavy query, a remote API, a computed result.
If every read is unique, or data must always be exactly current (an account balance at payment time), a cache adds risk without much benefit.
Where caches live
| Layer | Example | Good for |
|---|---|---|
| Client / browser | HTTP cache headers | Static assets, rarely changing API responses |
| CDN / edge | A CDN in front of images and pages | Media, static and semi-static content close to users |
| Application memory | An in-process LRU map | Tiny, very hot data such as config and feature flags |
| Distributed cache | Redis, Memcached | Shared data across many app servers |
| Database | Buffer pool, materialised views | Built in; you tune it rather than design it |
In an interview, name the layer and the reason: "Profile pictures go through a CDN; the profile JSON is cached in Redis because every service reads it."
The four read and write patterns
| Pattern | How it works | Trade-off |
|---|---|---|
| Cache-aside (lazy loading) | App reads cache; on a miss, reads the DB and fills the cache | Simple and the default choice; the first read after a miss is slow, and the data can be stale until expiry or invalidation |
| Read-through | The cache itself loads from the DB on a miss | Same behaviour as cache-aside, with the loading logic moved into the cache layer |
| Write-through | Every write goes to the cache and the DB together | Cache is always fresh; writes are slower, and you cache data nobody may read |
| Write-back (write-behind) | Writes go to the cache; the DB is updated later in batches | Very fast writes; data is lost if the cache fails before flushing |
Two related terms: write-around means writes skip the cache entirely (fine for data that's rarely re-read), and refresh-ahead means reloading popular keys shortly before they expire.
For most interview answers: cache-aside for reads, invalidate on write, with a TTL as a safety net.
Eviction: what leaves when the cache is full
- LRU (least recently used): the default and usually right.
- LFU (least frequently used): better when popularity is stable over time.
- TTL: every entry expires after a fixed time, bounding how stale data can get.
Mention size too: "The hot set is about 20 GB, so a small Redis cluster holds it with room to spare."
Invalidation: the hard part
With cache-aside, the usual write path is: update the database, then delete the cache key, so the next read reloads fresh data. Deleting is safer than updating the cache in place, because two concurrent writers can otherwise leave the older value in the cache.
There is still a race worth knowing:
- Reader A misses the cache and reads the old value from the DB.
- Writer B updates the DB and deletes the cache key.
- Reader A writes the old value into the cache.
The cache now holds stale data until the TTL expires. Mitigations: keep TTLs short for data that changes, delete the key a second time shortly after the write ("delayed double delete"), or version your values so an older write can't overwrite a newer one.
For data that must never be stale (inventory at checkout, balances), read from the source of truth at the moment it matters and use the cache only for display.
Failure modes interviewers ask about
Cache stampede (thundering herd). A hot key expires and thousands of requests miss at once and hit the database together. Fixes:
- Request coalescing: only one request rebuilds the key; others wait for it or get the stale value.
- Jittered TTLs: add randomness so keys don't expire at the same moment.
- Refresh ahead of expiry for known hot keys.
Hot keys. One key (a celebrity's profile, a viral post) gets so much traffic that a single cache node saturates. Fixes: replicate that key across nodes, or add a small in-process cache in front of Redis for the hottest items.
Cache penetration. Requests for keys that don't exist always miss and always hit the DB. A common attack pattern. Fixes: cache the "not found" result with a short TTL, or check a Bloom filter of valid keys first.
Cache down. If the cache disappears, can the database survive the full read load? If not, you need rate limiting or degraded responses so the outage doesn't cascade.
How to talk about caching in the interview
- Point to the read path and say what you cache and why ("feed pages; 95% of reads hit the same few thousand users' data").
- Pick the pattern (usually cache-aside) and the invalidation rule.
- State the staleness you accept ("a profile can be up to 60 seconds old").
- Name one failure mode and its fix (stampede and coalescing is a strong pick).
- Give the size: hot set, memory needed, and expected hit rate.
Practise it inside real designs. Prepd has a distributed cache question, an LRU cache LLD problem and a CDN question, each broken into steps with written feedback on every answer.