Caching

A cache trades freshness for latency and load. Every caching decision is really two decisions: how data gets in, and how stale data gets out.

Getting data in. Cache-aside (lazy loading) is the default: the app checks the cache, misses, reads the database, then writes the cache. It only caches what is actually requested, and the cache can be down without taking the app with it. Read-through pushes that logic into the cache layer. Write-through writes cache and database together on every write, so the cache is never stale but every write pays both costs. Write-behind buffers writes and flushes asynchronously, which is fast and risks losing data if the cache dies before the flush.

Getting data out. TTL expiry is the workhorse: cheap, and you accept bounded staleness. Explicit invalidation on write gives tighter consistency, but you have to find every key a write affects, which is where most cache bugs live. Versioned keys sidestep invalidation entirely by making new data write to a new key.

Where it lives. The same request usually passes several caches: the browser, a CDN edge, an application cache such as Redis, and the database's own buffer pool. Each layer you add cuts load on the one behind it and adds another place stale data can hide, so decide deliberately which layer owns freshness rather than setting a TTL at every level and hoping they agree.

Eviction and hit rate. A cache is bounded, so something has to leave. LRU evicts the least recently used entry and suits a working set that moves over time. LFU keeps what is popular over a longer horizon, so a one-off scan of cold keys cannot flush everything useful — the failure LRU is prone to. Watch the hit rate, but read it next to what a miss costs: a 90% hit rate on a 2ms query is worth less than a 60% hit rate on a 400ms one. And when the working set genuinely does not fit in memory, more memory helps and a cleverer eviction policy does not.

How it fails. A cache stampede (or thundering herd) happens when a hot key expires and every concurrent request misses at once, all hammering the origin together. Fixes: coalesce duplicate in-flight requests behind a single origin call, expire probabilistically a little early so one unlucky request refreshes before the deadline, or serve stale data while refreshing in the background.