Preventing Cache Stampedes and Thundering Herds

A cache stampede (or thundering herd) happens when a popular cached key expires and many requests miss the cache at the same moment. They all rush to the database to rebuild the value, overwhelming it.

Under high traffic, hundreds or thousands of concurrent misses can hit the database in milliseconds. The backend that normally serves zero load for that key suddenly takes the full firehose, causing latency spikes or outages.

The basic pattern reads the cache, and on a miss queries the DB and stores the result. Every concurrent miss runs this DB query — that is the vulnerability.

Let only the first missing request rebuild the value while others wait or serve stale. Redis SET NX grants a short-lived lock to exactly one client.

On a miss:

• Try to acquire the lock with SET NX
• If you got it, query the DB and repopulate the cache
• If not, briefly wait and re-read the cache

Only one DB query runs per expiry.

Store the value with a logical expiry earlier than its physical TTL. When the logical time passes, serve the stale value immediately and refresh it in the background, so users never wait on a miss.

Each request, with a small growing probability as expiry nears, voluntarily refreshes the key before it dies. This spreads recomputation across time so the herd never forms.

If many keys are written together (e.g. on deploy) they expire together, causing a synchronized stampede. Add random jitter to each TTL so expiries spread out.

After rebuilding, delete the lock. Give it a short TTL too, so a crashed worker does not hold it forever and block refreshes.

Guidelines:

• Lock: simplest, briefly delays some requests
• Stale-while-revalidate: best UX, needs background refresh
• Probabilistic + jitter: smooths load, no waiting

Combine them for very hot keys.

A related danger is the cache penetration miss: many requests for a key that does not exist in the DB always miss the cache and hammer the backend. Cache the not-found result briefly too, so repeated lookups are absorbed.

Test your stampede-prevention knowledge.

You learned to stop cache stampedes:

• A stampede floods the DB when a hot key expires
• A SET NX recompute lock limits rebuilds to one client
• Stale-while-revalidate serves old data while refreshing
• Probabilistic early expiry spreads recomputation out
• TTL jitter prevents synchronized mass expiry