Preventing Cache Stampedes and Thundering Herds
Mitigate stampedes with request coalescing, jittered TTLs, and probabilistic early expiration.
What Is a Cache Stampede?
A cache stampede (also called a thundering herd) happens when a popular cached value expires and many concurrent requests miss the cache at the same instant. They all rush to the database to recompute the same value.
- One key expires → 5,000 in-flight requests all see a miss
- 5,000 identical queries hit the database simultaneously
- The DB saturates, latency spikes, and sometimes the whole system falls over
The irony: the cache exists to protect the database, but the moment of expiry becomes the most dangerous moment of all.
Seeing the Problem in Code
Here is a classic naive cache-aside read. It works fine under low traffic, but under heavy concurrency every miss spawns its own loadFromDb call.
Notice there is nothing stopping 1,000 callers from running loadFromDb at once for the same key.
async function getUser(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) {
return JSON.parse(cached);
}
// STAMPEDE RISK: every concurrent miss runs this
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
}All lessons in this course
- Cache-Aside, Write-Through, and TTL Strategies
- Distributed Locks and the Redlock Algorithm
- Pub/Sub, Streams, and Rate Limiting with Redis
- Preventing Cache Stampedes and Thundering Herds