防止缓存击穿与惊群
通过请求合并、抖动 TTL 和概率性提前过期来缓解缓存击穿。
防止缓存击穿与惊群 是 CoddyKit 上的免费 Node.js Backend Development Bootcamp 课时。 这是第 4 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Node.js Backend Development Bootcamp 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Node.js Backend Development Bootcamp 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
What Is a Cache Stampede?
A cache stampede (also called a thundering herd) happens when a popular cached value expires and many concurrent requests miss the cache at the same instant. They all rush to the database to recompute the same value.
- One key expires → 5,000 in-flight requests all see a miss
- 5,000 identical queries hit the database simultaneously
- The DB saturates, latency spikes, and sometimes the whole system falls over
The irony: the cache exists to protect the database, but the moment of expiry becomes the most dangerous moment of all.
Seeing the Problem in Code
Here is a classic naive cache-aside read. It works fine under low traffic, but under heavy concurrency every miss spawns its own loadFromDb call.
Notice there is nothing stopping 1,000 callers from running loadFromDb at once for the same key.
async function getUser(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) {
return JSON.parse(cached);
}
// STAMPEDE RISK: every concurrent miss runs this
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
}Strategy 1: Request Coalescing
Request coalescing (a.k.a. single-flight or in-flight de-duplication) means: when many callers want the same key at the same time, only one of them actually computes the value. The rest await the same promise.
- Keep an in-memory map of
key → pending promise - First caller starts the work and stores its promise
- Concurrent callers find the pending promise and await it
- When it settles, delete the entry
This collapses N database calls into 1 per process.
Implementing Single-Flight
A minimal, dependency-free single-flight helper. Every concurrent caller for the same key shares one underlying promise. The finally block clears the entry so the next round recomputes.
This snippet is fully self-contained and demonstrates the coalescing behavior with simulated slow work.
function createSingleFlight() {
const inFlight = new Map();
return function run(key, fn) {
if (inFlight.has(key)) {
return inFlight.get(key);
}
const p = Promise.resolve()
.then(fn)
.finally(() => inFlight.delete(key));
inFlight.set(key, p);
return p;
};
}
async function main() {
const flight = createSingleFlight();
let dbCalls = 0;
const load = () => new Promise(res => {
dbCalls++;
setTimeout(() => res('value-' + dbCalls), 50);
});
// 5 concurrent callers, same key
const results = await Promise.all(
Array.from({ length: 5 }, () => flight('user:42', load))
);
console.log('results:', results);
console.log('actual db calls:', dbCalls);
}
main();Coalescing's Limit: It's Per-Process
In-memory single-flight only de-duplicates within one Node.js process. If you run 20 instances behind a load balancer, you can still get up to 20 simultaneous DB calls per expired key — one per process.
- Great for cutting N requests → 1 inside each instance
- Not enough alone for large horizontal fleets
- For cross-process protection you need a distributed lock in Redis
Combine coalescing (cheap, local) with distributed locking or probabilistic expiration (cluster-wide) for full coverage.
Strategy 2: Distributed Lock
A distributed lock lets a single instance (across the whole fleet) win the right to recompute. Use Redis SET key value NX PX ttl: it sets the key only if it does not exist, atomically.
- Winner recomputes and repopulates the cache
- Losers either wait-and-retry the cache, or serve stale data
- Always set a TTL on the lock so a crashed winner cannot deadlock the key forever
This is the cross-process complement to in-process coalescing.
async function getWithLock(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) return JSON.parse(cached);
const lockKey = 'lock:' + key;
const token = Math.random().toString(36).slice(2);
// NX = only if absent, PX = lock TTL in ms
const won = await redis.set(lockKey, token, 'NX', 'PX', 5000);
if (won === 'OK') {
try {
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
} finally {
// release only if we still own the lock
if (await redis.get(lockKey) === token) await redis.del(lockKey);
}
}
// Lost the race: briefly wait, then read the now-fresh cache
await new Promise(r => setTimeout(r, 50));
const retry = await redis.get(key);
return retry !== null ? JSON.parse(retry) : db.loadUser(id);
}Strategy 3: Jittered TTLs
If you warm 10,000 keys in a loop with the same TTL, they all expire at the same second — a synchronized stampede across many keys at once. TTL jitter spreads expirations out by adding a small random offset to each TTL.
- Base TTL 300s → actual TTL 270–330s, randomized per key
- Expirations scatter across a window instead of clustering on one tick
- Cheap, zero coordination, and it composes with every other strategy
Always jitter TTLs when you bulk-populate or refresh many related keys.
// Add +/- jitterPct random spread around a base TTL
function jitteredTtl(baseSeconds, jitterPct = 0.1) {
const spread = baseSeconds * jitterPct;
const offset = (Math.random() * 2 - 1) * spread; // -spread..+spread
return Math.max(1, Math.round(baseSeconds + offset));
}
// Demo: 5 keys warmed together get different lifetimes
for (let i = 0; i < 5; i++) {
console.log('key' + i + ' ttl =', jitteredTtl(300));
}Strategy 4: Probabilistic Early Expiration
Probabilistic early expiration (the XFetch algorithm) refreshes a key before it actually expires, with a probability that grows as expiry nears. So a single lucky request recomputes early while the old value is still served to everyone else.
The classic rule recomputes when:
now - delta * beta * ln(random()) ≥ expiry
Here delta is how long the last recompute took, and beta (default 1) tunes aggressiveness. Slow-to-compute keys (large delta) start refreshing earlier, which is exactly what you want.
XFetch in Code
To use XFetch you store, alongside the value, the recompute duration (delta) and the absolute expiry time. On each read you roll the probabilistic check. Most callers serve the cached value; occasionally one refreshes ahead of expiry.
This standalone demo shows that as we approach expiry, the early-refresh probability climbs toward 1.
function shouldRecompute(deltaMs, expiryMs, now, beta = 1) {
// XFetch: earlier refresh as we near expiry, scaled by recompute cost
const xfetch = now - deltaMs * beta * Math.log(Math.random());
return xfetch >= expiryMs;
}
const now = Date.now();
const delta = 200; // last recompute took 200ms
const expiry = now + 1000; // value expires in 1s
let refreshes = 0;
for (let i = 0; i < 1000; i++) {
// sample 'now' uniformly across the key's lifetime
const t = now + Math.random() * 1000;
if (shouldRecompute(delta, expiry, t)) refreshes++;
}
console.log('early refreshes out of 1000 reads:', refreshes);Combining the Strategies
These techniques are complementary layers, not competitors. A production cache read often stacks several:
- Coalescing — collapse duplicate work inside each process
- Distributed lock — one recompute across the whole fleet
- Jittered TTL — desynchronize bulk expirations
- Probabilistic early expiry — refresh hot keys before they ever miss
Start with jitter + coalescing (cheap, no coordination). Add a lock or XFetch for your hottest, most expensive keys.
Stale-While-Revalidate
A practical pattern that ties it together: stale-while-revalidate (SWR). Keep two lifetimes — a short fresh window and a longer stale window. While stale, immediately serve the old value and kick off a background refresh (de-duplicated by single-flight).
- Users almost never wait on a cold recompute
- The refresh runs once, off the request's critical path
- Pair with jitter so stale windows do not all end at once
SWR turns a hard miss (everyone waits) into a soft miss (one background refresh, everyone served instantly).
async function swrGet(redis, db, id, freshSec = 60, staleSec = 600) {
const key = 'user:' + id;
const raw = await redis.get(key);
if (raw) {
const { value, storedAt } = JSON.parse(raw);
const ageSec = (Date.now() - storedAt) / 1000;
if (ageSec > freshSec) {
// stale but usable: refresh in background, serve now
refreshInBackground(redis, db, id, key, staleSec);
}
return value;
}
return refreshInBackground(redis, db, id, key, staleSec);
}Quick Check
You run 30 Node.js instances behind a load balancer. A single extremely hot key expires and you must guarantee the database receives at most one recompute query for it. Which approach achieves this?
Recap
You learned how to defend against cache stampedes and thundering herds in Node.js:
- Request coalescing collapses concurrent duplicate work to one promise — but only per process.
- Distributed locks (
SET NX PX) extend that guarantee across the whole fleet; always give the lock a TTL. - Jittered TTLs desynchronize bulk expirations so many keys never expire on the same tick.
- Probabilistic early expiration (XFetch) refreshes hot, expensive keys before they ever miss.
- Stale-while-revalidate serves old data instantly and refreshes once in the background.
Layer them: jitter + coalescing as a baseline, then locks or XFetch for your hottest keys.
常见问题解答
「防止缓存击穿与惊群」课时是免费的吗？
是的 — 「防止缓存击穿与惊群」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Node.js Backend Development Bootcamp 课程的其余内容，请升级到 CoddyKit PRO。 Node.js Backend Development Bootcamp 课程共包含 4 节课。
「防止缓存击穿与惊群」这节课中我会学到什么？
通过请求合并、抖动 TTL 和概率性提前过期来缓解缓存击穿。 你通过在浏览器中直接运行的动手代码来练习 Node.js Backend Development Bootcamp，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Node.js Backend Development Bootcamp 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Node.js Backend Development Bootcamp 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 4 节课，共 4 节。
「防止缓存击穿与惊群」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Node.js Backend Development Bootcamp 课中编写并运行代码吗？
能。每节 Node.js Backend Development Bootcamp 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。