Предотвращение лавинного обращения к кэшу
Сдерживайте лавины запросов с помощью объединения запросов, TTL со случайным смещением и вероятностного досрочного истечения.
«Предотвращение лавинного обращения к кэшу» — бесплатный урок Node.js Backend Development Bootcamp на CoddyKit. Это урок 4 из 4. Ты можешь прочитать весь урок бесплатно ниже — а потом практиковать его прямо в браузере с встроенным редактором кода и ИИ-репетитором 24/7. Это часть пути обучения Node.js Backend Development Bootcamp, и твой прогресс синхронизируется между веб-версией и приложением CoddyKit. Курс Node.js Backend Development Bootcamp содержит 4 уроков всего.
Части этого урока еще не переведены и отображаются на английском.
What Is a Cache Stampede?
A cache stampede (also called a thundering herd) happens when a popular cached value expires and many concurrent requests miss the cache at the same instant. They all rush to the database to recompute the same value.
- One key expires → 5,000 in-flight requests all see a miss
- 5,000 identical queries hit the database simultaneously
- The DB saturates, latency spikes, and sometimes the whole system falls over
The irony: the cache exists to protect the database, but the moment of expiry becomes the most dangerous moment of all.
Seeing the Problem in Code
Here is a classic naive cache-aside read. It works fine under low traffic, but under heavy concurrency every miss spawns its own loadFromDb call.
Notice there is nothing stopping 1,000 callers from running loadFromDb at once for the same key.
async function getUser(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) {
return JSON.parse(cached);
}
// STAMPEDE RISK: every concurrent miss runs this
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
}Strategy 1: Request Coalescing
Request coalescing (a.k.a. single-flight or in-flight de-duplication) means: when many callers want the same key at the same time, only one of them actually computes the value. The rest await the same promise.
- Keep an in-memory map of
key → pending promise - First caller starts the work and stores its promise
- Concurrent callers find the pending promise and await it
- When it settles, delete the entry
This collapses N database calls into 1 per process.
Implementing Single-Flight
A minimal, dependency-free single-flight helper. Every concurrent caller for the same key shares one underlying promise. The finally block clears the entry so the next round recomputes.
This snippet is fully self-contained and demonstrates the coalescing behavior with simulated slow work.
function createSingleFlight() {
const inFlight = new Map();
return function run(key, fn) {
if (inFlight.has(key)) {
return inFlight.get(key);
}
const p = Promise.resolve()
.then(fn)
.finally(() => inFlight.delete(key));
inFlight.set(key, p);
return p;
};
}
async function main() {
const flight = createSingleFlight();
let dbCalls = 0;
const load = () => new Promise(res => {
dbCalls++;
setTimeout(() => res('value-' + dbCalls), 50);
});
// 5 concurrent callers, same key
const results = await Promise.all(
Array.from({ length: 5 }, () => flight('user:42', load))
);
console.log('results:', results);
console.log('actual db calls:', dbCalls);
}
main();Coalescing's Limit: It's Per-Process
In-memory single-flight only de-duplicates within one Node.js process. If you run 20 instances behind a load balancer, you can still get up to 20 simultaneous DB calls per expired key — one per process.
- Great for cutting N requests → 1 inside each instance
- Not enough alone for large horizontal fleets
- For cross-process protection you need a distributed lock in Redis
Combine coalescing (cheap, local) with distributed locking or probabilistic expiration (cluster-wide) for full coverage.
Strategy 2: Distributed Lock
A distributed lock lets a single instance (across the whole fleet) win the right to recompute. Use Redis SET key value NX PX ttl: it sets the key only if it does not exist, atomically.
- Winner recomputes and repopulates the cache
- Losers either wait-and-retry the cache, or serve stale data
- Always set a TTL on the lock so a crashed winner cannot deadlock the key forever
This is the cross-process complement to in-process coalescing.
async function getWithLock(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) return JSON.parse(cached);
const lockKey = 'lock:' + key;
const token = Math.random().toString(36).slice(2);
// NX = only if absent, PX = lock TTL in ms
const won = await redis.set(lockKey, token, 'NX', 'PX', 5000);
if (won === 'OK') {
try {
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
} finally {
// release only if we still own the lock
if (await redis.get(lockKey) === token) await redis.del(lockKey);
}
}
// Lost the race: briefly wait, then read the now-fresh cache
await new Promise(r => setTimeout(r, 50));
const retry = await redis.get(key);
return retry !== null ? JSON.parse(retry) : db.loadUser(id);
}Strategy 3: Jittered TTLs
If you warm 10,000 keys in a loop with the same TTL, they all expire at the same second — a synchronized stampede across many keys at once. TTL jitter spreads expirations out by adding a small random offset to each TTL.
- Base TTL 300s → actual TTL 270–330s, randomized per key
- Expirations scatter across a window instead of clustering on one tick
- Cheap, zero coordination, and it composes with every other strategy
Always jitter TTLs when you bulk-populate or refresh many related keys.
// Add +/- jitterPct random spread around a base TTL
function jitteredTtl(baseSeconds, jitterPct = 0.1) {
const spread = baseSeconds * jitterPct;
const offset = (Math.random() * 2 - 1) * spread; // -spread..+spread
return Math.max(1, Math.round(baseSeconds + offset));
}
// Demo: 5 keys warmed together get different lifetimes
for (let i = 0; i < 5; i++) {
console.log('key' + i + ' ttl =', jitteredTtl(300));
}Strategy 4: Probabilistic Early Expiration
Probabilistic early expiration (the XFetch algorithm) refreshes a key before it actually expires, with a probability that grows as expiry nears. So a single lucky request recomputes early while the old value is still served to everyone else.
The classic rule recomputes when:
now - delta * beta * ln(random()) ≥ expiry
Here delta is how long the last recompute took, and beta (default 1) tunes aggressiveness. Slow-to-compute keys (large delta) start refreshing earlier, which is exactly what you want.
XFetch in Code
To use XFetch you store, alongside the value, the recompute duration (delta) and the absolute expiry time. On each read you roll the probabilistic check. Most callers serve the cached value; occasionally one refreshes ahead of expiry.
This standalone demo shows that as we approach expiry, the early-refresh probability climbs toward 1.
function shouldRecompute(deltaMs, expiryMs, now, beta = 1) {
// XFetch: earlier refresh as we near expiry, scaled by recompute cost
const xfetch = now - deltaMs * beta * Math.log(Math.random());
return xfetch >= expiryMs;
}
const now = Date.now();
const delta = 200; // last recompute took 200ms
const expiry = now + 1000; // value expires in 1s
let refreshes = 0;
for (let i = 0; i < 1000; i++) {
// sample 'now' uniformly across the key's lifetime
const t = now + Math.random() * 1000;
if (shouldRecompute(delta, expiry, t)) refreshes++;
}
console.log('early refreshes out of 1000 reads:', refreshes);Combining the Strategies
These techniques are complementary layers, not competitors. A production cache read often stacks several:
- Coalescing — collapse duplicate work inside each process
- Distributed lock — one recompute across the whole fleet
- Jittered TTL — desynchronize bulk expirations
- Probabilistic early expiry — refresh hot keys before they ever miss
Start with jitter + coalescing (cheap, no coordination). Add a lock or XFetch for your hottest, most expensive keys.
Stale-While-Revalidate
A practical pattern that ties it together: stale-while-revalidate (SWR). Keep two lifetimes — a short fresh window and a longer stale window. While stale, immediately serve the old value and kick off a background refresh (de-duplicated by single-flight).
- Users almost never wait on a cold recompute
- The refresh runs once, off the request's critical path
- Pair with jitter so stale windows do not all end at once
SWR turns a hard miss (everyone waits) into a soft miss (one background refresh, everyone served instantly).
async function swrGet(redis, db, id, freshSec = 60, staleSec = 600) {
const key = 'user:' + id;
const raw = await redis.get(key);
if (raw) {
const { value, storedAt } = JSON.parse(raw);
const ageSec = (Date.now() - storedAt) / 1000;
if (ageSec > freshSec) {
// stale but usable: refresh in background, serve now
refreshInBackground(redis, db, id, key, staleSec);
}
return value;
}
return refreshInBackground(redis, db, id, key, staleSec);
}Quick Check
You run 30 Node.js instances behind a load balancer. A single extremely hot key expires and you must guarantee the database receives at most one recompute query for it. Which approach achieves this?
Recap
You learned how to defend against cache stampedes and thundering herds in Node.js:
- Request coalescing collapses concurrent duplicate work to one promise — but only per process.
- Distributed locks (
SET NX PX) extend that guarantee across the whole fleet; always give the lock a TTL. - Jittered TTLs desynchronize bulk expirations so many keys never expire on the same tick.
- Probabilistic early expiration (XFetch) refreshes hot, expensive keys before they ever miss.
- Stale-while-revalidate serves old data instantly and refreshes once in the background.
Layer them: jitter + coalescing as a baseline, then locks or XFetch for your hottest keys.
Часто задаваемые вопросы
Урок «Предотвращение лавинного обращения к кэшу» бесплатный?
Да — полный текст урока «Предотвращение лавинного обращения к кэшу» бесплатно доступен здесь в веб-версии. Чтобы практиковать его интерактивно (встроенный редактор кода и ИИ-репетитор 24/7) и разблокировать остальной курс Node.js Backend Development Bootcamp, подпишись на CoddyKit PRO. Курс Node.js Backend Development Bootcamp содержит 4 уроков всего.
Чему я научусь в уроке «Предотвращение лавинного обращения к кэшу»?
Сдерживайте лавины запросов с помощью объединения запросов, TTL со случайным смещением и вероятностного досрочного истечения. Ты практикуешь Node.js Backend Development Bootcamp с помощью реального кода, который запускаешь прямо в браузере, и ИИ-репетитор 24/7 отвечает на твои вопросы во время урока.
Нужен ли мне опыт, чтобы начать Node.js Backend Development Bootcamp?
Предыдущий опыт не требуется. Node.js Backend Development Bootcamp на CoddyKit структурирован для всех уровней — от новичков до продвинутых, поэтому ты можешь начать отсюда или с самого начала и учиться в своем темпе. Это урок 4 из 4.
Сколько времени занимает урок «Предотвращение лавинного обращения к кэшу»?
Большинство уроков CoddyKit занимают около 5–10 минут. Каждый из них компактный и интерактивный, поэтому ты постоянно делаешь прогресс и продолжаешь с того же места в веб-версии и приложении.
Можно ли писать и запускать код в этом уроке Node.js Backend Development Bootcamp?
Да. Каждый урок Node.js Backend Development Bootcamp включает встроенный редактор кода, поэтому ты пишешь и запускаешь реальный код прямо в браузере и получаешь моментальную обратную связь от AI — локальная установка не требуется.
Все уроки этого курса
- Стратегии Cache-Aside, Write-Through и TTL
- Распределённые блокировки и алгоритм Redlock
- Pub/Sub, потоки и ограничение частоты с Redis
- Предотвращение лавинного обращения к кэшу