Mencegah Cache Stampede dan Thundering Herd
Atasi stampede dengan penggabungan permintaan, TTL ber-jitter, dan kedaluwarsa awal probabilistik.
Mencegah Cache Stampede dan Thundering Herd adalah pelajaran Node.js Backend Development Bootcamp gratis di CoddyKit. Ini adalah pelajaran 4 dari 4. Kamu bisa membaca pelajaran lengkapnya di bawah secara gratis — lalu praktikkan langsung di browser dengan editor kode bawaan dan tutor AI 24/7. Ini adalah bagian dari jalur belajar Node.js Backend Development Bootcamp, dan progresmu tersinkronisasi di web dan aplikasi CoddyKit. Kursus Node.js Backend Development Bootcamp mencakup 4 pelajaran total.
Bagian dari pelajaran ini belum diterjemahkan dan ditampilkan dalam bahasa Inggris.
What Is a Cache Stampede?
A cache stampede (also called a thundering herd) happens when a popular cached value expires and many concurrent requests miss the cache at the same instant. They all rush to the database to recompute the same value.
- One key expires → 5,000 in-flight requests all see a miss
- 5,000 identical queries hit the database simultaneously
- The DB saturates, latency spikes, and sometimes the whole system falls over
The irony: the cache exists to protect the database, but the moment of expiry becomes the most dangerous moment of all.
Seeing the Problem in Code
Here is a classic naive cache-aside read. It works fine under low traffic, but under heavy concurrency every miss spawns its own loadFromDb call.
Notice there is nothing stopping 1,000 callers from running loadFromDb at once for the same key.
async function getUser(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) {
return JSON.parse(cached);
}
// STAMPEDE RISK: every concurrent miss runs this
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
}Strategy 1: Request Coalescing
Request coalescing (a.k.a. single-flight or in-flight de-duplication) means: when many callers want the same key at the same time, only one of them actually computes the value. The rest await the same promise.
- Keep an in-memory map of
key → pending promise - First caller starts the work and stores its promise
- Concurrent callers find the pending promise and await it
- When it settles, delete the entry
This collapses N database calls into 1 per process.
Implementing Single-Flight
A minimal, dependency-free single-flight helper. Every concurrent caller for the same key shares one underlying promise. The finally block clears the entry so the next round recomputes.
This snippet is fully self-contained and demonstrates the coalescing behavior with simulated slow work.
function createSingleFlight() {
const inFlight = new Map();
return function run(key, fn) {
if (inFlight.has(key)) {
return inFlight.get(key);
}
const p = Promise.resolve()
.then(fn)
.finally(() => inFlight.delete(key));
inFlight.set(key, p);
return p;
};
}
async function main() {
const flight = createSingleFlight();
let dbCalls = 0;
const load = () => new Promise(res => {
dbCalls++;
setTimeout(() => res('value-' + dbCalls), 50);
});
// 5 concurrent callers, same key
const results = await Promise.all(
Array.from({ length: 5 }, () => flight('user:42', load))
);
console.log('results:', results);
console.log('actual db calls:', dbCalls);
}
main();Coalescing's Limit: It's Per-Process
In-memory single-flight only de-duplicates within one Node.js process. If you run 20 instances behind a load balancer, you can still get up to 20 simultaneous DB calls per expired key — one per process.
- Great for cutting N requests → 1 inside each instance
- Not enough alone for large horizontal fleets
- For cross-process protection you need a distributed lock in Redis
Combine coalescing (cheap, local) with distributed locking or probabilistic expiration (cluster-wide) for full coverage.
Strategy 2: Distributed Lock
A distributed lock lets a single instance (across the whole fleet) win the right to recompute. Use Redis SET key value NX PX ttl: it sets the key only if it does not exist, atomically.
- Winner recomputes and repopulates the cache
- Losers either wait-and-retry the cache, or serve stale data
- Always set a TTL on the lock so a crashed winner cannot deadlock the key forever
This is the cross-process complement to in-process coalescing.
async function getWithLock(redis, db, id) {
const key = 'user:' + id;
const cached = await redis.get(key);
if (cached !== null) return JSON.parse(cached);
const lockKey = 'lock:' + key;
const token = Math.random().toString(36).slice(2);
// NX = only if absent, PX = lock TTL in ms
const won = await redis.set(lockKey, token, 'NX', 'PX', 5000);
if (won === 'OK') {
try {
const user = await db.loadUser(id);
await redis.set(key, JSON.stringify(user), 'EX', 60);
return user;
} finally {
// release only if we still own the lock
if (await redis.get(lockKey) === token) await redis.del(lockKey);
}
}
// Lost the race: briefly wait, then read the now-fresh cache
await new Promise(r => setTimeout(r, 50));
const retry = await redis.get(key);
return retry !== null ? JSON.parse(retry) : db.loadUser(id);
}Strategy 3: Jittered TTLs
If you warm 10,000 keys in a loop with the same TTL, they all expire at the same second — a synchronized stampede across many keys at once. TTL jitter spreads expirations out by adding a small random offset to each TTL.
- Base TTL 300s → actual TTL 270–330s, randomized per key
- Expirations scatter across a window instead of clustering on one tick
- Cheap, zero coordination, and it composes with every other strategy
Always jitter TTLs when you bulk-populate or refresh many related keys.
// Add +/- jitterPct random spread around a base TTL
function jitteredTtl(baseSeconds, jitterPct = 0.1) {
const spread = baseSeconds * jitterPct;
const offset = (Math.random() * 2 - 1) * spread; // -spread..+spread
return Math.max(1, Math.round(baseSeconds + offset));
}
// Demo: 5 keys warmed together get different lifetimes
for (let i = 0; i < 5; i++) {
console.log('key' + i + ' ttl =', jitteredTtl(300));
}Strategy 4: Probabilistic Early Expiration
Probabilistic early expiration (the XFetch algorithm) refreshes a key before it actually expires, with a probability that grows as expiry nears. So a single lucky request recomputes early while the old value is still served to everyone else.
The classic rule recomputes when:
now - delta * beta * ln(random()) ≥ expiry
Here delta is how long the last recompute took, and beta (default 1) tunes aggressiveness. Slow-to-compute keys (large delta) start refreshing earlier, which is exactly what you want.
XFetch in Code
To use XFetch you store, alongside the value, the recompute duration (delta) and the absolute expiry time. On each read you roll the probabilistic check. Most callers serve the cached value; occasionally one refreshes ahead of expiry.
This standalone demo shows that as we approach expiry, the early-refresh probability climbs toward 1.
function shouldRecompute(deltaMs, expiryMs, now, beta = 1) {
// XFetch: earlier refresh as we near expiry, scaled by recompute cost
const xfetch = now - deltaMs * beta * Math.log(Math.random());
return xfetch >= expiryMs;
}
const now = Date.now();
const delta = 200; // last recompute took 200ms
const expiry = now + 1000; // value expires in 1s
let refreshes = 0;
for (let i = 0; i < 1000; i++) {
// sample 'now' uniformly across the key's lifetime
const t = now + Math.random() * 1000;
if (shouldRecompute(delta, expiry, t)) refreshes++;
}
console.log('early refreshes out of 1000 reads:', refreshes);Combining the Strategies
These techniques are complementary layers, not competitors. A production cache read often stacks several:
- Coalescing — collapse duplicate work inside each process
- Distributed lock — one recompute across the whole fleet
- Jittered TTL — desynchronize bulk expirations
- Probabilistic early expiry — refresh hot keys before they ever miss
Start with jitter + coalescing (cheap, no coordination). Add a lock or XFetch for your hottest, most expensive keys.
Stale-While-Revalidate
A practical pattern that ties it together: stale-while-revalidate (SWR). Keep two lifetimes — a short fresh window and a longer stale window. While stale, immediately serve the old value and kick off a background refresh (de-duplicated by single-flight).
- Users almost never wait on a cold recompute
- The refresh runs once, off the request's critical path
- Pair with jitter so stale windows do not all end at once
SWR turns a hard miss (everyone waits) into a soft miss (one background refresh, everyone served instantly).
async function swrGet(redis, db, id, freshSec = 60, staleSec = 600) {
const key = 'user:' + id;
const raw = await redis.get(key);
if (raw) {
const { value, storedAt } = JSON.parse(raw);
const ageSec = (Date.now() - storedAt) / 1000;
if (ageSec > freshSec) {
// stale but usable: refresh in background, serve now
refreshInBackground(redis, db, id, key, staleSec);
}
return value;
}
return refreshInBackground(redis, db, id, key, staleSec);
}Quick Check
You run 30 Node.js instances behind a load balancer. A single extremely hot key expires and you must guarantee the database receives at most one recompute query for it. Which approach achieves this?
Recap
You learned how to defend against cache stampedes and thundering herds in Node.js:
- Request coalescing collapses concurrent duplicate work to one promise — but only per process.
- Distributed locks (
SET NX PX) extend that guarantee across the whole fleet; always give the lock a TTL. - Jittered TTLs desynchronize bulk expirations so many keys never expire on the same tick.
- Probabilistic early expiration (XFetch) refreshes hot, expensive keys before they ever miss.
- Stale-while-revalidate serves old data instantly and refreshes once in the background.
Layer them: jitter + coalescing as a baseline, then locks or XFetch for your hottest keys.
Belajar JavaScript dengan tutor AI — gratis
Tulis dan jalankan kode asli di browser kamu, dapatkan bantuan instan dari tutor AI 24/7, dan lanjutkan di mana kamu tinggalkan di web atau aplikasi.
- Kursus
- 22
- Pelajaran
- 92
Pertanyaan yang Sering Diajukan
Apakah pelajaran “Mencegah Cache Stampede dan Thundering Herd” gratis?
Ya — teks lengkap “Mencegah Cache Stampede dan Thundering Herd” gratis dibaca di sini di web. Untuk praktiknya secara interaktif (editor kode bawaan dan tutor AI 24/7) dan buka sisa kursus Node.js Backend Development Bootcamp, upgrade ke CoddyKit PRO. Kursus Node.js Backend Development Bootcamp mencakup 4 pelajaran total.
Apa yang akan aku pelajari di “Mencegah Cache Stampede dan Thundering Herd”?
Atasi stampede dengan penggabungan permintaan, TTL ber-jitter, dan kedaluwarsa awal probabilistik. Kamu berlatih Node.js Backend Development Bootcamp dengan kode praktik yang langsung kamu jalankan di browser, dan tutor AI 24/7 menjawab pertanyaanmu saat kamu mengerjakan pelajaran ini.
Apakah aku perlu pengalaman untuk memulai Node.js Backend Development Bootcamp?
Tidak diperlukan pengalaman sebelumnya. Node.js Backend Development Bootcamp di CoddyKit dirancang untuk pemula hingga pelajar tingkat lanjut, jadi kamu bisa memulai di sini atau dari awal dan belajar sesuai kecepatan kamu sendiri. Ini adalah pelajaran 4 dari 4.
Berapa lama pelajaran “Mencegah Cache Stampede dan Thundering Herd” memakan waktu?
Sebagian besar pelajaran CoddyKit memakan waktu sekitar 5–10 menit. Setiap pelajaran ringkas dan interaktif, jadi kamu membuat kemajuan stabil dan melanjutkan dari tempat kamu tinggalkan di web dan aplikasi.
Bisakah aku menulis dan menjalankan kode dalam pelajaran Node.js Backend Development Bootcamp ini?
Ya. Setiap pelajaran Node.js Backend Development Bootcamp menyertakan editor kode bawaan, jadi kamu menulis dan menjalankan kode nyata langsung di browser dan mendapatkan umpan balik AI instan — tidak diperlukan penyiapan lokal.
Semua pelajaran dalam kursus ini
- Strategi Cache-Aside, Write-Through, dan TTL
- Lock Terdistribusi dan Algoritma Redlock
- Pub/Sub, Stream, dan Pembatasan Laju dengan Redis
- Mencegah Cache Stampede dan Thundering Herd