0Pricing
Node.js Backend Development Bootcamp · レッスン

キャッシュスタンピードとThundering Herdの防止

リクエストの集約、ジッター付きTTL、確率的な早期期限切れでスタンピードを緩和します。

「キャッシュスタンピードとThundering Herdの防止」はCoddyKit上の無料Node.js Backend Development Bootcampレッスンです。 これはレッスン4/4です。 下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。 これはNode.js Backend Development Bootcamp学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Node.js Backend Development Bootcampコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

What Is a Cache Stampede?

A cache stampede (also called a thundering herd) happens when a popular cached value expires and many concurrent requests miss the cache at the same instant. They all rush to the database to recompute the same value.

  • One key expires → 5,000 in-flight requests all see a miss
  • 5,000 identical queries hit the database simultaneously
  • The DB saturates, latency spikes, and sometimes the whole system falls over

The irony: the cache exists to protect the database, but the moment of expiry becomes the most dangerous moment of all.

Seeing the Problem in Code

Here is a classic naive cache-aside read. It works fine under low traffic, but under heavy concurrency every miss spawns its own loadFromDb call.

Notice there is nothing stopping 1,000 callers from running loadFromDb at once for the same key.

async function getUser(redis, db, id) {
  const key = 'user:' + id;
  const cached = await redis.get(key);
  if (cached !== null) {
    return JSON.parse(cached);
  }
  // STAMPEDE RISK: every concurrent miss runs this
  const user = await db.loadUser(id);
  await redis.set(key, JSON.stringify(user), 'EX', 60);
  return user;
}

Strategy 1: Request Coalescing

Request coalescing (a.k.a. single-flight or in-flight de-duplication) means: when many callers want the same key at the same time, only one of them actually computes the value. The rest await the same promise.

  • Keep an in-memory map of key → pending promise
  • First caller starts the work and stores its promise
  • Concurrent callers find the pending promise and await it
  • When it settles, delete the entry

This collapses N database calls into 1 per process.

Implementing Single-Flight

A minimal, dependency-free single-flight helper. Every concurrent caller for the same key shares one underlying promise. The finally block clears the entry so the next round recomputes.

This snippet is fully self-contained and demonstrates the coalescing behavior with simulated slow work.

function createSingleFlight() {
  const inFlight = new Map();
  return function run(key, fn) {
    if (inFlight.has(key)) {
      return inFlight.get(key);
    }
    const p = Promise.resolve()
      .then(fn)
      .finally(() => inFlight.delete(key));
    inFlight.set(key, p);
    return p;
  };
}

async function main() {
  const flight = createSingleFlight();
  let dbCalls = 0;
  const load = () => new Promise(res => {
    dbCalls++;
    setTimeout(() => res('value-' + dbCalls), 50);
  });

  // 5 concurrent callers, same key
  const results = await Promise.all(
    Array.from({ length: 5 }, () => flight('user:42', load))
  );
  console.log('results:', results);
  console.log('actual db calls:', dbCalls);
}

main();

Coalescing's Limit: It's Per-Process

In-memory single-flight only de-duplicates within one Node.js process. If you run 20 instances behind a load balancer, you can still get up to 20 simultaneous DB calls per expired key — one per process.

  • Great for cutting N requests → 1 inside each instance
  • Not enough alone for large horizontal fleets
  • For cross-process protection you need a distributed lock in Redis

Combine coalescing (cheap, local) with distributed locking or probabilistic expiration (cluster-wide) for full coverage.

Strategy 2: Distributed Lock

A distributed lock lets a single instance (across the whole fleet) win the right to recompute. Use Redis SET key value NX PX ttl: it sets the key only if it does not exist, atomically.

  • Winner recomputes and repopulates the cache
  • Losers either wait-and-retry the cache, or serve stale data
  • Always set a TTL on the lock so a crashed winner cannot deadlock the key forever

This is the cross-process complement to in-process coalescing.

async function getWithLock(redis, db, id) {
  const key = 'user:' + id;
  const cached = await redis.get(key);
  if (cached !== null) return JSON.parse(cached);

  const lockKey = 'lock:' + key;
  const token = Math.random().toString(36).slice(2);
  // NX = only if absent, PX = lock TTL in ms
  const won = await redis.set(lockKey, token, 'NX', 'PX', 5000);

  if (won === 'OK') {
    try {
      const user = await db.loadUser(id);
      await redis.set(key, JSON.stringify(user), 'EX', 60);
      return user;
    } finally {
      // release only if we still own the lock
      if (await redis.get(lockKey) === token) await redis.del(lockKey);
    }
  }
  // Lost the race: briefly wait, then read the now-fresh cache
  await new Promise(r => setTimeout(r, 50));
  const retry = await redis.get(key);
  return retry !== null ? JSON.parse(retry) : db.loadUser(id);
}

Strategy 3: Jittered TTLs

If you warm 10,000 keys in a loop with the same TTL, they all expire at the same second — a synchronized stampede across many keys at once. TTL jitter spreads expirations out by adding a small random offset to each TTL.

  • Base TTL 300s → actual TTL 270–330s, randomized per key
  • Expirations scatter across a window instead of clustering on one tick
  • Cheap, zero coordination, and it composes with every other strategy

Always jitter TTLs when you bulk-populate or refresh many related keys.

// Add +/- jitterPct random spread around a base TTL
function jitteredTtl(baseSeconds, jitterPct = 0.1) {
  const spread = baseSeconds * jitterPct;
  const offset = (Math.random() * 2 - 1) * spread; // -spread..+spread
  return Math.max(1, Math.round(baseSeconds + offset));
}

// Demo: 5 keys warmed together get different lifetimes
for (let i = 0; i < 5; i++) {
  console.log('key' + i + ' ttl =', jitteredTtl(300));
}

Strategy 4: Probabilistic Early Expiration

Probabilistic early expiration (the XFetch algorithm) refreshes a key before it actually expires, with a probability that grows as expiry nears. So a single lucky request recomputes early while the old value is still served to everyone else.

The classic rule recomputes when:

  • now - delta * beta * ln(random()) ≥ expiry

Here delta is how long the last recompute took, and beta (default 1) tunes aggressiveness. Slow-to-compute keys (large delta) start refreshing earlier, which is exactly what you want.

XFetch in Code

To use XFetch you store, alongside the value, the recompute duration (delta) and the absolute expiry time. On each read you roll the probabilistic check. Most callers serve the cached value; occasionally one refreshes ahead of expiry.

This standalone demo shows that as we approach expiry, the early-refresh probability climbs toward 1.

function shouldRecompute(deltaMs, expiryMs, now, beta = 1) {
  // XFetch: earlier refresh as we near expiry, scaled by recompute cost
  const xfetch = now - deltaMs * beta * Math.log(Math.random());
  return xfetch >= expiryMs;
}

const now = Date.now();
const delta = 200;          // last recompute took 200ms
const expiry = now + 1000;  // value expires in 1s

let refreshes = 0;
for (let i = 0; i < 1000; i++) {
  // sample 'now' uniformly across the key's lifetime
  const t = now + Math.random() * 1000;
  if (shouldRecompute(delta, expiry, t)) refreshes++;
}
console.log('early refreshes out of 1000 reads:', refreshes);

Combining the Strategies

These techniques are complementary layers, not competitors. A production cache read often stacks several:

  • Coalescing — collapse duplicate work inside each process
  • Distributed lock — one recompute across the whole fleet
  • Jittered TTL — desynchronize bulk expirations
  • Probabilistic early expiry — refresh hot keys before they ever miss

Start with jitter + coalescing (cheap, no coordination). Add a lock or XFetch for your hottest, most expensive keys.

Stale-While-Revalidate

A practical pattern that ties it together: stale-while-revalidate (SWR). Keep two lifetimes — a short fresh window and a longer stale window. While stale, immediately serve the old value and kick off a background refresh (de-duplicated by single-flight).

  • Users almost never wait on a cold recompute
  • The refresh runs once, off the request's critical path
  • Pair with jitter so stale windows do not all end at once

SWR turns a hard miss (everyone waits) into a soft miss (one background refresh, everyone served instantly).

async function swrGet(redis, db, id, freshSec = 60, staleSec = 600) {
  const key = 'user:' + id;
  const raw = await redis.get(key);
  if (raw) {
    const { value, storedAt } = JSON.parse(raw);
    const ageSec = (Date.now() - storedAt) / 1000;
    if (ageSec > freshSec) {
      // stale but usable: refresh in background, serve now
      refreshInBackground(redis, db, id, key, staleSec);
    }
    return value;
  }
  return refreshInBackground(redis, db, id, key, staleSec);
}

Quick Check

You run 30 Node.js instances behind a load balancer. A single extremely hot key expires and you must guarantee the database receives at most one recompute query for it. Which approach achieves this?

Recap

You learned how to defend against cache stampedes and thundering herds in Node.js:

  • Request coalescing collapses concurrent duplicate work to one promise — but only per process.
  • Distributed locks (SET NX PX) extend that guarantee across the whole fleet; always give the lock a TTL.
  • Jittered TTLs desynchronize bulk expirations so many keys never expire on the same tick.
  • Probabilistic early expiration (XFetch) refreshes hot, expensive keys before they ever miss.
  • Stale-while-revalidate serves old data instantly and refreshes once in the background.

Layer them: jitter + coalescing as a baseline, then locks or XFetch for your hottest keys.

よくある質問

「キャッシュスタンピードとThundering Herdの防止」レッスンは無料ですか？

はい。「キャッシュスタンピードとThundering Herdの防止」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Node.js Backend Development Bootcampコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Node.js Backend Development Bootcampコースには全4レッスンが含まれています。

「キャッシュスタンピードとThundering Herdの防止」で何を学びますか？

リクエストの集約、ジッター付きTTL、確率的な早期期限切れでスタンピードを緩和します。 ブラウザで直接実行するハンズオンコードでNode.js Backend Development Bootcampを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

Node.js Backend Development Bootcampを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのNode.js Backend Development Bootcampは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。 これはレッスン4/4です。

「キャッシュスタンピードとThundering Herdの防止」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このNode.js Backend Development Bootcampレッスンでコードを書いて実行できますか？

はい。すべてのNode.js Backend Development Bootcampレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

  1. Cache-Aside、Write-Through、TTL戦略
  2. 分散ロックとRedlockアルゴリズム
  3. RedisによるPub/Sub、Streams、レート制限
  4. キャッシュスタンピードとThundering Herdの防止
← Node.js Backend Development Bootcampに戻る