Node.js Backend Development Bootcamp · 강의

캐시 스탬피드 및 우르르 몰림 방지

요청 병합, 지터를 적용한 TTL 및 확률적 조기 만료로 요청 폭주를 완화합니다.

캐시 스탬피드 및 우르르 몰림 방지은(는) CoddyKit의 무료 Node.js Backend Development Bootcamp 강의입니다. 이것은 4개 중 4번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 Node.js Backend Development Bootcamp 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. Node.js Backend Development Bootcamp 강의에는 총 4개의 강의가 포함되어 있습니다.

이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.

What Is a Cache Stampede?

A cache stampede (also called a thundering herd) happens when a popular cached value expires and many concurrent requests miss the cache at the same instant. They all rush to the database to recompute the same value.

One key expires → 5,000 in-flight requests all see a miss
5,000 identical queries hit the database simultaneously
The DB saturates, latency spikes, and sometimes the whole system falls over

The irony: the cache exists to protect the database, but the moment of expiry becomes the most dangerous moment of all.

Seeing the Problem in Code

Here is a classic naive cache-aside read. It works fine under low traffic, but under heavy concurrency every miss spawns its own loadFromDb call.

Notice there is nothing stopping 1,000 callers from running loadFromDb at once for the same key.

async function getUser(redis, db, id) {
  const key = 'user:' + id;
  const cached = await redis.get(key);
  if (cached !== null) {
    return JSON.parse(cached);
  }
  // STAMPEDE RISK: every concurrent miss runs this
  const user = await db.loadUser(id);
  await redis.set(key, JSON.stringify(user), 'EX', 60);
  return user;
}

Strategy 1: Request Coalescing

Request coalescing (a.k.a. single-flight or in-flight de-duplication) means: when many callers want the same key at the same time, only one of them actually computes the value. The rest await the same promise.

Keep an in-memory map of key → pending promise
First caller starts the work and stores its promise
Concurrent callers find the pending promise and await it
When it settles, delete the entry

This collapses N database calls into 1 per process.

Implementing Single-Flight

A minimal, dependency-free single-flight helper. Every concurrent caller for the same key shares one underlying promise. The finally block clears the entry so the next round recomputes.

This snippet is fully self-contained and demonstrates the coalescing behavior with simulated slow work.

function createSingleFlight() {
  const inFlight = new Map();
  return function run(key, fn) {
    if (inFlight.has(key)) {
      return inFlight.get(key);
    }
    const p = Promise.resolve()
      .then(fn)
      .finally(() => inFlight.delete(key));
    inFlight.set(key, p);
    return p;
  };
}

async function main() {
  const flight = createSingleFlight();
  let dbCalls = 0;
  const load = () => new Promise(res => {
    dbCalls++;
    setTimeout(() => res('value-' + dbCalls), 50);
  });

  // 5 concurrent callers, same key
  const results = await Promise.all(
    Array.from({ length: 5 }, () => flight('user:42', load))
  );
  console.log('results:', results);
  console.log('actual db calls:', dbCalls);
}

main();

Coalescing's Limit: It's Per-Process

In-memory single-flight only de-duplicates within one Node.js process. If you run 20 instances behind a load balancer, you can still get up to 20 simultaneous DB calls per expired key — one per process.

Great for cutting N requests → 1 inside each instance
Not enough alone for large horizontal fleets
For cross-process protection you need a distributed lock in Redis

Combine coalescing (cheap, local) with distributed locking or probabilistic expiration (cluster-wide) for full coverage.

Strategy 2: Distributed Lock

A distributed lock lets a single instance (across the whole fleet) win the right to recompute. Use Redis SET key value NX PX ttl: it sets the key only if it does not exist, atomically.

Winner recomputes and repopulates the cache
Losers either wait-and-retry the cache, or serve stale data
Always set a TTL on the lock so a crashed winner cannot deadlock the key forever

This is the cross-process complement to in-process coalescing.

async function getWithLock(redis, db, id) {
  const key = 'user:' + id;
  const cached = await redis.get(key);
  if (cached !== null) return JSON.parse(cached);

  const lockKey = 'lock:' + key;
  const token = Math.random().toString(36).slice(2);
  // NX = only if absent, PX = lock TTL in ms
  const won = await redis.set(lockKey, token, 'NX', 'PX', 5000);

  if (won === 'OK') {
    try {
      const user = await db.loadUser(id);
      await redis.set(key, JSON.stringify(user), 'EX', 60);
      return user;
    } finally {
      // release only if we still own the lock
      if (await redis.get(lockKey) === token) await redis.del(lockKey);
    }
  }
  // Lost the race: briefly wait, then read the now-fresh cache
  await new Promise(r => setTimeout(r, 50));
  const retry = await redis.get(key);
  return retry !== null ? JSON.parse(retry) : db.loadUser(id);
}

Strategy 3: Jittered TTLs

If you warm 10,000 keys in a loop with the same TTL, they all expire at the same second — a synchronized stampede across many keys at once. TTL jitter spreads expirations out by adding a small random offset to each TTL.

Base TTL 300s → actual TTL 270–330s, randomized per key
Expirations scatter across a window instead of clustering on one tick
Cheap, zero coordination, and it composes with every other strategy

Always jitter TTLs when you bulk-populate or refresh many related keys.

// Add +/- jitterPct random spread around a base TTL
function jitteredTtl(baseSeconds, jitterPct = 0.1) {
  const spread = baseSeconds * jitterPct;
  const offset = (Math.random() * 2 - 1) * spread; // -spread..+spread
  return Math.max(1, Math.round(baseSeconds + offset));
}

// Demo: 5 keys warmed together get different lifetimes
for (let i = 0; i < 5; i++) {
  console.log('key' + i + ' ttl =', jitteredTtl(300));
}

Strategy 4: Probabilistic Early Expiration

Probabilistic early expiration (the XFetch algorithm) refreshes a key before it actually expires, with a probability that grows as expiry nears. So a single lucky request recomputes early while the old value is still served to everyone else.

The classic rule recomputes when:

now - delta * beta * ln(random()) ≥ expiry

Here delta is how long the last recompute took, and beta (default 1) tunes aggressiveness. Slow-to-compute keys (large delta) start refreshing earlier, which is exactly what you want.

XFetch in Code

To use XFetch you store, alongside the value, the recompute duration (delta) and the absolute expiry time. On each read you roll the probabilistic check. Most callers serve the cached value; occasionally one refreshes ahead of expiry.

This standalone demo shows that as we approach expiry, the early-refresh probability climbs toward 1.

function shouldRecompute(deltaMs, expiryMs, now, beta = 1) {
  // XFetch: earlier refresh as we near expiry, scaled by recompute cost
  const xfetch = now - deltaMs * beta * Math.log(Math.random());
  return xfetch >= expiryMs;
}

const now = Date.now();
const delta = 200;          // last recompute took 200ms
const expiry = now + 1000;  // value expires in 1s

let refreshes = 0;
for (let i = 0; i < 1000; i++) {
  // sample 'now' uniformly across the key's lifetime
  const t = now + Math.random() * 1000;
  if (shouldRecompute(delta, expiry, t)) refreshes++;
}
console.log('early refreshes out of 1000 reads:', refreshes);

Combining the Strategies

These techniques are complementary layers, not competitors. A production cache read often stacks several:

Coalescing — collapse duplicate work inside each process
Distributed lock — one recompute across the whole fleet
Jittered TTL — desynchronize bulk expirations
Probabilistic early expiry — refresh hot keys before they ever miss

Start with jitter + coalescing (cheap, no coordination). Add a lock or XFetch for your hottest, most expensive keys.

Stale-While-Revalidate

A practical pattern that ties it together: stale-while-revalidate (SWR). Keep two lifetimes — a short fresh window and a longer stale window. While stale, immediately serve the old value and kick off a background refresh (de-duplicated by single-flight).

Users almost never wait on a cold recompute
The refresh runs once, off the request's critical path
Pair with jitter so stale windows do not all end at once

SWR turns a hard miss (everyone waits) into a soft miss (one background refresh, everyone served instantly).

async function swrGet(redis, db, id, freshSec = 60, staleSec = 600) {
  const key = 'user:' + id;
  const raw = await redis.get(key);
  if (raw) {
    const { value, storedAt } = JSON.parse(raw);
    const ageSec = (Date.now() - storedAt) / 1000;
    if (ageSec > freshSec) {
      // stale but usable: refresh in background, serve now
      refreshInBackground(redis, db, id, key, staleSec);
    }
    return value;
  }
  return refreshInBackground(redis, db, id, key, staleSec);
}

Quick Check

You run 30 Node.js instances behind a load balancer. A single extremely hot key expires and you must guarantee the database receives at most one recompute query for it. Which approach achieves this?

Recap

You learned how to defend against cache stampedes and thundering herds in Node.js:

Request coalescing collapses concurrent duplicate work to one promise — but only per process.
Distributed locks (SET NX PX) extend that guarantee across the whole fleet; always give the lock a TTL.
Jittered TTLs desynchronize bulk expirations so many keys never expire on the same tick.
Probabilistic early expiration (XFetch) refreshes hot, expensive keys before they ever miss.
Stale-while-revalidate serves old data instantly and refreshes once in the background.

Layer them: jitter + coalescing as a baseline, then locks or XFetch for your hottest keys.

자주 묻는 질문

“캐시 스탬피드 및 우르르 몰림 방지” 강의는 무료인가요?

네 — “캐시 스탬피드 및 우르르 몰림 방지” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 Node.js Backend Development Bootcamp 강의 전체를 잠금 해제할 수 있습니다. Node.js Backend Development Bootcamp 강의에는 총 4개의 강의가 포함되어 있습니다.

“캐시 스탬피드 및 우르르 몰림 방지”에서 뭘 배우나요?

요청 병합, 지터를 적용한 TTL 및 확률적 조기 만료로 요청 폭주를 완화합니다. 브라우저에서 직접 실행하는 실습 코드로 Node.js Backend Development Bootcamp을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.

Node.js Backend Development Bootcamp을(를) 시작하는 데 경험이 필요한가요?

사전 경험은 필요하지 않습니다. CoddyKit의 Node.js Backend Development Bootcamp은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 4번째 강의입니다.

“캐시 스탬피드 및 우르르 몰림 방지” 강의는 얼마나 걸리나요?

대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.

이 Node.js Backend Development Bootcamp 강의에서 코드를 작성하고 실행할 수 있나요?

네. 모든 Node.js Backend Development Bootcamp 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.

이 강의의 모든 강의

← Node.js Backend Development Bootcamp(으)로 돌아가기