0Pricing
Node.js Backend Development Bootcamp · Lesson

Health Checks, Readiness Probes, and Load Shedding

Signal liveness and readiness correctly and shed excess load to protect overwhelmed services.

Health Checks, Readiness Probes, and Load Shedding is a free Node.js Backend Development Bootcamp lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Node.js Backend Development Bootcamp learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Liveness vs Readiness

Production orchestrators (Kubernetes, ECS, Nomad) ask your service two different questions, and conflating them causes outages.

  • Liveness: "Is this process broken beyond repair?" If it fails, the orchestrator restarts the container.
  • Readiness: "Can this process serve traffic right now?" If it fails, the orchestrator removes the pod from the load balancer but does NOT kill it.

The classic mistake: putting a database ping in the liveness probe. When the DB has a transient hiccup, every replica fails liveness simultaneously and Kubernetes restarts your entire fleet at once, turning a 5-second blip into a full outage. Dependency checks belong in readiness, not liveness.

A Minimal Liveness Endpoint

A liveness check should be cheap, local, and dependency-free. It answers one thing: is the event loop alive and the process able to respond? Anything that reaches over the network does not belong here.

Below is a tiny dependency-free HTTP server (no framework) exposing /livez. It always returns 200 unless the process is so broken it cannot respond at all — which is exactly the signal a restart should be based on.

const http = require('http');

const server = http.createServer((req, res) => {
  if (req.url === '/livez') {
    // Liveness: no I/O, no DB, just "am I able to answer?"
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'alive' }));
    return;
  }
  res.writeHead(404);
  res.end();
});

server.listen(3000, () => {
  console.log('liveness server on :3000');
  // Self-test so an online judge sees output and exits.
  http.get('http://localhost:3000/livez', (r) => {
    let body = '';
    r.on('data', (c) => (body += c));
    r.on('end', () => {
      console.log('GET /livez ->', r.statusCode, body);
      server.close();
    });
  });
});

Readiness With Dependency Checks

Readiness aggregates the health of the things you need to serve a request: the database pool, a cache, a critical downstream API. If any required dependency is down, return 503 so the load balancer stops routing to you.

Key rules:

  • Run dependency checks in parallel with Promise.allSettled so one slow check does not serialize the rest.
  • Apply a per-check timeout — a hanging DB probe must not hang the readiness endpoint.
  • Distinguish required vs optional dependencies; a degraded optional cache should not pull you out of rotation.
function withTimeout(promise, ms, name) {
  let t;
  const timeout = new Promise((_, reject) => {
    t = setTimeout(() => reject(new Error(name + ' timed out')), ms);
  });
  return Promise.race([promise, timeout]).finally(() => clearTimeout(t));
}

async function checkReadiness(deps) {
  const results = await Promise.allSettled(
    deps.map((d) => withTimeout(d.probe(), 500, d.name).then(() => d))
  );

  const failures = results
    .map((r, i) => ({ r, dep: deps[i] }))
    .filter((x) => x.r.status === 'rejected' && x.dep.required);

  return { ready: failures.length === 0, failures: failures.map((f) => f.dep.name) };
}

// Demo with fake probes
const deps = [
  { name: 'postgres', required: true, probe: () => Promise.resolve() },
  { name: 'redis', required: false, probe: () => Promise.reject(new Error('down')) },
];

checkReadiness(deps).then((r) => console.log(JSON.stringify(r)));

Cache Readiness Results to Protect Dependencies

Probes are called frequently — every replica, every few seconds, from kubelet and sometimes from external monitors too. If each readiness call pings Postgres, your health checks alone can hammer a struggling database and make the incident worse.

Cache the dependency-check result for a short TTL (e.g. 1-2 seconds). The probe reads the cached verdict; a background refresh updates it. This decouples probe frequency from real dependency load.

function cachedCheck(checkFn, ttlMs) {
  let cache = { value: null, at: 0 };
  let inflight = null;

  return async function get() {
    const now = Date.now();
    if (cache.value && now - cache.at < ttlMs) return cache.value;
    if (inflight) return inflight; // coalesce concurrent calls
    inflight = Promise.resolve(checkFn())
      .then((value) => {
        cache = { value, at: Date.now() };
        return value;
      })
      .finally(() => { inflight = null; });
    return inflight;
  };
}

let calls = 0;
const readiness = cachedCheck(async () => { calls++; return { ready: true }; }, 1000);

Promise.all([readiness(), readiness(), readiness()]).then(async (rs) => {
  console.log('results', rs.length, 'actual checks', calls); // 3 results, 1 check
});

Startup Probes Avoid Premature Restarts

Slow-booting apps (warming caches, running migrations, JIT compiling) need a startup probe. Kubernetes runs only the startup probe until it succeeds once; liveness and readiness are suspended during this window.

Without it, a liveness check with a short period can kill a process that simply hasn't finished booting — a crash loop that never lets the app come up. Model a started flag separately from ready:

  • /startupz → 200 once boot completes (give it a generous failure threshold).
  • /livez → only meaningful after startup succeeds.
  • /readyz → gates traffic after the app is both started and dependencies are healthy.
const state = { started: false, shuttingDown: false };

async function boot() {
  // Simulate migrations / cache warm-up
  await new Promise((r) => setTimeout(r, 50));
  state.started = true;
}

function startupz() {
  return state.started ? 200 : 503;
}

boot().then(() => {
  console.log('before boot done? startupz would have been 503');
  console.log('startupz now ->', startupz()); // 200
});

Why You Need Load Shedding

Even a healthy service has a finite capacity. When arrival rate exceeds what you can process, requests pile up in queues. Latency climbs, timeouts fire, clients retry, and the added retry load pushes you further over the edge — a congestion collapse. Throughput can drop to near zero while CPU stays pinned.

Load shedding is the deliberate choice to reject some requests fast (with 503 + Retry-After) so the rest succeed. Rejecting cheaply is far better than accepting work you cannot finish. The goal: keep goodput (successfully completed requests) high under overload, instead of letting it collapse.

Shedding on Event-Loop Lag

In Node, the single-threaded event loop is the bottleneck. When you're overloaded, the loop falls behind and event-loop delay rises. That delay is an excellent, cheap, app-agnostic overload signal — better than CPU%, because it directly measures "am I keeping up?"

Use perf_hooks.monitorEventLoopDelay to sample the lag, and reject new requests when the smoothed delay crosses a threshold. The library @cabhishek/toobusy or toobusy-js wraps this pattern, but you can do it in a few lines.

const { monitorEventLoopDelay } = require('perf_hooks');

const h = monitorEventLoopDelay({ resolution: 20 });
h.enable();

const MAX_LAG_MS = 70;
function tooBusy() {
  // mean is in nanoseconds
  const lagMs = h.mean / 1e6;
  return lagMs > MAX_LAG_MS;
}

// Simulate a blocking burst, then measure.
function blockFor(ms) {
  const end = Date.now() + ms;
  while (Date.now() < end) {} // intentional CPU block
}

blockFor(120);
setTimeout(() => {
  console.log('mean lag (ms):', (h.mean / 1e6).toFixed(1));
  console.log('tooBusy ->', tooBusy());
  h.disable();
}, 100);

Bounded Concurrency: A Semaphore

Event-loop lag catches CPU overload, but many backends are bottlenecked on a downstream (DB pool, an upstream API). There, the right primitive is a concurrency limiter: cap the number of in-flight requests, and shed anything beyond the cap immediately.

This bounds queue depth, which by Little's Law bounds latency. A request that can't get a slot is rejected fast instead of waiting behind a long line. Prefer rejecting over unbounded queuing under overload.

class Semaphore {
  constructor(max) { this.max = max; this.inUse = 0; }
  tryAcquire() {
    if (this.inUse >= this.max) return false;
    this.inUse++;
    return true;
  }
  release() { if (this.inUse > 0) this.inUse--; }
}

async function handle(sem, id) {
  if (!sem.tryAcquire()) {
    return { id, status: 503 }; // shed: no capacity
  }
  try {
    await new Promise((r) => setTimeout(r, 20));
    return { id, status: 200 };
  } finally {
    sem.release();
  }
}

const sem = new Semaphore(2);
Promise.all([1, 2, 3, 4].map((i) => handle(sem, i))).then((rs) =>
  console.log(rs.map((r) => r.id + ':' + r.status).join(' '))
);

Prioritized Shedding: Protect Critical Traffic

Not all requests are equal. Under load you want to shed low-priority work (analytics writes, prefetch, background sync) before critical work (checkout, login). Implement tiers with separate thresholds so health checks and critical paths survive longest.

A common pattern: reserve capacity. Allow critical requests up to the hard cap, but start shedding low-priority requests at a lower watermark. This keeps headroom for what matters.

function admit({ inFlight, priority }) {
  const HARD_CAP = 100;     // absolute ceiling
  const LOW_PRIO_CAP = 70;  // shed low-prio earlier

  if (priority === 'critical') return inFlight < HARD_CAP;
  return inFlight < LOW_PRIO_CAP;
}

const cases = [
  { inFlight: 65, priority: 'low' },
  { inFlight: 80, priority: 'low' },
  { inFlight: 80, priority: 'critical' },
  { inFlight: 100, priority: 'critical' },
];

for (const c of cases) {
  console.log(`inFlight=${c.inFlight} ${c.priority} -> ${admit(c) ? 'ADMIT' : 'SHED 503'}`);
}

Graceful Shutdown Ties It Together

When a pod is told to stop (SIGTERM on deploy/scale-down), readiness and shutdown must cooperate so no request is dropped:

  • Flip readiness to 503 first. The load balancer notices and stops sending new traffic — but this takes a few seconds to propagate.
  • Keep serving in-flight requests during that window; do NOT close the server immediately.
  • Stop accepting new connections, drain, then close the DB pool.
  • Add a hard timeout so a stuck request can't block shutdown forever; force-exit after it.

Liveness must keep returning 200 during drain, or Kubernetes will SIGKILL you mid-drain.

A Complete Graceful Shutdown Handler

This standalone example shows the full sequence: a SIGTERM-style trigger flips ready to false, waits for the load balancer to notice, stops new work, drains in-flight requests, and force-exits after a hard deadline. The same ready flag is what your /readyz endpoint should read.

const state = { ready: true, inFlight: 0 };

function readyz() { return state.ready ? 200 : 503; }

async function gracefulShutdown({ drainWaitMs, hardDeadlineMs }) {
  console.log('readyz before:', readyz());
  state.ready = false; // 1) fail readiness, LB stops new traffic
  console.log('readyz after :', readyz());

  await new Promise((r) => setTimeout(r, drainWaitMs)); // 2) let LB propagate

  const start = Date.now();
  while (state.inFlight > 0) { // 3) drain in-flight work
    if (Date.now() - start > hardDeadlineMs) {
      console.log('hard deadline hit, forcing exit with', state.inFlight, 'in flight');
      return 'forced';
    }
    await new Promise((r) => setTimeout(r, 10));
  }
  console.log('drained cleanly');
  return 'clean';
}

state.inFlight = 1;
setTimeout(() => { state.inFlight = 0; }, 30); // a request finishes
gracefulShutdown({ drainWaitMs: 20, hardDeadlineMs: 500 }).then((r) =>
  console.log('shutdown result:', r)
);

Quick Check

Test your understanding of where dependency checks belong.

Recap

You learned to signal health and shed load correctly:

  • Liveness = restart-if-broken; keep it cheap and dependency-free.
  • Readiness = gate traffic; aggregate required dependencies in parallel, with per-check timeouts, and cache results so probes don't hammer dependencies.
  • Startup probes protect slow-booting apps from premature liveness restarts.
  • Load shedding preserves goodput under overload by rejecting fast with 503 + Retry-After.
  • Shed on event-loop lag for CPU-bound overload and on bounded concurrency for downstream-bound overload; bounding queue depth bounds latency.
  • Prioritize: shed low-priority work earlier, reserve capacity for critical paths.
  • Graceful shutdown: flip readiness to 503 first, drain in-flight requests, keep liveness 200, and force-exit after a hard deadline.

Frequently asked questions

Is the “Health Checks, Readiness Probes, and Load Shedding” lesson free?

Yes — the full text of “Health Checks, Readiness Probes, and Load Shedding” is free to read here on the web, and the Node.js Backend Development Bootcamp course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Node.js Backend Development Bootcamp course, upgrade to CoddyKit PRO.

What will I learn in “Health Checks, Readiness Probes, and Load Shedding”?

Signal liveness and readiness correctly and shed excess load to protect overwhelmed services. You practise Node.js Backend Development Bootcamp with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Node.js Backend Development Bootcamp?

No prior experience is required. Node.js Backend Development Bootcamp on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Health Checks, Readiness Probes, and Load Shedding” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Node.js Backend Development Bootcamp lesson?

Yes. Every Node.js Backend Development Bootcamp lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Timeouts, Retries, and Exponential Backoff with Jitter
  2. The Circuit Breaker Pattern and Bulkhead Isolation
  3. Graceful Shutdown and In-Flight Request Draining
  4. Health Checks, Readiness Probes, and Load Shedding
← Back to Node.js Backend Development Bootcamp