Health Checks, Readiness Probes, and Load Shedding
Signal liveness and readiness correctly and shed excess load to protect overwhelmed services.
Liveness vs Readiness
Production orchestrators (Kubernetes, ECS, Nomad) ask your service two different questions, and conflating them causes outages.
- Liveness: "Is this process broken beyond repair?" If it fails, the orchestrator restarts the container.
- Readiness: "Can this process serve traffic right now?" If it fails, the orchestrator removes the pod from the load balancer but does NOT kill it.
The classic mistake: putting a database ping in the liveness probe. When the DB has a transient hiccup, every replica fails liveness simultaneously and Kubernetes restarts your entire fleet at once, turning a 5-second blip into a full outage. Dependency checks belong in readiness, not liveness.
A Minimal Liveness Endpoint
A liveness check should be cheap, local, and dependency-free. It answers one thing: is the event loop alive and the process able to respond? Anything that reaches over the network does not belong here.
Below is a tiny dependency-free HTTP server (no framework) exposing /livez. It always returns 200 unless the process is so broken it cannot respond at all — which is exactly the signal a restart should be based on.
const http = require('http');
const server = http.createServer((req, res) => {
if (req.url === '/livez') {
// Liveness: no I/O, no DB, just "am I able to answer?"
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ status: 'alive' }));
return;
}
res.writeHead(404);
res.end();
});
server.listen(3000, () => {
console.log('liveness server on :3000');
// Self-test so an online judge sees output and exits.
http.get('http://localhost:3000/livez', (r) => {
let body = '';
r.on('data', (c) => (body += c));
r.on('end', () => {
console.log('GET /livez ->', r.statusCode, body);
server.close();
});
});
});All lessons in this course
- Timeouts, Retries, and Exponential Backoff with Jitter
- The Circuit Breaker Pattern and Bulkhead Isolation
- Graceful Shutdown and In-Flight Request Draining
- Health Checks, Readiness Probes, and Load Shedding