优雅关闭与在途请求排空
处理 SIGTERM,排空连接、完成工作,并在部署期间干净地关闭资源。
优雅关闭与在途请求排空 是 CoddyKit 上的免费 Node.js Backend Development Bootcamp 课时。 这是第 3 节课,共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Node.js Backend Development Bootcamp 学习路径的一部分,你的进度在网页和 CoddyKit 应用中同步。 Node.js Backend Development Bootcamp 课程共包含 4 节课。
本课时的部分内容尚未翻译,以英文显示。
Why Graceful Shutdown Matters
During a deploy, your orchestrator (Kubernetes, ECS, systemd) sends your process a signal and gives it a short window before forcibly killing it. If you exit immediately, you drop in-flight requests, abort half-written database transactions, and return broken responses to users.
A graceful shutdown means:
- Stop accepting new connections.
- Let in-flight requests finish (drain).
- Close resources cleanly: DB pools, message brokers, timers.
- Exit with code
0before the kill timer fires.
This lesson builds a production-grade shutdown path step by step.
The Signals: SIGTERM vs SIGINT vs SIGKILL
Process managers communicate shutdown intent via POSIX signals. You must handle the right ones:
SIGTERM— the polite "please stop" sent by Kubernetes, Docker, and systemd during a rollout. This is the one you handle.SIGINT— sent byCtrl+Cin a terminal. Handle it too for local dev parity.SIGKILL(signal 9) — cannot be caught, blocked, or ignored. The OS terminates you instantly. This is what fires if you miss the grace window.
Node lets you register listeners for catchable signals on process.
process.on('SIGTERM', () => {
console.log('Received SIGTERM, starting graceful shutdown');
shutdown();
});
process.on('SIGINT', () => {
console.log('Received SIGINT, starting graceful shutdown');
shutdown();
});
function shutdown() {
// close servers, drain requests, release resources
}server.close() Drains, It Does Not Kill
The core primitive is server.close([callback]) on an http.Server. Its behavior is exactly what draining requires:
- It stops the server from accepting new connections immediately.
- It keeps existing connections alive until their in-flight requests complete.
- The callback fires only once all connections are closed.
The trap: with HTTP keep-alive, idle connections stay open, so close()'s callback can hang. We solve that shortly. First, the happy path with a complete standalone server.
const http = require('http');
const server = http.createServer((req, res) => {
// simulate a slow in-flight request
setTimeout(() => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('done\n');
}, 500);
});
server.listen(3000, () => console.log('listening on 3000'));
process.on('SIGTERM', () => {
console.log('SIGTERM: closing server, draining requests');
server.close(() => {
console.log('all connections drained, exiting');
process.exit(0);
});
});The Hard Deadline: Never Drain Forever
Your orchestrator will SIGKILL you when its terminationGracePeriodSeconds elapses (default 30s in Kubernetes). A misbehaving client holding a connection open could otherwise make server.close() hang past that window, turning a clean shutdown into a hard kill.
The rule: always pair draining with a hard timeout that is shorter than the orchestrator's grace period. If draining wins, you exit 0. If the timeout wins, you exit non-zero so the failure is visible.
Use timer.unref() so the timeout itself never keeps the event loop alive.
function gracefulShutdown(server, { timeoutMs = 25000 } = {}) {
let shuttingDown = false;
return () => {
if (shuttingDown) return;
shuttingDown = true;
console.log('draining...');
const forceExit = setTimeout(() => {
console.error('drain timed out, forcing exit');
process.exit(1);
}, timeoutMs);
forceExit.unref();
server.close((err) => {
clearTimeout(forceExit);
if (err) { console.error(err); process.exit(1); }
console.log('drained cleanly');
process.exit(0);
});
};
}Idempotent Shutdown: Guard Against Double Signals
Signals can arrive more than once: an operator hits Ctrl+C twice, or both SIGTERM and SIGINT fire. If shutdown runs twice you'll call server.close() on an already-closing server and may double-close pools, throwing errors mid-shutdown.
Make the handler idempotent with a boolean guard (shown in the previous scene as shuttingDown). The first signal starts the drain; subsequent signals are ignored. Here is a tiny, fully standalone demonstration of the guard pattern you can run directly.
let shuttingDown = false;
function shutdown(reason) {
if (shuttingDown) {
console.log('already shutting down, ignoring ' + reason);
return;
}
shuttingDown = true;
console.log('shutdown started by ' + reason);
}
// simulate two rapid signals
shutdown('SIGTERM');
shutdown('SIGINT');
shutdown('SIGTERM');
console.log('handler ran exactly once');Stop the Load Balancer From Sending New Traffic
There is a race during rollouts: the orchestrator sends SIGTERM at roughly the same time it removes your pod from the load balancer's endpoint list. For a brief moment the LB may still route new requests to a pod that is already draining.
The common fix is a readiness probe that flips to "not ready" the instant shutdown begins. The LB stops sending new traffic before server.close() rejects connections, avoiding spurious 502s.
Flip a flag, let the probe fail, then begin draining (optionally after a short delay so the LB observes the change).
let isShuttingDown = false;
// readiness endpoint the orchestrator polls
app.get('/readyz', (req, res) => {
if (isShuttingDown) {
return res.status(503).send('shutting down');
}
res.status(200).send('ok');
});
process.on('SIGTERM', () => {
isShuttingDown = true; // probe now fails -> LB stops new traffic
setTimeout(beginDrain, 5000); // give the LB a few probe cycles
});Taming HTTP Keep-Alive Connections
HTTP keep-alive keeps TCP connections open between requests. During shutdown an idle keep-alive connection has no in-flight request, yet it prevents server.close()'s callback from firing because the socket is still open.
Two robust options:
- Send
Connection: closeon responses once shutdown begins, so clients stop reusing the socket. - Set
server.keepAliveTimeoutandheadersTimeoutsensibly, and track sockets to destroy idle ones during drain.
Tracking sockets gives you precise control: finish active requests, but forcibly end idle ones.
const http = require('http');
const connections = new Set();
const server = http.createServer((req, res) => {
if (isShuttingDown) res.setHeader('Connection', 'close');
res.end('ok\n');
});
server.on('connection', (socket) => {
connections.add(socket);
socket.on('close', () => connections.delete(socket));
});
function destroyIdleConnections() {
for (const socket of connections) {
if (socket._isIdle) socket.destroy();
}
}Marking Sockets Idle vs Active
To safely destroy only idle sockets, mark a socket active when a request starts and idle when its response finishes. During drain you destroy the ones still idle; active ones are left alone to complete.
This is precisely the strategy battle-tested libraries like stoppable and http-terminator implement for you. Understanding the mechanism lets you debug it when it misbehaves.
const connections = new Set();
server.on('connection', (socket) => {
socket._isIdle = true;
connections.add(socket);
socket.on('close', () => connections.delete(socket));
});
server.on('request', (req, res) => {
req.socket._isIdle = false; // active: a request is in flight
res.on('finish', () => {
req.socket._isIdle = true; // response sent: back to idle
if (isShuttingDown) req.socket.destroy(); // no reuse during drain
});
});
function closeIdle() {
for (const s of connections) if (s._isIdle) s.destroy();
}Closing Downstream Resources in Order
After the HTTP server has drained, release everything else. Order matters: close the HTTP server first so no new work arrives, then drain queues/consumers, then close the database pool last (in-flight handlers may still need it while finishing).
A typical teardown sequence:
server.close()— stop accepting requests, drain HTTP.- Stop message consumers (Kafka/RabbitMQ/BullMQ) so no new jobs start.
- Wait for in-flight jobs, then close broker connections.
await pool.end()— drain and close the DB connection pool.- Flush logs/metrics, then
process.exit(0).
async function shutdown(server, pool, broker) {
await new Promise((resolve, reject) =>
server.close((err) => (err ? reject(err) : resolve()))
);
console.log('http drained');
await broker.stopConsuming(); // no new jobs
await broker.drainInFlight(); // finish started jobs
await broker.close();
console.log('broker closed');
await pool.end(); // close DB pool last
console.log('db pool closed');
}A Promisified, Timed Shutdown Orchestrator
Put it together as one reusable function. It races the full async teardown against a hard deadline using Promise.race. Whoever finishes first decides the exit code. This pattern is framework-agnostic and works with Express, Fastify, or a raw http.Server.
Key properties: idempotent via the guard, bounded by timeoutMs, and explicit about success (exit 0) versus forced exit (exit 1).
function installGracefulShutdown(teardown, { timeoutMs = 25000 } = {}) {
let started = false;
const run = async (signal) => {
if (started) return;
started = true;
console.log('shutdown via ' + signal);
const deadline = new Promise((_, reject) => {
const t = setTimeout(() => reject(new Error('timeout')), timeoutMs);
t.unref();
});
try {
await Promise.race([teardown(), deadline]);
console.log('clean shutdown');
process.exit(0);
} catch (err) {
console.error('forced shutdown:', err.message);
process.exit(1);
}
};
process.on('SIGTERM', () => run('SIGTERM'));
process.on('SIGINT', () => run('SIGINT'));
}Don't Forget Crashes: uncaughtException
Graceful shutdown handles deliberate termination. Fatal errors are different. An uncaughtException or unhandledRejection leaves the process in an undefined state; the safe response is to log, attempt a best-effort resource flush, and exit non-zero so the orchestrator restarts you.
Do not try to keep serving traffic after an uncaught exception. The official Node guidance is to treat it as a crash. Here is a complete, runnable illustration of the listener firing exactly once before exit.
let crashing = false;
process.on('uncaughtException', (err) => {
if (crashing) return;
crashing = true;
console.error('uncaught:', err.message);
// best-effort: flush logs/metrics here, then exit non-zero
console.log('exiting with code 1');
process.exit(1);
});
// simulate a fatal bug somewhere deep in the app
setImmediate(() => {
throw new Error('boom: null pointer in handler');
});
console.log('server running, awaiting the crash');Quick Check: The Drain Race
You deploy a Node service to Kubernetes with terminationGracePeriodSeconds: 30. On SIGTERM you immediately call server.close(() => process.exit(0)) and nothing else. In production you still observe occasional 502 errors for clients during rollouts. What is the most likely cause and the correct fix?
Recap: The Graceful Shutdown Checklist
You now have a complete, production-ready shutdown path. The essentials:
- Handle SIGTERM and SIGINT, and make the handler idempotent with a guard.
- Fail the readiness probe first so the load balancer stops sending new traffic before you drain.
- server.close() stops new connections and drains in-flight requests; tame keep-alive by sending
Connection: closeand destroying idle sockets. - Always set a hard timeout shorter than the orchestrator's grace period, with
unref(), exiting non-zero if draining hangs. - Close resources in order: HTTP server, then consumers/broker, then the DB pool, then flush telemetry.
- Treat uncaughtException/unhandledRejection as crashes: best-effort cleanup, then exit non-zero for a restart.
Get these right and zero-downtime deploys stop dropping user requests.
常见问题解答
「优雅关闭与在途请求排空」课时是免费的吗?
是的 — 「优雅关闭与在途请求排空」的完整文本可在网页上免费阅读。要进行交互式练习(内置代码编辑器和全天候 AI 导师)并解锁 Node.js Backend Development Bootcamp 课程的其余内容,请升级到 CoddyKit PRO。 Node.js Backend Development Bootcamp 课程共包含 4 节课。
「优雅关闭与在途请求排空」这节课中我会学到什么?
处理 SIGTERM,排空连接、完成工作,并在部署期间干净地关闭资源。 你通过在浏览器中直接运行的动手代码来练习 Node.js Backend Development Bootcamp,全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Node.js Backend Development Bootcamp 需要有经验吗?
无需任何先前经验。CoddyKit 上的 Node.js Backend Development Bootcamp 课程适合初学者到高级学习者,你可以从这里开始或从头开始,按照自己的节奏学习。 这是第 3 节课,共 4 节。
「优雅关闭与在途请求排空」课时需要多长时间?
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动,所以你能稳步进步,并在网页和应用中从离开的地方继续。
我能在这节 Node.js Backend Development Bootcamp 课中编写并运行代码吗?
能。每节 Node.js Backend Development Bootcamp 课都包含内置代码编辑器,你可以在浏览器中直接编写并运行真实代码,并获得即时 AI 反馈 — 无需本地设置。
此课程中的所有课时
- 超时、重试与带抖动的指数退避
- 熔断器模式与舱壁隔离
- 优雅关闭与在途请求排空
- 健康检查、就绪探针与负载丢弃