Scale Horizontally with Sessions

A single server process can only do so much. To serve far more traffic, you run many instances of it side by side. 📈

Adding more copies of your server is horizontal scaling. It is often cheaper and safer than buying one ever-bigger machine.

A load balancer sits in front and spreads incoming requests across your instances, so no single copy gets overwhelmed.

Horizontal scaling needs a network transport, so MCP servers run over Streamable HTTP rather than the local-only stdio transport.

MCP keeps per-client state in a session, identified by a session id the client sends back on every follow-up request.

If a session lives only in one instance's memory, a request routed elsewhere loses its state. That mismatch is the stickiness problem.

One fix is sticky sessions: the load balancer always sends a given session id back to the same instance that created it.

Better for scale is a shared store like Redis. Any instance can load a session, so requests route freely and survive restarts.

Keep instances stateless, holding nothing critical only in local memory. Then you can add, remove, or replace them at any time.

The balancer pings each instance's health endpoint and stops sending traffic to any copy that fails, keeping users on healthy ones.

With stateless instances, you can autoscale: add copies when load rises and remove them when it falls, paying only for what you use.

What lets you route any request to any instance?

You ran many instances behind a load balancer, shared sessions in a central store, and kept copies stateless to autoscale on demand. 🚀