Load Balancing and Service Discovery
Understand how SaaS backends distribute traffic across many instances using load balancers, health checks, and service discovery.
Load Balancing and Service Discovery is a free SaaS Architecture & Startup Engineering lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the SaaS Architecture & Startup Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Need for Load Balancing
When one server cannot handle all traffic, you run many copies. A load balancer sits in front and spreads incoming requests across these instances.
This is the backbone of horizontal scaling for SaaS backends.
How a Load Balancer Works
Clients connect to a single address. The load balancer accepts the request and forwards it to one of the backend servers, then relays the response back.
To the client, the cluster looks like one powerful server.
Round Robin
The simplest algorithm is round robin: requests are handed to servers in rotation. Each server gets an equal share.
const servers = ['s1', 's2', 's3'];
let i = 0;
function next() {
const s = servers[i % servers.length];
i++;
return s;
}
console.log(next(), next(), next(), next());Least Connections
Least connections routes each new request to the server currently handling the fewest active connections.
This adapts better than round robin when requests have uneven durations.
Layer 4 vs Layer 7
Load balancers operate at different network layers:
- Layer 4 (transport) — routes by IP and port, very fast
- Layer 7 (application) — inspects HTTP, can route by URL path or headers
Layer 7 enables smart routing like sending /api to one pool and /static to another.
Health Checks
A load balancer must avoid sending traffic to dead servers. It runs periodic health checks against each instance.
A common pattern is a /health endpoint returning 200 OK when the app is ready.
app.get('/health', (req, res) => {
res.status(200).json({ status: 'ok' });
});Sticky Sessions
Some apps store session state in server memory. Sticky sessions pin a client to the same server so their session persists.
Better practice: keep servers stateless and store sessions in a shared store, so any server can handle any request.
Service Discovery
In dynamic environments, servers come and go constantly. Service discovery keeps an up-to-date registry of which instances exist and are healthy.
Tools like Consul, etcd, or Kubernetes services automate this.
Client-Side vs Server-Side Discovery
Two models:
- Server-side — clients hit a load balancer that consults the registry
- Client-side — clients query the registry directly and pick an instance themselves
Server-side is simpler; client-side reduces a network hop.
Autoscaling Integration
Load balancers pair with autoscaling: when traffic rises, new instances spin up, register with discovery, and the balancer starts routing to them automatically.
When traffic drops, instances are removed gracefully after draining connections.
Graceful Draining
Before shutting down an instance, the balancer should stop sending new requests but let existing ones finish. This is connection draining.
It prevents dropped requests during deployments and scale-down events.
Quick Check
Test your load balancing knowledge.
Recap
You learned how SaaS backends distribute traffic:
- Load balancers with round robin and least connections
- Layer 4 vs Layer 7 routing and health checks
- Stateless design, service discovery, and graceful draining
Together these let a backend scale horizontally and survive failures.
Frequently asked questions
Is the “Load Balancing and Service Discovery” lesson free?
Yes — the full text of “Load Balancing and Service Discovery” is free to read here on the web, and the SaaS Architecture & Startup Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the SaaS Architecture & Startup Engineering course, upgrade to CoddyKit PRO.
What will I learn in “Load Balancing and Service Discovery”?
Understand how SaaS backends distribute traffic across many instances using load balancers, health checks, and service discovery. You practise SaaS Architecture & Startup Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start SaaS Architecture & Startup Engineering?
No prior experience is required. SaaS Architecture & Startup Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Load Balancing and Service Discovery” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this SaaS Architecture & Startup Engineering lesson?
Yes. Every SaaS Architecture & Startup Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Horizontal Scaling Techniques
- Message Queues & Event-Driven
- Serverless Architecture Basics
- Load Balancing and Service Discovery