Upstream Servers & Load Balancing

As your web application grows, a single server might not be enough to handle all incoming requests. This can lead to slow response times or even crashes.

To prevent this, we use multiple backend servers to share the load. This improves both performance and reliability.

Nginx can act as a reverse proxy, distributing requests to a group of backend servers. These backend servers are often called upstream servers.

• Scalability: Handle more traffic by adding more servers.
• High Availability: If one server fails, others can still respond.
• Load Balancing: Distribute requests efficiently among servers.

In Nginx, you define a group of backend servers using the upstream block. This block gives a name to your group of servers, which you can then reference in your location blocks.

Think of it as a named pool of resources that Nginx can forward requests to.

Here's how to define a simple upstream block named my_backends with two servers. Each server directive specifies the address (IP or hostname) and port of a backend server.

Once an upstream block is defined, you can use it within a location block in your Nginx server configuration. The proxy_pass directive tells Nginx to forward requests to the specified upstream group.

Notice we use http://my_backends to reference our defined group.

By default, Nginx uses the Round Robin load balancing method. This means requests are distributed to upstream servers in a rotating, sequential manner.

• Request 1 goes to Server A.
• Request 2 goes to Server B.
• Request 3 goes to Server A.
• ...and so on.

It's simple and effective for evenly distributed loads.

What if some servers are more powerful or should handle more traffic? You can use the weight parameter in the server directive to assign different priorities.

A server with a higher weight will receive a proportionally larger share of requests.

In this example, server 1 has a weight of 3, and server 2 has a weight of 1. This means server 1 will receive 3 times as many requests as server 2.

Out of every 4 requests, 3 will go to server 1 and 1 to server 2.

Beyond weight, other directives can fine-tune server behavior:

• max_fails=N: Number of failed attempts before Nginx considers the server unavailable.
• fail_timeout=Xs: Time for which the server is considered unavailable after max_fails attempts.
• backup: Marks a server as a backup, used only when all primary servers are down.
• down: Marks a server as permanently unavailable.

These help Nginx manage server health automatically.

Consider the following Nginx configuration snippet:

upstream api_servers {
  server backend1.example.com:8080 weight=2;
  server backend2.example.com:8080;
}

server {
  listen 80;
  location /api {
    proxy_pass http://api_servers;
  }
}

If 9 requests arrive at the /api endpoint, how many requests will be sent to backend1.example.com:8080?

You've learned how Nginx uses upstream blocks to group multiple backend servers, enabling load balancing and improving application resilience.

• upstream block: Defines a named group of servers.
• proxy_pass: Connects a location to an upstream group.
• Round Robin: Nginx's default load balancing method.
• weight directive: Customizes request distribution among servers.

Next, we'll explore how Nginx can improve performance with proxy buffering and caching!