How to Configure Nginx as a Load Balancer

The short answer

Load balancing in Nginx has two parts: an upstream block that lists your servers and a proxy_pass directive that sends requests to it. A minimal working setup:

upstream app_backend {
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://app_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

By default requests are distributed in turn (round-robin). From here, tune the setup to your traffic.

How to choose a balancing method

The method goes on the first line inside upstream.

For most web applications, least_conn is a sensible starting point.

Passive health checks

Open source Nginx checks servers passively, based on real requests. The parameters are set per server:

upstream app_backend {
    least_conn;
    server 10.0.0.11:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:3000 max_fails=3 fail_timeout=30s;
}

• max_fails — how many failed attempts within fail_timeout mark a server as unavailable.
• fail_timeout — both the window for counting failures and how long the server stays excluded.

What counts as a failure is controlled by proxy_next_upstream. By default that is a connection error or timeout. You can add response codes:

proxy_next_upstream error timeout http_502 http_503;
proxy_next_upstream_tries 2;

Be careful with non-idempotent requests: retrying a POST on another server can create a duplicate order. Nginx does not retry them by default, so enable non_idempotent only deliberately. Active checks (health_check) exist only in the commercial NGINX Plus; with open source Nginx, use external monitoring instead.

Weights and backup servers

If your servers differ in capacity, use weight:

upstream app_backend {
    server 10.0.0.11:3000 weight=3;
    server 10.0.0.12:3000 weight=1;
    server 10.0.0.13:3000 backup;
    server 10.0.0.14:3000 down;
}

• weight=3 — the server gets roughly three times as many requests.
• backup — used only when all primary servers are unavailable. It cannot be combined with hash, ip_hash or random.
• down — the server is temporarily out of rotation, handy for maintenance.

Sticky sessions

Pinning a user to a server matters when sessions live in application memory. Open source Nginx offers two options:

# by client IP
ip_hash;

# by session cookie
hash $cookie_sessionid consistent;

ip_hash works poorly when many users share one IP (an office, a mobile carrier). Cookie-based affinity is more precise. The best option, though, is to drop sticky sessions entirely and store sessions in Redis or a database: any server can then handle any request, and scaling and deployments get simpler.

Connections to the backend

To avoid opening a new TCP connection for every request, enable keepalive:

upstream app_backend {
    least_conn;
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
    keepalive 32;
}

location / {
    proxy_pass http://app_backend;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Common mistakes

• Not passing Host and X-Forwarded-For — the app sees the wrong domain and the balancer's IP.
• max_fails=0 — checks are off, and requests keep hitting a dead server.
• In-memory sessions without affinity — users get logged out at random.
• Changing the config without testing it: always run nginx -t, then nginx -s reload.