Load Balancing

A load balancer distributes requests across healthy backends. The two axes that matter are which layer it operates at and how it chooses a backend.

Layer 4 balances on IP and port. It is fast, protocol-agnostic, and cheap, but it cannot see the request: no routing by path, no header inspection, no per-request retries. Layer 7 parses HTTP, so it can route /api to one pool and /static to another, terminate TLS, retry idempotent requests, and inject headers. You pay in CPU and latency.

Choosing a backend. Round robin is fine when requests are uniform. Least connections handles uneven request durations far better, because a backend stuck on slow requests stops receiving new ones. Consistent hashing routes the same key to the same backend, which is what you want in front of a cache tier, and it only remaps 1/n of keys when a node joins or leaves instead of reshuffling everything.

Health checks are what make it a load balancer rather than a traffic splitter. Passive checks observe real traffic for errors; active checks probe an endpoint on a timer. The subtle failure is a health check that only proves the process is up. It should exercise the dependencies the request path actually needs, or you will happily route traffic to a server whose database connection pool is exhausted.

Sticky sessions pin a user to one backend. They make scaling and deploys harder, and they are usually a sign that session state belongs somewhere shared instead.