One address, several machines
A single server grows until the price curve turns vertical, and then stops. Growing sideways instead means running the same application on several ordinary machines and placing something in front of them that hands each arriving request to one of the group. That something is the balancer. It holds the public address, forwards each request to a backend according to a policy you configure, and repeatedly probes each backend with a health check so that a machine which stops answering is removed from rotation before visitors notice. Two capabilities hide under the single name. A transport-level balancer, working at layer 4, forwards connections quickly and without reading them, which suits databases and anything that is not HTTP. An application-level balancer at layer 7 parses the request and can therefore route by hostname or path, terminate TLS on the way through, rewrite headers, retry an idempotent call against a different backend, and hold a request briefly while a backend recovers. This is why the same software that serves a single site as a web server, typically nginx or HAProxy, becomes a load balancer the moment you give it more than one upstream, and why every cloud platform sells the managed equivalent as a product you tick rather than install. The hard part is not any of that. It is the assumption the whole arrangement rests on: any server must be able to answer any request. An application that keeps sessions, uploaded files or caches on the local disk of whichever machine handled the last request violates that assumption immediately, and the symptom is a user who appears logged out every second click. Fix the state first, with a shared session store, signed tokens and object storage for uploads, and the balancer becomes a configuration detail.
| Policy | How it chooses | Reach for it when |
|---|---|---|
| Round robin | Next backend in the list, in turn | Backends are equal and requests cost roughly the same |
| Weighted round robin | In turn, but proportional to a weight you set | The machines differ in size, or you are draining one slowly |
| Least connections | Fewest open connections wins | Request durations vary wildly, as with uploads or long queries |
| Least response time | Fewest connections, tie-broken by measured latency | Backends are nominally identical but demonstrably are not |
| Source IP hash | Hash of the client address pins it to one backend | State is stuck on the backend and you need time to fix it properly |
| URI or consistent hash | Hash of the path sends identical requests to one backend | Backends cache, and you want a cache hit rather than three copies |
| Random with two choices | Picks two at random, keeps the less busy one | Many backends, and you want spread without a central counter |