The traffic director
One server can only grow so far — bigger CPU, more memory, until the price curve turns vertical. The alternative is horizontal: several ordinary servers running the same application behind one entry point that spreads requests among them. That entry point is the load balancer. It answers on the public address, forwards each request to a backend by some policy (round-robin, least connections, client affinity), and quietly removes any backend whose health checks fail — visitors never learn a machine died.
Two levels of sophistication exist under one name. Transport-level (L4) balancers forward network connections fast and blindly; application-level (L7) balancers read the HTTP request itself and can route by path or host, terminate TLS, and retry failed calls — which is why the same software that fronts single sites as a web server, like nginx or HAProxy, doubles as an L7 balancer. Every cloud platform sells the managed version as a checkbox product.
The catch is architectural, not operational: balancing assumes any server can serve any request. Applications that keep session state in local memory break that assumption — user logged in on server A, next request lands on B. The fixes (shared session stores, stateless tokens) are why 'stateless application design' and load balancing are effectively one topic.
