Load balancing is the practice of putting one entry point in front of several interchangeable servers and distributing incoming requests among them by a chosen policy, so that no single machine carries the traffic and no single machine's failure takes the service down.
Our take. Buy availability before you buy capacity: two modest servers behind a balancer beat one large machine long before raw traffic demands it, because the second machine is also how you reboot for updates without an outage. And unless the platform is doing the work anyway, we would rather operate nginx or HAProxy whose health-check behaviour we can read in a config file than a managed balancer whose timeout semantics are a support ticket away.
One address, several machines
A single server grows until the price curve turns vertical, and then stops. Growing sideways instead means running the same application on several ordinary machines and placing something in front of them that hands each arriving request to one of the group. That something is the balancer. It holds the public address, forwards each request to a backend according to a policy you configure, and repeatedly probes each backend with a health check so that a machine which stops answering is removed from rotation before visitors notice. Two capabilities hide under the single name. A transport-level balancer, working at layer 4, forwards connections quickly and without reading them, which suits databases and anything that is not HTTP. An application-level balancer at layer 7 parses the request and can therefore route by hostname or path, terminate TLS on the way through, rewrite headers, retry an idempotent call against a different backend, and hold a request briefly while a backend recovers. This is why the same software that serves a single site as a web server, typically nginx or HAProxy, becomes a load balancer the moment you give it more than one upstream, and why every cloud platform sells the managed equivalent as a product you tick rather than install. The hard part is not any of that. It is the assumption the whole arrangement rests on: any server must be able to answer any request. An application that keeps sessions, uploaded files or caches on the local disk of whichever machine handled the last request violates that assumption immediately, and the symptom is a user who appears logged out every second click. Fix the state first, with a shared session store, signed tokens and object storage for uploads, and the balancer becomes a configuration detail.
Policy
How it chooses
Reach for it when
Round robin
Next backend in the list, in turn
Backends are equal and requests cost roughly the same
Weighted round robin
In turn, but proportional to a weight you set
The machines differ in size, or you are draining one slowly
Least connections
Fewest open connections wins
Request durations vary wildly, as with uploads or long queries
Least response time
Fewest connections, tie-broken by measured latency
Backends are nominally identical but demonstrably are not
Source IP hash
Hash of the client address pins it to one backend
State is stuck on the backend and you need time to fix it properly
URI or consistent hash
Hash of the path sends identical requests to one backend
Backends cache, and you want a cache hit rather than three copies
Random with two choices
Picks two at random, keeps the less busy one
Many backends, and you want spread without a central counter
In practice
A worked before and after, using the commonest case on the web. Before: one machine running WordPress, with PHP sessions written to the local filesystem and media uploaded into wp-content. You add a second machine and a balancer set to round robin, expecting twice the headroom. What you get is a site that logs editors out on alternate clicks and serves images that exist on one machine and return a not-found error from the other, because each request now lands somewhere the previous request's side effects never reached. After: sessions move to a shared Redis, media moves to object storage or a shared volume, deployments push identical code to both machines, and the health check asks for a page that actually touches the database rather than a static file that will cheerfully return success on a server whose database has died. The same two machines now survive losing either one, which halves capacity rather than ending the service, and lets you patch and reboot them one at a time in the middle of the afternoon.
A CDN spreads copies of your content across locations to shorten the distance; a balancer spreads requests across servers that all hold the same thing.
Same software, different job. One upstream and it is a reverse proxy in front of an application; several upstreams with health checks and it is balancing.
An SLA is a promise about availability with money attached; balancing is one of the mechanisms you build to keep that promise.
Key takeaways
→A balancer owns the public address and hands each request to one of several equivalent backends, with health checks turning a dead machine into a routing decision.
→Layer 4 forwards connections blindly and fast; layer 7 reads the request and can route, terminate TLS and retry.
→The distribution policy is a small decision, while removing local state from the application is the real project.
A rotating selection from the course directory, drawn from the subcategories where this concept is taught rather than picked for it. Details, price and the provider link are on the course page.
Welcome to Kubernetes on AWS using Amazon EKS course.This course is completely focused on Amazon EKS - which is a mange…
Udemy
FAQ
When does a site genuinely need one?
When either capacity or availability runs out, and availability almost always runs out first. If a single machine's reboot is a visible outage, or if losing it for an hour would cost real money, a second backend pays for itself before traffic ever justifies it.
Is a load balancer just a reverse proxy?
Balancing is simply what a reverse proxy starts doing once it has several backends and a rule for picking between them. The words blur because one nginx or HAProxy install does both jobs, and which name you use depends entirely on how many upstreams are configured.
Doesn't it become a single point of failure itself?
It would, which is why the balancer is either run in a pair sharing a floating address, or bought as a managed service where the provider keeps redundant nodes behind one address. A lone balancer in front of three backends has moved the risk rather than removed it.
What should the health check actually test?
Something that fails when the service is genuinely unusable. A check for a static file passes on a machine whose database connection is gone, so point it at a small endpoint that touches the dependencies, keep it cheap, and set the failure threshold high enough that one slow response does not eject a healthy machine.
Sources
The primary text this definition rests on. Read it before you trust ours.
