Load Balancing: How Websites Handle Millions of Users
One server is not enough. Distribute the load.
One server is not enough. Distribute the load.
Click to send requests to a single server. Watch what happens as traffic increases: the queue grows, response times spike, and eventually requests start failing.
Now the same traffic is distributed across 3 servers. Each server handles a fraction of the load. Toggle between algorithms to see how they differ.
What if a server crashes? The load balancer pings each server regularly. If one stops responding, traffic is automatically routed to the healthy ones.
The LB sends a request to each server every few seconds: "Are you alive?" If a server fails to respond, it is removed from the pool.
When a server goes down, its share of traffic is redistributed to the remaining servers. Users might not even notice the failure.
When the failed server comes back, the LB detects it and starts sending traffic again. The pool grows and shrinks automatically.
Every website you use relies on load balancing.
Thousands of servers across the globe. Load balancers route you to the closest, least-busy server. You stream at full speed because the load is distributed.
Online games use load balancers to assign players to game servers. When one server fills up, new players are routed to another.
Retailers spin up extra servers for holiday traffic spikes. Load balancers distribute the flood across all of them. Without this, sites would crash in seconds.
Google handles over 8.5 billion searches per day. That is 99,000 searches per second. No single server could handle even 1% of that. Google uses a massive network of load balancers routing traffic to data centers on every continent.
You've seen why one server cannot handle everything, how load balancers distribute traffic across multiple servers, and why different algorithms suit different situations. Every high-traffic website uses this.
Every server has a maximum number of requests it can handle. Exceed that and requests queue up, slow down, or get rejected. This is why popular sites crash on launch day.
A load balancer sits in front of multiple servers and sends each incoming request to one of them. The user never knows which server they hit.
Send request 1 to Server A, request 2 to Server B, request 3 to Server C, then back to A. Simple and fair, but does not account for server load.
Send each request to whichever server currently has the fewest active connections. Naturally balances uneven workloads.
Put your new knowledge into practice!