What it is
A load balancer spreads incoming requests across a pool of servers so no single one is overwhelmed, and routes around dead servers using health checks. It's what makes horizontal scaling work: without it, "add more servers" is a wish, not a design.
In an interview the load balancer itself is cheap to draw and expensive to defend. The scoring questions live one level down: L4 or L7 and why, which algorithm and when it matters, how the LB detects a dead server, and what happens when the LB itself fails. WebSocket-heavy designs (chat, live dashboards) are where the L4 vs L7 choice stops being trivia and starts being the answer.
I'll put an L7 load balancer in front of the stateless app tier — least-connections, health checks every ten seconds — and for the WebSocket path I'll switch to L4 because those connections are long-lived and the balancer shouldn't be parsing them.
The picture
The variants and when to pick each
L4 vs L7 — the decision rule is connection lifetime
L4 balances at the transport layer: it routes on IP and port without reading the payload. That makes it very fast — millions of concurrent connections per device — and the right choice for persistent connections like WebSockets, where per-request inspection buys nothing. AWS NLB and HAProxy in TCP mode live here.
L7 understands HTTP: it can route by path or header, terminate TLS, compress responses, and do sticky sessions by cookie. That flexibility costs CPU per request. nginx, Envoy and AWS ALB are the standard examples, and L7 is the sensible default for ordinary request/response traffic.
Algorithms: round robin until requests stop being equal
Round robin is the default and is correct when servers and requests are roughly uniform. Weighted round robin handles heterogeneous machines. Least connections wins when request cost varies — long-lived or slow requests pile up on one server under round robin, and least-connections routes new work away from it. IP hash gives stickiness and cache locality at the cost of uneven spread.
The interview move is not reciting the list — it's naming the workload property that changes the answer: uniform short requests, round robin; variable-duration requests or streaming, least connections.
Health checks — how the LB knows a server died
Active checks probe each backend on an interval — typically every 5–30 seconds against a /health endpoint — and eject servers after a few consecutive failures. Passive detection watches real traffic and marks a backend down on connection errors or timeouts, which reacts faster but only after users ate some errors.
The detail worth volunteering: a health endpoint should verify the server can do real work (DB reachable, dependencies up), not merely that the process answers — otherwise the LB happily routes to a zombie.
The LB is itself a single point of failure
One load balancer in front of ten servers has moved the SPOF, not removed it. The standard fix is an active-passive pair sharing a virtual IP via VRRP/keepalived — the passive node takes over the IP within seconds of the active one dying. At larger scale: DNS round-robin across multiple LB pairs, and GeoDNS or anycast to steer users to the nearest region.
Also worth one sentence: an API gateway is essentially an L7 balancer plus auth, rate limiting and routing — interviewers accept it as a single box, and it rarely deserves deeper discussion unless the question is about the gateway itself.
Numbers worth memorising
| nginx/HAProxy simple proxying | ~100k+ RPS per box |
| L4 device concurrent connections | millions per appliance |
| Health check interval | 5–30 s typical, eject after 2–3 failures |
| LB failover (VRRP pair) | seconds to shift the virtual IP |
| NY→London network floor | ~80 ms — why geo-LB exists |
Where the interviewer pushes
The reliable push is "WebSockets behind your load balancer — what breaks?" The strong answer: an L7 balancer must support the connection upgrade and then hold the connection open, which defeats per-request balancing — so I either use L4 and balance at connection-establishment time, or L7 with connection-aware routing. Then the follow-through: connections are sticky by nature, so when a server dies its clients must reconnect and re-authenticate, and I need a service registry or consistent hashing so reconnects spread evenly instead of stampeding one node.
The second push is "what if the load balancer dies?" Weak answers add "another LB" and stop. The strong answer names the mechanism: active-passive pair with a shared virtual IP via keepalived, failover in seconds, plus DNS-level spread across pairs for the tier above. And the honest caveat — DNS failover is slow because clients cache resolutions for the TTL, so it's a coarse tool for region-level failure, not server-level.