Interactive guide · 6 chapters · ~16 min read
Load Balancer Engineering Insights
Design decisions, trade-offs, comparisons, and the mistakes that show up again and again in real load-balanced systems.
Mangesh MandlikLast updated:
01 · Terminology
Load Balancer vs Reverse Proxy
These terms overlap, but they answer different design questions: a reverse proxy answers what sits in front of your origins, while a load balancer answers how traffic gets spread across them.
A reverse proxy is defined by where it sits and what it shields: in front of origins, forwarding requests on the client's behalf. A load balancer is defined by its primary job: distributing traffic across multiple backends for scaling and availability. One product can do both, but neither label automatically implies the other in every deployment. For the deeper reverse-proxy treatment, see Reverse Proxy Fundamentals.
02 · Trade-offs
Layer 4 vs Layer 7 — The Real Trade-off
The choice isn't about which is "better" — it's about how much you need to see.
L4 is simpler because it routes using IP, port, and connection metadata and does not parse HTTP or other application payloads. That means it cannot route on URL path, header, or cookie, and it often preserves end-to-end TLS via passthrough. L7 spends more CPU because it decrypts or terminates and understands HTTP/gRPC, which is what enables path-based routing, header/cookie steering, redirects, retries, and caching.
03 · Tooling
NGINX vs HAProxy
Both are mature, widely deployed, and capable of load balancing — the differences are in origin and default strengths.
NGINX
Originated as a web server and reverse proxy; load balancing was added on top. Excellent at serving static content, caching, and general-purpose HTTP reverse proxying. Configuration is declarative and widely documented.
HAProxy
Purpose-built as a load balancer from day one, at both L4 and L7. Often chosen for very high connection counts, fine-grained health checking, and detailed traffic statistics (check current docs and benchmark your own workload before treating this as a ranking). Preferred in environments where load balancing (not serving content) is the primary job.
| Choose… | When | Watch for |
|---|---|---|
| NGINX | You already serve static content or cache, and need an HTTP reverse proxy with simple upstream pools. | Open-source NGINX health checking is passive (failures observed on real traffic); active probes are an NGINX Plus feature. |
| HAProxy | Load balancing is the primary job, at L4 or L7, with fine-grained health checks and rich statistics. | Not a content server or cache; you pair it with one if you need that. |
| Envoy | Microservices needing dynamic configuration, per-request retries, circuit breaking and strong gRPC support. | Configuration complexity; usually run through a control plane or mesh. |
| Managed cloud LB (ALB / NLB) | You want multi-AZ, auto-scaling, managed TLS certificates and no balancer fleet to operate. | Less control over algorithms and per-request logic; cost and quotas scale with traffic. |
04 · Deployment model
Hardware vs Software Load Balancers
Before cloud-native load balancing was ubiquitous, dedicated hardware appliances (F5, Citrix ADC) were the default.
Hardware appliances
Purpose-built devices with dedicated silicon for TLS offload and packet processing — extremely high throughput, but expensive, slow to reconfigure, and hard to scale elastically (you buy capacity in discrete boxes).
Software load balancers
Run on commodity servers or as managed cloud services (ALB/NLB, Cloud Load Balancing). Elastic, easy to automate, and straightforward to version-control as config — the default choice for most new cloud-native deployments.
05 · Pitfalls
Common Mistakes
The same handful of mistakes recur across almost every load-balanced system that has an outage.
06 · Wrap-up
FAQs & Real-World Architectures
A few recurring questions and how production systems typically compose everything covered so far.
Frequently asked questions
"Can a load balancer be a single point of failure?"
Yes — which is why production load balancers are themselves deployed redundantly (active-active pairs, or a managed multi-AZ cloud service), with a floating/virtual IP or DNS failover in front of them.
"Do I need a load balancer if I only have one server?"
Not strictly. But a load balancer or reverse proxy can still be useful if you want a stable entry point, centralized TLS, or blue/green deploys before you scale out later.
"Is a CDN a load balancer?"
A CDN is a specialized global reverse proxy and cache network, often built on the same anycast or GeoDNS techniques used for global load balancing — but its primary job is caching and edge delivery, not distributing compute load across an application server pool. See CDN Fundamentals for the fuller model.
A typical production architecture
Global anycast/GeoDNS routes users to the nearest region → a regional L7 load balancer (with TLS termination and path-based routing) fronts a set of application server auto-scaling groups → health checks and connection draining keep the pool accurate through deploys and scaling events → session/cart data lives in a shared Redis store, so no sticky sessions are required → blue/green or canary deployments roll out new versions safely.
Key takeaways
- 01Reverse proxy and load balancer are overlapping roles that are often implemented by the same software, but they are named for different responsibilities.
- 02L4 vs L7 is a visibility/cost trade-off, not a strictly-better-or-worse choice.
- 03NGINX, HAProxy, and Envoy each have a natural home — general reverse proxying, dedicated load balancing, and microservice-heavy traffic management, respectively.
- 04Hardware load balancers still exist, but software and managed services are the default for most new cloud-native systems.
- 05The recurring production mistakes are mostly lifecycle and policy problems: draining, health check tuning, certificates, and timeouts.