API Gateway Engineering Insights

The gateway is not free — it's a deliberate trade of one extra hop for a lot of centralised leverage.

Calling services directly is genuinely simpler when you have two services and one client — no extra component, no extra hop, nothing new to operate. The gateway earns its keep as the system grows: more services, more client types, and more cross-cutting policy that you refuse to copy-paste. The break-even point is when duplicated auth/rate-limiting/logging across services costs more than running and operating the gateway.

The real design question is not "which one wins?" but which layer should own which decision.

If one public endpoint only needs to spread traffic across replicas of a single backend tier, a load balancer is enough. Introduce a gateway when the same entry point must front many services or enforce shared API policy. Most internet-facing microservice systems use both. For balancing mechanics, see Load Balancer Fundamentals.

Putting auth at the gateway is powerful and slightly dangerous — you gain consistency but create a trust boundary you must defend.

Centralized auth at the gateway means one implementation, one place to fix bugs, and services that trust an already-verified identity header. The risk is the confused-deputy / trust boundary problem: if a request can reach a service without passing the gateway (e.g. via internal network), and the service blindly trusts the identity header, an attacker who bypasses the gateway is trusted. The mitigation is network policy (services only accept traffic from the gateway) or signed identity tokens the service re-verifies.

Decentralized auth (each service verifies the token itself) removes the single trust boundary but reintroduces the duplication the gateway was meant to kill. The common hybrid: the gateway does coarse authentication and rejects obvious bad tokens, while sensitive services still re-verify a signed token for defence in depth.

Fewer round trips for the client, but a fatter, more coupled gateway — this one's a real judgment call.

Aggregation shines for high-latency clients (mobile on cellular) where each round trip is expensive: one call, parallel fan-out, one response. But it moves response-shaping logic into the edge, couples the gateway to backend schemas, and makes the gateway a bigger, more stateful thing to reason about. Over-aggregation turns your gateway into a monolith with extra steps.

Every request now pays for one more network hop and one more thing that can fail — is the centralisation worth it?

The gateway adds work on every request, but the cost depends on what the gateway actually does. A thin self-hosted gateway on the same network may add roughly sub-millisecond to low-single-digit milliseconds (illustrative; it varies with hardware, TLS, plugins and load); cross-zone hops, token introspection, large body transforms, aggregation, or managed-gateway control paths can add noticeably more. It also adds an availability dependency — if the gateway tier is down, everything behind it is effectively down. The trade is centralised policy and observability versus extra latency and a new failure domain, so measure the actual p95/p99 overhead instead of assuming it is trivial or fatal.

For customer-facing production APIs, the answer is usually clear — but understanding why frames the whole HA design.

A single gateway is simpler to run and reason about, and fine for a demo or an internal tool. For customer-facing production traffic it is both a single point of failure and a throughput ceiling: one process, one blast radius. The cluster answer — stateless instances behind a load balancer, shared config, and shared rate-limit state with discovery/control-plane data outside the instance — is more moving parts but the standard way to survive an instance dying and to scale past one machine.

The concepts are universal; the products differ in language, extensibility, and how much you operate yourself.

The landscape

Common mistakes

FAQs & real-world architectures