API Gateway Fundamentals

An API gateway is a single entry point that sits between clients and a fleet of backend services, receiving every request and forwarding it to the right service while handling the concerns every service would otherwise duplicate.

From a client's perspective, the gateway looks like one API at one address. Behind that address there may be dozens of microservices — users, orders, payments, search — each with its own address, its own team, and its own deployment cadence. The gateway hides all of that. The client says GET /orders/42 to a single host; the gateway figures out which service owns /orders, checks that the caller is allowed in, and forwards the request.

API Gateway Architecture Diagram: One Entry Point, Many Services

The diagram above shows the gateway architecture: many clients send every request to the gateway, which authenticates the caller, applies policies such as rate limits, and routes the request to the service that owns that path. Services can change behind it without clients noticing.

The core responsibilities

Every API gateway story starts the same way: a handful of services, then clients calling each of them directly — and the same problems copy-pasted into every service.

The direct-call pain

When clients call services directly, three problems appear almost immediately. First, every service re-implements the same cross-cutting logic — token validation, rate limiting, logging — and they drift out of sync, so an auth bug fixed in orders-service still ships in payments-service. Second, clients must know the address of every service, so every new service or relocation means a client change. Third, there is no single place to enforce policy — you cannot throttle an abusive client, roll out a new API version, or trace a request end-to-end from one spot.

The blast radius problem

Direct coupling also spreads failure. A mobile client that hardcodes five service addresses breaks in five different ways, and a security fix has to be coordinated across five teams. The gateway collapses all of that into one contract the client depends on and one place the platform team controls.

A gateway and a load balancer can both sit near the edge, but they solve different problems and commonly appear in the same request path.

A load balancer distributes traffic across replicas of the same service or tier to improve throughput and availability. Every backend behind it is intended to be interchangeable, so the balancer's core question is "which healthy instance should take this request?" For the mechanics of L4/L7 balancing, health checks, and algorithms, see Load Balancer Fundamentals.

An API gateway is an L7 entry point for many backend services. It routes requests by path, method, host, header, or version and adds API-level policy: authentication, authorization, rate limiting, request aggregation, protocol translation, and versioning. A gateway may route to services that are themselves load-balanced, so it complements rather than replaces load balancing.

A gateway and a reverse proxy overlap at the forwarding layer, but the intent is different: generic backend proxying versus API-specific policy at the edge.

A reverse proxy accepts requests on behalf of backend servers and forwards them, often terminating TLS and sometimes doing caching, compression, or simple path-based forwarding. Its core job is fronting backends and managing HTTP/TCP traffic. For the baseline reverse-proxy model, see Reverse Proxy Fundamentals.

An API gateway builds on that reverse-proxy pattern and focuses on API concerns: token validation and authorization, per-client rate limiting, request aggregation across services, API versioning, service discovery integration, and per-route observability. Many gateways (Kong, for example) are implemented on top of a reverse-proxy core plus a policy/plugin layer.

Before designing a gateway, separate what it must do from how well it must do it — that framing keeps the design reasoning clear.

Functional requirements

Non-functional requirements

Understanding the fixed order of stages a request passes through is the key mental model — most gateway features are just a stage in this pipeline.

A request flows through the gateway as an ordered pipeline. The order matters: you authenticate before you authorize, you rate-limit before you do expensive work, and you attach a trace id early so every later stage and downstream call is correlated.

Routing is the gateway's headline job: turn an incoming request into a decision about which backend service should handle it.

A route table is a set of rules. Each rule matches on attributes of the request — path pattern, HTTP method, host, or header — and names a target service. When several rules could match, precedence decides: some gateways forward to the first matching rule in configured order, others prefer a more specific match (for example exact before prefix), so check how your gateway resolves overlaps. Because matching is per-request, a single client connection can reach many different services over its lifetime.

Centralising identity checks at the gateway is one of the biggest wins — services stop each re-implementing (and mis-implementing) auth.

Authentication — who are you?

The gateway validates the caller's credential — typically a JWT or an API key. For a JWT it verifies the signature and expiry, and for opaque tokens it may call an auth service to introspect them. A request that fails authentication is rejected immediately with 401 Unauthorized, before it ever touches a backend service. That single choke point means a token bug is fixed once, at the gateway, not in every service, though sensitive services may still re-verify signed identity or only trust requests that can arrive through the gateway path.

Authorization — are you allowed?

Authentication only proves identity; authorization decides what that identity may do. After a token is validated, the gateway checks the caller's roles or scopes against what the route requires. A valid but under-privileged token — a normal user hitting an admin-only route — is rejected with 403 Forbidden. The distinction is worth memorising: 401 means "I don't know who you are", 403 means "I know who you are, and you can't do this".

Rate limiting protects backends from overload and enforces fair-use quotas, and the gateway is the natural place to do it once for everyone.

The most common algorithm is the token bucket. Each client (keyed by API key, user id, or IP) has a bucket that refills at a steady rate up to a capacity. Every request consumes one token; if the bucket is empty, the request is rejected with 429 Too Many Requests. The bucket's capacity allows short bursts while the refill rate caps the sustained throughput — a good match for real API traffic.

Three features that turn a router into a real platform layer: fusing responses, evolving the API safely, and finding services dynamically.

Request aggregation (the BFF pattern)

A mobile home screen might need user profile, recent orders, and payment status. Instead of the client making three round trips, it makes one call to the gateway, which fans out to three services in parallel and fuses the results into a single response. This is the Backend-for-Frontend (BFF) pattern — fewer round trips, less client complexity, and a payload shaped for that specific client.

API versioning

The gateway can route /v1/... and /v2/... to different service versions simultaneously. This decouples client migration from backend deploys: v2 ships, a slice of traffic moves to it, and v1 keeps serving everyone else until they migrate. Versioning at the edge means internal services don't have to carry every historical shape forever.

Service discovery

Hardcoding backend addresses breaks the moment instances autoscale or move. Instead, services register themselves in a registry (Consul, etcd, Eureka, or the platform's own) with health status, and the gateway consults a registry-fed view of healthy instances. Implementations usually cache or subscribe to registry updates rather than hitting the registry on every request, so new instances can start receiving traffic quickly while unhealthy ones are removed without redeploying the gateway.

A gateway on every request is both the best observation point in the system and its most dangerous single point of failure — you must get both right.

Observability

Because every request passes through it, the gateway is the ideal place to mint a trace id and attach it to the request before forwarding. Each downstream service propagates that id, so logs, metrics, and traces across the whole call graph share one correlation key. Centralised metrics (latency, error rate, throughput per route) and distributed tracing turn a fog of microservices into a single readable timeline.

Scaling & high availability

One gateway box is a single point of failure — if it dies, the whole API goes dark. The fix is a cluster of stateless gateway instances behind a load balancer, with configuration (routes, keys, limits) synced from a shared control plane. Any instance can serve any request, so the load balancer's health checks simply route around a failed instance while the rest carry the traffic. Shared state (rate-limit buckets, discovery cache) lives in an external store like Redis so instances stay interchangeable.