API Gateway Interview & Revision

EVOLVE: (1) Clients call users/orders/payments services directly — note duplicated auth and clients knowing every address. (2) Introduce ONE gateway as the single entry point that routes by path. (3) Move authentication to the gateway (reject bad tokens at the edge, 401). (4) Add authorization (403 for under-privileged). (5) Add token-bucket rate limiting per key (429). (6) Add request aggregation for chatty clients. (7) Add /v1 vs /v2 routing for safe rollout. (8) Replace hardcoded addresses with service discovery via a registry. (9) Mint a trace id for observability. (10) Cluster the gateway (stateless instances behind a load balancer, shared config + rate-limit state) so it isn't a single point of failure.

EVOLVE: start with a reverse proxy (NGINX/OpenResty) doing plain path-based forwarding. Add a plugin pipeline so cross-cutting concerns are pluggable: an auth plugin (JWT/key), a rate-limiting plugin backed by shared storage, a logging/tracing plugin. Split into a control plane (declare routes/plugins) and a data plane (NGINX workers executing them). Finally cluster the data plane behind a load balancer with config synced from the control plane. Kong is 'reverse proxy core + plugin policy engine + control/data-plane split' — walk that build-up.

EVOLVE: begin with a single JVM proxy in front of services. Introduce a filter chain — pre-filters (auth, rate limiting, routing decision), route-filters (forward to the chosen service), and post-filters (add headers, record metrics). Integrate Eureka for service discovery so routes resolve to healthy instances. Move Zuul 1's blocking IO to Zuul 2's async/non-blocking model for high connection counts. Then run multiple Zuul instances behind an ELB. The story is 'filter chain → discovery → async → cluster'.

EVOLVE: (1) Direct calls expose the duplication/coupling pain. (2) Add a gateway for a single stable entry point and routing. (3) Layer cross-cutting concerns one at a time exactly as in 'Design an API Gateway'. (4) Distinguish north-south (this gateway) from east-west traffic (a service mesh with sidecars). (5) Make the entry point a stateless cluster behind a load balancer with shared state. The point: the 'entry point' IS an API gateway grown one concern at a time.

LB spreads traffic across replicas of one service or tier (which healthy instance?). Gateway routes different API requests to different services and adds auth, rate limiting, aggregation, and versioning (which service, and is it allowed?). They usually compose: LB in front of a gateway cluster, and services behind the gateway may be load-balanced too.

A route table maps path/method/host patterns to services, and evaluated per request. API gateways use implementation-specific route-matching strategies. Some optimize path matching using prefix trees or other indexing structures, while others evaluate configured rules and priorities. Matching complexity and route precedence depend on the gateway and its configuration.

So services stop each re-implementing (and mis-implementing) token validation. The gateway verifies the JWT/API key once and rejects bad tokens at the edge with 401 before any backend work. Distinguish 401 (not authenticated) from 403 (authenticated, not authorized).

A token bucket per client key refills at a fixed rate up to a capacity; empty bucket → 429. In a cluster the bucket state must live in shared storage (Redis) or each instance grants a full quota. Return Retry-After and X-RateLimit-* headers.

One client call fans out to several services in parallel and the gateway fuses the responses into one payload (often in a BFF-style layer), cutting client round trips. Use per-call timeouts, circuit breakers, and partial-response degradation.

Instead of hardcoded addresses, services register in a health-checked registry (Consul/etcd/Eureka) and the gateway resolves from a cached or watch-updated healthy-instance set, so autoscaled instances join and dead ones drop without redeploying the gateway.

Keep instances stateless, run a cluster behind a load balancer, sync config from a control plane, and keep shared rate-limit state outside the instance (for example Redis). Discovery/config truth lives in the control plane or registry, while instances can keep disposable local caches, so losing one instance does not cause an outage.

In one sentence each, how does an API gateway differ from a load balancer and from a plain reverse proxy? Answer: Load balancer: spreads traffic across interchangeable replicas of one tier. Reverse proxy: generic L7 forwarding, TLS termination and caching in front of servers. Gateway: a reverse proxy plus API-level policy (auth, limits, aggregation, versioning) across many services.

A request arrives with a valid token but for a user without admin rights hitting an admin route. What status does the gateway return, and is that authentication or authorization? Answer: 403 Forbidden. The token was valid (authentication succeeded) but the caller lacks permission, which is authorization.

You scaled the gateway from 1 to 4 instances and clients suddenly get 4× their rate limit. What's the bug and the fix? Answer: Per-instance counters: each of the 4 instances counts independently, so a client can get up to 4× its limit. Fix: shared state (an atomic Redis counter/bucket or a rate-limit service).

Why is aggregation done with parallel fan-out plus per-call timeouts rather than sequential calls, and what should happen when one backend is down? Answer: Parallel fan-out makes latency about the slowest call instead of the sum. Per-call timeouts and circuit breakers stop a slow backend stalling everything, and a down backend yields a partial response with nulls/flags rather than a 500.

How does service discovery let a newly autoscaled instance receive traffic without redeploying the gateway? Answer: The instance registers in the health-checked registry and starts receiving traffic once it passes checks; the gateway's cached or watched healthy-instance set picks it up without a redeploy.

Where should the trace id be created, and why does it break observability if a downstream service drops it? Answer: At the gateway, the first hop, so every request gets an id. If a downstream service drops it, the trace breaks there: you see the request arrive and vanish.

Walk the EVOLUTION you'd present for 'Design an API Gateway' in the correct order of concerns. Answer: Direct calls → one gateway routing by path → authentication → authorization → rate limiting → aggregation → versioning → discovery → observability → clustered HA gateway.