Reverse Proxy · Production Engineering

Forwarding is not a redirect. The proxy maintains two independent connections and copies bytes between them, which is what makes every other feature possible.

When a request arrives, the proxy accepts the client connection, parses the request line and headers, and decides where it goes. It then borrows a connection to the chosen backend from a keep-alive pool (or opens a new one), rewrites hop-by-hop headers, adds forwarding headers like X-Forwarded-For and X-Forwarded-Proto, and streams the request upstream.

On the way back it buffers the response so a slow client can't tie up a precious backend worker — the backend finishes fast and moves on, while the proxy dribbles bytes to the client at whatever rate it can accept. Connection pooling, buffering, and header rewriting are the unglamorous mechanics that let a small backend serve many slow clients.

Terminating TLS at the proxy centralises the most error-prone public-edge concern in a web service: certificates and handshake policy.

The client and proxy perform the TLS handshake — certificate presentation, key exchange, cipher negotiation, and ALPN if needed. Once that finishes, the proxy holds the decrypted HTTP request in memory and can route, cache, compress, inspect, or reject it before any backend sees it. That L7 visibility is exactly why TLS usually terminates here rather than at a blind L4 hop.

What happens next is a separate design choice. In the common termination pattern, the proxy forwards plain HTTP over a trusted private network. In stricter environments it opens a new HTTPS or mTLS connection upstream, so public TLS still terminates at the proxy but internal traffic stays encrypted too. If you instead use TCP/TLS pass-through, the backend keeps the certificate and the proxy loses most HTTP-aware features.

Because the certificate and private key live only on the public proxy tier, rotation is a single automated job (for example, ACME/Let's Encrypt renewals), TLS version and cipher policy is one config, and OCSP stapling and session resumption are configured once. Backends either avoid public crypto material entirely or only handle the internal trust relationship for re-encryption.

The routing engine turns a request's host and path into a backend decision, and the order of rules is everything.

Nginx matches location blocks by a defined precedence: an exact (=) match ends the search immediately; otherwise nginx remembers the longest matching prefix location, then checks regex locations in file order and uses the first regex that matches — unless the remembered prefix has the ^~ modifier, in which case regexes are skipped. If no regex matches, the remembered longest prefix is used. A request is resolved to exactly one location, which selects an upstream. A broad / prefix does not shadow a specific /api prefix, but a regex location can override a prefix location, silently sending API traffic to the wrong place.

Beyond static paths, the engine can route on headers, query args, and rewritten URIs, and it can strip or preserve prefixes before forwarding. The discipline is to keep rules explicit and ordered, and to test them, because routing bugs are invisible until a user lands on the wrong backend.

Serving static files from the proxy is the cheapest performance win available, but only if you get caching headers and file access right.

For a static request the proxy maps the URL to a file on disk and streams it with sendfile, letting the kernel copy bytes straight from the file cache to the socket with zero application involvement. Correct Cache-Control/Expires headers then let browsers and CDNs cache the file too, so most requests never even reach the proxy again.

The two operational concerns are freshness and safety. Fingerprinted filenames (app.4f2c.js) let you set long TTLs without stale-content risk. And the file root must be locked down so path traversal can't escape the intended directory.

Proxy caching trades a little staleness for a lot of backend relief — the engineering is all in the cache key and the TTL.

The proxy builds a cache key (typically method + host + URI, sometimes plus selected headers), and on a hit within the TTL it returns the stored response without touching the backend. On a miss it fetches, stores, and serves. Request collapsing (in nginx, proxy_cache_lock on, which is off by default) ensures that when a cached item expires, a thundering herd of simultaneous requests results in one backend fetch, not thousands. To keep serving the stale copy to everyone else while that single refresh runs, add proxy_cache_use_stale updating (typically with proxy_cache_background_update on).

Tuning is about correctness first: never cache per-user or authenticated responses under a shared key, honour Cache-Control from the backend, and use short TTLs with stale-while-revalidate for content that changes. Measure hit ratio continuously — it is the single number that tells you whether the cache is earning its keep.

The proxy is your perimeter. Everything you enforce here is enforced for every backend at once.

Standard controls layer up at the edge: TLS with modern ciphers, security headers (HSTS, CSP, X-Frame-Options), IP allow/deny lists, connection and request rate limiting, request size and timeout limits, and optionally a WAF module to block known exploit patterns. Because backends have no public address, all of this is enforced before a hostile request can reach an application.

Rate limiting deserves special attention: leaky-bucket or token-bucket limits per IP or per key blunt brute-force and scraping, and connection limits stop a single client exhausting worker slots. The proxy is ideal for coarse edge protection because it sees every connection. If you also run an API gateway, that gateway may still apply finer-grained limits per consumer, token, or product plan deeper in the stack.

One domain, many services: the proxy composes independent backends into a single coherent site.

Each service gets its own upstream pool and its own location rules, so teams deploy and scale independently while presenting one URL space to users. The proxy can also do canary and blue-green routing — sending 5% of /api traffic to a new version by weighting upstreams — and can rewrite paths so a backend's internal routes differ from the public ones.

The trade-off is that the proxy config becomes a shared, high-stakes artifact. Keep it in version control, review changes, and validate (nginx -t) before every reload, because one bad route affects the whole site.

The proxy tier scales horizontally and independently — but only if you design it to be stateless and front it with a load balancer.

Run several identical proxy instances behind a dedicated load balancer so no single proxy is a point of failure and so you can add capacity by adding instances. Keep proxies stateless: push shared cache to a distributed tier if needed, and keep TLS session state resumable. Health checks let the load balancer eject a sick proxy automatically.

Scale the layers on their own axes — proxies for connection and edge-work capacity, backends for compute. Because proxies absorb static, cached, and blocked traffic, the backend tier you have to scale is only the genuinely dynamic remainder.

For each failure mode: what breaks, the user impact, how you detect it, how you recover, and the scaling strategy that prevents it next time.