Global CDN — Architecture Deep Dive

A cache hit is the path you are buying when you invest in a CDN. Everything else exists to make this path common and predictable.

Request arrival at the Edge PoP

The client reaches the nearest healthy Edge PoP via DNS or Anycast Routing. TLS terminates locally, the request is normalized, and routing metadata identifies the customer, hostname, and cache behavior that apply.

Cache key computation

The edge computes a deterministic key from the hostname, path, and an approved set of vary dimensions such as device class or normalized query parameters. The key must exclude attacker-controlled or irrelevant headers or the cache becomes fragmented and unsafe.

Memory-first lookup

The edge checks the in-memory tier first because it is the lowest-latency path. If the object is present and still fresh by TTL, the edge can stream bytes immediately. If it is not in RAM, a second local SSD lookup may still produce a hit without leaving the PoP.

Serve and account

Once the object is found, the edge serves it, updates hit counters, and records metrics such as latency, key class, and byte size. Hot objects can be promoted within the local cache hierarchy so the next request is even cheaper.

Freshness metadata check

Finding the object is not the same as being allowed to serve it. Before returning bytes, the edge checks freshness metadata such as remaining max-age, the Age header accumulated so far, and any per-object policy override. An object can be present in cache and still fail this check if its freshness lifetime has expired — that outcome routes into revalidation, not a plain hit.

Representation and range handling

If the object varies by an approved dimension — compressed encoding, image format, or device class — the edge selects the matching stored representation rather than a single canonical body. For range requests, a fresh hit can often be served directly from the stored object without contacting upstream, provided the requested byte range is fully present locally.

A miss is unavoidable sometimes, but it should be carefully managed because it is where latency, origin load, and consistency rules all collide.

Lookup failure and miss classification

The edge detects that the object is absent, expired beyond what stale policies allow, or intentionally bypassed. It then classifies the miss: cold miss, expired object, revalidation candidate, or bypass because the request is personalized or explicitly uncacheable.

Shield lookup and request coalescing

Instead of every edge opening an origin connection, the miss usually targets a Regional Shield Cache. If the shield already has the object, the origin stays untouched. If multiple identical misses arrive together, the shield can perform request coalescing so only one upstream fetch proceeds while followers await the result.

Origin fetch and response policy evaluation

When the shield also misses, it fetches from the Origin Server or Origin Load Balancer + replicas. The returned headers determine cacheability, TTL, revalidation rules, range support, and whether the response belongs in memory, SSD, or only the shield tier.

Store, serve, and propagate warmth

The response is streamed back to the client while being stored in shield and edge caches when allowed. That one slow trip creates many future fast trips, which is the core economic model of a CDN.

Bounded origin concurrency and connection reuse

Upstream fetches reuse pooled, keep-alive connections rather than opening a new connection per miss, and the number of concurrent fetches toward a given origin is bounded. Timeouts and a limited retry budget prevent one slow origin path from tying up shield or edge resources indefinitely, and retries are capped so a struggling origin is not met with retry amplification on top of its existing load.

Streamed fill and partial-fetch handling

A miss response can be streamed to the client while it is simultaneously being written into cache, so the client does not wait for the full object to land locally first. If the client disconnects mid-fill, the fill itself can continue so the object still gets cached for the next request. An incomplete or failed fill should not be treated as a valid cached object — a partial or errored fetch must not be marked reusable, or a future request could be served a truncated response.

A cache hit and a cold miss are the two ends of the spectrum. Between them sits revalidation: a cheap conditional check that confirms whether an aging object is still good, without re-downloading it.

The freshness signals involved

Cache-Control: max-age (and related shared-cache directives) tells intermediaries how long a response can be reused. Expires is an older, secondary freshness signal that max-age generally takes priority over. Age reports how long a response has already been held by a cache, which lets a client or downstream cache compute remaining freshness on its own. ETag is an opaque identifier for a specific representation; Last-Modified is a timestamp-based alternative. Together they let a client ask an origin, cheaply, 'has this changed since I last saw it?'

The revalidation flow

The sequence is: (1) the object exists locally, (2) its freshness lifetime has expired, (3) the edge sends a conditional request upstream carrying If-None-Match (with the stored ETag) or If-Modified-Since (with the stored Last-Modified value), (4) the origin replies 304 Not Modified if nothing changed, or a new 200 with a fresh representation if it did, and (5) the edge refreshes its freshness metadata — and body, if a new one arrived — accordingly.

Why revalidation is its own category

Revalidation is cheaper than a cold miss because a 304 response typically carries no body, just updated headers — the network cost is a fraction of a full re-fetch. It is also distinct from a plain hit: the edge does contact upstream, it just avoids re-transferring bytes it already has. Treating revalidation as a named third path, rather than folding it into 'hit' or 'miss,' makes both capacity planning and monitoring more honest.

Sometimes the better answer to an expired object is not 'block and refetch' but 'serve what you have and quietly fix it in the background.'

Stale-while-revalidate

A stale object can be served for a bounded window while exactly one request triggers an asynchronous refresh. This hides refill latency from users, prevents many concurrent requests from all waiting on the same slow refresh, and composes naturally with request coalescing at the shield. The risk is real and explicit: for that bounded window, a user can receive content that is technically out of date.

Stale-if-error

If the upstream or origin path is unhealthy, selected content classes may continue serving stale cached bytes for a bounded period rather than failing outright. This must be a policy decision made per content class, not a blanket default — stale authentication state, financial data, security-sensitive responses, or rapidly changing metadata may be unsafe to serve stale even briefly, while a marketing image or a static bundle usually is not.

The cache key is one of the most consequential and least discussed decisions in CDN design: it determines which requests are allowed to share a cached response.

What can go into a key

Typical inputs include scheme and host where relevant, a normalized path, a deliberately selected set of query parameters, selected headers or representation dimensions, and — only where explicitly required — device or image-format variants. Every dimension added to the key narrows how broadly a cached response can be reused.

Query parameter normalization

A safe policy ignores tracking parameters that do not affect the response, and sorts or canonicalizes the parameters that remain so equivalent requests map to the same key. Two failure modes sit on either side of this: blindly including every query parameter destroys hit ratio by fragmenting the cache into near-duplicate keys, while blindly ignoring parameters that do change the response causes the wrong content to be served.

Header normalization and Vary

Vary tells a cache which request headers affect the response representation. Varying on a low-cardinality, well-understood header (like a coarse device class) is manageable. Varying on a high-cardinality or attacker-influenceable header can explode the number of distinct cached representations for a single logical object and destroy cache efficiency.

Auth, cookies, and shared caching

Public shared caching must not accidentally reuse an identity-specific response. If auth-bearing headers or cookies are not explicitly excluded from the key — or the response is not correctly marked private/bypassed — one user's personalized response can be cached and replayed to another.

Cache-key explosion and poisoning

Any attacker-controlled or high-cardinality dimension included in the key can be exploited to fragment the cache and create resource pressure on its own. In the opposite direction, missing a dimension that genuinely matters can cause one representation to be served to a user it was never meant for — a correctness failure, not just an efficiency one.

Freshness and eviction are separate concerns. A production CDN must answer both 'is this object still valid?' and 'is this object still worth keeping locally?'

Expiration vs eviction

TTL-based expiration is a correctness rule: after some time the object is stale unless revalidated. Eviction is a capacity rule: under space pressure, the node must choose which objects to discard even if they are still technically fresh.

LRU, LFU, and TinyLFU-style ideas

LRU favors recency and is easy to implement, but can be fooled by one-time scans. LFU captures frequency, but can retain old popularity too long. Modern designs often borrow TinyLFU-style admission or frequency sketches so a burst of cold large files does not evict truly hot content.

Memory vs SSD tiering

RAM should hold very hot small items; SSD should hold a larger warm set. Promotion and demotion policies matter: repeated SSD hits can promote an object to memory, while cold memory objects can sink to SSD before full eviction.

Size-aware eviction

Bytes matter. One 500 MB file should not evict ten thousand popular 50 KB icons just because it was fetched recently. Real CDNs combine recency/frequency with size, object class, and sometimes customer priority.

Admission is a separate decision from eviction

Eviction asks 'what do we remove when the cache is full?' Admission asks a question that comes earlier: 'should this object enter the cache at all?' Not every fetched object should necessarily displace existing content. A one-time huge download can be streamed to the client without ever being written into RAM or SSD, so it does not push out genuinely hot objects.

What admission can weigh

At a conceptual level, an admission decision can consider recency, observed frequency, object size, expected future reuse, and the customer or object class involved. None of this requires a specific production algorithm — the point is that entry into the cache is itself a policy choice, not an automatic consequence of being fetched.

Caching only successful 200 responses is an incomplete design — some of the most repetitive load on an origin comes from requests that never succeed.

What can be cached negatively

Selected classes of unsuccessful responses can be short-lived cache candidates: 404 Not Found, redirects, and — depending on policy — some upstream failure responses. A common example is bots or crawlers repeatedly requesting a nonexistent path; without negative caching, every one of those requests reaches origin.

The correctness risk

Negative caching protects origin, but it introduces its own failure mode: a resource that was just created can remain invisible to users until the negative-cache TTL expires, and caching errors too aggressively during an incident can quietly prolong the outage by hiding the fact that the origin has already recovered.

Status-aware policy

The decision to negatively cache a response — and for how long — should depend on the response status and the content class involved, not be a single blanket TTL applied to every non-200 response.

An Edge PoP is the physical place where routing, caching, TLS, and security all meet.

Ingress and reverse proxy layer

Front-line proxy nodes accept TCP/QUIC connections, terminate TLS, apply routing and WAF checks, and dispatch requests toward local cache services. This layer must be extremely predictable because every request crosses it, hit or miss.

Local cache store

Behind ingress sits the Edge Cache with an in-memory tier and an SSD tier. The cache store holds the actual bytes plus metadata such as TTL, vary dimensions, ETag, compression variants, and whether the object is range-friendly for video.

Config sync and health agents

Each PoP runs agents that pull or receive versioned config snapshots: domain mappings, cert bundles, origin maps, purge markers, and WAF rules. Separate health agents export metrics and can mark specific subsystems unhealthy so traffic drains before a total outage.

The request pipeline inside a PoP

Internally, a request moves through a fairly consistent pipeline: network ingress → connection/TLS/QUIC handling → WAF/rate controls → request normalization → cache key computation → local cache lookup → upstream/shield path (on miss) → response processing → metrics/logging. Naming these stages explicitly makes it easier to reason about where a given failure or slowdown is actually occurring.

Local resilience

A PoP should keep functioning even when parts of it are unhealthy. That means retaining a last-known-good config snapshot if a new one fails validation, keeping certificate and key material available locally rather than fetched per handshake, and isolating the health of individual subsystems. A single failing process or node inside a PoP should not necessarily take down the entire location — that is a different failure than the whole PoP going dark.

The edge is the natural place to terminate client-facing transport protocols, because it is already the first hop that owns TLS termination and connection handling.

HTTP/2 over TCP

HTTP/2 multiplexes many logical request/response streams over a single TCP connection, which avoids the overhead of many separate connections. The trade-off is that TCP itself has no concept of independent streams: packet loss anywhere on that connection can stall every multiplexed stream sharing it, even ones that were otherwise ready to proceed.

HTTP/3 over QUIC

HTTP/3 carries the same HTTP semantics over QUIC, which runs on UDP. QUIC gives streams independent loss recovery — a lost packet on one stream does not stall the others — and offers connection establishment and connection-migration properties (such as surviving a client's network change) that TCP-based HTTP does not.

It is a design trade-off, not a universal upgrade

HTTP/3 is not universally faster than HTTP/2 — its benefits are most visible on lossy or mobile networks and workloads with many concurrent streams, and it introduces its own operational considerations (UDP handling, middlebox behavior, new congestion-control code paths). It is also worth noting that the protocol between edge and client can differ from the protocol the edge uses to talk to the origin or shield — that choice is independent.

The regional tier exists because edge caches are numerous and geographically sparse, while origins are few and precious.

Why the shield tier exists

Edge caches are excellent for locality, but each one sees only a slice of global traffic. The Regional Shield Cache aggregates demand across many PoPs so long-tail objects still become hot somewhere before origin sees them repeatedly.

Shield TTLs and storage choices

Shields often keep objects longer than an individual edge because they are closer to origin and serve as the second chance before refill. They still need tiering and eviction policies, but the working set is different: broader geographic demand, fewer client connections, heavier refill focus. Refill mechanics — coalescing, bounded concurrency, backpressure — are covered in Origin Shielding below.

Origin shielding is the policy decision that says every refill should go through a controlled intermediary instead of letting every edge improvise direct origin fetches.

Collapsing N misses into 1 fetch

Suppose a homepage hero image expires everywhere at the same minute. Without shielding, dozens or hundreds of PoPs can hit origin at once. With shielding, all of those requests converge on one regional layer that either serves from shield cache or coalesces them behind a single refill.

Protecting fragile origins

Many application origins were never built to absorb internet-scale fan-out. Shielding protects them from both hot-content bursts and routine synchronized expirations. It also centralizes connection pooling, origin auth, and backpressure.

Failure isolation

If one shield region is unhealthy, routing can send misses to another shield or directly to a backup origin path. Because shielding is explicit, failover is controllable rather than every edge simultaneously discovering and hammering a secondary origin on its own.

Connection reuse and bounded concurrency

The shield keeps pooled, reused connections toward origin rather than opening one per request, and bounds how many concurrent refills a given origin can receive. Backpressure — slowing or queuing further refills when origin signals it is struggling — keeps the shield from becoming an amplifier instead of a protector.

Avoiding failover stampedes

If the primary shield path fails, naively redirecting every affected edge straight to origin (or to a single backup shield) can overwhelm that fallback path immediately. A safer pattern moves traffic gradually, keeps coalescing active at the fallback, and applies the same bounded-concurrency budget there too.

"Origin" is a role, not a single kind of system — the CDN needs to select among possible origins as carefully as it selects among edges.

What origin can be

Depending on the customer, an origin might be object storage, an application service, a dedicated media store, or a multi-region replicated origin fleet. The CDN's job is the same regardless: know where to refill from, and know when that place is unhealthy.

Selection inputs

Origin selection can depend on customer/domain configuration, live health signals, a configured priority or failover order, regional or locality preferences where applicable, and the request type or path being served.

Bounded failover

Failover must be bounded. If every edge independently detects a failing primary origin and independently retries a secondary, that secondary can be hit with the full aggregate load all at once — turning a partial origin problem into a full one. Failover logic needs shared budgets and pacing, not just a per-edge retry list.

Protecting origin from being bypassed

Origin protection mechanisms — authenticated CDN-to-origin requests, mTLS or shared signing, IP allowlisting or private connectivity where available — exist so the origin cannot be trivially reached directly from the public internet, bypassing the CDN's caching and security controls entirely.

Anycast is how a global CDN makes one service IP feel local from many geographies at once.

BGP announcement model

Each participating Edge PoP advertises the same IP prefix to upstream peers. Routers on the public internet then choose the best path according to BGP policy and topology. To the client it looks like one IP; operationally it is many possible edge locations.

How the first packet lands at a nearby PoP

When a client's TCP SYN or QUIC packet enters the network, BGP path selection forwards it toward whichever advertisement the network considers topologically or policy-preferred — this is often a nearby PoP, but BGP does not inherently understand application latency, cache warmth, or edge CPU utilization, so "preferred path" is not the same guarantee as "geographically nearest" or "lowest latency."

Route withdrawal and failover

If a PoP is degraded, the CDN withdraws its route or stops advertising it. Reachability shifts as that withdrawal propagates, but convergence is not instantaneous everywhere — different networks observe and react to the change at different speeds. Existing TCP connections can break outright during a route change; QUIC's connection-migration properties can behave differently depending on the specific routing and path conditions, but are not a blanket guarantee of a seamless transition.

Versioned or fingerprinted assets are one of the simplest and highest-leverage techniques in CDN design because they turn freshness into a naming problem instead of a global purge race.

Fingerprinting strategy

Build pipelines append a content hash to asset names such as app.3f92c1.js. If the bytes change, the URL changes too. That lets the CDN cache the object for a year with immutable semantics because new deployments naturally point at a new key.

Build and deploy integration

The application manifest or HTML references the hashed filenames produced by the build system. Deploy orchestration uploads the new objects first, then flips HTML or manifests to reference them. This minimizes broken references during rollout.

Rollback behavior

Rollback becomes safer too: the old hashed asset likely still exists and is still cached. Instead of purging globally under time pressure, the app can often revert references to the prior fingerprinted version.

Versioning does not eliminate the need for purge

Immutable, fingerprinted assets sharply reduce how often purge is needed, but they do not remove it entirely. The mutable entry points that reference those assets — HTML pages, JS/CSS manifests — still need a short freshness window, revalidation, or occasional purge so that clients actually discover the new fingerprinted filenames in the first place.

Purge feels simple at the API boundary and complex everywhere behind it.

Purge API internals

A purge request writes an intent record, expands the target set by domain/path rules, assigns work to regional distributors, and publishes version bumps or tombstones that edge nodes can apply locally. The request should be durable before any fan-out begins.

Acknowledgement tracking

Edges or regional relays can ack receipt and application separately. That distinction matters: 'we sent the purge' is not the same as 'every PoP stopped serving the old object.' Operational dashboards should show both.

Eventual consistency across thousands of PoPs

Even a fast control plane will have stragglers because networks, relays, and PoPs are not perfectly synchronized. That means operators must reason about purge propagation windows, stale-if-error behavior, and whether the application can tolerate a short mix of old and new content.

Purge scope: exact-key vs broader invalidation

A purge job can target an exact cache key, or something broader conceptually similar to path/prefix, tag, or surrogate-key invalidation — clearing every cached object associated with a shared label rather than one URL at a time. Processing must be idempotent and retry-safe: a purge message delivered twice, or replayed after a transient failure, should not corrupt state or double-apply in a way that breaks the version/tombstone model tracking which objects are still valid.

What acknowledgement actually means

A purge "acknowledgement" can mean several different things, and conflating them causes confusion during incidents: accepted durably (the intent is recorded and will not be lost), delivered to a region (a regional distributor has received it), and applied at edge (a specific PoP has actually stopped serving the old object). Operator-visible convergence tracking — and honest straggler tracking for the PoPs that haven't confirmed yet — should distinguish all three.

Everything an Edge PoP needs to behave correctly — except the object bytes themselves — arrives through this control-plane path.

What gets distributed

Objects distributed to PoPs include cache rules, origin maps, routing policy, WAF rules, certificates/keys (or references to securely held key material), and purge markers.

The distribution mechanic

A practical sequence looks like: validate → version → durable source of truth → distribute → locally stage → atomically activate → acknowledge → observe → rollback. Validation and versioning happen before anything leaves the control plane; staging lets a PoP hold the new config without switching to it yet; activation is atomic so a PoP is never serving a half-applied ruleset.

Why staged rollout matters

Malformed config must not be able to take down the whole fleet — a partial, staged rollout is safer than pushing a change globally and instantly. The edge should always retain its last-known-good state so it can keep serving correctly if a new push turns out to be bad, and rollback needs to be fast rather than requiring a full redeploy cycle.

Certificates need stricter handling

Certificates and key material deserve tighter controls than ordinary cache-rule config — how exactly that secure storage and access control works is deployment-specific and out of scope here, but the general principle is that sensitive key material should never be treated as casually as, say, a TTL override.

Video is one of the workloads that makes CDN architecture obvious: the files are large, the audience is global, and demand spikes are brutal.

Manifest and bitrate ladder

Video is packaged into a manifest plus multiple bitrate renditions. The player fetches the manifest, measures conditions, and chooses segment qualities dynamically. This works well with CDNs because small segments are easier to cache and route than one giant file.

Segment caching

Popular segments — intro sequences, common timestamps, and hot live-event windows — become natural CDN objects. The edge can cache them independently, and shields can coalesce misses when many viewers land on the same point in a stream.

Range requests and large-object handling

For downloads or partially cached assets, the CDN may need to honor byte-range requests. Large-object support affects cache admission, SSD layout, readahead policy, and whether partially fetched objects can still be reused efficiently.

Cache keys and range requests for segments

Each segment and rendition typically needs its own position in the cache key — a 480p segment and a 1080p segment of the same moment in a video are different cached objects. Range requests add another dimension: an edge can choose to cache full objects and serve ranges out of them, or treat partial-object caching as an explicit implementation choice, but either way it is a design decision rather than universal built-in behavior.

Admission and eviction pressure from large objects

Large segments and downloads compete for the same cache capacity as everything else. This is exactly where the admission policy discussed earlier matters: a huge one-time object should not automatically displace a large volume of smaller, genuinely hot content.

First byte vs sustained throughput

Video has two distinct latency goals that can pull in different directions: time-to-first-byte (how fast playback can start) and sustained throughput (whether the stream can keep up without rebuffering once playing). A design that only optimizes the first can still fail the viewer during the second.

Live-event hot windows

During a live event, a narrow window of segments becomes extremely hot simultaneously across a huge audience, then goes cold again moments later as the stream moves forward. This is a short-lived, extreme version of the hot-object problem, and it stresses request coalescing and shield capacity specifically around the live edge of the stream.

The edge is where the CDN decides who should receive bytes, who should be challenged, and who should be dropped entirely.

Signed URL validation

The edge verifies token expiry, signature, path scope, and sometimes user or IP binding before serving protected assets. This keeps premium media or licensed downloads from being scraped simply because the bytes are globally cached.

WAF rule evaluation

WAF rules can inspect method, path, headers, query parameters, body patterns, bot signals, and rate patterns. Because the logic runs close to ingress, bad requests can be blocked before they consume shield or origin resources.

Volumetric absorption and rate limiting

A globally distributed edge has much more aggregate ingress capacity than a single origin region. That is why it is effective for DDoS scrubbing: the attack is diffused across many PoPs, filtered locally with rate limiting at ingress, and only clean traffic continues deeper into the stack.

Origin hiding as a security control, not just a performance one

Keeping origin unreachable from the open internet — the same mechanism discussed under Origin Architecture and Selection — is also a security control: it removes an entire class of attacks that would otherwise target origin directly and bypass edge protections altogether.

Per-tenant quotas and rollout safety

Because many customers share the same edge infrastructure, per-tenant/per-customer quotas keep one customer's traffic or attack surface from degrading service for everyone else. Security rule changes need the same staged-rollout discipline as any other config — pushing a new WAF rule globally and instantly during active incident response is itself a risk, not just a fix.

Cache deception and key/routing rule interaction

Cache-deception style attacks are really a cache-key problem wearing a security hat: if routing or key rules can be tricked into treating a sensitive response as publicly cacheable, the result is effectively the same as the cache-poisoning failure discussed earlier — the fix lives in key design and normalization, not in a separate security subsystem.

Volumetric vs application-layer attacks

A volumetric or connection-exhaustion flood is a different problem from an application-layer flood of individually valid but expensive requests. The distributed edge surface absorbs the former well simply by being large; the latter needs request-level inspection and rate/behavioral controls rather than raw capacity.

A production CDN runs many customers on shared infrastructure, which means one customer's behavior can affect another's experience unless the system actively prevents it.

What needs isolating

Concretely: per-tenant bandwidth and concurrency controls, fairness over shared cache capacity, isolation of WAF/compute cost between tenants, per-tenant origin connection budgets, and the ability to isolate a suddenly hot customer so its load does not spill into everyone else's. Priority or reserved capacity can be layered on top where the business model calls for it.

Global health can hide a local problem

A fleet can look entirely healthy in aggregate — normal CPU, normal bandwidth, normal error rates — while one tenant or one PoP is quietly exhausting a specific shared bottleneck (a connection pool, a rate-limit budget, a cache-capacity allocation) that other tenants happen to depend on too. Per-tenant metrics, not just fleet-wide ones, are what surfaces this.