Global CDN — Trade-offs & Decisions

Content changes, but edge caches want stability. How should the CDN decide when an object is no longer safe to serve?

Problem

We need a freshness model for cached objects. The CDN can let objects age out naturally by TTL, or it can support explicit purge so applications can force replacement immediately.

Option A — TTL-based expiration

Cheap, decentralized, and easy for the edge to enforce. Every PoP can answer freshness locally. The downside is that bad or outdated content can linger until TTL expiry unless you set very short TTLs, which hurts hit ratio.

Option B — Explicit invalidation / purge

Gives operators fast control over stale content, but turns freshness into a global control-plane fan-out problem. Purge traffic, acknowledgement tracking, and propagation lag all become first-class concerns.

Decision

Why

TTL keeps the hot path simple and preserves cache efficiency. Purge exists for operational correctness when content changes out-of-band or under incident pressure.

Trade-offs

Long TTLs improve offload but widen stale windows unless paired with revalidation. Frequent purge improves freshness control but adds control-plane fan-out load, acknowledgement-tracking overhead, and propagation-lag risk across thousands of PoPs.

The CDN must choose how clients reach a nearby edge location in the first place.

Problem

We need a global front door that sends users toward low-latency, healthy Edge PoPs and can reroute around failures quickly.

Option A — DNS-based Geo-Routing

Simple mental model, easier debugging, and good enough for smaller footprints. But DNS answers stick in recursive resolvers and client caches, so reroute speed is bounded by TTL behavior and resolver geography accuracy.

Option B — Anycast Routing

Many PoPs announce the same IP, so BGP steers clients toward a nearby path and reacts faster to route withdrawals. Operationally stronger at scale, but requires networking maturity and careful traffic engineering.

Decision

Why

DNS gets the product launched without deep network machinery. Anycast becomes compelling once the CDN is large enough that packet-time steering and fast route withdrawal provide clear operational value — but Anycast's route selection is topology/policy-based, not a guarantee of the geographically nearest or lowest-latency path, which is why the two approaches are frequently combined rather than sequenced as a strict replacement.

Trade-offs

DNS is easier but stickier, since answers are cached by resolvers and clients. Anycast reacts faster at the routing layer, but is harder to operate correctly because BGP path preference is influenced by the wider internet's routing policy, not only your application's latency goals.

Should content be proactively copied into the CDN, or should the edge fetch it lazily on the first miss?

Problem

We need a refill strategy for cached content. Some workloads are predictable enough to pre-seed globally; others are better fetched only when actually requested.

Option A — Push CDN

In practice, "push" usually means explicit pre-positioning or pre-warming of specific objects ahead of expected demand — not that every edge permanently stores every object. This is attractive for planned software releases or a known video premiere, but operationally expensive if the predicted hotspots are wrong.

Option B — Pull CDN

Edges fetch on the first Cache Miss and then keep the object warm as demand proves itself. This is simpler and naturally adapts to real traffic, which is why it is the default for most general web CDNs.

Decision

Why

Most objects never become globally hot enough to justify proactive distribution. Pull keeps storage focused on observed demand and pairs naturally with shield caches.

Trade-offs

Pull pays a cold-start penalty on the first request. Push reduces that penalty for known hotspots, but wastes bandwidth and storage when demand forecasts are wrong.

Should the CDN rely only on local edge caches, or introduce a second cache tier between edges and origins?

Problem

A globally distributed edge gives low latency, but each PoP sees only local demand. We need to decide whether a second cache layer is worth the extra hop on misses.

Option A — Edge cache only

Simpler request path and fewer moving parts. Good for small footprints or tolerant origins, but synchronized expirations and long-tail objects can still hammer origin because each edge behaves independently.

Option B — Edge cache + Regional/Shield Cache

Adds another tier and another operational surface, but dramatically improves long-tail hit ratio, request coalescing, and origin protection.

Decision

Why

The edge optimizes user latency; the shield optimizes origin efficiency. Those are related goals, but not identical ones.

Trade-offs

The shield adds complexity and a miss-time hop. In return it prevents origin traffic from scaling linearly with the number of edge locations.

Edge caches need speed and size, but RAM and SSD optimize different parts of that trade-off.

Problem

We need to decide what storage medium should back the edge cache. Pure RAM is fast but small and expensive; pure SSD is larger but slower.

Option A — Memory cache only

Excellent latency for tiny hot objects and simple lookup logic. But capacity is limited, and large-object workloads can crowd out everything else or become uneconomical.

Option B — SSD cache only

Much larger working set and better economics for images, downloads, and video segments, but misses the extreme low-latency advantage of RAM for the hottest micro-objects.

Decision

Why

The working set of a CDN is naturally skewed. A small fraction of objects is extremely hot; a larger long tail is worth caching but not worth pinning in RAM.

Trade-offs

Tiering increases implementation complexity and requires good admission/eviction logic. The reward is better hit ratio and better economics than either medium alone.

Purge-sensitive content wants instant global consistency, but globally synchronized control is expensive and slow.

Problem

We need a consistency model for cache invalidation and config rollout specifically — this is narrower than implying all CDN data follows one single "global consistency" model.

Option A — Strong global consistency

In theory it gives a clean mental model, but in practice it couples request or deploy progress to the slowest region, the worst network path, or the least healthy edge.

Option B — Eventual consistency with observability

This keeps the hot path fast and the control plane resilient, at the cost of a visible propagation window rather than an instantaneous global switch.

Decision

Why

The alternative makes every global update hostage to perfect coordination, which is unrealistic across thousands of PoPs.

Trade-offs

Some users may see old content briefly while others see new content. The system must make that window visible and small rather than pretending it never exists.

Some content should be globally cacheable by anyone; some content must be authorized at the edge before a single byte leaves cache.

Problem

We need to decide how the CDN handles protected downloads, premium media, and expiring access grants versus broadly public assets like logos or JS bundles.

Option A — Public assets

Operationally simple and ideal for immutable static content. Anyone who knows the URL can fetch it, and the edge can maximize cacheability because auth is not part of the decision.

Option B — Signed URLs / Token Auth

Signed URLs are one mechanism for this; signed cookies or bearer tokens validated at the edge are others. The underlying architectural decision is public shared access vs edge-authorized access — not necessarily URL signing specifically. Whichever mechanism is used, it adds validation work at the edge and token/key lifecycle management, but lets the CDN serve cached protected objects only to authorized users for a bounded time window.

Decision

Why

The public path stays simple and maximally cacheable, while the protected path keeps access control close to users without forcing every authorized read back to origin.

Trade-offs

Signed URLs increase edge logic and token-key management. Public assets are simpler but inappropriate for paid, private, or revocable content.

The more aggressively you cache, the better the offload. The more carelessly you cache, the more likely you are to serve the wrong bytes.

Problem

We need a policy for mixed workloads: static assets, API responses, personalized HTML, signed media, and rapidly changing metadata.

Option A — Cache everything

Maximizes hit ratio in theory, but quickly runs into correctness and security issues because not all responses are reusable across users or over time.

Option B — Selective caching by content type and rules

Requires explicit policy, but lets the CDN be aggressive where content is immutable or public and conservative where responses depend on identity, cookies, or short-lived state.

Decision

Why

Correctness and security come first. A slightly lower hit ratio is acceptable; globally replaying user-specific content is not.

Trade-offs

Selective caching demands better metadata, rule testing, and cache-key hygiene. The reward is safe offload instead of accidental cross-user leakage.

An object is no longer fresh, or the upstream that would refresh it is slow or unavailable. What should the edge do right now?

Problem

We need a policy for what happens the moment freshness expires and upstream is not immediately able to confirm or refresh the object.

Option A — Block and revalidate

The request waits for a conditional check against upstream before responding. Correct by construction, but the client pays full round-trip latency, and this option is unusable if upstream is genuinely down.

Option B — Stale-while-revalidate

Serve the stale object immediately while one request refreshes it in the background. Hides latency and works well with coalescing, but risks briefly serving outdated content.

Option C — Stale-if-error

If upstream is unhealthy, continue serving stale content for a bounded window rather than failing. Improves availability, but is unsafe for content classes where staleness itself is a correctness or security problem.

Option D — Refuse stale content

Fail closed rather than risk serving anything outdated. Safest for sensitive content, but the least available option under upstream trouble.

Decision

Why

Availability and correctness trade against each other differently depending on what is actually being served. Treating this as one blanket policy for the whole CDN ignores that difference.

Trade-offs

Being permissive about staleness improves availability and hides latency, at the cost of occasionally serving outdated bytes. Being strict improves correctness guarantees, at the cost of failing harder when upstream struggles.

Every dimension added to a cache key improves correctness for varying responses, and fragments the cache a little more.

Problem

We need a cache-key strategy that captures genuine response variation without needlessly multiplying the number of distinct cached copies of essentially the same object.

Option A — URL/path-only key

Maximizes hit ratio and is simple to reason about, but is unsafe the moment any response actually varies by query parameter, header, or auth state — different users or contexts can collide on the same cached entry.

Option B — Selected normalized query/header dimensions

Includes only the query parameters and headers that are known to change the response, normalized (sorted, canonicalized) so equivalent requests still share a key. More correct than path-only, and still hit-ratio-friendly if the selected set is genuinely small.

Option C — Overly broad, request-derived key

Includes many or all query parameters and headers defensively. Reduces the risk of missing a real variation dimension, but can destroy hit ratio and open the door to cache-key-explosion attacks from attacker-controlled high-cardinality inputs.

Decision

Why

Correctness requires capturing real variation; efficiency requires not capturing anything beyond that. An explicit, minimal-but-sufficient key is the only option that satisfies both.

Trade-offs

A minimal explicit key requires ongoing discipline — every new response header or query parameter that changes behavior needs to be deliberately added to policy. The reward is a cache that stays both correct and efficient as the application evolves.