Skip to content
SeeItFlow
System Design · Resilience · ~12 min read

Netflix: Failure & Scale Scenarios

At global scale, failures are not edge cases — they are routine operational events. This section walks through the most important failure modes and how a well-designed streaming architecture handles each.

Mangesh MandlikLast updated:
Scenario 01

CDN Edge Node Failure

A CDN edge node becomes unavailable. What happens to users routing through it?

CDN edges are stateless caches, so a failed edge holds no unique data — see the CDN Architecture deep dive for how caching and Open Connect work.

CDN networks use health checks — either DNS-based or anycast routing — to detect node failures, typically within seconds to tens of seconds depending on probe intervals. Traffic is automatically redirected to the next nearest available edge. Because CDN edges hold only cached content (no unique state), no data is lost. The only impact is higher latency for affected users until they are rerouted.

Why CDN is stateless

Each edge caches content that was fetched from origin. That content is reproducible — any edge can fill its cache from origin. An edge failure means no data loss, only a cache warmup period after rerouting.

Scenario 02

Regional Failure

An entire cloud region — including origin storage replicas and control-plane services — becomes unavailable.

Swipe to pan
Regional Failure — Traffic automatically reroutesUS UserEU UserUS CDN⛔ DOWNEU CDNabsorbing ↑Originmulti-regionControl Planererouting DNSblockedreroutedUS users rerouted to EU edge · Higher latency, but streams continue · No data loss
Regional failure: DNS is updated to point to alternate CDN edges once cached answers expire — affected users see higher latency, not an outage

The control plane must be multi-region to survive this scenario. When a region fails, DNS routing is updated (by DNS and health-check services that do not depend on the failed region) to direct traffic to the next available region. Resolvers and clients that cached the old answer keep using it until its TTL expires, so recovery is not instantaneous and depends on the TTL. Origin storage must also be replicated across regions (or at minimum have recent backups available in other regions).

CDN edges in unaffected regions continue serving cached content — most users in those regions experience no disruption. Users in the affected region are redirected to farther CDN edges at higher latency until the region recovers.

Key takeaways
  1. 01Multi-region control plane is required for regional resilience
  2. 02CDN edges in unaffected regions continue operating independently
  3. 03Users in the failed region see higher latency, not a complete outage
  4. 04Recovery: the region comes back online and CDN traffic gradually shifts back
Scenario 03

Origin Overload

A massive traffic spike drives origin request rates beyond its capacity.

This scenario is unlikely with a well-configured CDN and healthy cache hit rates. If the cache hit rate is high (for example an assumed 99%, illustrative), origin requests are about 1% of total edge segment requests — manageable even during the largest launches.

If it does happen — during a cold cache scenario (new CDN region brought online) or after a full cache flush — the CDN hits the same cache stampede problem described in general caching theory: many concurrent requests for the same missing key, all falling through to origin at once. The fix is request coalescing: if 1,000 concurrent requests arrive for the same segment, collapse them into a single origin request. Only one request goes to origin; the response is fanned out to all 1,000 waiting clients.

Scenario 04

Network Degradation at Scale

A major network congestion event reduces available bandwidth for millions of users simultaneously.

Swipe to pan
Adaptive Bitrate Streaming — Quality follows the network1080p720p480p1080pStrong WiFiMediumWeak 3GWiFi recoveredAt each 2-4 second segment boundary, the player measures download speed and selects thehighest quality rendition it can download faster than it plays. No buffering — quality adapts.Trade-off: Accept visible quality reduction to prevent playback interruption. Always the right call.
Network degradation handled automatically — players switch to lower quality renditions at segment boundaries, no buffering

Adaptive bitrate streaming handles this gracefully. Every affected player measures reduced download speeds and switches to lower quality renditions. Quality drops across the board — from 1080p to 480p or lower — but playback continues without buffering.

The architecture has no server-side change to make. ABR is entirely player-driven. When congestion clears, players detect improved bandwidth and gradually ramp quality back up.