Trade-offs & Decisions
Every meaningful architectural choice in this design, revisited as a standalone decision: the problem it responds to, the realistic alternatives, their advantages and disadvantages, what was chosen, and — most importantly — why it fits this specific workload rather than being universally correct.
Application Server vs. Direct Object-Storage Upload
Should large video uploads pass through the application server, or go directly to object storage?
Through the application server
Advantages: simple mental model; the server can inspect every byte as it arrives, which can matter for validation or content scanning done inline. Disadvantages: consumes app-tier bandwidth and connections for pure data transfer; couples upload scaling to business-logic-server scaling.
Direct-to-object-storage (chosen)
Advantages: app servers stay lightweight and only issue credentials; object storage is purpose-built for this bandwidth; upload and application scaling become independent. Disadvantages: requires a short-lived, scoped upload credential and more orchestration; any inline inspection/validation the app tier would have done in-path now has to happen at a different point (e.g. at finalize, or asynchronously after).
Why it fits: upload volume and per-file size are both large enough that app-tier bandwidth becomes a real, avoidable cost, and the orchestration/security cost of scoped credentials is modest by comparison. At very low upload volume, this decision matters much less.
Synchronous vs. Asynchronous Processing
Should transcoding happen inside the upload request, or after it, off to the side?
Synchronous
Advantages: no job-tracking infrastructure needed; simpler mental model. Disadvantages: upload latency scales with video length/complexity; CPU-bound transcoding blocks an otherwise I/O-bound service; one slow job can starve unrelated requests.
Asynchronous via a durable handoff (chosen)
Advantages: the upload path acknowledges as soon as the upload is accepted/finalized — not once the video is fully ready; processing workers scale independently; retryable and trackable. Disadvantages: video isn't immediately playable in all qualities; requires job-status tracking and a creator-facing 'processing' state.
Why it fits: transcoding is CPU-intensive and duration-variable — exactly the kind of work that shouldn't share a request/response path with anything latency-sensitive.
Database vs. Object Storage for Video Binaries
Where should the actual video bytes — originals, renditions, segments, thumbnails — live?
Primary database
Advantages: one system to operate; can be a reasonable choice for small media blobs in a small prototype. Disadvantages: relational databases are not built for multi-gigabyte binary blobs at real video-platform scale; every read/write competes with transactional metadata traffic; far more expensive per gigabyte.
Object storage (chosen)
Advantages: durable via replication; highly scalable; dramatically cheaper per gigabyte; purpose-built for large binary objects. Disadvantages: no relational queries — the database still needs pointers/metadata to make objects discoverable.
Why it fits: for a video platform's actual object sizes and volume, object storage is the appropriate architecture — small prototypes storing a handful of small media blobs directly in a database aren't wrong for their scale, but that approach does not carry forward as the platform grows into real video-sized binaries and real upload volume.
Origin-Only Delivery vs. CDN
Should viewers stream directly from origin, or from CDN edge locations?
Origin only
Advantages: simplest possible setup, no caching layer to reason about. Disadvantages: origin bandwidth becomes the bottleneck under real viewer volume; high latency for viewers far from origin; no resilience to a video suddenly going viral.
CDN in front of origin (chosen)
Advantages: popular content served from edges near viewers; massively reduced origin traffic; lower latency; far higher aggregate throughput. Disadvantages: cache misses still reach further upstream; cache invalidation and freshness need to be reasoned about; additional infrastructure and cost.
Origin shield changes the failure calculus
A CDN alone still concentrates every edge's misses onto origin. Adding an origin shield/tiered cache changes what 'CDN failure' means for origin protection — see Direct Origin vs. Origin Shield below.
Why it fits: at large, globally distributed viewer scale, a CDN isn't an optimization — it's the difference between an architecture that scales and one that doesn't.
Whole-File Streaming vs. Segmented Streaming
Should the player download one large rendition file, or many small time-sliced segments?
Whole-file streaming
Advantages: simpler to produce (no segmentation step). Disadvantages: poor buffering behavior and clumsy seeking compared to segmented delivery, and it can't support adaptive bitrate switching mid-playback.
Segmented streaming (chosen)
Advantages: efficient buffering, fast seeking to any timestamp, partial retrieval, resilient recovery from a dropped segment, and it enables adaptive bitrate switching between segments. Disadvantages: more files to produce, store, and cache-key; slightly more complex manifest/playlist logic.
Why it fits: every one of segmentation's real benefits — ABR, cache granularity, seeking, and failure isolation — is directly required by the non-functional requirements around startup latency and playback resilience.
Fixed Quality vs. Adaptive Bitrate
Should the player request one fixed rendition for the whole video, or switch dynamically?
Fixed quality
Advantages: simplest player logic. Disadvantages: stalls playback on weak/variable networks; wastes bandwidth on strong ones; can't respond to changing conditions mid-playback.
Adaptive bitrate (chosen)
Advantages: the client picks a sustainable quality per segment based on real-time bandwidth and buffer measurements, smoothing playback across a wide range of network conditions. Disadvantages: requires the multi-resolution rendition ladder and aligned segmentation to already exist; more complex client-side logic.
Why it fits: low startup latency and a globally diverse audience on wildly different networks make fixed quality a poor fit almost everywhere except a controlled, uniformly high-bandwidth environment.
Fixed Worker Pool vs. Independently Scalable Workers
Should the transcoding worker fleet be a fixed size, or scale dynamically with demand?
Fixed pool
Advantages: predictable capacity and cost. Disadvantages: a burst of uploads creates an unbounded backlog with no automatic relief; over-provisioning for peak load wastes money the rest of the time.
Independently, dynamically scalable workers (chosen)
Advantages: worker count tracks demand automatically; backlog drains without manual intervention; cost scales with actual load. Disadvantages: autoscaling has real startup delay (new capacity isn't instantly available); queue depth alone can be a misleading signal (see Architecture Deep Dive -> Processing Backpressure); worker capacity can be constrained by CPU/GPU/codec-specific hardware rather than scaling freely; a backlog can still grow even at maximum practical worker count during a genuine capacity-exceeding spike.
Why it fits: upload volume is inherently bursty, and processing is a stateless, queue-driven workload well suited to autoscaling — but the autoscaling signal needs to be work-weighted, and the design needs an honest answer for what happens when even elastic capacity is exceeded, not an assumption that scaling always keeps up.
Database Queries vs. a Dedicated Search Index
Should video search query the metadata database directly, or a purpose-built index?
Database queries
Advantages: no additional system to operate; relational databases can support full-text search reasonably well at moderate scale. Disadvantages: ranking sophistication, query-pattern flexibility, and scale/isolation from transactional traffic all become harder as requirements grow.
Dedicated search index (chosen)
Advantages: purpose-built for text search and relevance ranking; isolates search load entirely from metadata writes/reads. Disadvantages: the index lags metadata by an indexing delay, and it's another system to keep in sync and operate.
Why it fits: once the catalog and query sophistication justify richer ranking, query flexibility, and load isolation, a dedicated index earns its cost — see Search & Autocomplete for the indexing/ranking mechanics themselves, which this module deliberately does not re-teach.
Synchronous Counter vs. Asynchronous Aggregation
Should every view synchronously increment a database row, or flow through an aggregation pipeline?
Synchronous per-view write
Advantages: lower lag between a view happening and the count reflecting it — closer to a direct update. Disadvantages: severe write contention on a single row for any popular video; under concurrency and failure, 'exactly current' is a stronger claim than a synchronous counter actually delivers once retries, failures, and concurrent writers are accounted for.
Asynchronous aggregation (chosen)
Advantages: batches many increments into a handful of writes; scales independently of raw view rate; naturally absorbs bursts. Disadvantages: the displayed count lags reality by an aggregation window; requires bounded, identity-based de-duplication of view events, which suppresses duplicates within its retained window without being a guarantee of exact lifetime counts.
Why it fits: a view count that's briefly stale is imperceptible to users, while row-level write contention under a real traffic spike is a genuine outage risk. See Exact vs. Approximate View Count below for how to decide which guarantee a given use case actually needs.
Single Region vs. Multi-Region + CDN
Is a CDN alone sufficient for global delivery, or does the platform also need multi-region origin infrastructure?
Single region + CDN
Advantages: simpler to operate; CDN already solves most delivery-latency pain for cached content. Disadvantages: origin region failure risks a global outage for anything not already cached; distant creators pay real upload/processing latency and cross-region transfer cost.
Multi-region origin + CDN (requirement-dependent)
Advantages: origin availability and disaster recovery; upload/processing locality; independent regional scaling and failure isolation. Disadvantages: requires an explicit metadata write-ownership and replication model (see Architecture Deep Dive -> Multi-Region Ownership) and fencing before failover — without these, multi-region can introduce conflicting-write risk rather than removing risk; meaningfully higher operational complexity.
Why it's requirement-dependent: at moderate scale with a concentrated user base, a CDN alone is usually sufficient and multi-region origin infrastructure is premature complexity. At true global scale with strict availability targets, it becomes necessary — but only once the ownership/replication/fencing model is actually designed, not as a default 'add more regions' move.
Monolithic Job vs. Processing DAG
Should one processing job represent 'transcode this video,' or should the pipeline be modeled as a graph of independent, trackable stages?
One job per video
Advantages: simplest possible job model; nothing to orchestrate between stages. Disadvantages: a failure anywhere (one bad rendition, a thumbnail bug) looks the same as total failure; no parallelism between independent stages; retrying means redoing everything, including work that already succeeded.
Stage/rendition DAG (chosen)
Advantages: independent stages (and often independent renditions) can run in parallel; partial failure is precisely diagnosable and partially retryable; per-stage state gives real visibility into where a video actually is in processing. Disadvantages: more orchestration complexity; needs a scheduler/workflow layer capable of expressing dependencies between stages.
Why it fits: as the rendition ladder and pipeline stages grow (multiple representations, audio packaging, thumbnails, validation), the cost of coarse, undiagnosable all-or-nothing failures grows with it — a DAG model pays for itself once there's more than a trivial number of independent stages.
Precompute vs. On-Demand Renditions
Should every rendition in the ladder be produced at upload time, or should some be generated lazily, on first request?
Precompute everything at upload
Advantages: every rendition is ready the moment processing completes; no first-viewer latency penalty; simpler playback-time logic (nothing to trigger on demand). Disadvantages: pays full compute and storage cost for every rendition even if some are rarely or never requested — a real cost for the long tail of content.
On-demand / lazy generation for some renditions
Advantages: avoids compute/storage cost for renditions that turn out to be unpopular. Disadvantages: the first viewer requesting a not-yet-generated rendition pays real added latency; adds real implementation complexity (a generation-on-miss path, coordination with the safe-publication boundary, and cache-ability of a rendition that didn't exist a moment ago).
Why it's a trade-off, not a default: lazy generation is a legitimate cost lever at extreme long-tail scale, but it is not a universal best practice — the first-view latency cost and added implementation complexity need to be worth it for the specific content population in question.
Single Codec vs. Multi-Codec
Should the platform standardize on one codec, or produce renditions in more than one?
Single codec
Advantages: simpler pipeline, less compute/storage per video, fewer compatibility edge cases to test. Disadvantages: risks poor compatibility or poor efficiency on some portion of the device/browser population, depending which single codec is chosen.
Multi-codec
Advantages: can serve each device population its best-supported, most efficient option. Disadvantages: multiplies compute and storage cost per additional codec; more combinations to validate and keep aligned in the rendition ladder.
Why it's a trade-off: this module does not prescribe one specific codec as universally correct — the right choice depends on the target device/browser population's decode support and the platform's tolerance for the added compute/storage cost of supporting more than one.
Mutable vs. Versioned Media Objects
Should a reprocess overwrite the existing rendition objects, or publish a new, separately-versioned set?
Mutate in place
Advantages: no extra storage held for superseded versions. Disadvantages: unsafe under a long-lived CDN cache key — a viewer or edge that already cached the old bytes under that key can end up with an inconsistent view, and there's no clean way to guarantee every reader has moved to the new bytes atomically.
Immutable, versioned objects (chosen)
Advantages: safe under long-TTL CDN caching, since a version's bytes genuinely never change once published; supports the safe-publication and blue/green-style reprocessing patterns described in Architecture Deep Dive; makes rollback to a prior known-good version simple. Disadvantages: superseded versions need an explicit garbage-collection policy so storage doesn't grow unbounded.
Why it fits: given that segments are cached at CDN edges for potentially long TTLs, mutating bytes behind an already-cached key is close to unworkable at scale — immutable/versioned outputs are what make aggressive CDN caching safe in the first place.
Direct Origin Access vs. Origin Shield
Should every CDN edge miss go straight to origin, or through an intermediate shield/tiered cache?
Direct origin access from every edge
Advantages: one fewer layer to operate. Disadvantages: a viral or newly-published video creates independent cold misses at every edge worldwide, each hitting origin directly — origin sees miss amplification proportional to the number of edges, not the number of distinct objects.
Origin shield / tiered cache (chosen)
Advantages: collapses cross-edge miss fan-in — many edges' misses resolve against the shield instead of each independently reaching origin; meaningfully reduces peak origin load during a cold, suddenly-popular object. Disadvantages: another caching layer to operate and reason about; adds a hop for a genuine shield-miss.
Why it fits: without a shield, a single viral video's cold start can multiply origin load by roughly the number of edges that each independently miss — exactly the scenario the shield layer exists to prevent.
TTL-Based vs. Versioned Cache Keys
For immutable media objects, should caching rely on a TTL expiring an otherwise-fixed key, or on the object's version being part of the key itself?
TTL-based caching of a fixed key
Advantages: familiar, simple caching model. Disadvantages: requires actively invalidating or waiting out the TTL whenever the underlying bytes change — awkward for content that's supposed to be genuinely immutable, and risky if invalidation is missed (see Failure & Scale -> Stale Manifest).
Versioned cache keys (chosen for segments/finalized renditions)
Advantages: because the key itself encodes the version, there's never a 'stale vs fresh' ambiguity for a given key — old versions simply have different keys, and the object behind any given key genuinely never changes. This removes the need for invalidation entirely for that object, in favor of very long TTLs. Disadvantages: superseded-version objects need their own garbage-collection story (see Storage Lifecycle) since old keys can't just be 'refreshed' — they're abandoned, not updated.
Why it fits: immutable content and versioned keys are a natural pair — reducing invalidation complexity to a garbage-collection problem is a meaningfully simpler operational story than actively invalidating a mutable key everywhere it's cached.
Public Origin vs. Signed Playback Delivery
For restricted (private/unlisted/subscriber-only) content, should delivery be openly accessible, or gated by a signed credential?
Public/open delivery
Advantages: simplest possible CDN/origin configuration; fine for genuinely public content. Disadvantages: cannot enforce any access restriction — anyone with a URL can retrieve the content indefinitely, regardless of what the metadata layer says about visibility.
Signed URL / cookie / token delivery (chosen for restricted content)
Advantages: the delivery layer itself enforces a short-lived, scoped credential, so the CDN/origin can't be used to bypass the authorization decision made by the metadata/auth service. Disadvantages: added issuance/validation complexity; cache-key design must account for the credential without fragmenting the cache (see Architecture Deep Dive -> CDN Cache Key).
Why it fits: for public content, the simplicity of open delivery is a reasonable and common choice. For anything restricted, delivery-layer enforcement is what actually makes 'private' mean something — otherwise the CDN cache itself becomes an unintentional authorization bypass.
Exact vs. Approximate View Count
Does the product actually need an exact lifetime view count, or is an approximate, eventually-consistent number good enough?
Exact counting
Advantages: defensible, auditable number — necessary wherever the count has financial or contractual consequences (this module treats billing/financial counters as a stricter, separate concern, out of scope). Disadvantages: requires durable, globally-scoped deduplication and much stronger consistency guarantees than a simple bounded dedupe window provides — meaningfully more engineering investment.
Approximate, eventually-consistent counting (chosen for the user-facing display count)
Advantages: bounded, identity-based dedupe within an aggregation window is simple, cheap, and good enough for a popularity signal users glance at. Disadvantages: does not guarantee an exact lifetime count — a sufficiently delayed duplicate or a dedupe-state loss can still cause a small discrepancy.
Why it's tied to the requirement, not a fixed choice: the user-facing view count is a good fit for approximate semantics; anything with financial/contractual stakes needs a stricter design entirely, not a stretched version of the same aggregation pipeline.
Replicate All Content vs. Selective Replication
Should every video's original and renditions replicate to every region, or only some?
Replicate everything, everywhere
Advantages: uniformly fast regional access and uniform disaster-recovery posture for every video. Disadvantages: multiplies storage and cross-region transfer cost by the number of regions, for content that may never actually be requested outside its home region.
Selective replication (chosen as the general approach)
Advantages: replication follows durability requirements, data-residency constraints, actual popularity, and cost — long-tail content stays near its origin region, popular content earns broader replication. Disadvantages: a cache miss for content that isn't replicated to the requesting region has to reach back to a region that does have it, which can mean higher latency or, in a regional-failure scenario, real unavailability if the only copy is in the failed region.
Why it fits: replication cost scales with region count, while most content's actual demand does not spread evenly across regions — selective replication aligns cost with actual access patterns rather than a uniform worst-case assumption.
Single CDN vs. Multi-CDN
Should the platform depend on one CDN provider, or run more than one for availability/cost reasons?
Single CDN
Advantages: simplest operational model — one cache-key scheme, one set of provider-specific behaviors to reason about. Disadvantages: a provider-wide CDN outage removes the platform's entire edge-caching layer at once (see Failure & Scale -> CDN Provider Outage).
Multi-CDN (advanced option, not baseline)
Advantages: can route around a single provider's outage or regional weak spot; can optimize cost/performance per region or per provider's strengths. Disadvantages: meaningfully higher operational complexity — cache-key behavior, invalidation, and origin-protection controls all need to be consistent (or deliberately reconciled) across providers; genuinely harder to reason about and test.
Why it's advanced-only: multi-CDN meaningfully improves availability and can improve cost/performance, but the operational complexity is real — most platforms should treat this as a scale/availability upgrade to reach for once single-CDN limitations are actually measured, not as a starting-point default.