Walkthrough: Video Platform

"Design YouTube" is really two designs stapled together: an upload pipeline (get a huge file in, process it) and a serving path (play it smoothly anywhere). Keep them separate in your head and the question becomes manageable.

Step 1: Requirements

Functional:

1. Users can upload videos (assume up to tens of GB).
2. Users can watch videos with smooth playback on varying network speeds.
3. Users can view video metadata (title, views) — search is out of scope; confirm that.

Non-functional:

• Uploads are resumable — a dropped connection at 90% must not restart from zero.
• Playback starts fast and adapts to bandwidth (no endless buffering on mobile).
• Scale: ~1M uploads/day (≈12/sec) and 100M views/day (≈1,200 QPS average) — heavily read-skewed.

Step 2: Core entities

User, Video (the raw and transcoded bytes — lives in blob storage), VideoMetadata (title, status, manifest URL — lives in the database). That split — metadata in the DB, bytes in the blob store — is the load-bearing decision of this whole question.

Step 3: API

POST /v1/videos                   -> { video_id, presigned_upload_urls[] }
GET  /v1/videos/:id               -> { metadata, manifest_url }

Step 4: High-level design — the upload path

The most common mid-level mistake here: routing file bytes through your API servers. A 10GB upload through an app server ties up memory and connections, doubles bandwidth (in, then out to storage), and times out. Instead:

1. Client calls POST /videos. The app server creates a metadata row (status: uploading) and returns presigned URLs — temporary, scoped links that let the client upload directly to S3 (blob storage: cheap, durable to 11 nines).
2. The client uploads via multipart upload in 5–10MB chunks, in parallel.
3. When the upload completes, an S3 event notification pings the backend — no polling.

The bytes never touch your servers. This pattern is explicitly expected at mid-level.

Step 5: The processing pipeline

Raw uploads can't be served as-is. On the S3 event, an orchestrator kicks off a processing DAG (a dependency graph of jobs):

1. Split the video into few-second segments.
2. Transcode each segment into a ladder of formats — multiple resolutions/bitrates (1080p, 720p, 480p...) and codecs. Every segment × format job is independent, so this is embarrassingly parallel: throw it at a queue with a worker pool and it scales linearly.
3. Generate manifests — index files listing every segment at every quality level.
4. Mark the metadata row status: ready.

Queues between stages absorb the 12-uploads/sec average and any spike, and a failed job retries without redoing the whole video.

Step 6: The serving path — adaptive bitrate streaming

Why segments at all, instead of downloading the file? Three reasons interviewers fish for: fast startup (play segment 1 while fetching segment 2), seeking (jump to minute 40 = fetch that segment, not 40 minutes of file), and adaptive quality.

Playback uses HLS/DASH (the two standard streaming protocols): the player fetches the manifest, then requests segments one at a time, choosing the bitrate per segment based on measured bandwidth. Wi-Fi drops to 3G mid-video? The player simply requests the next segment at 480p. No server smarts needed — the client adapts.

Both segments and manifests are served from a CDN (edge servers caching content near users). Video is the perfect CDN workload: large, immutable, read 100M times/day. The origin (S3) is only touched on cache misses.

Deep dive 1: Resumable uploads

Track per-chunk state in the metadata DB: each chunk gets a fingerprint (SHA-256 hash) and a status. On reconnect, the client asks "which chunks do you have?", compares, and uploads only the gaps. S3 returns an ETag (a content hash) per uploaded part, which the backend verifies against the fingerprint — corruption caught per-chunk, not per-file.

Deep dive 2: Hot video metadata

A viral video sends millions of players to GET /videos/:id at once — a thundering herd on one row. Cache metadata in a distributed LRU cache (evicts least-recently-used entries when full), and for the very hottest IDs, replicate the entry across all cache nodes so no single shard melts.

A common tangent: view counting. Incrementing a DB counter 100M times/day per hot video is needless write load — batch increments in Redis and flush periodically; approximate is fine here (say that consistency call out loud).

Wrap-up

Recap: uploads are direct-to-S3, chunked, fingerprinted, and resumable; processing is a parallel transcode DAG fed by queues; playback meets the smooth-anywhere requirement via CDN-served HLS segments with client-side bitrate adaptation. With more time: search, recommendations, and regional pre-positioning of hot content.