Design a video platform
Design a YouTube-style video platform from scratch: requirements, estimates, direct uploads, a transcoding pipeline, adaptive bitrate streaming through a CDN, and where it breaks.
On this page
The problem: huge files, every kind of screen
Creators upload videos. Millions of people watch them on phones on a train, laptops on hotel Wi-Fi and 4K TVs on fibre. Each viewer expects the video to start in a second or two and keep playing, even when the connection gets worse.
Creators upload once; viewers everywhere watch
- Now a video platform, like YouTube. Creators upload videos; millions of people search, browse and watch them on phones, laptops and TVs, on fast and slow connections.
What makes this design interesting is the size of the data. A single upload can be bigger than every other piece of data a user will ever create, and watching it moves gigabytes. Most of the design is about keeping those bytes on the right path: never through your app servers, rarely from your own storage, mostly from caches near the viewer.
This is a first design. The full case study (linked at the end) goes deeper.
Step 1: pin down the requirements
List the functional and non-functional requirements you would agree on before designing.
Compare with our answer
A strong answer mentions: upload a video with title and description · watch with quality that adapts to the connection · search and browse · views and likes counts · start time under ~2 s and little rebuffering · uploads never lost · a processing delay after upload is acceptable · out of scope: live streaming, comments, recommendations, monetisation.
Functional
- Creators upload videos (up to a few GB) with a title and description.
- Viewers watch on any device; quality adjusts to their connection.
- Viewers search and browse; videos show view and like counts.
Non-functional
- Playback starts in under ~2 seconds, with rare rebuffering.
- Uploaded videos are never lost (high durability).
- A video may take minutes to become watchable after upload.
- Watching must be highly available; view counts can be slightly behind.
Out of scope: live streaming, comments, recommendations, ads.
Step 2: quick estimates
Assume 50 million daily viewers watching 5 videos each, and 500,000 uploads a day averaging 10 minutes.
| Quantity | Working | Result |
|---|---|---|
| Video starts | 250 M ÷ 86,400 s | ≈ 3,000 / s (peaks ~6,000) |
| Uploads | 500,000 ÷ 86,400 s | ≈ 6 / s |
| Storage per upload | original (~1 GB) + about 5 renditions (~1 GB together) | ≈ 2 GB |
| New storage | 500,000 × 2 GB | ≈ 1 PB / day |
| Watch traffic | 250 M views × ~5 min × ~3 Mbit/s ≈ 110 MB each | ≈ 28 PB / day ≈ 2.6 Tbit/s average |
The numbers tell the story. Uploads are rare (6 a second) but each is heavy. Watching is the giant: terabits per second, which no single datacentre should serve directly. Metadata (titles, counts) is tiny by comparison.
Step 3: split metadata from media
Small metadata in a database, huge media in object storage
- Split the problem in two. Metadata, meaning titles, likes and comments, is small and goes in a database. The media, the video files themselves, is huge and goes in object storage.
The first decision is the same as in every media-heavy design: metadata in a database, media in object storage.
| Table | Key | Notes |
|---|---|---|
videos | video_id | owner, title, description, duration, status (uploading / processing / ready / failed), storage prefix |
renditions | (video_id, quality) | 240p … 4K, bitrate, codec, manifest key |
view_counts | video_id | Updated in batches, not per view (see below) |
POST /v1/videos {title, description, size} → {video_id, upload_urls[]}
POST /v1/videos/{id}/complete {part_etags[]}
GET /v1/videos/{id} → {title, counts, manifest_url}
GET /v1/search?q=...&cursor=...
Step 4: build the design
The upload path
Uploads go straight to storage with presigned URLs, in resumable parts
The app server never carries video bytes. It creates the videos row with status = uploading and hands the client presigned URLs (signed, time-limited permission to write one object). The client uploads directly to a "raw" bucket using multipart upload: the file goes up in parts (say 16 MB each), several in parallel, and an interrupted upload resumes from the last finished part instead of starting a 2 GB file again.
Transcoding
Transcoding runs on workers fed by a queue
- Uploaded videos can’t be streamed as they are. They must be transcoded into many versions: 240p for weak connections, 1080p and 4K for big screens. That takes minutes per video, so it runs on workers fed by a queue.
The original file cannot be streamed as it is: it may be in any format, and one 4K file is useless on a phone with a weak signal. Transcoding converts it into a ladder of renditions (versions at different resolutions and bitrates), for example 240p at ~0.4 Mbit/s up to 1080p at ~5 Mbit/s and 4K at ~15–20 Mbit/s.
When the upload completes, an event puts a job on a queue. Transcoding is CPU-heavy (a rough rule: several CPU-minutes per minute of video for a full ladder), so 500,000 ten-minute videos a day keep thousands of cores busy. It is done as a pipeline:
Split into chunks, encode in parallel, then package
- Split the original into chunks of a few seconds.
- Encode every chunk into every rendition, in parallel across many workers. A 10-minute video finishes in about a minute instead of an hour, and a failed chunk is retried alone.
- Package the results as short segments (typically 2–6 s) plus a manifest: a small playlist file listing every rendition and its segments. The two standard formats are HLS (
.m3u8playlists) and MPEG-DASH (.mpd). - Write renditions to storage, set
status = ready, and index the metadata for search.
The watch path
Viewers stream from a nearby CDN, and quality adapts to their connection
- Viewers stream from a CDN, close to where they live. The player watches your connection and switches between versions as it speeds up or slows down: adaptive bitrate streaming. That’s why a video drops to blurry instead of stopping.
The player fetches the manifest, then requests segments one at a time from a CDN. After each segment it measures its download speed and how many seconds are buffered, and picks the next segment's quality: adaptive bitrate streaming (ABR). When the train enters a tunnel, quality drops for a few segments instead of the video stopping. Each segment is a normal, cacheable HTTP file, which is exactly what CDNs are good at.
At 2.6 Tbit/s, the CDN is not an optimisation, it is the design. With a 95% edge hit rate, only about 130 Gbit/s reaches the origin storage, and popular videos are almost never read from it.
The browse path, and the whole picture
Three paths: upload through a queue and workers, watch through a CDN, browse through app servers, a cache and search
- Finding videos needs a search index, fed from the metadata. Popular video pages are cached. Putting it together: an upload path through a queue and workers, a watch path through a CDN, and a browse path through app servers, a cache and search.
Video pages (title, counts, channel) are served by stateless app servers from the metadata database, with caching in Redis for popular videos. A search index is fed from metadata changes. Thumbnails come from object storage via the CDN, like everything else heavy.
| Path | Volume | Components |
|---|---|---|
| Upload | ~6/s, GBs each | Presigned multipart upload → raw bucket → event → queue → transcoding workers → renditions bucket |
| Watch | ~3,000 starts/s, Tbit/s of segments | Player → CDN edge → origin storage |
| Browse | thousands of page views and searches/s | Load balancer → app servers → cache → metadata DB; search index |
Key decisions
How many renditions, and when?
| Option | How | Good at | Weak at |
|---|---|---|---|
| Full ladder up front | Encode every rendition at upload | Every video plays well everywhere immediately | Wasted compute and storage: most videos get few views |
| Basic first, more later | Encode a few common renditions (e.g. 360p, 720p) quickly; add 4K and better codecs when a video gets popular | Fast to publish; spend compute where views are | Two-stage pipeline; popular videos briefly lack top quality |
| On demand (just-in-time) | Package or transcode when first requested, then cache | Minimal storage for the long tail | First viewer waits; risky if many first views happen at once |
Choice: basic first, more later. Views follow a long tail: a small share of videos gets most of the watching, so it pays to spend extra compute (higher resolutions, more efficient codecs like AV1, which save bandwidth but cost more to encode) only on those.
How is video delivered?
| Option | Good at | Weak at |
|---|---|---|
| One progressive MP4 file | Simple; any player | One fixed quality: stalls on slow links, wastes bandwidth on small screens |
| ABR with HLS/DASH segments | Adapts to the connection; segments cache perfectly in CDNs; seek by jumping to a segment | Many small files; a manifest to manage |
Choice: ABR. It is the industry standard for on-demand video.
Counting views
Writing UPDATE videos SET views = views + 1 for 3,000 starts a second, all on a few viral rows, creates hot rows that lock and slow the database. Instead, players send view events to a log such as Kafka; a stream job deduplicates and aggregates them per video per minute, and writes one update per video per minute. Counts lag by a minute or so, which users never notice.
Where this breaks
In the interview
The platform now wants live streaming: a creator broadcasts and up to 1 million people watch, with at most ~10 seconds of delay. What changes, and what can you reuse?
Compare with our answer
Ingest changes: the creator streams continuously (usually RTMP or SRT) to an ingest server near them, which transcodes in real time into a ladder and cuts short segments (2 s or less) as they arrive. The manifest is a live playlist that updates every segment. Delivery is reused: the same HLS/DASH segments through the CDN with ABR; with 1 million viewers requesting the newest segment at the same moment, an origin shield and request coalescing are essential. There is no chunk-parallel transcoding of a finished file, so encoder capacity per stream must be reserved. Low-latency HLS or DASH with partial segments can get the delay down to a few seconds. The finished stream is saved as a normal video afterwards.
A strong answer mentions: a real-time ingest and transcoding step · short segments and a continuously updated manifest · reusing CDN delivery and ABR · origin shield and request coalescing for the synchronised spike · the latency trade-off of segment length.
A design: the mobile app uploads the video to the API server, which saves it to local disk and transcodes it into one 1080p MP4 in the request handler; viewers download that file from the API servers; each view runs UPDATE videos SET views = views + 1. List the problems.
Compare with our answer
- Uploads through the API server: ties up connections and bandwidth for minutes; no resume. Use presigned multipart uploads to object storage.
- Local disk: other servers can't see the file and a disk failure loses it. Store in object storage.
- Transcoding in the request: takes minutes and blocks the server. Queue the job and use workers, chunk-parallel.
- One 1080p file: stalls on slow connections. Produce an ABR ladder with HLS/DASH segments.
- Serving from API servers: Tbit/s of traffic from one place. Serve through a CDN.
- Per-view UPDATE: hot rows on viral videos. Aggregate view events through a stream.
A strong answer mentions: direct uploads to object storage · queue plus workers for transcoding · a rendition ladder with ABR · CDN delivery · aggregated view counts.
Where to go next
- The full case study: Design YouTube, with deeper dives into the transcoding DAG, CDN strategy and recommendations.
- The concepts used here: object storage, CDNs, message queues, caching and search.
- Metadata (title, status, counts) in a database; media in object storage. Never put video bytes through app servers.
- Estimates: 250 M views/day → ~3,000 starts/s; ~110 MB per view → ~2.6 Tbit/s. 500 K uploads/day × ~2 GB ≈ 1 PB/day.
- Upload: presigned URLs, multipart (resumable parts) → raw bucket → event → queue.
- Transcode: split into chunks → encode each into a ladder (240p … 4K) in parallel → package HLS/DASH segments (2–6 s) + manifest →
ready. - Watch: player reads the manifest, fetches segments from a CDN, picks quality per segment (ABR).
- Spend compute on popular videos: basic renditions first, higher quality and better codecs later.
- View counts: events → stream aggregation → batched updates, not a per-view UPDATE.
- Breaks: viral cold start (origin shield, coalescing), poison uploads (retry limits, dead-letter queue), abandoned uploads (lifecycle rules).
1. Your platform serves 250 million views a day, each moving about 110 MB. Why is the CDN central to the design rather than an optimisation?
250 M × 110 MB ≈ 28 PB a day ≈ 2.6 Tbit/s. Edges near viewers absorb almost all of it; origin sees only the misses.
2. A viewer's connection drops from 20 Mbit/s to 2 Mbit/s mid-video. With adaptive bitrate streaming, what happens?
ABR works because every rendition is cut into short segments with a shared timeline. The player chooses each segment's quality from measured throughput and buffer level.
3. A 10-minute upload takes an hour to transcode on one worker. How do you make it available in minutes?
Chunk-parallel encoding turns one long job into many short ones; a failed chunk is retried alone. Streaming originals breaks on slow connections and unusual formats.
4. Most videos get very few views, while a small share gets most of the watching. How should that shape transcoding?
Spending compute and storage where the views are is the long-tail answer. Waiting for 1,000 views would leave early viewers with an unplayable original.
5. Viewers start 3,000 videos a second. Each start runs UPDATE videos SET views = views + 1. What goes wrong first?
Concentrated writes to one row serialise on its lock. Send view events to a stream, aggregate per minute, and write once per video per minute.
6. A brand-new video goes viral and hundreds of CDN edges all miss and fetch the same segments from origin at once. What helps most?
A shield turns hundreds of edge misses into one origin fetch per segment, and coalescing collapses simultaneous requests. Shorter segments mean more requests, not fewer.
Score 60% or more and the lesson is marked complete.