Interview patterns · Design 6

Large files and media

How to upload, process and serve large files: presigned URLs, multipart and resumable uploads, processing pipelines, CDN delivery, and where the money actually goes.

IntermediateInterview priority: High30 min
On this page
  1. The problem: a 2 GB video through an app server
  2. Signals that you are facing it
  3. Separate the bytes from the metadata
  4. Direct uploads with presigned URLs
  5. Multipart and resumable uploads
  6. Processing pipeline
  7. Delivery through a CDN
  8. Worked example: the cost of a video platform
  9. When not to use these tools
  10. Which case studies use it
  11. In the interview
  12. Quiz

The problem: a 2 GB video through an app server

A video app lets users upload clips of up to 2 GB from their phones. The first version sends the file in one HTTP request to the API, which writes it to disk and copies it to storage.

Three things go wrong. An upload on a mobile connection at 20 Mbit/s takes over 13 minutes, and for all that time an app server holds the connection, buffers and memory; a few hundred uploads at once fill the fleet, and normal API calls queue behind them. When the train enters a tunnel at 95%, the upload starts again from zero. And once the file is stored, a 4K original is useless to a viewer on a phone: it has to be converted into smaller versions before anyone can watch it.

Large files need a different path from everything else: bytes go straight to object storage, uploads are split and resumable, processing happens asynchronously, and delivery goes through a CDN.

Signals that you are facing it

  • Photos, video, audio, documents, backups, datasets: anything above a few MB.
  • Mobile clients on unreliable networks.
  • Processing after upload: thumbnails, transcoding, virus scanning, text extraction.
  • Many viewers per file: the same video watched millions of times.
  • Storage and bandwidth growing by terabytes a day, which means cost will come up.

Say early: "Metadata goes in the database, bytes go in object storage. Clients upload directly with presigned URLs, and a pipeline processes files after upload."

Separate the bytes from the metadata

Object storage (Amazon S3, Google Cloud Storage, Azure Blob) stores files as objects under keys, durable across several data centres and cheap per GB. It is the right home for bytes. The database keeps only the metadata: the owner, title, size, status, and the object key, such as videos/8f3a.../original.mp4. A 2 GB file never passes through a database, and rarely through your own servers at all.

Direct uploads with presigned URLs

A presigned URL is a link to one object in storage with a signature attached, created by your server using its storage credentials. Anyone holding it can upload (or download) that one object until it expires, typically after minutes. The client never gets credentials, and the bytes never touch your app servers.

Through the app server, then straight to storage with a presigned URL

The flow:

  1. The client asks the API to start an upload. The API checks the user, creates a metadata row with status = uploading, and returns a presigned URL for a new object key.
  2. The client uploads directly to object storage.
  3. Storage emits an event when the object is complete (S3 event notifications to a queue, for example), and the pipeline takes over.

Constrain what the URL allows: a short expiry, the exact key, and where supported a size limit (S3 presigned POST policies accept a content-length-range). Never trust the client's claimed file type; the pipeline checks the real content.

Multipart and resumable uploads

A multipart upload sends a large object as independent parts. In S3, parts are 5 MB to 5 GB each, up to 10,000 parts per object. The client starts an upload (getting an upload ID), uploads parts in parallel, each with its own presigned URL, and finally calls "complete" with the list of parts, after which storage joins them into one object.

Parts upload in parallel and fail independently

What this buys:

  • Resume: if the connection drops, the client asks which parts storage already has and sends only the rest. A failure costs at most one part, not the whole file.
  • Speed: several parts in flight use the bandwidth better, especially over long-distance links.
  • Retry per part: one bad part is resent on its own.

Resumable upload protocols such as tus and Google Cloud's resumable uploads offer the same idea through a single endpoint. For sync products like Dropbox, files are split into fixed-size blocks identified by their hash, so an edit re-uploads only the changed blocks, and a block the service already has is not uploaded at all (deduplication).

Processing pipeline

When the original lands, a pipeline turns it into what viewers need. For video:

  • Validate and scan: check the real format, size and duration; scan for malware; run moderation.
  • Transcode into a bitrate ladder: several resolutions (240p to 1080p or 4K) at different bitrates.
  • Segment each version into short chunks (2–6 seconds) for adaptive streaming with HLS or DASH: the player fetches a manifest listing the chunks and switches quality chunk by chunk as the network changes.
  • Generate thumbnails and extract metadata.
  • Mark the row status = ready.

Process after upload, then serve from the CDN

Transcoding is slow (a long video can take many minutes of CPU per version), so it runs on a queue with autoscaled workers. Big files are split into segments and transcoded in parallel across workers, then stitched, so a two-hour film doesn't wait for one machine. Jobs must be idempotent: a retried job overwrites the same output keys rather than creating duplicates (see idempotency).

Delivery through a CDN

A CDN caches files in many cities and serves viewers from the nearest one. Popular videos and images are served almost entirely from the edge; only misses reach storage. For private content, the API hands out signed CDN URLs or cookies that expire, so links can't be shared forever. Large CDNs also use an origin shield: one central cache layer between the edges and storage, so a new video requested by 200 edge locations at once causes one fetch from storage, not 200.

Worked example: the cost of a video platform

Assume 100,000 uploads a day, 500 MB each on average, and roughly a 1.5× overhead for the transcoded versions.

ItemCalculationResult
New originals100,000 × 500 MB50 TB a day
With transcoded versions× 1.575 TB a day, ~27 PB a year
Upload bandwidth50 TB ÷ 86,400 s~580 MB/s ≈ 4.6 Gbit/s on average
Storage cost after a year27 PB at roughly $0.02 per GB-month~$550,000 a month, growing
Delivery30 PB watched a month at roughly $0.01–0.03 per GB (CDN, at volume)~$300,000–900,000 a month

Two lessons hide in the table. Delivery usually costs more than storage for a popular service, which is why big video companies run their own CDNs and peer directly with internet providers. And storage keeps growing forever, so tiering matters: originals and rarely watched versions move to infrequent-access or archive classes (several times cheaper per GB, with a fee or a delay to read), and some versions can be deleted and regenerated on demand. Exact prices vary by provider, region and volume; quote them as rough figures.

When not to use these tools

  • Small files. A 50 KB avatar can go through the API in one request; multipart parts must be at least 5 MB anyway.
  • Sensitive data that must be inspected before storage. Some compliance rules require scanning before anything is written; upload to a quarantine bucket, scan, then move.
  • No repeated reads. Backups and logs are rarely downloaded, so a CDN adds nothing; cheap storage tiers matter more.
  • Live streaming. Live video is ingested continuously (RTMP, SRT or WebRTC) and segmented on the fly; it is not a file upload.

Which case studies use it

  • Design YouTube: resumable uploads, a transcoding pipeline, adaptive streaming and a CDN.
  • Design Dropbox: block-level upload, hashing and deduplication, sync of changed blocks only.
  • Design Instagram: photo upload, resizing into several versions, CDN delivery.
  • Design WhatsApp: media uploaded once to storage, recipients download it by reference.
  • The video platform concept lesson.

In the interview

Your turn: a document-signing service

Users upload PDFs of up to 100 MB to be signed by others. Documents are confidential; only invited signers may view them, for 7 days. After signing, the final PDF must never change. Design the upload, processing and access path.

Compare with our answer
  • Upload with a presigned URL (or a few parts) into a private bucket; metadata row with owner, signers and status.
  • Pipeline: check it is a real PDF, scan for malware, render page thumbnails, compute a hash of the file for integrity.
  • Access: the API checks the user is an invited signer, then returns a short-lived presigned download URL (minutes), not a public link. No CDN caching of private content, or a CDN with signed URLs only.
  • Expiry: the API simply stops issuing URLs after 7 days; a lifecycle rule archives or deletes.
  • Immutability: write the signed PDF under a new key and enable object lock or versioning so it can't be overwritten; store its hash with an audit log.

A strong answer mentions: private bucket, presigned upload · validation and scanning · short-lived download URLs after an authorisation check · hash for integrity · immutability via object lock or versioning · expiry handled by the API and lifecycle rules.

Your turn: spot the bug

A team's upload flow: the client uploads to storage with a presigned URL, then calls POST /uploads/{id}/done, which marks the row ready and enqueues transcoding. Support finds thousands of videos that exist in storage but never appear in the app, and the storage bill includes terabytes nobody can see. What is wrong?

Compare with our answer

Processing depends on the client's "done" call. Clients that crash, lose connectivity or are killed by the phone's OS after the upload finishes never send it, so the rows stay in uploading. Separately, abandoned multipart uploads leave invisible parts that are still billed.

A strong answer mentions:

  • Trigger processing from the storage completion event, not the client.
  • Make the "done" call optional and idempotent (it can only speed things up).
  • A reconciliation job for rows stuck in uploading and objects without rows.
  • A lifecycle rule to abort incomplete multipart uploads after a few days.
  • Alerting on the count of stuck uploads.
Large files on one page
  • Bytes in object storage; metadata (key, size, status) in the database.
  • Presigned URL: signed, short-lived permission for one object; client uploads directly, app never sees bytes.
  • Multipart: S3 parts 5 MB–5 GB, up to 10,000; parallel, retry per part, resume after the last finished part.
  • Dedup and delta sync: hash fixed-size blocks; upload only new ones.
  • Pipeline from storage events: scan, transcode a bitrate ladder, segment for HLS/DASH, thumbnails, status = ready. Idempotent jobs.
  • CDN for delivery; signed URLs or cookies for private content; origin shield for cold-edge stampedes.
  • Cost: delivery usually beats storage; tier old content; lifecycle rules for incomplete uploads.
  • Numbers: 2 GB at 20 Mbit/s ≈ 13 min; 100 K uploads × 500 MB = 50 TB/day.
Check what stuck

1. An API server relays every user upload of up to 1 GB to object storage, and API latency spikes whenever many users upload. What is the best change?

2. A mobile upload of a 1.5 GB file fails at 90% on a train. With S3 multipart upload in 16 MB parts, what does the client need to resend?

3. A video platform's bill shows storage at $400,000 a month and CDN delivery at $900,000. Which change saves the most with least impact on viewers?

4. Thousands of uploaded videos are stuck in the "uploading" status even though the files exist in storage. Which design flaw most likely explains it?

5. A newly launched trailer is requested by viewers in 150 cities within a minute, and storage request rates spike. Which feature targets this?

6. Users report that links to their private videos still work weeks after sharing them with a friend. What is the best fix?

Score 60% or more and the lesson is marked complete.

Passing the quiz also marks it complete. Progress is saved in this browser only.