Object storage
Where photos, videos and backups belong: block vs file vs object storage, how 11 nines of durability work, blobs plus metadata, presigned uploads, CDNs and storage tiers.
On this page
The problem: files do not fit where rows live
A photo app lets users upload pictures. The first version stores each image as a BLOB column in Postgres, or as a file on the app server's disk. Both break quickly.
Big files don't belong in the database or on one server's disk
- Apps store more than rows of text. Photos, videos, PDFs and backups are big files: a single video can be bigger than your whole user table. Where should they live?
- Putting them in the database makes it huge, slow to back up and expensive. Keeping them on an app server’s disk is fragile: if that server dies, the files go with it, and other servers can’t see them.
- In the database: a 2 MB photo is about 10,000 times bigger than the row that describes it. The database balloons, backups take hours, replicas lag while copying megabytes, and expensive database memory is wasted on bytes nobody queries.
- On an app server's disk: only that server has the file. Add a second server behind the load balancer and half the requests cannot find it. If the disk dies, the photos die with it.
What you want is storage that every server can reach, that never loses files, that grows without limit, and that is cheap per gigabyte. That is object storage.
Three kinds of storage
| Kind | What you get | Accessed by | Good at | Examples |
|---|---|---|---|---|
| Block | A raw virtual disk; you put a filesystem or database on it | One machine at a time, mounted as a drive | Low latency (sub-millisecond on SSD), small random reads and writes | AWS EBS, a local NVMe SSD |
| File | A shared filesystem with folders and permissions | Many machines, through NFS or SMB | Shared files for tools that expect a normal filesystem | AWS EFS, an NFS server |
| Object | A flat store of whole objects under string keys | Any client, through an HTTP API (PUT, GET, DELETE) | Huge scale, high durability, low cost per GB | Amazon S3, Google Cloud Storage, Azure Blob, MinIO |
Databases sit on block storage because they rewrite small pages in place. User-uploaded media, backups, logs and data-lake files go in object storage.
Objects live in a bucket, each under a key
- Object storage, like Amazon S3 or Google Cloud Storage, is built for this. You put files, called objects, into a bucket, each under a name called its key, like photos/user42/beach.jpg.
An object store has buckets (top-level containers) holding objects, each under a key like photos/user42/beach.jpg. The slashes look like folders, but the namespace is flat: the key is just a string, and "folders" are only a shared prefix. You cannot edit a byte in the middle of an object; you upload a new version of the whole thing. In exchange you get practically unlimited capacity: one S3 object can be up to 5 TB, and a bucket can hold any number of them.
How it never loses your files
Every object is copied across machines and buildings
- It’s cheap, it scales to practically unlimited size, and it’s extremely durable: every object is copied across several machines and buildings, so losing one is astronomically unlikely.
Amazon S3 is designed for 99.999999999% (eleven nines) durability per object per year. In plain terms: store 10 million objects and you would expect to lose one about every 10,000 years. That comes from two techniques, plus constant checking.
Three full copies, or erasure coding at half the space
| Technique | How | Storage cost | Survives | Catch |
|---|---|---|---|---|
| Replication (3 copies) | Full copy on 3 disks in different zones | 3× | Losing 2 copies | Expensive at petabyte scale |
| Erasure coding (e.g. 6 data + 3 parity) | Split into fragments plus parity computed from them; any 6 of 9 rebuild the object | 1.5× | Losing any 3 fragments | Rebuilding a fragment reads 6 others; more CPU and network on repair and reads |
An availability zone is a separate datacentre (or group of them) with its own power and network. Spreading copies or fragments across zones means a fire in one building loses nothing. The system also scrubs continuously, re-reading data and comparing checksums, and repairs any fragment that has silently gone bad (bit rot) before a second failure can hit the same object.
Large stores use erasure coding for most data because at petabyte scale halving the disks is a huge saving; replication is still common for small or very hot objects. Note that durability is not availability: S3 Standard is designed for 99.99% availability, so a request can briefly fail even though the data is safe. Retry with backoff.
Blobs plus metadata
The database keeps the key; the bucket keeps the bytes
- Your database stores only the key, a tiny string. The app uses it to find the file when needed.
The pattern every design uses: the bytes (the "blob") go in object storage, and a small metadata row goes in your database.
CREATE TABLE photos (
id BIGINT PRIMARY KEY,
owner_id BIGINT NOT NULL,
storage_key TEXT NOT NULL, -- photos/42/9f1c2e.jpg
size_bytes BIGINT,
content_type TEXT,
sha256 TEXT,
status TEXT NOT NULL, -- pending | ready | deleted
created_at TIMESTAMPTZ NOT NULL
);
The database answers questions ("Ana's photos, newest first"); object storage serves the bytes. Use generated keys (an ID or content hash), not user-supplied filenames, so names never collide and nobody can guess other users' paths.
Uploading without carrying the bytes
A presigned URL lets the phone upload straight to the bucket
- For uploads, the app can hand the user a pre-signed URL: a temporary, permission-limited link. The phone uploads straight to the bucket, so your app servers never have to carry the heavy file.
A presigned URL is a normal storage URL with a signature attached: your server signs "PUT to this exact key, valid for 10 minutes" with its secret credentials, and gives the link to the client. The client uploads directly to object storage, and your app servers never touch the bytes. The full flow:
- Client asks the API to upload. The API inserts a
photosrow withstatus = pendingand returns a presigned URL for a fresh key. - Client
PUTs the file straight to the bucket. - The bucket emits an "object created" event. A worker checks the size and type, scans it, makes thumbnails and sets
status = ready.
Limit what a presigned link allows: one key, a short expiry, and where the provider supports it, a size limit (S3's presigned POST policies can enforce a content-length-range). Always validate the file after upload as well.
Large files go up in parts, in parallel, and resume after failures
For big files, use multipart upload: the file is split into parts (S3 allows up to 10,000 parts of 5 MB to 5 GB each) uploaded in parallel and retried individually. A flaky phone connection at 95% then resumes from the last part instead of starting a 2 GB video again.
Serving downloads: CDNs and tiers
Lifecycle rules move cold data to cheaper storage; a CDN serves the hot data
- Files are often hot when new and rarely touched later. Lifecycle rules move old objects to cheaper cold storage automatically. And serving files through a CDN keeps downloads fast worldwide.
Serving every view from the bucket is slow for distant users and expensive: internet egress from cloud storage costs roughly $0.05–0.09 per GB. A CDN caches objects at edge servers near users. Name objects by content (9f1c2e.jpg) so they never change, and you can cache them for a year. For private files, the CDN can check signed URLs or signed cookies.
Most files are hot for days and then almost never read. Lifecycle rules move objects between storage classes automatically:
| Class (S3 names) | Rough price per GB-month | Read access | Use for |
|---|---|---|---|
| Standard | ~$0.023 | Milliseconds | New and popular media |
| Infrequent Access | ~$0.0125 | Milliseconds, plus a per-GB retrieval fee | Old photos, last year's videos |
| Glacier Deep Archive | ~$0.001 | Hours to restore | Compliance archives, old backups |
Worked example: a photo app's storage bill
1 million uploads a day × 2 MB = 2 TB a day, about 730 TB a year. Upload bandwidth averages 2 TB ÷ 86,400 s ≈ 23 MB/s, with peaks several times higher.
All in Standard after one year: 730,000 GB × $0.023 ≈ $17,000 a month, and growing. With a rule that moves photos older than 90 days to Infrequent Access, about 550 TB of that year is cheaper: 180 TB × $0.023 + 550 TB × $0.0125 ≈ $4,100 + $6,900 = $11,000 a month. Downloads matter as much: 50 TB of views a month at $0.09/GB from the bucket is $4,500, which a CDN with a high hit rate cuts substantially (CDN egress is cheaper per GB, and most requests never reach the bucket).
When not to use object storage
- Small structured data you query or update in place: rows belong in a database.
- Low-latency random writes: databases and anything rewriting small parts of a file need block storage. First-byte latency from object storage is typically tens of milliseconds.
- Software that expects a filesystem (shared home directories, legacy tools that append to files): use file storage.
- Tiny objects in huge numbers: billions of 100-byte objects cost more in per-request fees than in storage; batch them into larger files or keep them in a database.
In the interview
Lawyers upload contracts (1–50 MB PDFs), 200,000 a day. Documents are private, must be kept for 7 years, are read a lot in the first month and rarely after. Some must be impossible to delete before the 7 years end. Sketch the storage design.
Compare with our answer
Bytes in object storage under generated keys; a documents table with owner, matter, key, size, checksum, status and retention date. Uploads via short-lived presigned URLs (multipart for large files), validated and virus-scanned by a worker on the upload event. Downloads through presigned GET URLs that expire in minutes, checked against permissions in the app. Lifecycle: Standard for 30 days, Infrequent Access after, archive tier after a year. For "cannot be deleted", use the store's write-once retention lock (S3 Object Lock in compliance mode). Volume: 200,000 × ~10 MB ≈ 2 TB/day.
A strong answer mentions: bytes and metadata separated · presigned URLs for both upload and private download · validation after upload · lifecycle tiers matching the access pattern · a retention lock for the legal requirement · a rough daily volume.
Your photo app stores originals in object storage. Product now wants users to edit a photo (crop, filters) and keep the history of edits, and wants every photo to load in under 100 ms worldwide. What changes?
Compare with our answer
Objects are immutable, so each edit writes a new object under a new key (or uses bucket versioning); the metadata gets a photo_versions table pointing at each key, and the photo row points at the current version. Keep the original untouched. For speed: serve through a CDN with content-hashed keys so each version is cached forever and a new edit simply has a new URL (no purge needed); generate resized variants once per version. Old versions can move to a cheaper tier by lifecycle rule.
A strong answer mentions: objects are immutable, so edits create new objects · a versions table in the database · content-addressed keys so the CDN never serves a stale edit · pre-generated sizes · lifecycle rules for old versions.
- Block = a disk for one machine (databases). File = a shared filesystem (NFS). Object = whole objects under keys over HTTP (media, backups, logs).
- Bucket + key; flat namespace; objects are replaced, not edited. S3: up to 5 TB per object; strong read-after-write consistency since 2020.
- Durability (eleven nines, design target) from replication (3×) or erasure coding (e.g. 6+3 = 1.5×, survives 3 losses), across zones, with scrubbing. Durability ≠ availability.
- Bytes in the bucket, metadata (key, size, type, checksum, status) in the database.
- Upload: presigned URL (one key, short expiry, size limit), multipart for big files, event → worker validates →
ready. - Download: CDN with content-hashed keys and long TTLs; signed URLs or cookies for private files.
- Lifecycle tiers: ~$0.023 → ~$0.0125 → ~$0.001 per GB-month. Egress costs roughly $0.05–0.09/GB.
- Keep metadata and bytes in sync:
pendingstatus, sweepers, abort incomplete multipart uploads, reconciliation.
1. A team stores user videos (up to 2 GB) as BLOBs in Postgres. Which problem appears first as they grow?
Media bloats the database: backups, restores and replicas all copy it, and the buffer pool fills with bytes nobody queries. Object storage holds the bytes; the database keeps a key.
2. Your mobile app uploads 500 MB videos through your API servers, which forward them to S3. Uploads fail often on weak connections and the servers' network is saturated. What should change?
Presigned URLs take the API servers out of the data path, and multipart upload lets a flaky connection resume from the last finished part instead of starting over.
3. Why do large object stores use erasure coding such as 6 data + 3 parity fragments for most data?
Erasure coding gives similar or better fault tolerance for half the space of triple replication. The cost is extra work on reads and repairs, which need several fragments.
4. A user requests an upload URL, the app inserts a pending row, but the user closes the app before uploading. What is the best way to handle such cases?
The database and the bucket cannot share a transaction, so design for drift: a status column, an event-driven ready transition and cleanup jobs. Client confirmations are unreliable and can be forged.
5. Photos are viewed heavily for a week, then rarely. The bill for storage is growing fast. What helps most?
Lifecycle rules match price to access pattern automatically. Durability internals are the provider’s choice, and silently degrading users’ originals is not an acceptable cost fix.
6. An avatar is served through a CDN with a one-year cache lifetime. Users who change their avatar still see the old one. What is the cleanest fix?
Content-addressed keys make every version a new URL, so long cache lifetimes are safe and no purge is needed. Purging works but is slower and easy to miss; a 10-second TTL throws away most of the CDN's benefit.
Score 60% or more and the lesson is marked complete.