Messaging and social · Design 16

Design Instagram

Design Instagram in a 45-minute interview: presigned photo uploads, a resize pipeline, CDN delivery, hybrid feed fan-out, sharded ids and like counters.

IntermediateInterview priority: High50 min
On this page
  1. How to use this page
  2. 1. Clarify the requirements
  3. 2. Back-of-the-envelope estimates
  4. 3. API
  5. 4. Data model
  6. 5. High-level design
  7. 6. Deep dives
  8. 7. Bottlenecks, failures and monitoring
  9. 8. Follow-up questions
  10. Quiz

How to use this page

This page follows the order of a real system design interview. At each step there is a Your turn box: write your own answer first, then open ours and compare. Instagram has two halves that interviewers probe: a media pipeline that moves hundreds of terabytes of photos a day without touching your API servers, and a feed that must load in half a second for half a billion people.

PhaseTime in a 45-min interviewWhat you produce
1. Clarify requirements5 minUpload, follow, feed, likes; what's out of scope
2. Estimate5 minUploads/s, feed loads/s, storage/day, image bandwidth
3. API5 minUpload URL, create post, feed with a cursor
4. Data model5 minPosts, follows, feed cache, likes, photo objects
5. High-level design10 minUpload path, fan-out path, feed path
6. Deep dives10–15 minUploads, image sizes and CDN, feed building, ids, counters
7. Wrap up2–3 minBottlenecks, failures, monitoring

Users upload photos with a caption, follow other users, and see a home feed of recent posts from people they follow. They can like and comment. Skip stories, reels, direct messages and ads.

1. Clarify the requirements

Your turn: scope it

The interviewer says: "Design Instagram." Write the questions you'd ask, then 4–5 functional and 4–5 non-functional requirements with targets, and what you'd leave out.

Compare with our answer

Useful questions: Photos only, or video too? Is the feed chronological or ranked? How large can an upload be? How quickly must a new post appear for followers? Are some accounts private?

A strong answer mentions: upload, follow, feed, like/comment, profile grid · feed latency target · photos must never be lost (durability is separate from availability) · eventual consistency for feeds · stories, reels, DMs and ads out of scope.

Functional: what it must do

  • Upload a photo with a caption: JPEG/HEIC up to ~20 MB
  • Follow and unfollow users
  • Home feed: Posts from people you follow, newest or ranked
  • Like and comment
  • Profile page: A user’s own posts in a grid

Out of scope

Stories and reelsDirect messagesAds

Non-functional: how well it must work

QualityTargetWhy
Feed latency< 500 ms (p99)Scrolling must feel smooth
Availability99.99%A down feed is a down app
ConsistencyEventual (a new post may take seconds to appear)Nobody notices a few seconds’ delay
DurabilityPhotos never lostThey’re people’s memories
Read-heavy~50 feed loads per uploadOptimise the read path
Scale500 M daily usersGlobal, mobile, spiky

Notice the two different reliability words. Availability is "can I load the feed right now?"; durability is "will my wedding photo still exist in ten years?". You can relax the first a little during an outage; you can never relax the second. That's why photos go to replicated object storage, not to a disk on an app server.

2. Back-of-the-envelope estimates

Your turn: the numbers

Assume 500 M daily users, 0.2 posts per user per day, 10 feed loads per user per day, 2 MB per original photo, resized copies adding 50% more, and ~10 images of ~150 KB shown per feed load. Work out uploads/s, feed loads/s, storage per day and image bandwidth.

Compare with our answer
  • Uploads: 500 M × 0.2 = 100 M/day ÷ ~86,400 ≈ 1,200/s (peaks of 3,000+ on New Year's Eve).
  • Feed loads: 500 M × 10 = 5 B/day ≈ 58 K/s: about 50 reads per upload.
  • Storage: 100 M × 2 MB × 1.5 = 300 TB/day, about 110 PB/year before replication. Object storage, not a database.
  • Image bandwidth: 5 B × 10 × 150 KB ÷ 86,400 ≈ 87 GB/s (about 700 Gbps) on average, more at peak. Only a CDN can serve this.
  • Metadata (captions, likes) is tiny by comparison: a post row is well under 1 KB.

A strong answer mentions: reads ≫ writes · storage dominated by images · bandwidth as the real cost · metadata small enough for sharded SQL · peaks 2–3× average.

Assumptions: daily active users 500 M; posts per user per day 0.2; feed loads per user per day 10; photo size (original) 2 MB; stored sizes per photo (x original) 1.5.

NumberEstimateWhy it matters
Uploads per second1.16 K / s100 M a day ≈ 1,200/s.
Feed loads per second57.9 K / s5 B a day ≈ 58,000/s, peaks higher.
New photo storage per day300 TB≈ 300 TB a day: object storage, not a database.
Photo storage per year110 PB
Image bandwidth (feed)86.8 GB/s~10 images per feed load at ~150 KB each, mostly from the CDN.
  • Images are the big numbers; metadata (captions, likes) is tiny by comparison.
  • Peaks are 2–3× the average; plan capacity for the peak.

Two numbers set the shape of the design. 300 TB/day says photos live in object storage and never in a database. ~700 Gbps of image traffic says almost every image byte must come from a CDN edge, not from your servers.

3. API

Your turn: the API

Design the calls for posting a photo. Should the photo bytes go through your API at all? What does the client call, in what order?

Compare with our answer

A strong answer mentions: a call that returns a presigned upload URL · the client uploading straight to object storage · a separate "create post" call with the media id and caption · a cursor-paginated feed · idempotent like (liking twice is a no-op) · follow/unfollow.

CallRequestResponseNotes
POST /v1/uploads{ contentType, size }{ uploadUrl (presigned), mediaId }The app uploads straight to object storage
POST /v1/posts{ mediaId, caption }201 { postId }Called after the upload finishes
GET /v1/feed?cursor=–{ posts[], nextCursor }Cursor pagination; ~10 posts per page
POST /v1/users/{id}/follow–204DELETE to unfollow
POST /v1/posts/{id}/likes–204Idempotent: liking twice is a no-op
POST /v1/posts/{id}/comments{ text }201 { commentId }
GET /v1/users/{id}/posts?cursor=–{ posts[], nextCursor }Profile grid

Posting is two steps: get an upload URL, upload, then create the post. A presigned URL is a link to the object store signed by our server with a short expiry (say 15 minutes) that allows exactly one action: "PUT one object at this key, up to this size, of this type". The phone talks to storage directly; our API never handles the bytes.

4. Data model

Your turn: the tables

What do you store, where, and keyed how? Which tables are hit on every feed load?

Compare with our answer

A strong answer mentions: posts sharded by author with a time-sortable id · follows stored in both directions · a feed cache of post ids per user · likes as rows plus a fast counter · photos as objects keyed by media id with one key per size · the feed path touching only the cache and a post cache.

posts Sharded SQL or Cassandra (by user_id)

  • post_id (time-sortable id)
  • user_id (shard key)
  • media_keys: {thumb, medium, full}
  • caption, created_at

Posts are always fetched by author or by id; sharding by author keeps a profile on one shard.

follows Graph-friendly store (two tables)

  • following(user_id, followee_id)
  • followers(user_id, follower_id)

Both directions are needed: “who do I follow” for the feed, “who follows me” for fan-out.

feed cache Redis (list per user)

  • feed:{userId} → [postId, postId, …]

Precomputed feeds make the hot read path a single lookup. Store ids only, newest ~500.

likes and counts Cassandra + Redis counters

  • likes(post_id, user_id)
  • counter:likes:{postId}

Popular posts get thousands of likes a second: count in memory, persist in batches.

photos Object storage + CDN

  • photos/{mediaId}/thumb.jpg
  • …/medium.jpg, …/full.jpg

Hundreds of terabytes a day of immutable blobs, served from edge caches worldwide.

Comments on the data:

  • Posts are sharded by user_id, so a profile grid is one shard. A home feed hydrates posts from many authors anyway, through a post cache, so cross-shard reads there are by id and cheap.
  • Photo keys never change. photos/{mediaId}/medium.jpg is written once and never overwritten. An immutable object can be cached by the CDN forever, so you never have to invalidate a cache. To "edit" a photo you'd write a new media id.
  • The feed cache stores ids, ~500 per user, for active users only, exactly as in Design Twitter.
  • Likes are rows plus a counter. The row (post_id, user_id) makes a like idempotent and answers "did I like this?"; the counter answers "how many?".

5. High-level design

Your turn: draw it

Sketch three flows: uploading a photo, getting the new post to followers, and loading the feed. Which servers touch the image bytes?

Compare with our answer

Only the object store, the resizer and the CDN touch image bytes. The API servers handle small JSON. People often forget the resize step (phones would otherwise download 4,000-pixel originals) and the queue between creating a post and fanning it out.

The architecture, then uploading a photo straight to storage

  1. The architecture splits writes from reads. The post service and an image pipeline handle uploads. A queue and fan-out workers push new posts into followers’ cached feeds. The feed service reads those caches, ranks, and returns pages, while every image is served by the CDN.
  2. Uploading: the app asks for a presigned URL and sends the photo straight to object storage, not through our servers. A resizer makes thumbnail, medium and full sizes. Then the app creates the post, which is saved and announced on a queue.
ComponentWhat it does
API gatewayAuth, rate limits, routing to services
Post serviceCreates posts; issues presigned upload URLs
Object store + CDNOriginal and resized photos; served from the edge
Image resizerEvent-driven workers making thumbnail/medium/full sizes
Fan-out queue + workersPush new post ids into followers’ feeds asynchronously
Follow graphfollowers and following tables, cached
Feed cacheRedis list of post ids per user
Feed serviceReads, hydrates and ranks the feed
RankingScores posts (recency, relationship, engagement)

Fan-out to followers, then loading the feed

  1. Fan-out on write: workers read the event, fetch the author’s followers, and push the new post id onto each follower’s cached feed. When Ana opens the app later, her feed is already built.
  2. Loading the feed: the feed service reads the precomputed list of post ids, hydrates them with post details from cache, ranks them, and returns one page. The images themselves come from the CDN.

Notice how the paths are separated. The write path (post service, queue, fan-out workers) can lag a few seconds without anyone noticing. The read path (feed service, feed cache, post cache, CDN) is what users feel on every scroll, so it does as little work as possible: one cache read, one multi-get, a ranking pass, and the images come from an edge server a few milliseconds away.

6. Deep dives

Your turn: the hard parts

Write two or three sentences on each before reading: How does a 5 MB photo get from the phone into storage? What sizes do you store and why? How is the home feed built when one account has 300 M followers? How do you count likes on a viral post?

Compare with our answer

A strong answer mentions:

  • Presigned direct upload, an event-driven resizer, and what the feed shows while resizing is in progress.
  • A few fixed sizes in modern formats, immutable keys, served by a CDN.
  • Hybrid fan-out: push for normal accounts, pull for celebrities, merge and rank on read.
  • Likes as idempotent rows plus an in-memory counter flushed in batches.

The upload pipeline

The problem. A phone on a weak mobile network uploads a 5 MB photo. If it goes through an API server, that server holds a connection open for many seconds and copies every byte twice (in from the phone, out to storage).

OptionHowCatch
Upload through the APIPhone → API server → object storeAPI servers tied up by slow uploads; double bandwidth
Store in the databasePhoto as a blob in the posts table300 TB/day destroys any database and its backups
Presigned direct upload (chosen)API signs a short-lived URL; phone PUTs to object storageOne more round trip; must handle uploads that never become posts

The pipeline, step by step:

  1. POST /v1/uploads returns {uploadUrl, mediaId}. The URL is limited to one key, a size cap and a content type.
  2. The phone uploads. For large files or video, use a multipart, resumable upload so a dropped connection resends one part, not the whole file.
  3. The object store emits an "object created" event onto a queue. Resizer workers decode the image, strip location metadata (EXIF GPS data is a privacy leak), convert HEIC to web formats, and write thumbnail, medium and full sizes.
  4. POST /v1/posts {mediaId, caption} creates the post. The post service checks the media exists and belongs to this user.

Image sizes and the CDN

A phone showing a 400-pixel-wide thumbnail should not download a 4,000-pixel original: that's 10–20× the bytes. Pre-generate a few fixed sizes (for example 150, 640 and 1,080 pixels wide) once at upload, in modern formats such as WebP or AVIF, which are typically 25–50% smaller than JPEG at similar quality. The app asks for the size that matches its screen.

Then a CDN serves them. Because each key is immutable, CDN entries can have a very long TTL, and the hit ratio for feed images can be very high: a new post is fetched from the origin once per edge location, then served from the edge to everyone nearby. See CDNs for how edges and origins work.

Private accounts add one twist: image URLs must not be shareable forever. Serve them with signed CDN URLs that expire after some hours, so a leaked link stops working.

Building the home feed

The problem. Ana follows 300 accounts; most have hundreds of followers, one has 300 M.

OptionHowCatch
Fan-out on readOn load, query recent posts of all 300 and mergeHundreds of reads per load at 58 K loads/s
Fan-out on writeOn post, push the id into every follower's feed cacheOne celebrity post = 300 M writes
Hybrid (chosen)Push for normal accounts; pull celebrity posts at read time; mergeTwo paths; the merge must de-duplicate

This is the same trade-off as Twitter, and the Twitter page goes deeper into the threshold and cache rebuilds. What's different here is ranking: Instagram's feed has been ranked rather than chronological since 2016. The feed service collects a few hundred candidates (cached ids plus celebrity pulls), hydrates them, scores each one for predicted interest (how often Ana interacts with the author, recency, engagement so far), and returns the top 10. To keep scrolling stable, it caches the ranked list for the session, so the next page continues the same order instead of re-ranking.

Ids and sharding the metadata

Posts need ids that are unique across thousands of shards and sort by time, so "newest first" is a sort by id. Instagram's well-known scheme (described in 2012) generates them inside PostgreSQL, with no separate service:

BitsPart
41Milliseconds since a custom epoch
13Logical shard id (up to 8,192 shards)
10Per-shard sequence (1,024 ids per ms per shard)

The trick is logical shards: data is split into thousands of logical shards, each a Postgres schema, and many logical shards share one physical server. To scale, you move logical shards to new servers; the shard id inside every post id never changes, so you always know where a post lives.

Counting likes on a viral post

UPDATE posts SET likes = likes + 1 on every like makes one row on one shard the hottest thing in the system. Instead: insert the like row (idempotent per user), and increment a counter in Redis (INCR likes:{postId}) that a job flushes to the database every few seconds. For extreme posts, split the counter into N sub-keys and sum them on read. Counts are briefly approximate; exactness here isn't worth a hot row.

Serving the world: regions and the read path

Half a billion users are spread across every continent, and a cross-continent round trip is 100–150 ms. With a 500 ms budget, the feed cannot make several sequential calls to a data centre on another continent. So:

  • Images come from the nearest CDN edge everywhere (already covered).
  • Feed reads are served from the nearest region. Each region has its own feed caches and read replicas of the post and follow data, fed by asynchronous replication. A post can appear a second later on another continent, which the consistency requirement allows.
  • Writes (new posts, follows, likes) go to the region that owns the user's shard, or are written locally and replicated, depending on how much conflict you can tolerate. Likes are simple: counters merge by adding. Posts are owned by one author, so conflicts are rare.
Durability: how photos survive disasters

Object stores like S3 are designed for 99.999999999% (eleven nines) durability by storing each object redundantly across multiple facilities, usually with erasure coding (data split into fragments plus parity, so any k of n fragments can rebuild it) rather than plain triple copies. Add cross-region replication for originals so a regional disaster can't lose them. Resized copies are derived data: if they were lost you could regenerate them from the original, so some systems store them with cheaper, less redundant settings.

7. Bottlenecks, failures and monitoring

Scaling bottlenecks

BottleneckFix
Celebrity postsHybrid fan-out: pull celebrity posts at read time
Image bandwidthCDN with high cache hit rate; small formats; lazy loading in the app
Feed cache memoryStore ids only, cap at ~500, skip inactive users
Posts DB growthShard by user_id; time-sortable ids; archive cold data
Upload spikes (New Year’s Eve)Queue-based resizing absorbs bursts; autoscale workers

When things fail

FailureWhat happens
Fan-out workers fall behindPosts appear in feeds a bit late; the queue holds the backlog safely
Feed cache node lostFeeds for those users are rebuilt with fan-out on read until the cache refills
Resizer crashes mid-jobThe event is retried; resizing is idempotent
A region goes downDNS sends users to another region; photos are replicated across regions

What to monitor

Feed latency (p99)Fan-out queue lagCDN cache hit ratioUpload success rateResize job failuresFeed cache hit ratio

8. Follow-up questions

Your turn: the interviewer pushes

Answer in order before opening ours. 1) Add video posts up to 60 seconds. What changes? 2) A user deletes a post. What must happen, and how fast? 3) Users in a new country complain that feeds load slowly. How do you find out why, and what do you do?

Compare with our answer

A strong answer mentions:

  1. Resumable multipart uploads; a transcoding pipeline producing several bitrates in short segments; adaptive streaming from the CDN (see Design YouTube); much higher storage and bandwidth per post.
  2. Mark the post deleted so hydration drops it at once (no scrubbing of feed caches); delete objects asynchronously; purge or let expire CDN copies (signed URLs help); honour legal deletion deadlines.
  3. Measure first: time to first byte for the API vs image load time per region; likely causes are no nearby CDN edge or a far-away API region; add edges or a regional deployment with replicated metadata.
How would you add stories that vanish after 24 hours?

Store them separately with a 24-hour TTL; a story tray per user built like a tiny feed; viewers list per story.

How would ranking work?

A model scores candidate posts on relationship strength, recency and predicted engagement; the feed service fetches ~500 candidates and keeps the top ones.

How do you show a new post in my own feed immediately?

Read-your-writes: the app inserts it locally, and the feed service merges the user’s own latest posts at read time.

More senior follow-ups
  • Explore / discovery. A second feed built only from accounts you don't follow: candidate generation from embeddings and co-engagement, a heavier ranking model, and aggressive caching, since it doesn't need fan-out at all.
  • Abuse and moderation. The resizer is the natural hook: run image classifiers and hash matching (for known illegal content) asynchronously before the post is fanned out, and hold the post if a check fails.
  • Cost. At ~110 PB a year, move originals older than a few months to colder storage classes, and drop rarely viewed derived sizes that can be regenerated.
Instagram on one page
  • Numbers: 100 M uploads/day ≈ 1.2 K/s; 5 B feed loads/day ≈ 58 K/s; 300 TB new photos/day; ~700 Gbps of feed images.
  • Upload: presigned URL → direct PUT to object storage (multipart/resumable for big files) → "created" event → resizer → create post.
  • Resizer: fixed sizes, WebP/AVIF, strip EXIF location, idempotent so retries are safe; lifecycle rule deletes orphan uploads.
  • CDN: immutable keys, long TTLs, origin shield and request coalescing for viral posts; signed URLs for private accounts.
  • Feed: hybrid fan-out into ~500 ids per active user; pull celebrities; rank a few hundred candidates; stable order per session.
  • Ids: 41-bit time + 13-bit logical shard + 10-bit sequence; logical shards move between physical servers.
  • Likes: idempotent rows + Redis counter flushed in batches; split hot counters.
  • Durability ≠ availability: replicated object storage for originals; resized copies can be regenerated.
Mock interviewPractise it as a mock interview: uploads, CDN and the feedOpens in the ByteAtlas lab. Your progress is saved in this browser.
Check what stuck

1. The API servers start timing out during peak hours, and profiling shows most of their time is spent receiving photo uploads from phones. What is the best fix?

2. A new post appears in followers' feeds with broken images for a few seconds. What is the most likely cause?

3. Why can feed images be cached by the CDN with a very long TTL?

4. 100 M photos a day at 2 MB, with resized copies adding 50%. Roughly how much new storage is that per day?

5. A celebrity posts, and within a minute millions of phones request the same new image. Which mechanism protects the origin storage most directly?

6. Instagram's post ids include a 13-bit logical shard id. What does that make easy?

7. A post gets 50,000 likes a minute and the shard holding it slows down. What is the best change?

Score 60% or more and the lesson is marked complete.

Passing the quiz also marks it complete. Progress is saved in this browser only.