Design a news feed
Design an Instagram- or X-style home feed from scratch: requirements, estimates, fan-out on write vs read, the celebrity problem, a feed cache, and media through a CDN.
On this page
The problem: hundreds of timelines, in under a second
Open Instagram or X and the home feed appears almost instantly: recent posts from everyone you follow, newest or most interesting first. It looks like a simple list. Behind it is one of the most common system design questions, because the obvious solution collapses under load and the fix forces a real trade-off.
The goal: one feed built from hundreds of accounts, in well under a second
- Now a news feed, like Instagram or X. Open the app and see recent posts from everyone you follow, newest and most interesting first, in well under a second.
This is a first design: we build it step by step, make the key decisions, and look at where it breaks. The full case study (linked at the end) goes deeper.
Step 1: pin down the requirements
Before reading on, write down the functional requirements (what users can do) and non-functional requirements (how well it must work) you would agree with the interviewer.
Compare with our answer
A strong answer mentions: post text and photos · follow and unfollow · a home feed of followed accounts, newest first, paginated · a latency target for loading the feed · the scale (users, posts, follows) · how fresh the feed must be (seconds are fine) · what is out of scope (comments, likes ranking, ads, search).
Functional
- Users create posts (text plus optional photos).
- Users follow and unfollow other users.
- The home feed shows posts from followed accounts, newest first, 20 at a time, with "load more".
Non-functional
- Feed loads in under ~300 ms at p99.
- Highly available: a slightly stale feed is far better than an error.
- Eventual consistency is fine: a new post may take a few seconds to reach followers' feeds.
Out of scope for now: comments, likes, ranking by machine learning, ads, notifications.
Step 2: quick estimates
The numbers: reads dwarf writes
- The numbers: 12,000 feed loads a second, but only 600 new posts. Posts include photos, and you follow hundreds of people. Building each feed from scratch on every load would mean reading hundreds of timelines every time.
Assume 200 million daily active users who open the feed about 5 times a day, and one in four post once a day.
| Quantity | Working | Result |
|---|---|---|
| Feed loads | 200 M × 5 ÷ 86,400 s | ≈ 12,000 / s average (peaks 2–3× higher) |
| New posts | 50 M ÷ 86,400 s | ≈ 600 / s |
| Read : write ratio | 12,000 : 600 | 20 : 1 |
| Fan-out writes (if we copy each post to followers, ~300 each) | 600 × 300 | ≈ 180,000 feed inserts / s |
| Post metadata | 50 M × ~1 KB | ≈ 50 GB / day, ~18 TB / year |
| Photos | half the posts × ~1 MB (all sizes) | ≈ 25 TB / day |
Two conclusions shape everything: the system is read-heavy, so reads must be cheap; and photos are by far the largest data, so they must stay out of the database and app servers.
Step 3: the API and the data
POST /v1/posts {text, media_ids[]} → {post_id}
POST /v1/users/{id}/follow
GET /v1/feed?cursor=<last_post_id>&limit=20 → {posts[], next_cursor}
The feed uses a cursor (the ID of the last post you saw) rather than a page number: new posts arrive constantly, and "page 2" would shift under the user, showing duplicates.
| Table | Key | Notes |
|---|---|---|
users | user_id | Profile, follower count |
follows | (follower_id, followee_id) | Plus a reverse index to list someone's followers |
posts | post_id | author_id, created_at, text, photo keys. Sharded by post_id |
| Feed cache | feed:{user_id} | The latest ~500 post IDs for each user, in Redis |
Post IDs should be roughly time-ordered (for example Snowflake-style IDs: a timestamp plus a machine ID and a counter), so sorting by ID sorts by time and the cursor works across shards.
Step 4: build the design
First attempt: build each feed when it is read
The simplest design, fan-out on read (also called pull): when you open the app, look up the 300 accounts you follow, fetch each one's recent posts, merge them by time and return the top 20.
SELECT * FROM posts WHERE author_id IN (/* 300 ids */)
ORDER BY created_at DESC LIMIT 20;
With posts sharded across many databases, those 300 authors live on many shards, so each feed load becomes dozens of queries plus a merge. At 12,000 loads a second that is 3.6 million author lookups per second, for a result that barely changes between loads. Reads are the hot path, and this makes them the expensive one.
Fan-out on write: do the work when someone posts
Fan-out on write: copy each new post's ID into followers' ready-made feeds
- The classic trick is fan-out on write. When someone posts, workers copy the post’s ID into a ready-made feed for each follower. Reading a feed then becomes a single fast lookup.
- Those ready-made feeds live in a cache, so almost every feed load is a memory read. The writes happen in the background through a queue and workers, so posting stays instant.
Flip it. When someone posts:
- The post service saves the post and puts a "fan-out" job on a queue, then replies to the author at once.
- Fan-out workers read the author's follower list and push the post ID into each follower's feed list in Redis (
LPUSHthenLTRIMto keep the newest 500). - Loading a feed is now one Redis read for 20 IDs, then a batch fetch of those posts from a post cache (most posts being read are recent and hot).
The work moves to the write side, where it runs in the background: 180,000 small inserts a second spread across a Redis cluster, which is very manageable. See message queues for why the queue matters: posting stays fast even when fan-out falls behind.
Sizing the feed cache. 500 IDs × 8 bytes = 4 KB per user raw, roughly 10 KB with Redis overhead. For 200 M active users that is about 2 TB of RAM: for example 32 shards of 64 GB, each with a replica. Users inactive for a month are dropped from the cache and their feed is rebuilt (by pull) when they come back, which saves a lot of memory and fan-out work.
The celebrity problem
Celebrity posts are pulled at read time instead of being copied
- The catch is the celebrity problem. One star with 50 million followers would trigger 50 million copies per post. So celebrity posts are skipped during fan-out and merged in when you read your feed: fan-out on read, just for them.
A star with 50 million followers posts. Fan-out on write means 50 million inserts for one post; at a few hundred thousand inserts a second it takes minutes, and ordinary users' posts queue up behind it. A few such accounts posting at once and the whole pipeline falls behind.
The standard answer is a hybrid: push for normal accounts, pull for the few huge ones.
Pull merges at read time; the hybrid pushes for most accounts and pulls only celebrities
- Accounts above a threshold (say 100,000 followers) are marked as celebrities and skipped during fan-out.
- Each celebrity's recent post IDs are kept in one small cached list per author.
- On feed load, the service reads your precomputed list plus the recent lists of the (usually few) celebrities you follow, merges by time, and returns the top 20.
Media and the full picture
The whole design: app servers, feed cache, fan-out queue and workers, object storage and a CDN
- Photos go to object storage and are served through a CDN, so images load fast and never pass through your app servers. Put together: app servers, a feed cache, a queue with fan-out workers, sharded storage, object storage and a CDN.
Photos never pass through the app servers. The client gets a presigned URL, uploads straight to object storage, and a worker makes resized versions. Feeds contain only photo URLs; the images are served by a CDN from edge servers near the user.
| Component | Job |
|---|---|
| Load balancer + stateless app servers | API, auth, assembling feeds |
Post DB (sharded by post_id) + post cache | Source of truth for posts; hot posts in Redis |
| Follow graph DB | Who follows whom, in both directions |
| Fan-out queue + workers | Copy post IDs to followers' feeds in the background |
| Feed cache (Redis cluster) | Precomputed lists of post IDs per user |
| Object storage + CDN | Photos, uploaded directly, served from the edge |
Key decisions
Push, pull or hybrid?
| Option | How | Good at | Weak at |
|---|---|---|---|
| Pull (fan-out on read) | Build the feed at read time from followed accounts | Cheap writes; always fresh; nothing wasted on inactive users | Slow, expensive reads at 12,000/s; many shards per request |
| Push (fan-out on write) | Copy each post ID into followers' feeds at write time | Feed load is one cache read | Celebrity posts explode into millions of writes; work wasted on inactive users |
| Hybrid | Push for most accounts, pull for celebrities | Fast reads and bounded write cost | Two code paths; a merge step on every read |
Choice: hybrid. It is what large feed systems use in practice, and it is the answer interviewers hope to hear, with the threshold and the merge explained.
Where do the feeds live?
| Option | Good at | Weak at |
|---|---|---|
| Redis lists or sorted sets | Sub-millisecond reads; cheap trims; simple | RAM is expensive; needs replicas, and a rebuild path if lost |
Wide-column store (Cassandra) with (user_id, post_id) rows | Cheap disk, huge capacity, durable | Reads take milliseconds, not microseconds |
| No stored feed (pure pull) | No extra storage | Expensive reads, as above |
Choice: Redis for active users, treated as a cache: if a shard is lost, feeds are rebuilt by pull from the post database. Some designs keep a durable copy in Cassandra behind it.
Chronological or ranked?
Start chronological: simple, explainable, easy to paginate. A ranked feed adds a step on read: take a few hundred candidate posts (from the same precomputed list and celebrity lists), score them with a model (likes, your past interactions, freshness) and return the best 20. The fan-out design stays the same; only the read path changes.
Where this breaks
In the interview
Product decides the feed must be ranked by predicted interest, not just newest first, and must include "suggested posts" from accounts you don't follow. What changes in the design, and what stays the same?
Compare with our answer
Fan-out on write stays; it now produces a candidate set rather than the final order. The read path gains a ranking step: gather ~500 candidates (precomputed list, celebrity posts, and suggestions from a separate recommendation service), fetch features (likes, author affinity, age) from a feature store, score with a model, return the top 20, and cache the ranked page briefly so "load more" is consistent. Cursor pagination must now refer to the ranked list (e.g. a session ID with an offset), since post IDs are no longer in order. Latency budget gets tighter, so ranking must be fast (tens of ms) with a fallback to chronological if it times out.
A strong answer mentions: candidates from the existing fan-out · a separate source for suggested posts · a ranking step with a feature store and a model · pagination over a cached ranked list · a latency budget with a chronological fallback.
A design: posting writes the post to the database, then the API server loops over all followers and inserts the post ID into each feed before replying. Feeds are stored without any size limit. Deleting a post leaves it in feeds. What goes wrong?
Compare with our answer
- Fan-out inside the request: a user with 10,000 followers waits seconds to post, and a celebrity's post times out. Put fan-out on a queue with workers and reply at once.
- No celebrity handling: millions of inserts per post. Use the hybrid with a follower threshold.
- Unbounded feeds: memory grows forever for lists nobody scrolls to the end of. Trim to the newest ~500 IDs.
- Deletes leak: filter deleted, unfollowed and blocked posts at read time, and clean up lazily.
- No plan for inactive users: fan-out to them is wasted; skip them and rebuild on return.
A strong answer mentions: asynchronous fan-out via a queue · the celebrity hybrid · trimming feed lists · read-time filtering for deletes · skipping inactive users.
Where to go next
- The full case study: Design Twitter/X timeline, with deep dives on ranking, the follow graph and failure handling.
- The concepts used here: caching, message queues, sharding, object storage, CDNs and scaling reads.
- 200 M DAU → ~12,000 feed loads/s vs ~600 posts/s: 20:1 read-heavy. Make reads one cache lookup.
- Fan-out on write: post saved → queue → workers push the post ID into each follower's Redis list (
LPUSH+LTRIM500). ~180,000 inserts/s. - Celebrity problem: 50 M followers = 50 M inserts. Hybrid: skip accounts over ~100 K followers in fan-out; merge their recent posts at read time.
- Feed cache: ~10 KB per user in Redis → ~2 TB for 200 M users; drop inactive users and rebuild by pull.
- Feed load: read 20 IDs → hydrate from the post cache → filter deleted/unfollowed/blocked → return with a cursor.
- Time-ordered post IDs (Snowflake-style) make cursors work across shards.
- Photos: presigned upload to object storage; resized by workers; served by a CDN.
- Watch fan-out queue lag; handle hot posts with local caches or key copies.
1. Your feed service gets 12,000 feed loads and 600 new posts per second. Why is fan-out on write a good default?
Reads outnumber writes 20 to 1, so paying once per post to make every read cheap is the right trade. It makes celebrity posts expensive, which is why the hybrid exists.
2. A singer with 40 million followers posts, and ordinary users' posts take minutes to reach their followers. What is the standard fix?
The hybrid model pushes for normal accounts and pulls for the few huge ones, bounding the write cost. Throwing workers at it is wasteful, and rate-limiting a creator is a product decision, not a design fix.
3. Each user's feed list holds the newest 500 post IDs, about 10 KB in Redis. Roughly how much memory do 200 million active users need?
200 M × 10 KB = 2,000 GB ≈ 2 TB: a cluster of, say, 32 shards of 64 GB plus replicas. Dropping inactive users shrinks it further.
4. A user deletes a post that was already fanned out to 2 million feeds. What is the most practical way to stop it showing?
Read-time filtering hides the post immediately at almost no cost; background cleanup can follow. Synchronous removal is as expensive as the original fan-out.
5. Why does the feed API use a cursor (the last post ID seen) rather than ?page=2?
With a constantly growing list, "page 2" changes between requests. A cursor says "posts older than this one", which stays stable.
6. Fan-out workers keep feeds for every registered user, including 300 million who haven't opened the app in a year. What should change?
Fan-out to people who never read is wasted writes and memory. Rebuilding a returning user's feed once, by pull, is cheap by comparison.
7. Photos in posts are uploaded through the app servers and stored as BLOBs in the posts database. Which change helps most?
Media is the largest data by far: object storage holds it cheaply and durably, presigned uploads keep it off the app servers, and the CDN serves it from near the user.
Score 60% or more and the lesson is marked complete.