Messaging and social · Design 17

Design Twitter / X

Design the Twitter (X) home timeline in a 45-minute interview: fan-out on write vs read, the celebrity hybrid, the timeline cache, Snowflake ids and ranking.

AdvancedInterview priority: High55 min
On this page
  1. How to use this page
  2. 1. Clarify the requirements
  3. 2. Back-of-the-envelope estimates
  4. 3. API
  5. 4. Data model
  6. 5. High-level design
  7. 6. Deep dives
  8. 7. Bottlenecks, failures and monitoring
  9. 8. Follow-up questions
  10. Quiz

How to use this page

This page follows the order of a real system design interview. At each step there is a Your turn box: write your own answer first, then open ours and compare. The Twitter question looks like "a CRUD app with followers", and the whole interview is really about one decision: when do you build a timeline, at write time or at read time?

PhaseTime in a 45-min interviewWhat you produce
1. Clarify requirements5 minHome vs user timeline, ordering, what's out of scope
2. Estimate5 minTweets/s, timeline reads/s, fan-out writes/s, cache size
3. API5 minPost, follow, read timelines with cursors
4. Data model5 minTweets, user tweet lists, the social graph, cached timelines
5. High-level design10 minWrite path, fan-out path, read path
6. Deep dives10–15 minPush vs pull and the celebrity hybrid, the cache, ids, ranking
7. Wrap up2–3 minBottlenecks, failures, what you would monitor

Users post tweets of up to 280 characters, follow other users, and read a home timeline of tweets from people they follow, newest first. They can like and retweet. Skip search, trends, direct messages and ads.

1. Clarify the requirements

Your turn: scope it

The interviewer says: "Design Twitter." Write down the questions you would ask, then 4–5 functional requirements, 4–5 non-functional requirements with targets, and what you would put out of scope.

Compare with our answer

The questions that change the design: Is the home timeline newest-first or ranked? How many followers can one account have? How fresh must a timeline be? Must I see my own tweet immediately? Do we need search and trends?

A strong answer mentions: home timeline vs user timeline as separate things · latency target for the home timeline · eventual consistency for followers but read-your-writes for the author · the follower distribution (0 to 100 M+) · search, trends and DMs out of scope.

Functional: what it must do

  • Post a tweet: Up to 280 characters, optional media
  • Follow / unfollow
  • Home timeline: Tweets from people you follow, newest first
  • User timeline: All tweets by one user
  • Like and retweet

Out of scope

SearchTrendsDirect messagesAds

Non-functional: how well it must work

QualityTargetWhy
Timeline latency< 200 ms (p99)The main screen of the app
Availability99.99%Breaking news drives huge spikes
ConsistencyEventual: a tweet may take a few seconds to reach followersExcept your own tweet, which you must see at once
Read-heavy~10 timeline loads per tweet postedEach load shows ~20 tweets, so ~200 tweet reads per tweet written
Scale250 M daily users, 500 M tweets/dayFollowers per user range from 0 to 100 M+

Two terms to fix on the board straight away. The user timeline is one person's own tweets (their profile page): easy, one index read. The home timeline is the merged feed of everyone you follow: this is the hard part, and the rest of the interview is about it.

2. Back-of-the-envelope estimates

Your turn: the numbers

Assume 250 M daily users who each post 2 tweets and load their home timeline 20 times a day, an average of 200 followers, and a cache of 800 tweet ids (8 bytes each) per active user. Work out tweets/s, timeline loads/s, timeline writes/s if every tweet is pushed to every follower, and the cache size.

Compare with our answer
  • Tweets: 250 M × 2 = 500 M/day ÷ ~86,400 ≈ 5,800/s average. Big events spike far higher (Twitter's record was about 143 K tweets/s in 2013), so plan for 3× and absorb the rest in a queue.
  • Timeline loads: 250 M × 20 = 5 B/day ≈ 58 K/s.
  • Fan-out writes: 5,800 × 200 ≈ 1.2 M timeline inserts/s. The heaviest write load in the system, and every one is a cache write, not a database write.
  • Cache: 250 M × 800 × 8 bytes = 1.6 TB of raw ids. Real Redis lists add per-entry overhead and you keep replicas, so budget several times that: still a modest Redis cluster.
  • Tweets stored: ~300 bytes with metadata × 500 M ≈ 150 GB/day, about 55 TB/year before replication. Media goes to object storage and a CDN.

A strong answer mentions: reads ≫ writes · fan-out multiplies writes by follower count · ids not tweets in the cache · only active users get a cache · peaks well above the average.

Assumptions: daily active users 250 M; tweets per user per day 2; timeline loads per user per day 20; average followers 200; tweet ids kept per cached timeline 800.

NumberEstimateWhy it matters
Tweets per second5.79 K / s500 M ÷ ~86 K ≈ 5,800/s, with spikes during big events.
Timeline reads per second57.9 K / s≈ 58,000/s: reads dominate.
Fan-out writes per second1.16 M / s≈ 1.2 M/s: fan-out is the heaviest write load.
Timeline cache size1.6 TB≈ 1.6 TB: fits in a Redis cluster.
Tweet text per year54.8 TB~300 bytes per tweet with metadata; media is separate.
  • Store ids, not tweets, in timelines: 8 bytes vs ~300 bytes.
  • Only cache timelines for active users; rebuild on demand for others.

The number that drives the design is the 1.2 M fan-out writes per second, and it hides a skew: the average is 200 followers, but one account with 100 M followers would turn a single tweet into 100 M writes. Averages hide the celebrity problem; always ask for the distribution.

3. API

Your turn: the API

Write the 5–6 calls the app needs. How does the client page through an endlessly scrolling timeline without seeing duplicates when new tweets arrive?

Compare with our answer

A strong answer mentions: POST /tweets returning the new id · home and user timelines as separate GETs · cursor pagination using the last tweet id seen (not page numbers) · follow/unfollow · like and retweet as idempotent calls · rate limiting on posting.

CallRequestResponseNotes
POST /v1/tweets{ text, mediaIds?, replyTo? }201 { tweetId, createdAt }Rate-limited; text validated server-side
GET /v1/timeline/home?cursor=–{ tweets[], nextCursor }Cursor = last seen tweet id
GET /v1/users/{id}/tweets?cursor=–{ tweets[], nextCursor }User timeline
POST /v1/users/{id}/follow–204
POST /v1/tweets/{id}/like–204Idempotent
POST /v1/tweets/{id}/retweet–201 { tweetId }A retweet is a tiny tweet pointing at the original

Why a cursor and not ?page=3? New tweets arrive at the top all the time. With page numbers, page 3 shifts down and the user sees tweets twice. A cursor says "give me 20 tweets older than id 1789…", which stays correct however many new tweets arrive. Because tweet ids are time-ordered (see Snowflake below), "older than this id" is the same as "earlier than this time".

4. Data model

Your turn: the tables

What do you store, keyed how? Think about the two questions the system asks constantly: "who follows X?" (on every post) and "what should Y see?" (on every timeline load).

Compare with our answer

A strong answer mentions: tweets keyed by tweet id in a sharded store · a per-user list of their own tweet ids, newest first · followers and following stored in both directions · cached home timelines as lists of ids in Redis · like and retweet counters kept apart from the tweet row.

tweets Sharded store by tweet_id (Manhattan / Cassandra)

  • tweet_id (Snowflake: time + machine + sequence)
  • user_id, text, media_ids
  • reply_to, retweet_of
  • created_at

Fetched by id when hydrating timelines; time-sortable ids make “newest first” natural.

user tweets index Wide-column (partition by user_id)

  • user_id (partition)
  • tweet_id (clustering, descending)

User timeline = one partition read, newest first.

social graph Graph store (FlockDB-style) + cache

  • followers(user_id, follower_id)
  • following(user_id, followee_id)
  • follower_count

Fast “followers of X” for fan-out and “who do I follow” for celebrity merge.

home timelines Redis lists

  • timeline:{userId} → [tweetId × 800]

Precomputed; one read per timeline load.

engagement counts Counters (Redis) + batch persistence

  • likes:{tweetId}
  • retweets:{tweetId}

Likes and retweets on viral tweets are extremely hot.

Comments on these choices:

  • Tweets are sharded by tweet_id, not by user. A home timeline hydrates 20 ids from 20 different authors, so there is no locality to gain from sharding by author, and sharding by id spreads a celebrity's write and read load evenly. The user timeline gets its own index, partitioned by user_id, so "Sam's tweets" is still one partition read.
  • The social graph is stored twice: followers(user → follower) for fan-out and following(user → followee) for the celebrity merge at read time. Each is a simple adjacency list; Twitter built FlockDB for exactly this.
  • Home timelines hold ids only. A tweet id is 8 bytes; a full tweet is ~300. Storing ids also means an edited or deleted tweet changes in one place.
  • Counters live apart from the tweet row, so a viral tweet's likes don't lock the row that every timeline read needs.

5. High-level design

Your turn: draw it

Sketch the boxes for two flows: Sam posts a tweet that should reach his 200 followers, and Ana opens her home timeline. Where does the work happen in each?

Compare with our answer

Compare with the figures below. The pieces people forget: the queue between posting and fan-out (so posting is fast and spikes are absorbed), the hydration step (ids → tweets), and that the author's own tweet must appear for them immediately.

The architecture, then posting a tweet and fanning it out

  1. The architecture: a tweet service with an id generator writes tweets, and a queue drives fan-out workers that fill per-user timeline caches. The timeline service reads those caches, merges celebrity tweets on the fly, and hydrates ids into tweets.
  2. Posting: the tweet service gets a time-ordered Snowflake id, saves the tweet, and puts a “new tweet” event on a queue. Fan-out workers look up followers and push the id onto each follower’s cached timeline.
ComponentWhat it does
Tweet serviceValidates and stores tweets, appends to the author’s tweet list, emits events
ID generator (Snowflake)64-bit ids: time + machine + sequence, roughly sortable by time
Fan-out queue + workersPush tweet ids into followers’ timeline caches
Social graphFollowers / following with counts, heavily cached
Timeline cacheRedis list of ~800 tweet ids per active user
Timeline serviceReads cached ids, pulls followed celebrities’ recent tweets from the tweets store, merges, hydrates, paginates

Reading the home timeline, and what changes when a celebrity tweets

  1. Reading: the timeline service takes the precomputed ids from the cache, merges in recent tweets from any celebrities you follow, hydrates the ids into full tweets, and returns a page.
  2. When an account with a hundred million followers tweets, we skip fan-out entirely. The tweet just lands in that account’s tweet list, and followers pull it in when they load their timelines.

Trace the post flow out loud:

  1. Sam's app calls POST /v1/tweets. The tweet service gets a Snowflake id, writes the tweet and appends the id to Sam's own tweet list, then returns 201. Posting is now done from Sam's point of view.
  2. A "new tweet" event goes onto a partitioned queue (Kafka, for example).
  3. A fan-out worker reads Sam's followers in pages of a few thousand and, for each follower with a cached timeline, does LPUSH timeline:{follower} id followed by LTRIM to 800 entries, batched in Redis pipelines.

And the read flow:

  1. Ana's app calls GET /v1/timeline/home. The timeline service reads her cached id list (one Redis call).
  2. It merges in recent tweets from the few celebrities she follows (see the hybrid below) and from Ana herself.
  3. It hydrates the ids: a multi-get from a tweet cache (memcached in front of the tweet store) for ~20 tweets, plus author names and counts.
  4. It filters (deleted tweets, blocked or muted accounts, unfollowed authors) and returns a page with a cursor.

6. Deep dives

Your turn: the hard parts

Before reading on, write two or three sentences on each: a user with 100 M followers tweets; what happens? What exactly is in the timeline cache, and for whom? How are tweet ids generated across thousands of servers? How would you rank instead of sorting by time?

Compare with our answer

A strong answer mentions:

  • Push for normal accounts, pull for accounts over a follower threshold, merged at read time.
  • Ids only, ~800 per user, only for recently active users; rebuild on a miss.
  • Snowflake: timestamp + machine id + sequence, 64 bits, roughly time-sorted, no coordination.
  • Ranking as candidate generation → scoring → filtering, on top of the same timeline machinery.

Push or pull: when is a timeline built?

The problem. Ana follows 300 accounts. Building her timeline at read time means 300 queries per load, at 58 K loads/s. Building it at write time means one tweet from a 100 M-follower account is 100 M writes.

OptionHowCatch
Fan-out on read (pull)On each load, fetch recent tweets of everyone Ana follows and mergeHundreds of reads per load; p99 latency suffers as people follow more accounts
Fan-out on write (push)On each tweet, insert the id into every follower's cached timelineReads become one lookup, but a celebrity tweet is 100 M writes and minutes of lag
Hybrid (chosen)Push for accounts below a follower threshold; for the few above it, skip fan-out and pull their recent tweets at read timeTwo code paths, and the read path must merge

The choice: hybrid. Over 99.9% of accounts have modest follower counts, so pushing is cheap for them. The handful of huge accounts (say above ~1 M followers; the exact threshold is tuned from metrics) are fetched at read time: Ana follows perhaps 5 of them, so the merge is 5 small reads of "latest tweets by X", which are themselves among the most cached data in the system.

Even pushing has a cost for big-but-not-huge accounts. A 500 K-follower account means 500 K Redis inserts: split across many workers with pipelining that takes seconds, which the requirement ("a few seconds is fine") allows.

The timeline cache: what, for whom, and what if it's lost

Each active user gets a Redis list timeline:{userId} of the newest ~800 tweet ids. Three rules keep it affordable:

  • Ids, not tweets (8 bytes vs ~300).
  • Capped length. Hardly anyone scrolls past a few hundred tweets; older pages fall back to a slower pull.
  • Only for active users (logged in within the last few weeks). Fan-out workers skip followers with no cached timeline. When an inactive user returns, the timeline service rebuilds their list once with fan-out on read, then pushes keep it fresh.

Twitter ran this on Redis with several replicas of each timeline, because losing a cache node means rebuilding millions of timelines at once.

Tweet ids without coordination: Snowflake

Thousands of servers create tweets at once. A single auto-increment counter is a bottleneck and a single point of failure; random UUIDs are 128 bits and not ordered by time. Snowflake packs a 64-bit id from three parts:

BitsPartPurpose
41Milliseconds since a custom epochIds sort by time; lasts ~69 years
10Machine idUp to 1,024 generators, no coordination between them
12Per-machine sequenceUp to 4,096 ids per millisecond per machine

Sorting by id is (roughly) sorting by time, which is what makes "newest first", cursors and merging of pushed and pulled lists cheap. "Roughly", because two machines' clocks differ by a few milliseconds.

Seeing your own tweet, and other consistency details

Fan-out takes seconds, but Sam must see his tweet the instant he posts. The fix is read-your-writes at read time: the timeline service always merges Sam's own latest few tweets (from his user tweet list) into his home timeline. The app can also insert the tweet locally from the 201 response.

Unfollows and deletes are handled the same lazy way. Already-cached ids from an unfollowed account are filtered out during hydration and age out of the 800-entry list. A deleted tweet is marked deleted in the tweet store; hydration drops it, so no one needs to scrub 100 M cached lists.

Ranking: "For you" instead of newest-first

Ranking reuses everything above and adds a funnel:

  1. Candidate generation. Gather a pool of a few hundred to ~1,500 tweets: the cached ids, pulled celebrity tweets, and (for a "For you" feed) tweets from accounts you don't follow, found through your graph and interests. Twitter's 2023 write-up described about 1,500 candidates, roughly half from outside your network.
  2. Scoring. A machine-learned model predicts engagement (like, reply, retweet, dwell) per candidate from features such as author relationship, recency and past interactions.
  3. Filtering and mixing. Remove blocked or muted authors and near-duplicates, limit consecutive tweets from one author, insert ads.
Why ranking costs more than it looks

Scoring 1,500 candidates with a neural network for each of 58 K loads per second is around 90 M model evaluations a second. Real systems use a cheap first-stage ranker to cut the pool, then a heavy model on a few hundred, and cache the ranked result for a few minutes so that a refresh doesn't re-rank from scratch. In an interview, sketch the funnel and say which stage is expensive; you're not expected to design the model.

Hot counters

A viral tweet can get tens of thousands of likes a minute. UPDATE tweets SET likes = likes + 1 would lock one row on one shard. Instead, record each like (likes(tweet_id, user_id) so it's idempotent per user) and increment an in-memory counter that is flushed to storage every few seconds. The displayed count is briefly approximate, which nobody notices.

7. Bottlenecks, failures and monitoring

Scaling bottlenecks

BottleneckFix
Celebrity fan-outFollower threshold → pull at read time
Fan-out write volume (1 M+/s)Partitioned queue, many workers, batched Redis pipelines
Breaking-news spikesQueue absorbs bursts; timelines lag slightly rather than fail
Hot tweet countersIn-memory counters with periodic flush
Tweet storageShard by tweet id; cold tweets in cheaper storage

When things fail

FailureWhat happens
Fan-out workers downTweets are saved; timelines catch up when workers return
Timeline cache lostRebuild from the graph + recent tweets (fan-out on read) while it refills
ID generator node downOther nodes keep issuing ids; each has its own machine bits

What to monitor

Home timeline latency (p99)Fan-out lag (seconds)Queue depthCache hit ratioTweets per secondError rate per endpoint

8. Follow-up questions

Your turn: the interviewer pushes

Answer in order before opening ours. 1) The World Cup final ends and tweets jump to 10× normal for five minutes. What happens? 2) A user with 50 K followers is suddenly retweeted by a celebrity and gains 5 M followers in a day. What changes? 3) Product wants tweets editable for 30 minutes. What breaks?

Compare with our answer

A strong answer mentions:

  1. Posting stays fast because it only writes the tweet and enqueues; the fan-out queue grows and timelines lag by tens of seconds; autoscale workers, protect timeline reads first, alert on lag rather than errors.
  2. The account crosses the threshold and moves to pull; de-duplicate during the merge; the follower-count check must use a cached, approximate count so it's cheap per tweet.
  3. Very little in timelines, because they store ids and hydrate at read time; edits update one row; caches of hydrated tweets need invalidation or a short TTL; keep the edit history for display.
How would you rank the timeline instead of newest-first?

Fetch a few hundred candidates (cached ids + celebrity tweets), score them with a model, return the top 20.

How would you implement search?

A separate pipeline: tweets stream into an inverted index (like Earlybird / Elasticsearch), sharded by time.

How do retweets work?

A retweet is a small tweet with retweet_of = original id; timelines hydrate the original and show “Sam retweeted”.

More senior follow-ups
  • Multi-region. Each region keeps its own timeline caches, filled by fan-out workers consuming a replicated event stream; the tweet store replicates asynchronously. A tweet can appear a second later in another region, which the consistency requirement already allows.
  • Protected accounts and blocks. Fan-out only goes to approved followers, and hydration re-checks blocks, because a block can happen after the id was pushed.
  • Lists and topics. A list is another timeline: the same push/pull machinery keyed by list_id instead of user_id.
  • Why not one big SQL query? SELECT … WHERE author IN (300 ids) ORDER BY created_at LIMIT 20 across shards is a scatter-gather on every load; it works at small scale and is the honest first version to mention before the cache.
Twitter on one page
  • Numbers: 500 M tweets/day ≈ 5.8 K/s; 58 K timeline loads/s; ×200 followers ≈ 1.2 M fan-out writes/s; 800 ids × 8 B × 250 M ≈ 1.6 TB raw cache.
  • Two timelines: user timeline = one index read; home timeline = the hard part.
  • Hybrid fan-out: push ids for normal accounts via queue + workers; pull celebrity tweets at read time and merge; de-dupe by id.
  • Timeline cache: Redis list of ~800 ids, active users only, replicated; rebuild on miss with fan-out on read.
  • Read path: cached ids + celebrity pulls + own tweets → hydrate (multi-get) → filter → page with cursor.
  • Ids: Snowflake = 41-bit ms + 10-bit machine + 12-bit sequence; time-sortable; guard against clock going backwards.
  • Consistency: eventual for followers, read-your-writes for the author; deletes and unfollows filtered at hydration.
  • Ranking: candidates (~hundreds to 1.5 K) → model score → filter and mix.
  • Watch: fan-out lag, timeline p99, cache hit ratio, queue depth.
Mock interviewPractise it as a mock interview: push or pull, and the celebrity problemOpens in the ByteAtlas lab. Your progress is saved in this browser.
Check what stuck

1. A tweet from an account with 100 M followers is fanned out on write like every other tweet. What is the main consequence?

2. Ana follows 300 accounts, 4 of them above the celebrity threshold. What does loading her home timeline involve?

3. Why does the timeline cache hold tweet ids rather than full tweets?

4. 250 M users each have 800 cached ids at 8 bytes. A colleague proposes caching timelines for all 1.5 B registered accounts instead. What is the best response?

5. Sam posts a tweet and immediately refreshes. Fan-out takes 3 seconds. How do you make sure he sees it?

6. Why are Snowflake ids preferred over random UUIDs for tweets?

7. A Redis node holding 5% of timelines fails at peak and its replica is also down. What is the biggest risk?

Score 60% or more and the lesson is marked complete.

Passing the quiz also marks it complete. Progress is saved in this browser only.