Design a chat app
Design a WhatsApp-style chat app from scratch: WebSocket gateways, routing a message to the right server, ordering, offline push, sharded history, and where it fails.
On this page
The problem: "anything new for me?" two hundred thousand times a second
Asha sends Ben "on my way!". Ben should see it within a second, and if his phone is off, it must be waiting for him when he turns it on. The obvious first version is a normal web API: Asha's phone POSTs the message, and Ben's phone asks GET /messages?since=… every few seconds.
That is polling, and at scale it fails twice. With 200,000 phones online polling every 2 seconds, the servers answer 100,000 requests a second, nearly all of them "nothing new". And a message still waits up to 2 seconds to be noticed. Poll faster and the load goes up; poll slower and chat feels broken.
The goal, and the numbers we design for
- Next: a chat app like WhatsApp. One person sends a message; the other should see it almost instantly. If they’re offline, they get a push notification and the message waits for them.
- The numbers: 200,000 people online at once, 5,000 messages sent per second, and every message stored forever: that adds up to about 12 terabytes a year. Unlike the URL shortener, this is write-heavy.
This lesson is a first design that applies APIs and WebSockets, sharding, message queues and caching. Unlike the URL shortener, which was read-heavy, chat is write-heavy: every message is a new row.
Step 1: requirements
| Kind | Requirement | Consequence |
|---|---|---|
| Functional | 1:1 chats and small groups (up to ~100 members) | Groups mean one message, many deliveries |
| Functional | Real-time delivery when the recipient is online | The server must be able to push |
| Functional | Offline recipients get a push notification and the message later | Store first, deliver later |
| Functional | Full history kept on the server, loadable on a new phone | Storage grows forever |
| Non-functional | Delivered in under 500 ms when both are online | No polling; persistent connections |
| Non-functional | An acknowledged message is never lost | Store durably before saying "sent" |
| Non-functional | Messages appear in the same order for everyone in a chat | Sequence numbers, not phone clocks |
Out of scope: end-to-end encryption, voice and video calls, media beyond a note on where it goes. (WhatsApp keeps messages only until delivered and encrypts end to end; we design the Slack or Messenger style that keeps history.)
Step 2: quick estimates
| Quantity | Working | Result |
|---|---|---|
| Online at once | given | 200,000 connections |
| Messages sent | given | 5,000 / s average, ~15,000 at peak |
| History loads | opening a chat, scrolling back | 3,000 / s |
| Messages per day | 5,000 × 86,400 | ≈ 430 million |
| Storage per year | 430 M × ~80 bytes (text + IDs + timestamps) × 365 | ≈ 12 TB a year, ×3 with replication |
| Gateways | 200,000 ÷ 50,000 per server | 4, plus one spare |
| Push notifications | messages to offline users | ≈ 200 / s |
The numbers say three things. Connections decide how many gateway servers you need. Writes are steady and never stop, so storage must be split. And reads cluster on recent messages, which makes them easy to cache.
Step 3: the API
Two channels, used for different things:
- One WebSocket per online phone for everything real-time. Client → server:
send {client_msg_id, chat_id, text}. Server → client:ack {client_msg_id, seq},message {chat_id, seq, sender, text, sent_at},receipt {chat_id, seq, delivered|read}. - Plain HTTP for everything else:
GET /v1/chats/{id}/messages?before_seq=…&limit=50for history,GET /v1/sync?since=…after being offline,POST /v1/chatsto create a group.
The client_msg_id is a UUID made by the phone. If the ack is lost and the phone resends, the server recognises the ID and does not store the message twice.
Step 4: build it up
Hold a connection to every online phone
The server can only push if the phone keeps a connection open. A WebSocket starts as an HTTP request and is then "upgraded" into a long-lived, two-way channel over the same TCP connection.
Gateways hold the open connections
- Instant delivery means the server must push, so every online phone keeps a WebSocket open. Special gateway servers hold those connections, around 50,000 each.
Gateways are servers whose job is to hold those connections: they terminate TLS, check heartbeats (a tiny ping every 30–60 s so dead connections are noticed and mobile networks don't close idle sockets), and pass messages to and from the chat logic. Keeping them separate means a chat-logic deploy does not drop 200,000 connections. The load balancer in front works at layer 4 (TCP) so each connection stays pinned to one gateway.
| Option | How | Latency | Cost at 200 K online |
|---|---|---|---|
| Short polling | Ask every few seconds | Up to the interval | ~100 K mostly empty requests/s |
| Long polling | Request hangs until there is news | Low | A new request after every message |
| Server-Sent Events | One-way server → phone stream | Low | Sending still needs separate requests |
| WebSocket | One persistent two-way connection | Lowest | One open connection per online user |
Choice: WebSocket, with HTTP for history and sync.
Find the recipient's connection
Asha is on gateway 1; Ben is on gateway 3. How does gateway 1 know that? When a phone connects, its gateway writes user → gateway into a session map in Redis, with a TTL that each heartbeat refreshes.
Store, ack, look up, forward, deliver
The order of those steps is the heart of the design:
- Store first. The message is written durably with its sequence number.
- Then ack the sender (one tick). An ack before the write would be a promise you can't keep if the server dies.
- Then deliver. Look up the recipient; if online, forward to their gateway; if not, queue a push notification.
- The recipient's ack produces the "delivered" receipt (two ticks).
Keep messages in order
Phone clocks disagree and messages travel through different servers, so timestamps can't define order. Each chat gets a counter, and every stored message takes the next number: seq 41, 42, 43. Phones display by seq, and a gap (41, then 43) tells the phone to fetch 42.
| Option | How | Catch |
|---|---|---|
| Sender's timestamp | Sort by the time the phone attached | Clocks are wrong by seconds or minutes |
Redis INCR per chat | Atomic counter in memory | After a failover the new primary can reissue numbers |
| Counter row on the chat's shard | In one transaction: bump chats.last_seq, insert the message | Serialises writes within one chat (fine: a chat gets a few messages a second) |
Choice: the counter row, which works because of how we shard (next step). Assigning the number in the same transaction that stores the message means a number never exists without its message.
BEGIN;
UPDATE chats SET last_seq = last_seq + 1 WHERE chat_id = 81 RETURNING last_seq; -- 42
INSERT INTO messages (chat_id, seq, client_msg_id, sender_id, body, sent_at)
VALUES (81, 42, 'c0f1…', 7, 'on my way!', now())
ON CONFLICT (chat_id, client_msg_id) DO NOTHING; -- a retry stores nothing new
COMMIT;
If the insert hits the conflict, the transaction should be rolled back and the original seq looked up and re-acked, so a retry does not burn a number. Ordering is only promised within a chat, which is what keeps it cheap.
Store the history: shard by chat
Messages sharded by conversation ID
- Five thousand writes a second, all day, pushes one database leader hard, and 12 more terabytes every year soon outgrow one machine. So shard the messages, by conversation ID, so each chat’s history lives together on one shard.
Writes never stop and 12 TB a year outgrows one machine, so messages are sharded. The shard key decides which queries stay on one machine:
| Shard key | Good at | Weak at |
|---|---|---|
chat_id | A chat's history is one range read on one shard; the seq counter and the messages share a transaction | A huge, busy group concentrates on one shard |
user_id | "All my chats" is local | A message between two users belongs to two shards: write it twice or look elsewhere |
message_id | Perfectly even spread | Loading one chat's history asks every shard |
Choice: chat_id, hashed so chats spread evenly. Postgres or MySQL shards work well at this size; at much larger scale, teams move to wide-column stores built for append-heavy, time-ordered data (Discord stores messages partitioned by channel and time bucket in Cassandra, later ScyllaDB). A separate small table, sharded by user_id, lists each user's chats and their latest seq, so a phone can ask "which chats changed since I was last online?".
Offline users: queue the push
A queue and workers send push notifications
- If Ben is offline, the chat service puts a “notify Ben” job on a queue. Workers send the push notification, without slowing down message delivery for everyone else.
If the session map has no entry for Ben, the message is already stored, so nothing is lost. The chat service puts a "notify Ben" job on a queue; push workers call Apple's APNs or Google's FCM. The queue keeps a slow push provider from ever slowing message delivery, and absorbs bursts.
When Ben's phone reconnects, it calls sync with the last seq it has for each chat and receives everything newer, in order.
The full picture
Gateways, chat service, cache, shards and the push queue
- Recent messages are read most, so a cache of each chat’s last screen of messages saves many database reads. The full picture: gateways for live connections, app servers, a cache, sharded storage, and a queue for notifications.
A Redis list per chat holding the last ~50 messages answers most "open a chat" requests from memory, so the 3,000 history loads a second mostly skip the database. Groups use the same path: store the message once under the group's chat_id, then deliver to each member through the session map, with offline members getting a push. Photos and files never travel through the chat servers: the phone uploads them to object storage and sends a small message holding the link.
Where this breaks
Two more to name if time allows: a hot group (a 100-member group in a live event puts every write on one chat's counter row and one shard, so cap group size and batch deliveries per gateway), and read receipts (each message in a group produces a receipt per member, so receipts can outnumber messages; batch them, and send "read up to seq 57" rather than one event per message).
When to keep it simpler
- A few thousand users: one server can hold every connection and one Postgres can store every message. Skip the gateway split, sharding and session map until the numbers demand them.
- Notifications, not conversation: if users only receive updates (order status, alerts), Server-Sent Events or push alone is enough; you don't need two-way connections.
- Huge broadcast channels (millions of followers) are a feed problem, not a chat problem: fan out on read, as in the news feed.
In the interview
A design says: "The gateway receives a message, forwards it to the recipient's gateway, acks the sender, then writes it to the database in the background. Messages are sorted by the sender's timestamp. Push notifications are sent inside the request, before the ack." Find every problem.
Compare with our answer
- Ack before the durable write: if the server dies after the ack, an "acknowledged" message is lost.
- Background write after forwarding: the recipient may see a message that never reaches the database, so it vanishes on their next device.
- Sender timestamps disagree across phones, so two people can see different orders; use a per-chat
seq. - Push inside the request: a slow APNs or FCM call delays the ack and couples delivery to a third party; queue it.
- No de-duplication is mentioned, so retries create duplicates.
A strong answer mentions: store before ack · forward after storing · server-assigned sequence numbers · push through a queue · client message IDs for retries.
Product wants multiple devices per user (phone, laptop and tablet) with all of them showing new messages live, and "read on one device" clearing the unread badge on the others. What changes?
Compare with our answer
- The session map becomes per device:
user → [(device, gateway)], each with its own heartbeat TTL. - Delivery fans out to every online device of each recipient, and also to the sender's other devices, so the laptop shows what the phone sent.
- Each device keeps its own last-synced
seqper chat, so sync works independently. - "Read up to seq N" becomes an event pushed to the user's other devices, and stored so offline devices pick it up.
- Load grows with devices, not users: ~2–3× more connections and deliveries.
A strong answer mentions: per-device sessions · fan-out to all devices including the sender's · per-device sync position · read state synced as an event · gateway capacity re-estimated.
Where to go next
The full interview-length version, with a 45-minute plan, store-and-forward, end-to-end encryption and deeper ordering, is Design WhatsApp. Next in this module: design a news feed.
- Numbers: 200 K online → 4 gateways at 50 K + spare; 5,000 msgs/s (write-heavy); ~12 TB/year; 3,000 history loads/s; ~200 pushes/s.
- Transport: WebSocket per online phone, L4 load balancer, heartbeats every 30–60 s; HTTP for history and sync.
- Routing: session map in Redis
user → gateway, TTL refreshed by heartbeat. - Write path: store with
seq→ ack (✓) → look up → forward → recipient ack (✓✓). Forwarding is not delivery. - Exactly once (effectively): retries +
client_msg_idde-duplication. - Ordering: per-chat
seqassigned in the same transaction as the insert; gaps trigger a fetch. - Storage: shard by
chat_id; per-user chat list sharded byuser_id; last ~50 messages per chat in Redis. - Offline: queue → push workers → APNs/FCM; on reconnect, sync from last
seqper chat. - Failures: stale sessions, reconnect storms (jittered backoff), hot groups, receipt amplification.
1. 200,000 phones poll for new messages every 2 seconds. What is the main problem compared with WebSockets?
Polling wastes requests and adds delay at the same time. A persistent connection lets the server push the moment a message is stored. TLS works for both.
2. When should the server send the "sent" acknowledgement (one tick) to the sender?
One tick promises the message won't be lost, so it must follow the durable write. Waiting for the recipient would make the tick depend on whether their phone is on.
3. Asha's phone is on gateway 1 and Ben's on gateway 3. How does gateway 1 deliver her message?
Each gateway records its connected users in the session map. Broadcasting multiplies traffic by the number of gateways; polling brings back the delay.
4. Which shard key fits chat message history best?
Loading a chat reads one range on one shard, and the counter and insert share a transaction. Message ID spreads writes but makes every history load ask every shard; date puts all writes on today's shard.
5. A phone sends a message, the server stores it, the ack is lost, and the phone resends. What stops a duplicate?
Retries give at-least-once delivery; de-duplicating by client_msg_id makes it effectively exactly-once. TCP only de-duplicates within one connection, and a retry usually comes on a new one.
6. A gateway holding 50,000 connections crashes and all those phones reconnect in the same second. What is the best protection?
A reconnect storm is a burst of handshakes, session writes and syncs. Spreading the retries out in time and limiting the accept rate stops one failure from cascading into the next gateway.
7. Why assign each message's sequence number in the same transaction that inserts it?
If the number were taken first and the insert then failed, phones would see a gap that can never be filled. Numbers are per chat, not global.
Score 60% or more and the lesson is marked complete.