Design it yourself · Chapter 21

Design a chat app

Design a WhatsApp-style chat app from scratch: WebSocket gateways, routing a message to the right server, ordering, offline push, sharded history, and where it fails.

IntermediateInterview priority: High40 min
On this page
  1. The problem: "anything new for me?" two hundred thousand times a second
  2. Step 1: requirements
  3. Step 2: quick estimates
  4. Step 3: the API
  5. Step 4: build it up
  6. Where this breaks
  7. When to keep it simpler
  8. In the interview
  9. Where to go next
  10. Quiz

The problem: "anything new for me?" two hundred thousand times a second

Asha sends Ben "on my way!". Ben should see it within a second, and if his phone is off, it must be waiting for him when he turns it on. The obvious first version is a normal web API: Asha's phone POSTs the message, and Ben's phone asks GET /messages?since=… every few seconds.

That is polling, and at scale it fails twice. With 200,000 phones online polling every 2 seconds, the servers answer 100,000 requests a second, nearly all of them "nothing new". And a message still waits up to 2 seconds to be noticed. Poll faster and the load goes up; poll slower and chat feels broken.

The goal, and the numbers we design for

  1. Next: a chat app like WhatsApp. One person sends a message; the other should see it almost instantly. If they’re offline, they get a push notification and the message waits for them.
  2. The numbers: 200,000 people online at once, 5,000 messages sent per second, and every message stored forever: that adds up to about 12 terabytes a year. Unlike the URL shortener, this is write-heavy.

This lesson is a first design that applies APIs and WebSockets, sharding, message queues and caching. Unlike the URL shortener, which was read-heavy, chat is write-heavy: every message is a new row.

Step 1: requirements

KindRequirementConsequence
Functional1:1 chats and small groups (up to ~100 members)Groups mean one message, many deliveries
FunctionalReal-time delivery when the recipient is onlineThe server must be able to push
FunctionalOffline recipients get a push notification and the message laterStore first, deliver later
FunctionalFull history kept on the server, loadable on a new phoneStorage grows forever
Non-functionalDelivered in under 500 ms when both are onlineNo polling; persistent connections
Non-functionalAn acknowledged message is never lostStore durably before saying "sent"
Non-functionalMessages appear in the same order for everyone in a chatSequence numbers, not phone clocks

Out of scope: end-to-end encryption, voice and video calls, media beyond a note on where it goes. (WhatsApp keeps messages only until delivered and encrypts end to end; we design the Slack or Messenger style that keeps history.)

Step 2: quick estimates

QuantityWorkingResult
Online at oncegiven200,000 connections
Messages sentgiven5,000 / s average, ~15,000 at peak
History loadsopening a chat, scrolling back3,000 / s
Messages per day5,000 × 86,400≈ 430 million
Storage per year430 M × ~80 bytes (text + IDs + timestamps) × 365≈ 12 TB a year, ×3 with replication
Gateways200,000 ÷ 50,000 per server4, plus one spare
Push notificationsmessages to offline users≈ 200 / s

The numbers say three things. Connections decide how many gateway servers you need. Writes are steady and never stop, so storage must be split. And reads cluster on recent messages, which makes them easy to cache.

Step 3: the API

Two channels, used for different things:

  • One WebSocket per online phone for everything real-time. Client → server: send {client_msg_id, chat_id, text}. Server → client: ack {client_msg_id, seq}, message {chat_id, seq, sender, text, sent_at}, receipt {chat_id, seq, delivered|read}.
  • Plain HTTP for everything else: GET /v1/chats/{id}/messages?before_seq=…&limit=50 for history, GET /v1/sync?since=… after being offline, POST /v1/chats to create a group.

The client_msg_id is a UUID made by the phone. If the ack is lost and the phone resends, the server recognises the ID and does not store the message twice.

Step 4: build it up

Hold a connection to every online phone

The server can only push if the phone keeps a connection open. A WebSocket starts as an HTTP request and is then "upgraded" into a long-lived, two-way channel over the same TCP connection.

Gateways hold the open connections

  1. Instant delivery means the server must push, so every online phone keeps a WebSocket open. Special gateway servers hold those connections, around 50,000 each.

Gateways are servers whose job is to hold those connections: they terminate TLS, check heartbeats (a tiny ping every 30–60 s so dead connections are noticed and mobile networks don't close idle sockets), and pass messages to and from the chat logic. Keeping them separate means a chat-logic deploy does not drop 200,000 connections. The load balancer in front works at layer 4 (TCP) so each connection stays pinned to one gateway.

OptionHowLatencyCost at 200 K online
Short pollingAsk every few secondsUp to the interval~100 K mostly empty requests/s
Long pollingRequest hangs until there is newsLowA new request after every message
Server-Sent EventsOne-way server → phone streamLowSending still needs separate requests
WebSocketOne persistent two-way connectionLowestOne open connection per online user

Choice: WebSocket, with HTTP for history and sync.

Find the recipient's connection

Asha is on gateway 1; Ben is on gateway 3. How does gateway 1 know that? When a phone connects, its gateway writes user → gateway into a session map in Redis, with a TTL that each heartbeat refreshes.

Store, ack, look up, forward, deliver

The order of those steps is the heart of the design:

  1. Store first. The message is written durably with its sequence number.
  2. Then ack the sender (one tick). An ack before the write would be a promise you can't keep if the server dies.
  3. Then deliver. Look up the recipient; if online, forward to their gateway; if not, queue a push notification.
  4. The recipient's ack produces the "delivered" receipt (two ticks).

Keep messages in order

Phone clocks disagree and messages travel through different servers, so timestamps can't define order. Each chat gets a counter, and every stored message takes the next number: seq 41, 42, 43. Phones display by seq, and a gap (41, then 43) tells the phone to fetch 42.

OptionHowCatch
Sender's timestampSort by the time the phone attachedClocks are wrong by seconds or minutes
Redis INCR per chatAtomic counter in memoryAfter a failover the new primary can reissue numbers
Counter row on the chat's shardIn one transaction: bump chats.last_seq, insert the messageSerialises writes within one chat (fine: a chat gets a few messages a second)

Choice: the counter row, which works because of how we shard (next step). Assigning the number in the same transaction that stores the message means a number never exists without its message.

BEGIN;
UPDATE chats SET last_seq = last_seq + 1 WHERE chat_id = 81 RETURNING last_seq;   -- 42
INSERT INTO messages (chat_id, seq, client_msg_id, sender_id, body, sent_at)
VALUES (81, 42, 'c0f1…', 7, 'on my way!', now())
ON CONFLICT (chat_id, client_msg_id) DO NOTHING;   -- a retry stores nothing new
COMMIT;

If the insert hits the conflict, the transaction should be rolled back and the original seq looked up and re-acked, so a retry does not burn a number. Ordering is only promised within a chat, which is what keeps it cheap.

Store the history: shard by chat

Messages sharded by conversation ID

  1. Five thousand writes a second, all day, pushes one database leader hard, and 12 more terabytes every year soon outgrow one machine. So shard the messages, by conversation ID, so each chat’s history lives together on one shard.

Writes never stop and 12 TB a year outgrows one machine, so messages are sharded. The shard key decides which queries stay on one machine:

Shard keyGood atWeak at
chat_idA chat's history is one range read on one shard; the seq counter and the messages share a transactionA huge, busy group concentrates on one shard
user_id"All my chats" is localA message between two users belongs to two shards: write it twice or look elsewhere
message_idPerfectly even spreadLoading one chat's history asks every shard

Choice: chat_id, hashed so chats spread evenly. Postgres or MySQL shards work well at this size; at much larger scale, teams move to wide-column stores built for append-heavy, time-ordered data (Discord stores messages partitioned by channel and time bucket in Cassandra, later ScyllaDB). A separate small table, sharded by user_id, lists each user's chats and their latest seq, so a phone can ask "which chats changed since I was last online?".

Offline users: queue the push

A queue and workers send push notifications

  1. If Ben is offline, the chat service puts a “notify Ben” job on a queue. Workers send the push notification, without slowing down message delivery for everyone else.

If the session map has no entry for Ben, the message is already stored, so nothing is lost. The chat service puts a "notify Ben" job on a queue; push workers call Apple's APNs or Google's FCM. The queue keeps a slow push provider from ever slowing message delivery, and absorbs bursts.

When Ben's phone reconnects, it calls sync with the last seq it has for each chat and receives everything newer, in order.

The full picture

Gateways, chat service, cache, shards and the push queue

  1. Recent messages are read most, so a cache of each chat’s last screen of messages saves many database reads. The full picture: gateways for live connections, app servers, a cache, sharded storage, and a queue for notifications.

A Redis list per chat holding the last ~50 messages answers most "open a chat" requests from memory, so the 3,000 history loads a second mostly skip the database. Groups use the same path: store the message once under the group's chat_id, then deliver to each member through the session map, with offline members getting a push. Photos and files never travel through the chat servers: the phone uploads them to object storage and sends a small message holding the link.

Where this breaks

Two more to name if time allows: a hot group (a 100-member group in a live event puts every write on one chat's counter row and one shard, so cap group size and batch deliveries per gateway), and read receipts (each message in a group produces a receipt per member, so receipts can outnumber messages; batch them, and send "read up to seq 57" rather than one event per message).

When to keep it simpler

  • A few thousand users: one server can hold every connection and one Postgres can store every message. Skip the gateway split, sharding and session map until the numbers demand them.
  • Notifications, not conversation: if users only receive updates (order status, alerts), Server-Sent Events or push alone is enough; you don't need two-way connections.
  • Huge broadcast channels (millions of followers) are a feed problem, not a chat problem: fan out on read, as in the news feed.

In the interview

Your turn: spot the bugs

A design says: "The gateway receives a message, forwards it to the recipient's gateway, acks the sender, then writes it to the database in the background. Messages are sorted by the sender's timestamp. Push notifications are sent inside the request, before the ack." Find every problem.

Compare with our answer
  • Ack before the durable write: if the server dies after the ack, an "acknowledged" message is lost.
  • Background write after forwarding: the recipient may see a message that never reaches the database, so it vanishes on their next device.
  • Sender timestamps disagree across phones, so two people can see different orders; use a per-chat seq.
  • Push inside the request: a slow APNs or FCM call delays the ack and couples delivery to a third party; queue it.
  • No de-duplication is mentioned, so retries create duplicates.

A strong answer mentions: store before ack · forward after storing · server-assigned sequence numbers · push through a queue · client message IDs for retries.

Your turn: the requirement changes

Product wants multiple devices per user (phone, laptop and tablet) with all of them showing new messages live, and "read on one device" clearing the unread badge on the others. What changes?

Compare with our answer
  • The session map becomes per device: user → [(device, gateway)], each with its own heartbeat TTL.
  • Delivery fans out to every online device of each recipient, and also to the sender's other devices, so the laptop shows what the phone sent.
  • Each device keeps its own last-synced seq per chat, so sync works independently.
  • "Read up to seq N" becomes an event pushed to the user's other devices, and stored so offline devices pick it up.
  • Load grows with devices, not users: ~2–3× more connections and deliveries.

A strong answer mentions: per-device sessions · fan-out to all devices including the sender's · per-device sync position · read state synced as an event · gateway capacity re-estimated.

Where to go next

The full interview-length version, with a 45-minute plan, store-and-forward, end-to-end encryption and deeper ordering, is Design WhatsApp. Next in this module: design a news feed.

Chat app on one page
  • Numbers: 200 K online → 4 gateways at 50 K + spare; 5,000 msgs/s (write-heavy); ~12 TB/year; 3,000 history loads/s; ~200 pushes/s.
  • Transport: WebSocket per online phone, L4 load balancer, heartbeats every 30–60 s; HTTP for history and sync.
  • Routing: session map in Redis user → gateway, TTL refreshed by heartbeat.
  • Write path: store with seq → ack (✓) → look up → forward → recipient ack (✓✓). Forwarding is not delivery.
  • Exactly once (effectively): retries + client_msg_id de-duplication.
  • Ordering: per-chat seq assigned in the same transaction as the insert; gaps trigger a fetch.
  • Storage: shard by chat_id; per-user chat list sharded by user_id; last ~50 messages per chat in Redis.
  • Offline: queue → push workers → APNs/FCM; on reconnect, sync from last seq per chat.
  • Failures: stale sessions, reconnect storms (jittered backoff), hot groups, receipt amplification.
Interactive labBuild it yourself on the design boardOpens in the ByteAtlas lab. Your progress is saved in this browser.
Check what stuck

1. 200,000 phones poll for new messages every 2 seconds. What is the main problem compared with WebSockets?

2. When should the server send the "sent" acknowledgement (one tick) to the sender?

3. Asha's phone is on gateway 1 and Ben's on gateway 3. How does gateway 1 deliver her message?

4. Which shard key fits chat message history best?

5. A phone sends a message, the server stores it, the ack is lost, and the phone resends. What stops a duplicate?

6. A gateway holding 50,000 connections crashes and all those phones reconnect in the same second. What is the best protection?

7. Why assign each message's sequence number in the same transaction that inserts it?

Score 60% or more and the lesson is marked complete.

Passing the quiz also marks it complete. Progress is saved in this browser only.