Design Dropbox
Design Dropbox in a 45-minute interview: chunking and content-defined chunking, deduplication, metadata vs block storage, the sync protocol, conflicts and change notifications.
On this page
How to use this page
This page follows the order of a real system design interview. At each step there is a Your turn box: write your own answer first (it stays in your browser), then open ours and compare. Reading a finished design feels easy; producing one under time pressure is the skill you are practising.
| Phase | Time in a 45-min interview | What you produce |
|---|---|---|
| 1. Clarify requirements | 5 min | Functional list, non-functional targets, what's out of scope |
| 2. Estimate | 5 min | Storage, upload bandwidth, number of blocks |
| 3. API | 5 min | The upload, commit, change-feed and download calls |
| 4. Data model | 5 min | Metadata tables, the journal, the block index, where bytes live |
| 5. High-level design | 10 min | Boxes and arrows, and the upload and sync flows traced through them |
| 6. Deep dives | 10–15 min | Chunking, sync, conflicts, notifications, durability |
| 7. Wrap up | 2–3 min | Bottlenecks, failures, what you would monitor |
Users have a folder that syncs across their devices. Files can be large. Changes on any device appear on the others. Users can share folders and restore old versions. Skip real-time co-editing (that’s Docs), search inside files, and the web preview of documents.
1. Clarify the requirements
The interviewer says: "Design Dropbox." Write down what you would ask, then 4–6 functional requirements, 4–5 non-functional requirements with a target, and 2–3 things you would put out of scope.
Compare with our answer
The questions that matter most: How large can a file be? Do we need version history, and for how long? Shared folders? Must it work offline? Real-time co-editing, or whole-file sync? That last one changes everything: co-editing is the Google Docs problem (operations merged character by character); Dropbox syncs whole files and keeps conflicting copies side by side.
A strong answer mentions: upload/download any size · sync to every device · sharing · version history · offline edits that catch up · durability as the first non-functional requirement · co-editing out of scope.
Functional: what it must do
- Upload and download files: Any size, from any device
- Automatic sync: Changes appear on all devices
- Share folders: With view or edit rights
- Version history: Restore earlier versions for 30 days
- Works offline: Sync catches up when back online
Out of scope
Real-time co-editingSearch inside filesDocument previews
Non-functional: how well it must work
| Quality | Target | Why |
|---|---|---|
| Durability | Eleven nines: files must never be lost | It’s people’s only copy |
| Sync latency | Seconds for small changes | |
| Bandwidth efficiency | Upload only what changed | Large files, slow uplinks |
| Consistency | Every device converges; conflicts kept, never silently overwritten | |
| Scale | 700 M users, exabytes stored |
2. Back-of-the-envelope estimates
Assume 700 M users storing 20 GB each, deduplication saving 30%, 15% of users active per day, and each active user changing 50 MB a day. Work out total storage, upload bytes per day and per second, and the number of 4 MB blocks.
Compare with our answer
- Raw storage: 700 M × 20 GB = 14 EB; after 30% dedup ≈ 10 EB. Physically more: erasure coding adds roughly 1.3–1.5× on top.
- Active users: 700 M × 15% = 105 M. Changed data: 105 M × 50 MB ≈ 5 PB/day.
- Bandwidth: 5 PB ÷ 86,400 s ≈ 60 GB/s on average (about 500 Gbps), more at peak.
- Blocks: 10 EB ÷ 4 MB ≈ 2.5 trillion blocks, each needing an entry in the block index.
- Metadata: a few hundred bytes per file version. Even 100 billion file versions is tens of TB: tiny next to the blocks, but read constantly.
A strong answer mentions: exabytes of blocks vs terabytes of metadata · bandwidth as the cost driver · the number of index entries · dedup saving · overhead of redundancy.
Assumptions: users 700 M; data per user 20 GB; saved by deduplication (%) 30; changed data per active user per day 50 MB; daily active share (%) 15.
| Number | Estimate | Why it matters |
|---|---|---|
| Total stored (after dedup) | 9.8 EB | ≈ 10 EB: exabyte-scale block storage. |
| Uploaded per day | 5.2 PB | With chunking, only changed blocks travel. |
| Average upload bandwidth | 60.8 GB/s | 5 PB ÷ 86,400 s ≈ 60 GB/s. |
| 4 MB blocks stored | 2.45 T | Each needs a hash → location entry in the block index. |
- Metadata (file trees, versions) is tiny compared with blocks, but it’s queried constantly: keep it separate.
- Many users store the same files (installers, popular PDFs): deduplicate by content hash.
The key insight: two kinds of data with completely different shapes. Blocks are exabytes, immutable and read rarely; metadata is small, mutable, transactional and queried thousands of times a second ("what changed in this folder?"). They must live in different systems. And at 60 GB/s of changes, sending only the changed parts of a file is not an optimisation, it is the budget.
3. API
A laptop edits a 2 GB file. List the calls it makes to upload the change, and the calls the phone makes to pick it up. Which call can fail with a conflict?
Compare with our answer
A strong answer mentions: ask which block hashes the server is missing · upload only those, addressed by hash · a separate commit that points the file at its new block list and carries the version it was based on · a change feed read from a cursor · a long poll (or push) that only says "something changed" · downloads by hash, through a CDN.
| Call | Request | Response | Notes |
|---|---|---|---|
POST /v1/blocks/check | { hashes[] } | { missing[] } | Which of these chunks does the server not have yet? |
PUT /v1/blocks/{hash} | 4 MB chunk | 201 | Upload only the missing chunks |
POST /v1/files/commit | { path, blockHashes[], baseVersion } | { fileId, version } or 409 conflict | Atomically point the file at its new list of blocks |
GET /v1/changes?cursor= | – | { entries[], cursor, hasMore } | Everything that changed since my cursor |
GET /v1/notify?cursor= | (long poll) | { changes: true } after up to 60 s | Wakes the client when something changes |
GET /v1/blocks/{hash} | – | chunk bytes | Download, often via CDN |
Three things to point out about this API:
- Blocks are addressed by their hash (
PUT /v1/blocks/{sha256}). Uploading the same block twice is harmless: the second upload is a no-op. That makes block upload naturally idempotent (safe to retry), which matters on flaky networks. - Upload and commit are separate. The file only changes when the commit succeeds, and the commit is rejected unless every block it names is already stored. So the metadata can never point at a missing block.
baseVersionon commit is the hook for conflict detection: "I edited version 7". If the server is at version 8, the commit gets409 Conflict.
4. Data model
What do you store about a file, what do you store about a block, and how does a device find out what changed since it was last online? Pick a database for each.
Compare with our answer
A strong answer mentions: a file table keyed by namespace and path holding the version and ordered list of block hashes · a per-namespace append-only journal with a sequence number · a hash → location index with a reference count · blocks in object storage · sharding metadata by namespace so one folder's changes are in one place.
file metadata Sharded SQL (by namespace / user)
namespace_id, path, file_idversion, block_hashes[]size, modified_at, deleted
File trees, versions and sharing are relational and need transactions; Dropbox ran this on sharded MySQL for years.
journal (changes) Sharded SQL, append-only per namespace
namespace_id, seqfile_id, op, version
An ordered list of every change; each device remembers its cursor in it.
block index Key-value store
sha256 → storage_location, refcount
hash → where the block lives; also tells us if we already have it (dedup).
blocks Object / block storage (Magic Pocket, S3)
blocks/{sha256}
Immutable 4 MB chunks, erasure-coded across data centres for durability.
Two terms in this model need defining:
- A namespace is a tree of files that syncs as a unit: your personal Dropbox is one namespace, and each shared folder is its own namespace that is mounted into each member's tree. Sharding metadata by namespace keeps every change to one folder on one shard, so a commit and its journal entry fit in one local transaction.
- The journal is an ordered, append-only list of every change in a namespace, numbered 1, 2, 3… A device remembers the last number it has applied: its cursor. "Sync" is then simply "give me journal entries after 5,812".
5. High-level design
Sketch the boxes: where does a file's content go, where does its description go, and how does the phone learn that the laptop changed something? Then step through ours.
Compare with our answer
Compare your sketch with the figure. The pieces people most often forget: the split between metadata and blocks, the journal and cursor (instead of comparing whole folder trees), and a notification channel that carries no data.
The architecture, then editing one slide of a 2 GB file
- The key idea is separating metadata from blocks. A metadata service with a sharded database and a journal tracks files, versions and changes. A block service stores immutable chunks addressed by their hash. A notification service tells devices when to sync.
- You change one slide in a 2 GB file. The client splits the file into 4 MB chunks, hashes each, and asks which hashes the server lacks. Only the one or two changed chunks are uploaded, then the file’s new block list is committed.
| Component | What it does |
|---|---|
| Client sync engine | Watches the folder, chunks and hashes files, keeps a local index |
| Metadata service | Files, folders, versions, sharing; commits are transactional |
| Journal | Ordered change log per namespace; devices sync from a cursor |
| Block service + index | Content-addressed chunks; dedup by hash |
| Block storage | Erasure-coded, multi-region, durable |
| Notification service | Long polls that wake devices when their namespaces change |
Trace the upload out loud:
- The laptop's sync engine notices
deck.pptxchanged. It splits the file into chunks, hashes each with SHA-256, and compares them with its local index of the previous version: 2 of 500 chunks are new. - It asks the block service which of those hashes the server lacks (
/blocks/check). Both are missing. - It uploads the two chunks (8 MB). The block service recomputes each hash to make sure the bytes match their name, then stores them.
- It commits:
path = /deck.pptx, blocks = [500 hashes], baseVersion = 7. The metadata service checks the version, writes version 8 and appends a journal entry in one transaction.
Another device picks up the change
- The phone keeps a long-poll request open to the notification service. When the journal changes, it’s woken, asks for changes since its cursor, downloads only the chunks it doesn’t already have, and rebuilds the file.
The download side mirrors it: the phone is woken, reads journal entries after its cursor, sees the new block list, fetches only the hashes it doesn't have (here, two), and rebuilds the file locally. Then it saves its new cursor.
6. Deep dives
Before reading on, write two or three sentences on each: how do you avoid re-uploading a 2 GB file for a small edit? What happens when two people edit the same file offline? How does a device learn about changes without polling all day?
Compare with our answer
A strong answer mentions:
- Chunk and hash; upload only missing hashes; content-defined boundaries so inserts don't shift every chunk.
- A commit that carries its base version; the loser of a race gets a conflict and saves a conflicted copy.
- A journal per namespace with a cursor per device; a long poll that only says "changed".
- Blocks first, then the metadata commit, so a crash can only leave unused blocks, never a broken file.
Chunking: why fixed-size chunks are not enough
Splitting files into 4 MB chunks named by their hash gives three things at once: delta sync (only changed chunks travel), deduplication (a chunk already stored, by you or anyone, is never stored twice) and resumable uploads (an interrupted upload keeps the chunks that finished).
Fixed-size chunks have one weakness: an insert. If you add a paragraph near the start of a file, every byte after it moves, so every 4 MB boundary falls on different bytes and every hash changes.
Fixed-size chunks against content-defined chunks
| Option | How | Catch |
|---|---|---|
| Whole file | Upload the file on every change | 2 GB for a one-slide edit; minutes on a home uplink |
| Fixed-size chunks | Cut every 4 MB, hash each | Inserts and deletes shift every later boundary |
| Content-defined chunks | A rolling hash over a small window (about 48 bytes); cut where it matches a bit pattern | Variable chunk sizes; more CPU; needs min/max sizes |
| Binary diff (rsync-style) | Send only the changed byte ranges against the old version | Server must hold or rebuild the old version; doesn't deduplicate |
A rolling hash (Rabin fingerprint, or the faster Gear hash used by FastCDC) is a hash of the last few dozen bytes that can be updated in constant time as the window slides one byte. Cutting wherever its low 22 bits are zero gives chunks of about 2<sup>22</sup> bytes = 4 MB on average, with a minimum and maximum size to avoid tiny or huge chunks.
Choice: content-defined chunking for the file contents, plus per-chunk compression. Dropbox itself used fixed 4 MB blocks with an rsync-style diff inside changed blocks; both answers are defensible if you explain the insert problem.
Deduplication and its privacy problem
Because blocks are named by their content, a chunk that already exists anywhere in the store is never uploaded or stored again: the /blocks/check call returns "not missing". Popular installers, PDFs and photos shared across many accounts are stored once.
Metadata vs blocks: the decision that shapes everything
| Metadata | Blocks | |
|---|---|---|
| Size | Tens of TB | Around 10 EB |
| Changes | Constantly (renames, new versions) | Never: a block is immutable |
| Needs | Transactions, "what changed since?" queries | Cheap, durable bytes |
| Store | Sharded SQL (Dropbox used MySQL, later its own Edgestore on top) | Object/block storage (Dropbox built Magic Pocket) |
Immutability is what makes blocks easy: no locking, perfect caching in a CDN (a hash never changes meaning), and integrity checking for free (recompute the hash). All the hard concurrency lives in the small metadata store, where SQL transactions handle it well.
Order matters: upload blocks first, commit metadata second. A crash in between leaves orphan blocks (reference count 0) for the garbage collector; the reverse order could leave a file pointing at bytes that don't exist.
The sync protocol: journal and cursor
The naive way to sync is to compare the device's whole folder tree with the server's. With 100,000 files, that is a lot of work every time. Instead:
- Each commit appends an entry to the namespace's journal in the same transaction (so a committed change always has an entry, and entries are in commit order).
- Each device stores a cursor per namespace: the last journal sequence it applied.
- Sync is
GET /changes?cursor=5812→ entries 5,813 onwards, a new cursor, andhasMorefor paging.
This makes sync resumable: a phone that was offline for a week just reads a longer page. If the cursor is too old (the journal was compacted), the server returns "reset" and the device falls back to a full tree comparison once.
The three trees in the sync engine
Dropbox's rewritten sync engine (Nucleus, 2020) describes sync as three trees on the client: the remote tree (the server's latest state, from the journal), the local tree (what's on disk, from file-system events) and the synced tree (the last state both agreed on). Comparing local with synced tells you what the user changed; comparing remote with synced tells you what others changed. If both changed the same file differently, that is a conflict. Keeping the "merge base" explicitly is what lets the engine tell "I deleted it" from "I never had it", which naive two-way comparisons get wrong.
Conflicts: two people edit the same file offline
Two offline edits to the same file
- Two people edit the same file while offline. The first to sync wins the commit. The second commit is based on an old version, so the server returns a conflict and the client saves a “conflicted copy” instead of overwriting anyone’s work.
| Option | How | Catch |
|---|---|---|
| Last writer wins | Accept every commit | Someone's work silently disappears |
| Lock the file while editing | Check out / check in | Impossible offline; forgotten locks block everyone |
| Merge automatically | Combine the two versions | Impossible for arbitrary binary files |
| Optimistic concurrency + conflicted copy | Commit carries baseVersion; on mismatch the client saves its version as "deck (Ben's conflicted copy).pptx" | A human has to reconcile two files |
The commit check is one conditional update: UPDATE files SET version = 8, blocks = ? WHERE namespace_id = ? AND path = ? AND version = 7. If no row matches, someone else won and the server returns 409. This is optimistic concurrency control: no locks, just a check at write time that the thing you read hasn't changed.
Change notifications: waking 100 M devices cheaply
Devices should hear about changes within seconds without polling. Each device keeps one long poll open (GET /notify?cursor=…): the server holds the request for up to about 60 seconds and answers early only if one of the device's namespaces moves past its cursor. The answer carries no file data, only "something changed"; the device then calls /changes. That separation keeps the notification service simple and stateless enough to hold tens of millions of idle requests.
Durability: eleven nines
Storing three full copies costs 3× the raw space. Erasure coding splits a block into, for example, 6 data fragments plus 3 parity fragments; any 6 of the 9 rebuild it. That survives 3 lost fragments at 1.5× overhead. Spread the fragments across machines, racks and zones, and scrub continuously: background jobs re-read blocks and recompute their hashes, so silent disk corruption is found and repaired before a second failure lands on the same block.
7. Bottlenecks, failures and monitoring
Scaling bottlenecks
| Bottleneck | Fix |
|---|---|
| Metadata hot spots (huge shared folders) | Shard by namespace; cache directory listings; rate-limit sync storms |
| Exabytes of blocks | Own storage system (Magic Pocket) with erasure coding; cold blocks on dense disks |
| Notification fan-out for shared folders | Notify per namespace; devices pull changes themselves |
| First sync of a new device | Download from the CDN in parallel; LAN sync between devices on the same network |
When things fail
| Failure | What happens |
|---|---|
| Upload interrupted | Chunks already uploaded are kept; resume with the rest |
| Commit fails after upload | Orphan blocks are garbage-collected later (refcount 0) |
| Metadata shard down | Sync pauses for those namespaces; local files keep working |
| Corrupted block detected | Rebuilt from erasure-coded fragments in other zones |
What to monitor
Sync latency (commit → other device)Upload/download throughputConflict rateBlock store durability checksMetadata DB latencyNotification fan-out lag
8. Follow-up questions
Answer these three in order before opening ours. 1) A 50 GB video upload fails at 80% on a hotel Wi-Fi. What happens when it resumes? 2) Version history must go from 30 days to 10 years for business customers. What changes? 3) A user renames a folder with 200,000 files. What does the journal look like?
Compare with our answer
A strong answer mentions:
- Chunks already uploaded are stored and found by
/blocks/check; only the remaining 20% is sent; nothing is committed until all blocks exist. - Old versions keep their blocks alive through reference counts, so storage grows; move old, rarely read blocks to colder, denser storage; GC runs on version expiry, per plan.
- Don't write 200,000 path changes: store a folder as an entry with an ID and children pointing at it by ID, so a rename is one metadata change and one journal entry; clients apply it locally.
How does deduplication interact with privacy?
Dedup by hash can reveal that someone stores a known file; many services dedup only within a user or encrypt per user.
How do you share a folder?
Shared folders are their own namespace; each member’s tree mounts it; the journal is per namespace.
How would you support LAN sync?
Devices on the same network discover each other and copy chunks directly, checking hashes, before falling back to the cloud.
- Numbers: ~10 EB of blocks after dedup (plus erasure-coding overhead), ~5 PB/day changed (~60 GB/s), ~2.5 trillion 4 MB blocks; metadata is only tens of TB.
- Split: metadata (sharded SQL, transactional) vs blocks (immutable, content-addressed object storage).
- Chunking: 4 MB chunks named by SHA-256; upload only missing hashes. Content-defined chunking (rolling hash) survives inserts.
- Dedup: by hash; privacy risk across users, so prove possession or dedup per account.
- Order: upload blocks → commit metadata. Crash leaves orphan blocks for GC, never a broken file.
- Sync: per-namespace journal + per-device cursor;
changes?cursor=is resumable. - Notify: long poll that says only "changed"; jitter for big shared folders.
- Conflicts: commit with
baseVersion; 409 → conflicted copy. Never silently overwrite. - Durability: erasure coding across zones (e.g. 6+3 at 1.5×), continuous hash scrubbing.
1. A user inserts one paragraph near the start of a 500 MB log file. With fixed 4 MB chunks, roughly how much is uploaded?
An insert moves every later byte, so fixed boundaries land on different content and every hash changes. Content-defined chunking avoids this by placing boundaries according to the bytes themselves.
2. Why does the client upload blocks before committing the file's new block list?
Orphan blocks are harmless and get garbage-collected. A committed file whose blocks don't exist would be a broken file on every device. The commit names hashes, not locations.
3. Two devices both edit version 7 of a file offline. The laptop syncs first. What should happen when the phone commits?
The commit carries baseVersion = 7 but the server is at 8. Optimistic concurrency rejects it, and the client keeps both versions so no work is lost.
4. A phone has been offline for three days. How does it find out what changed?
The per-namespace journal plus a device cursor makes sync incremental and resumable. A full tree comparison is the fallback only when the cursor is too old.
5. A storage team proposes cross-user deduplication where the client sends only a hash and the server links the existing block. What is the main risk?
SHA-256 collisions are not a practical concern; the problem is that a hash acts as a key to the content. Require proof of possession or deduplicate within one account.
6. Which choice best explains keeping metadata and blocks in separate stores?
The two have opposite shapes: tens of TB queried constantly vs exabytes of write-once bytes. Each scales best in its own kind of store.
7. A shared folder with 10,000 members changes. What is the most likely immediate problem?
Every member's devices are woken and all ask for the same journal entries. Jittered wake-ups and caching the hot namespace's latest page spread the load.
Score 60% or more and the lesson is marked complete.