Scope the requirements
Two problems hide inside this question: moving very large files reliably (the data plane) and telling every device what changed (the sync plane). Candidates who treat it as 'upload a file to S3' miss both, and the interviewer knows within five minutes.
Functional — what it must do
- Upload files from any device — up to 50 GB, resumable after disconnects.
- Download files from any device, ideally near the user.
- Share files with other users with access control.
- Automatic sync: edit a file on one device, see it on the others within seconds.
- Explicitly cut: real-time co-editing (that's Google Docs), version history UI, virus scanning — name them and offer them back.
Non-functional — the numbers that shape the design
- Availability over consistency: cross-device sync lag of a few seconds is fine; being unable to save is not.
- Durability is sacred — eleven nines on stored bytes; losing a user's file is the one unforgivable failure.
- A 50 GB upload must survive network drops without restarting — resumable by construction.
- Encryption in transit and at rest; sharing enforced by ACL, not obscurity.
- Scale: 500M users, 100M DAU, ~200 files each — on the order of 10 PB of content.
I'll split this into a control plane and a data plane immediately: metadata and permissions go through my services, but file bytes go directly between the client and blob storage. If a 50 GB file ever flows through my app servers, the design is already wrong.
Back-of-envelope estimates
The numbers force the two big decisions: file size makes chunking non-optional, and total volume makes 'store it in the database' laughable — blob storage plus a metadata store is the only shape that works.
| Quantity | Estimate | How you got there |
|---|---|---|
| Worst-case upload time | ~1 hour | 50 GB at 100 Mbps ≈ 1.1 hours. No connection lives that long untouched — resumability is a requirement, not a feature. |
| Chunks per large file | ~10,000 | 50 GB ÷ 5 MB chunks. Each chunk is independently uploadable, retryable and fingerprintable. |
| Total files | ~100 B | 500M users × 200 files. Most are small — average ~100 KB. |
| Content storage | ~10 PB | 100B files × ~100 KB. Blob storage territory; dedup will claw a meaningful slice back. |
| Metadata | ~100 TB | 100B files × ~1 KB of metadata each — too big for one box, fine for a partitioned store keyed by user. |
| Sync connections | ~1M concurrent | 100M DAU with a fraction online at once, each holding a cheap idle WebSocket or long-poll. |
One hour to upload a big file and ten petabytes to store settle the architecture on their own: chunked direct-to-blob uploads, a partitioned metadata store, and a CDN in front of downloads.
Sketch the API
The API's defining move is that upload and download endpoints return URLs, not bytes — presigned links that let clients talk to blob storage directly while my services stay in charge of who may do what.
uploading state; dedupe check on fingerprint happens here.- Presigned URLs are bearer tokens — anyone holding one can fetch the bytes. That's why they expire in minutes, are scoped to a single object and operation, and are minted per-request after an ACL check. Auth travels in headers, never in the body.
- The changes feed is cursor-paginated, not timestamp-paginated — timestamps collide and skew across writers; an opaque cursor over a monotonic per-user sequence guarantees a device never misses or double-applies a change.
Data model
Metadata is small, hot and relational-ish; content is huge and opaque. Model them separately and never let file bytes near the metadata store.
I'd start with Postgres partitioned by user_id — metadata access is always user-scoped, the ACL checks want transactions, and 100 TB partitions cleanly. Content goes to S3 with server-side encryption, fronted by a CDN. If metadata write volume outgrew Postgres partitioning, I'd move files and shared_files to DynamoDB with user_id as partition key — the access patterns are already key-value shaped, so I'd say that migration out loud rather than pre-paying for it.
High-level design
Control plane through the file service, data plane straight between client and S3, and a sync plane that pushes change notifications over WebSockets with polling as the safety net.
- Device A's sync agent notices a changed file, fingerprints it and its chunks, and POSTs the manifest → the file service checks quota and dedup, records metadata in
uploadingstate, and returns presigned URLs per chunk. - The client uploads chunks in parallel, directly to S3, retrying individual chunks on failure; a dropped connection resumes from the chunk list, not from byte zero.
- S3 emits events as parts land → the file service verifies ETags against S3 (never trusting the client's report), marks the file
uploaded, and appends a change record for every user with access. - The sync service fans the change out over WebSockets to Device B and every other online device; offline devices catch up via
GET /changes?cursor=…when they reconnect. - Device B pulls only the changed chunks through the CDN using short-lived signed URLs and patches its local copy.
Deep dives — where the interview is won
Uploading 50 GB: chunked, parallel, resumable — and verified server-side
The client fingerprints the whole file and each 5 MB chunk, then initiates a multipart upload: the file service returns an uploadId and a presigned URL per part, the client uploads parts in parallel, and completion stitches them into one object. This buys three things at once: progress (per-chunk status), parallelism (saturate the uplink), and resume (a reconnecting client asks which chunks are missing and sends only those).
The detail that separates candidates: never trust the client's word that a chunk landed. The client PATCHes chunk status as a hint, but the file service confirms against S3's own part listing and ETags before marking the file uploaded. A malicious or buggy client can misreport; the blob store is the referee.
Delta sync: content-defined chunking is the whole trick
When a user edits one paragraph of a large file, re-uploading everything is unacceptable. Chunk fingerprints solve it — compare local chunk hashes to the server's manifest and upload only what changed. But fixed-size chunks break on inserts: adding one byte at the front shifts every chunk boundary, so every hash changes and you're back to re-uploading the file.
The fix is content-defined chunking: a rolling hash (Rabin fingerprint) over a sliding window declares a chunk boundary wherever the hash hits a pattern, so boundaries are anchored to content, not offsets. An insert changes the chunks near the edit and nothing else. The same fingerprints give dedup for free — identical chunks across files, versions and even users are stored once and reference-counted.
Sync down: push for speed, pull for truth
Real-time sync wants push: devices hold a WebSocket (or SSE stream) to the sync service, and a change fans out to online devices in under a second. But push alone is not a sync protocol — connections drop silently, servers restart, messages get lost in fan-out. Any device that treats a pushed event as the source of truth will eventually diverge and stay diverged.
So the durable mechanism is the per-user change log consumed by cursor: every device periodically calls GET /changes?cursor=… and applies whatever it missed, and the WebSocket push is best-effort — a hint meaning 'poll now'. This hybrid is the standard answer: push provides latency, pull provides correctness, and the cursor makes catch-up exact regardless of how long a device slept.
Conflicts: last-write-wins is a data-loss policy — say so
Two devices edit the same file offline and both sync. Pure last-write-wins silently discards one user's work, which for a file product is quietly catastrophic. Dropbox's answer is the right one to cite: detect the conflict (the uploading device's base version no longer matches the server's head version) and fork — keep both, renaming one to a 'conflicted copy'. No merge, no data loss, and the human resolves it with full information.
Detection is cheap: every upload carries the version (or content fingerprint) it was based on, and the metadata service compares atomically at commit time. This is optimistic concurrency control applied to files — worth naming, because it connects to how the interviewer's follow-ups about versioning will go.
Follow-ups you should expect
Why availability over consistency here? A file system feels like it should be consistent.
How does dedup interact with 'delete my file'?
What if the S3 upload event never arrives — the file sits in 'uploading' forever?
uploading records, asks S3 directly which parts exist, and either completes or expires the upload. The client also re-PATCHes status on reconnect. Every async signal in the design has a polling truth-check behind it.How do you make downloads fast for a user in Singapore when the file was uploaded in Virginia?
Where does compression fit?
Where candidates lose the room
- Routing file bytes through app servers instead of presigned direct-to-S3 — the single most common failure; it turns your service tier into a 10 PB proxy.
- Chunking server-side after receiving the whole file, which defeats every reason chunking exists (resume, parallelism, delta sync).
- Fixed-size chunks when asked about syncing edits — one inserted byte shifts every boundary and forces a full re-upload; the interviewer is fishing for content-defined chunking.
- Trusting client-reported chunk status without verifying ETags against the blob store — a correctness and security hole in one.
- Sync via WebSocket push alone (diverges on dropped connections) or polling alone (seconds-to-minutes of lag) — the expected answer is push for latency plus cursor-based pull for truth.
- Using filename or path as the file's identity instead of an ID plus content fingerprint — renames become copies and dedup becomes impossible.