Blueprint
Social & feeds intermediate

Design Instagram

The feed core is borrowed from news-feed and stated in a minute — the marks are in the media pipeline: 200 TB a day of photos and video that must never pass through your app servers.

asked at Meta · Pinterest · Snap Frequently asked ~35 min walkthrough
01

Scope the requirements

This looks like the news feed with pictures, and treating it that way is the classic time-management failure. The feed machinery is assumed knowledge here; the interviewer wants the media path — how bytes get in, how they get transformed, and how half a billion people fetch them fast — because that's where all the real cost lives.

Functional — what it must do

  • Upload posts: photos up to ~8 MB and videos up to ~4 GB, with a caption.
  • Follow other users — asymmetric, no approval.
  • A chronological feed of followed users' posts — I'll state the fan-out design by reference to the news feed and spend my depth elsewhere.
  • View media quickly from anywhere in the world.
  • Explicitly cut: likes, comments, stories, search, DMs — named and offered back as extensions.

Non-functional — the numbers that shape the design

  • Feed load p99 under 500 ms; media starts rendering within ~200 ms — which forces a CDN, not a nice-to-have.
  • Media is never lost — durable storage is non-negotiable; feed staleness up to 1 minute is fine (availability over consistency).
  • Read-heavy at roughly 100:1 — views utterly dominate uploads.
  • Scale target: 500M DAU, 100M posts/day — the media volume this implies is the number that shapes the design.
  • A 4 GB video upload must survive a flaky mobile connection — resumability is a requirement, not polish.
Say this out loud

The thing I want on the table immediately: this is a media system wearing a social app's clothes. Text and metadata are a rounding error — the design is about moving 200 terabytes a day into object storage and out through a CDN without my servers ever holding the bytes.

02

Back-of-envelope estimates

One line of this arithmetic matters more than all the others: the media line. It's three orders of magnitude bigger than everything else and it dictates where every byte flows.

QuantityEstimateHow you got there
Post writes~1k/s100M posts/day ÷ ~86k s ≈ 1,200/s; peak ~5k/s. Trivial as metadata writes.
Feed reads~60k/s500M DAU × 10 feed loads = 5B/day; peak ~300k/s. The 100:1 skew, visible.
Media ingested~200 TB/day100M posts × ~2 MB average. This is the number the whole design answers to.
Media per year~70 PB200 TB × 365 — and transcoded variants roughly double it. Only object storage plays at this scale.
Metadata~100 GB/day100M posts × ~1 KB. Media outweighs metadata 2,000:1 — they must live in different systems.
Feed cache~1 TB~200 post IDs × ~2 KB × 500M users — same shape as the news feed, a modest Redis cluster.
Say this out loud

Two hundred terabytes a day of media against a hundred gigabytes of metadata — that ratio is the design. Bytes go client to S3 directly and S3 to client via CDN; my services only ever handle pointers.

03

Sketch the API

The interesting decision is that POST /posts doesn't accept media at all — it returns permission to upload. That one choice keeps the byte path off my servers forever.

POST/api/v1/postsbody: { caption, mediaType, size } → { postId, presignedUrls[] }. Creates a pending post and hands back presigned S3 URLs — no bytes in this request.
GET/api/v1/feed?limit=20&cursor={ts,postId}→ { items[], nextCursor }. Items carry CDN URLs for each variant, never raw media.
PUT/api/v1/users/{id}/followIdempotent follow edge; DELETE to unfollow. Same reasoning as the news feed.
GET/api/v1/posts/{postId}Single post metadata plus CDN URLs for thumbnail, feed and full-resolution variants.
  • Upload is a two-phase commit: create the post in pending status, client uploads straight to S3 with the presigned URL, and an S3 event — not the client's word — flips the post towards live. Trusting the client to report "done" leaves ghost posts pointing at missing objects.
  • Responses carry variant URLs, chosen per context: the feed hands out thumbnail and feed-resolution URLs; only the detail view links full resolution. Sending 8 MB originals to a phone rendering a 300-pixel square is the bandwidth bill nobody audits.
04

Data model

The rule that organises everything: the database holds pointers, S3 holds bytes. Post rows are about a kilobyte no matter how large the video they describe.

posts
postId PK (Snowflake, time-sortable) · userId · caption · mediaKeys[] · status (pending/processing/live) · createdAt
Index on (userId, createdAt) for author timelines and the celebrity pull path. status gates feed visibility.
follows
(followerId, followeeId) composite PK
Two KV indexes, both directions. No graph database — the access patterns are single-hop lookups.
media (S3)
original + variants: thumbnail, feed, full · multiple resolutions per video
Immutable once written. Keyed by content-addressable paths; served exclusively via CDN.
feed_cache
userId → ~200 postIds (Redis)
Identical to the news feed. A cache, not truth — rebuild by pull on loss.
The database call

Posts and follows go to DynamoDB or Cassandra — at 500M DAU the access is pure key-value with no read-time joins, and I want partitioning handled for me. Media goes to S3 without discussion; nothing else survives 70 PB a year. If the interviewer scoped this to a few million DAU I'd run the metadata on sharded Postgres, but the S3-plus-CDN media path wouldn't change at any scale — that part isn't a function of size, it's a function of what media is.

05

High-level design

Two byte paths that never cross my services: uploads go client to S3 with a presigned URL, downloads go S3 to client through the CDN. The async lane is the media pipeline — feed fan-out works exactly as in the news feed and is folded into the feed service here.

client edge service data media pipeline Uploader Viewer API gateway CDN media variants Post service presigned URLs Feed service hybrid fan-out + feed cache Metadata DB posts, follows S3 originals + variants Media events S3 → SQS Media workers scan · transcode · thumbs write variants insert pending fetch media POST /posts upload event GET /feed hydrate mark live presigned PUT
notice the bytes: the uploader writes straight to s3 and the viewer reads straight from the cdn (which pulls s3 on a miss) — no media ever crosses the service lane.
  1. Upload: client POSTs caption and media type → post service mints a Snowflake postId, inserts a pending row, and returns presigned S3 URLs. The request carries no media.
  2. The client PUTs bytes directly to S3 — multipart and resumable for video. My servers are not in this path at all.
  3. S3 emits an upload event onto a queue → media workers virus-scan, transcode into thumbnail, feed and full-resolution variants (multiple bitrates for video), write variants back to S3, and flip the post to live — only then does the post event reach feed fan-out.
  4. Feed read: viewer GETs /feed → feed service reads ~200 precomputed IDs, hydrates metadata, merges celebrity posts pulled live — the news-feed hybrid, unchanged — and returns items whose media fields are CDN URLs.
  5. The client fetches media from the nearest CDN edge; misses pull from S3 once and then live at the edge with long TTLs, because media is immutable.
06

Deep dives — where the interview is won

Phase 01

Presigned URLs: the bytes must never touch my app servers

The naive design streams uploads through the API tier: client → app server → S3. At 200 TB a day that turns stateless services into a byte-shovelling fleet — every upload holds a connection and buffers megabytes for its whole duration, autoscaling tracks bandwidth instead of requests, and a deploy mid-upload kills transfers. The fix is to make the API a control plane only: POST /posts returns a presigned S3 URL — a time-limited, signed grant to PUT one specific key — and the client talks to S3 directly. S3 is built to absorb exactly this; my services never see a byte.

The subtlety is knowing when the upload happened. I don't trust the client to tell me — I create the post as pending and let an S3 event notification drive the state machine forward. If the client dies mid-upload, the pending row and any orphaned parts get reaped by a janitor after a TTL. This is a small two-phase commit between my database and the object store, and naming it that way earns credit.

client app storage Uploader API signs url S3 bucket presign PUT bytes
app hands out a signed s3 url. bytes go client → s3 directly, never touch app servers.
Trade-offI give up synchronous validation — the server can't inspect bytes it never sees, so virus scanning and format checks move into the async pipeline, and a malicious upload exists in S3 briefly before scanning kills it. Quarantining pending objects from serving until they pass is the containment.
Phase 02

One photo becomes many: transcode fan-out, then a CDN doing the real serving

Serving the 8 MB original to every viewer is the silent design failure — a feed scroll on mobile would pull hundreds of megabytes nobody asked for. So the pipeline fans each upload out into variants: a thumbnail for grids, a feed-resolution image, full resolution for the detail view; videos additionally transcode into multiple resolutions and bitrates for device-appropriate delivery. Storage roughly doubles; egress drops by an order of magnitude. That trade pays for itself many times over, because at this scale bandwidth, not disk, is the bill that hurts.

Delivery is the CDN's job, and the property that makes it trivially effective is immutability: a media variant, once written, never changes. That means cache TTLs measured in months, no invalidation story at all, and a hit rate limited only by popularity skew. The 200 ms media-start target is unmeetable from a single region — physics says so — and entirely ordinary from an edge PoP. Long-tail originals that stop being viewed tier down to cold storage like Glacier; only variants stay hot.

s3 worker cdn Original Variants Transcode job fanout CDN edge event webp/mp4 on first hit
s3 object-created fires the transcoder. outputs land back in s3; cdn caches on first hit.
Trade-offVariants trade roughly 2× storage and transcode compute for an order of magnitude less egress and a fraction of the latency — and immutability means the one thing a CDN is bad at, invalidation, never comes up.
Phase 03

The 4 GB video: chunked, resumable, and verified server-side

A 4 GB upload on a mobile connection takes the better part of an hour, and a single-request upload that dies at 95% restarts from zero — users will not tolerate that twice. The answer is multipart upload: the client splits the file into 5–10 MB chunks, gets presigned URLs per part, uploads parts in parallel, and records progress. On reconnect it asks which parts landed and resumes from there. Chunking also buys parallelism on good networks — the same mechanics Dropbox and YouTube use, which is worth saying because it shows the pattern generalises.

The integrity detail interviewers probe: don't trust the client's account of what it uploaded. Each part has an ETag (content fingerprint); before completing the multipart upload, the server lists the parts S3 holds and verifies them against the manifest. Only then does the object seal and the pipeline start. Client-reported status is a UI convenience, never the source of truth.

client s3 Uploader Part 1 Part 2 Part N 10MB
large file split into 10 mb parts. each part uploaded independently, resumable on failure.
Trade-offChunking adds client complexity and per-part bookkeeping for a payoff that's invisible on a 2 MB photo — so I'd apply it above a size threshold rather than universally, and say the threshold out loud.
Phase 04

The feed is the news-feed answer — I say so, and spend the time saved on media

Fan-out here is deliberately by reference: push post IDs to followers' cached feeds via a queue, skip precompute for flagged celebrity accounts and merge their posts at read time, skip fan-out for dormant users entirely. It's the standard hybrid and I state it in a minute flat — rebuilding it from scratch is how candidates run out of time before the media pipeline this question grades.

Two things do change because posts are media-first. First, fan-out is gated on the pipeline: a post enters feeds only when its variants exist and status is live, otherwise followers see broken images — the queue consumes the worker's completion event, not the upload event. Second, the feed payload is pure pointers: IDs hydrate to metadata plus CDN URLs, so a feed page is a few kilobytes even though it fronts gigabytes of media, and the heavy fetches happen lazily from the edge as the user scrolls.

post fanout read Author posts Fanout writer Redis timelines Viewer feed push N LRANGE
feed fanout on write into per-user timeline lists. read path is one redis LRANGE.
Trade-offGating on transcode means a post is visible to followers seconds later than the upload finishes — I'm spending my one-minute staleness budget on correctness, which is exactly what it's for.
07

Follow-ups you should expect

How is this different from designing Twitter?
The skew of the bytes flips. Twitter is text-first — 30 GB a day of tweets with media as an attachment problem. Instagram is media-first: 200 TB a day where storage, transcoding and CDN egress dominate every cost line, and the metadata system is almost incidental. Same feed core, completely different centre of gravity — and the design time should follow the bytes.
A transcoding worker crashes halfway through a video. What does the user see?
Nothing broken — the post is still processing and hasn't entered any feed. The queue redelivers the job after the visibility timeout and another worker retries; transcoding is idempotent because variants are written to deterministic keys, so a re-run overwrites cleanly. Repeated failures go to a DLQ and the post surfaces to the user as failed rather than hanging forever.
What's your storage cost story at 70 PB a year?
Tier by access pattern. Variants stay in standard S3 because they serve traffic; originals are rarely re-read after transcoding, so lifecycle rules move them to infrequent-access and then Glacier within weeks. That cuts the dominant cost line severalfold. I'd also dedupe identical uploads by content hash — reposts and backups are common — which is cheap insurance at this volume.
Do you need strong consistency anywhere?
Almost nowhere, and I'd defend that. Feed staleness of a minute is invisible; follow edges propagating in seconds is fine. The one place I want read-your-writes is the author seeing their own post immediately after upload — served by reading the author's own timeline from the metadata store directly, not through the fan-out path. Everything else buys availability with eventual consistency.
How do you serve users far from your origin region?
For reads the CDN already solves it — edges are global and media is immutable. For uploads, single-region S3 means a user in Jakarta PUTs across an ocean, so I'd use transfer acceleration or regional ingest buckets that replicate to the primary asynchronously. Metadata can stay single-region-write far longer than people assume; the 500 ms feed budget survives one cross-region hop, and the CDN hides the rest.

Where candidates lose the room

  • Streaming uploads through the API servers instead of presigned direct-to-S3 — it turns the stateless tier into a bandwidth bottleneck and is the single most common failure on this question.
  • Serving one-size originals with no thumbnail or resolution variants — an order of magnitude of wasted egress that the interviewer reads as never having shipped a media product.
  • Skipping the CDN and serving images from S3 to a global audience — the 200 ms media target is physically impossible from one region.
  • Re-deriving the entire news-feed fan-out from scratch and leaving five minutes for the media pipeline — this question grades the pipeline.
  • Not estimating media storage at all — it dwarfs every other number by three orders of magnitude, and missing it means the whole design answers the wrong question.

Concepts this leans on

Found a bug or want more?

Tell me what's broken, missing, or wrong. Every message lands in my inbox.