Scope the requirements
Draw the seams: git object storage (write-once, cold), metadata (PRs, issues, users — transactional), and derived views (search, notifications, activity feeds). Each has different SLAs.
Functional — what it must do
- Host git repos: clone, push, pull, fetch — including large monorepos.
- Pull requests: branches, review threads, merge with strategies.
- Issues + comments with markdown + mentions.
- Code search across all public repos.
- Explicitly cut: Actions/CI, packages, wiki — offer them back.
Non-functional — the numbers that shape the design
- Clone latency for a warm 1 GB repo under 10 s end-to-end.
- 500M repos, ~100M active users, ~10k git pushes/s peak.
- Repo storage: petabytes. Access pattern is highly skewed — Linux kernel gets thousands of clones/day, most repos get zero.
- Search index over ~10B code files.
- 99.99% availability on read (clone/pull); write can retry.
Git is the hard part because the protocol has weird properties — content-addressed, cold, packed. Everything else is normal web scale.
Back-of-envelope estimates
Storage is petabyte-scale but bandwidth is modest for git operations. Search is what pushes hardware.
| Quantity | Estimate | How you got there |
|---|---|---|
| Repos | ~500M | public + private. Long-tail distribution; top 0.1% get 99% of the traffic. |
| Git ops/s | ~50k clones + 10k pushes peak | clones dominate; pushes small but expensive (fsync + pack rebuild). |
| Storage | ~500 PB | avg repo ~1 GB (skewed by monorepos); deduped across forks via shared objects. |
| Code files | ~10B | search index rebuilt continuously; ~50 TB of postings + ngrams. |
| Metadata rows | ~50B | issues, PRs, comments, reviews. Postgres cluster per shard. |
| Push egress | ~1 TB/s during release windows | big projects pushing tags trigger CI clones from thousands of workers. |
Storage is petabytes but read bandwidth is bounded — most reads hit git object cache, not disk. The real design question is packing and fork dedup.
Sketch the API
Git protocol on port 22 (SSH) or 443 (HTTPS); REST API for everything else. GraphQL sits on top of REST for expressive queries.
- Git protocol is bespoke: not HTTP semantics you can layer conventional caching on. Custom router picks the git backend holding the repo's shard.
- Merge is done server-side — the API applies the client's requested strategy (merge, squash, rebase) against fresh state, not the state the PR was opened against.
Data model
Two large storage systems: git object store (content-addressed, immutable) and metadata database (transactional). Search is a derived index over both.
The git object store is content-addressed — a custom layer on top of local disk + object storage. Refs go in a fast KV (Redis-backed with WAL for durability). Everything else is Postgres with CDC into Elasticsearch.
High-level design
Three tiers: an edge router that picks the git backend for a given repo, the git backend itself (holds objects + refs), and the metadata/web tier that handles PRs/issues.
- Clone: client → SSH/HTTPS → router looks up which shard holds
owner/repo→ forwards to that git backend. - Git backend reads refs from the fast store, computes packfile of missing objects for the client, streams the bytes.
- Push: client sends packfile → git backend verifies + writes new objects (immutable content-addressed) → atomically updates the ref.
- PR create: metadata service writes a row → CDC publishes to Kafka → search indexer updates ES + notifier fans out to subscribed users.
- Merge: metadata service checks CI status + reviews → asks git backend to perform the merge strategy on server side (produces a merge commit or squashed commit) → updates the target ref → posts merge event.
Deep dives — where the interview is won
Git storage: content-addressed dedup across forks
Every git object (blob, tree, commit) is named by the SHA-1 of its content. Identical content = same SHA = one copy. When you fork Linux, you don't copy 5 GB of history — you share the object store with the source repo and only add new objects when you push. This is why storage grows sub-linearly with repos.
The backend keeps 'alternates' — a config line pointing each fork's local store at the parent's shared pool. On clone, the packfile is composed from both. On push, only new objects go to the fork's own store. GitHub's storage bill would be astronomical without this — real repos with many forks share 90%+ of their objects.
Sharding a repo across a git farm without breaking clone locality
Repos vary in size by 5 orders of magnitude. A naive random-shard scheme puts a Linux-kernel-sized repo on one box that gets crushed. The design: assign each repo to a shard at creation (hash of owner_id gives locality — a user's repos co-locate), then move hot repos to dedicated shards proactively.
For very large monorepos, replicate across N read shards — clones hit any of them, pushes go through a leader with sync replication. The router (a small stateless service) reads a repo → shard map from a global config store and routes accordingly.
Code search over ~10B files without scanning them all
Grep across 500 PB is not a plan. Code search uses an inverted index: for every trigram (three-character sequence) in a file, record the file id. A search for func_name becomes a set intersection of trigram posting lists — fast because trigram sets are small per file and posting lists are compact.
The catch: code has more distinct trigrams than natural language (all those ___ ->s foo combinations). Index storage is ~5-10× the raw code. Sharding by repo_id keeps search results consistent for a single-repo query; global search fans out to all shards and merges.
Follow-ups you should expect
What happens on a force-push to a widely-cloned repo?
Merge conflicts — where does the resolution happen?
How do you scale from 100M repos to 1B?
How does the search index stay fresh on 10k pushes/s?
ref_updated event to Kafka. Indexer consumes, fetches the diff between old and new ref via the git backend, applies incremental updates to trigrams (add new files, remove deleted ones). Full re-index of a repo happens on schedule (weekly) and after major merges.What if a git backend shard dies?
Where candidates lose the room
- Storing every fork as a full copy — dedup via shared object pools is the whole storage story.
- Randomly sharding repos with no locality — a user's dashboard fans out to every shard for a listing query.
- Doing merges client-side and only accepting the resulting SHA — the server needs to enforce merge rules against fresh state, not the state when the PR opened.
- Full re-scan on code search — trigram indexes are the difference between milliseconds and seconds.
- Skipping the packfile compression — git without packing is 5-10× larger on disk and slower to clone.