Scope the requirements
Two seams: sandboxed execution (the interesting security + isolation problem) and submission workflow (queue + result). The interviewer scores on knowing why running arbitrary code needs more than 'run it in a container'.
Functional — what it must do
- Users submit code (Python, C++, Java, Go, JS) against a problem.
- Server runs code against N test cases with time + memory limits.
- Return verdict: Accepted / Wrong Answer / TLE / MLE / RE per test case.
- Persist submission history + leaderboard by problem.
- Explicitly cut: contests, discussion, editorial — offer them back.
Non-functional — the numbers that shape the design
- Verdict latency: under 10 s for typical problems (10 test cases).
- 50k submissions/hour peak (contest windows push this 10×).
- Isolation: user code cannot access the internet, other submissions, or the host.
- Fairness: contest submissions get FIFO queue; non-contest can be de-prioritised.
- 99.9% on submission accept; execution may retry.
The whole system is a queue on the front, a sandbox pool in the middle, and a durable submission log at the back. Sandbox design is where interviews live or die.
Back-of-envelope estimates
Submission volume is small; per-submission compute is the scale story. During contests, everything runs 10-100× hotter than steady state.
| Quantity | Estimate | How you got there |
|---|---|---|
| Submissions/hour | ~50k peak, 500k contest peak | one submission runs ~5 s across ~10 test cases. |
| Sandbox runtime | ~5 s per submission | 10 test cases × ~500 ms each. Time limit typically 1-3 s per case. |
| Concurrent sandbox workers | ~1000 during contests | 50k/h × 5 s / 3600 s × safety = 100-1000. |
| Storage submissions | ~5 TB/year | 50k/h × 24 × 365 × ~10 KB (code + verdict). Postgres. |
| Test cases per problem | ~10-100 | hidden + sample. Stored as tar.gz in S3. |
| Language runtimes | ~10 | each pre-baked into a base image with compilers + libs pinned. |
1000 sandbox workers × 5 s = 5000 CPU-seconds/s at contest peak — that's a lot of hardware but the shape is bounded and predictable.
Sketch the API
Submission is a POST that returns an id; verdict polls or subscribes. Streaming stdout during execution is a nice-to-have.
- Submission returns immediately with an id; execution is async. Client subscribes to the WS for live updates or polls.
- Test cases are never sent to the client — user code runs on our sandbox, we return only verdict + failing input for the first-failing case.
Data model
Small structured data (submissions, problems, users) + blob storage for code + test cases. The queue itself is the operational spine.
Postgres for submissions because they're relational + queried by user_id + problem_id. Kafka for the queue because it gives replay + priority partitions. S3 for test cases because they're immutable + large + read many times.
High-level design
Three tiers: web (accepts submissions), queue (distributes to workers), sandbox pool (runs code). The isolation layer is the load-bearing wall.
- User POST /submissions → submission service inserts row with status=queued → publishes id to priority queue → returns id.
- Client subscribes to /rt/submissions/{id} for live updates.
- Sandbox worker pulls next submission from queue → downloads test cases from S3 (cached locally per problem) → starts a fresh sandboxed container per submission.
- Runs user code against each test case with time + memory limits; streams progress events to realtime push service.
- On completion: writes final verdict + timing to Postgres, updates the problem's leaderboard ZSET in Redis, tears down the sandbox.
Deep dives — where the interview is won
Sandbox: containers alone are not enough
A Docker container is not a security boundary against hostile code — a kernel bug lets user code escape. For running untrusted code at scale, the standard is one of two layers: gVisor (a user-space kernel intercepting syscalls) or Firecracker microVMs (KVM-based, tiny, seconds to boot). Both isolate the guest kernel from the host.
Inside the sandbox, enforce cgroups CPU + memory limits, disable network namespaces (no internet), mount a read-only filesystem with just the test case as writable temp. Kill the sandbox at time limit + a grace period. Cleanup is nuclear — destroy the sandbox, don't reuse. Reuse leaks state between users.
Priority queue: contest submissions cannot wait
During a contest, submissions must be judged in FIFO order — a delayed verdict changes rankings. Non-contest submissions can wait 30+ seconds without complaint. Kafka partitioning by priority: queue.high for contest, queue.normal for daily practice, queue.low for exploration. Worker pool consumes with weighted allocation — say 80% workers on high, 15% normal, 5% low.
During a rush, high-priority queue absorbs all workers until it drains. This risks starving normal-priority; add a shedding mechanism: if normal queue lag exceeds ~5 min, redirect some workers there. The knob is the fairness policy: it's a business decision, not a technical one.
Test case delivery: never in the response, always at the sandbox
Test cases stay server-side forever. Sending them to the client would let users train against private tests. Sandbox workers download the test case archive (tar.gz from S3) on first run for a problem, then cache locally — most problems get many submissions, so the archive lives in the worker's disk warm cache.
Verdict returned to the user is coarse-grained: 'passed', 'failed on test 5', with the failing input possibly shown only for the first sample case (public). Hidden test failures show the case number but not the input. This is a leak-prevention design; a subtle test case is the whole IP.
Follow-ups you should expect
How do you support 10 languages?
Judging fairness across hardware — same code, different runtime?
taskset to avoid noisy neighbours, (2) run each submission twice on different boxes and take the median. Most competitive programming platforms do (1) and warn users about small runtime variance.Can users see test cases from stack traces or side channels?
What happens to a submission mid-crash of a worker?
Leaderboard integrity — someone submits, then edits code to game?
Where candidates lose the room
- Running user code in plain Docker — kernel escape is real; use gVisor or Firecracker.
- Reusing sandboxes across submissions — a subtle side effect leaks between users.
- Sending test cases to the client — you lose the IP the moment one leaks.
- Postgres as the submission queue — polling scales worse than Kafka's push semantics.
- One queue for all priorities — a contest spike blocks everyday practice submissions for hours.