The recommendation system design interview
A recommender design answer has a standard spine: what the system optimizes and how that becomes a label, a funnel that narrows millions of items to a handful (retrieval, ranking, re-ranking), the features each stage can afford, offline and online evaluation, and what goes wrong after launch. The candidates who do well say the objective and the latency budget before drawing any model.
Key points
- Define the objective as a business outcome, then turn it into a label you can collect.
- Use a funnel: cheap retrieval over the whole catalog, a heavier ranker over hundreds of candidates, rules last.
- Give each stage a share of the latency budget and say what that budget allows.
- Offline metrics choose models; an online test decides whether to ship.
- Plan for cold start and for feedback loops, where the model trains on what it chose to show.
Objective, label and scope
Take “recommend videos on the home screen of a learning app”. Ask who the users are, how many items exist, how fresh the catalog is, and what success means to the business. Clicks are easy to collect and easy to game with clickbait thumbnails. Completed lessons and return visits are closer to the goal but arrive later and more sparsely.
A common answer is a weighted objective: predict the chance of a click and the chance of finishing at least half the video, then rank by a combination. Say the weights are a product decision, tested online.
The funnel, with numbers
Assume 10 million videos, 5 million daily users and a 200 ms budget at the 99th percentile for the whole request. The table shows one reasonable split. Interviewers listen for the cost of each stage; the exact milliseconds are open to debate.
| Stage | Items in, items out | Model | Budget |
|---|---|---|---|
| Candidate retrieval | 10 million to about 1,000 | Two-tower embeddings with approximate nearest-neighbor search, plus a few rule sources (subscriptions, trending) | About 30 ms |
| Feature fetch | 1,000 candidates | Precomputed user and item features from a feature store | About 40 ms |
| Ranking | 1,000 to about 50 | A gradient-boosted model or a small neural network with user, item and cross features | About 60 ms |
| Re-ranking and rules | 50 to the 10 shown | Diversity, freshness, already-watched filters, policy rules | About 10 ms |
| Network and overhead | n/a | n/a | The remaining 60 ms |
Features and training data
- User: watch history embedding, topics followed, recent activity counts, device, language.
- Item: topic, length, age, creator, recent completion rate.
- Cross: has the user watched this creator before, similarity between the item and the last few items watched.
- Context: time of day, position on the screen.
- Training data: impressions with their outcomes. Log the features as they were at serving time, or training will see values the model never had when it decided (training and serving skew).
- Negatives: shown but not clicked is a better negative than a random item; position bias means low positions get fewer clicks regardless of quality, so record the position and correct for it.
Offline and online evaluation
Offline, retrieval is judged by recall at K (did the items the user later watched appear in the 1,000?) and ranking by metrics such as NDCG or AUC on a time-based split: train on weeks 1 to 4, test on week 5. A random split leaks the future into training.
Online, an A/B test on the business metric decides, with guardrails such as session length, creator diversity and complaint rate. Offline gains that do not show up online are a classic follow-up: the usual suspects are a leaky offline split, a proxy label that diverged from the goal, or a change that only helps users who were already engaged.
How it sounds out loud
“Before models: I’ll optimize for lessons completed, with clicks as an early signal, because clicks alone reward clickbait.”
“We have 10 million items and 200 milliseconds, so I need a funnel. Retrieval narrows to about a thousand in 30 milliseconds, the ranker sees those thousand, and rules pick the final ten.”
“I’ll go deeper on ranking first, because that is where the objective lives. Tell me if you would rather start with cold start.”
Cold start, feedback loops and drift
- New users: start from popular items in their language and a short onboarding choice of topics, then switch to personal signals after a few sessions.
- New items: content embeddings from the title, transcript and topic let a new video enter retrieval before anyone watches it; reserve a small share of impressions for exploration.
- Feedback loops: the model only learns about what it showed. Keep a little randomized traffic so you can measure items it would never pick.
- Drift: watch the score distribution, the click-through rate by position and the share of impressions per creator. Retrain on a schedule and when monitored metrics move.
Serving and freshness
Most of the heavy work can happen before the request arrives. Item embeddings are computed in batch and loaded into the nearest-neighbor index; user embeddings can be refreshed every few hours or computed at request time from recent activity if freshness matters more than cost. Ranking features that change slowly come from a feature store; a handful of real-time counters, such as what the user watched in the last ten minutes, come from a stream.
Say what happens when a dependency is slow. If the feature store misses its budget, the ranker can fall back to default feature values, and if ranking times out, the service can serve the retrieval order or a cached list of popular items. A recommender that returns something reasonable in 200 ms beats one that returns the best list in two seconds.
At 5 million daily users with, say, 10 home-screen loads each, that is 50 million requests a day, about 580 a second on average and several times that at peak. Say the peak factor you assume; it decides how many ranking replicas you need.
Common mistakes
- Naming a model architecture before the objective, the label and the scale.
- One model over the whole catalog, with no thought for latency.
- Training on clicks without saying how clickbait and position bias are handled.
- A random train and test split for data that has time in it.
- Treating an offline metric gain as proof that the change should ship.
- No plan for new users or new items.
Questions
Do I need to know two-tower models in detail?
Know what they do: one network embeds users, another embeds items, and retrieval finds items close to the user in that space. Then be ready to say how they are trained and why they suit retrieval but not final ranking.
How deep should the ranking model discussion go?
Start with a strong baseline such as gradient-boosted trees, say what features it needs, and only then propose something heavier with a reason, for example many sparse ID features.
What twist should I expect?
Common ones are a new item with no history, offline metrics up while the A/B test is flat, or latency that doubles after new features. Each one has a short, structured answer if you have practised it.
Practise this out loud with Vera
Mockich’s ML system design round runs 45 to 50 min on a shared whiteboard Vera can read, with ML questions such as recommendations, ranking, fraud, forecasting and LLM systems. Vera gives at most two small nudges and adds one twist at a time, such as a new item with no history. Today it uses the general system design stages and scorecard (8 skills, including the API and data model), not an ML-specific one; the report quotes your reasoning for each.
Related guides
Sources
We read these sources on 2026-11-03 for the usual structure of recommender designs and the follow-ups they report; the learning-app case, the funnel numbers and the spoken lines are our own.
- [1] Alireza Dirafzoon, AI/ML Interviews: ML system design (video and feed recommendation) (MIT)
- [2] Mercari, Machine learning system design pattern (MIT)
- [3] IGotAnOffer, ML System Design Interview
- [4] Eugene Yan, Applied ML (papers and write-ups on production ML) (MIT)