Recommendation Systems From Scratch

Candidate recall, exposure bias and constrained ranking

Candidate generation narrows a corpus, scoring estimates relevance or utility, and re-ranking applies policies such as diversity, freshness and eligibility. Retrieval recall limits what a ranker can ever select: an excellent item omitted upstream cannot be recovered by a better scoring model. Compare popularity/content/collaborative candidates and blend where cold-start coverage requires it. Filter eligibility without using user-controlled tenant identity.

Implicit feedback is exposure-biased: a nonclick may mean unseen, not disliked. Train/evaluate with time-based splits, point-in-time features and a well-defined label horizon. Precision/recall/NDCG depend on the candidate population and relevance labels; an offline gain is not a proven causal business improvement. Online experiments need stable assignment, guardrails and a policy for delayed outcomes/interference. Keep exploration, user control and diversity visible so click feedback does not merely amplify prior exposure. The design lesson’s local ranking code demonstrates deterministic score/filter behavior, not model-training performance.

Failure exercise and worked answer

A ranker has excellent precision on previously shown items. Why might it fail on a newly launched catalogue?

Compare the invariant, failure and recovery

The evaluation omitted unseen candidates/cold-start users and inherited the old exposure policy. Measure candidate coverage and evaluate new cohorts with leakage-safe features and appropriate exploration.

Who this is for

You know Python and basic features, training and evaluation. The first question is whether the catalog is large enough or the scorer expensive enough to justify separate retrieval. “Pick twenty out of ten million in 100 ms” is an example product constraint, not the shape of every recommender. A small eligible catalog can be scored directly.

The path

Part 1 · Design

Recommendation quality starts before ranking separates candidate generation, scoring and re-ranking. Run its exact small fixture and explain why adding a relevant candidate improves candidate recall but may leave nDCG at a fixed cutoff unchanged. Then compare co-occurrence, content and learned-vector candidates. The deliverable is a candidate-population definition, a ranking metric and a hard eligibility rule—not an arbitrary requirement that every system use a heavyweight model or heuristic rail.

Part 2 · Build

Building from initial signals is a worked propensity/signal study. Distinguish likely adoption from the incremental effect of contacting someone: a person likely to convert may do so without intervention. Define time splits, label maturity, feature availability and how preprocessing fitted on training data is reused. Inspect the lesson's execution boundary; a documentation-reviewed model pipeline is not a reported fit or measured business gain. Deliver a leakage checklist and a precisely defined lift calculation.

Part 3 · Teardown

Monolith paper study examines the named paper's embedding and online-training mechanisms. Read its diagrams with the assumptions and evaluation conditions attached. A paper describing a ByteDance system is not a complete disclosure of today's TikTok ranking product. A dictionary mock does not reproduce a distributed embedding service or its benchmark. Deliver a claim/source table separating what the paper reports, what a local example models and what has not been reproduced. Do not infer revenue or engagement improvements absent the paper's measured population, baseline and stated result.

How to read this series

Use the design → build → paper order, returning to feature-history and data-contract lessons when a failure trace reveals a gap. The hub is a study map, not a promise of fixed total reading time or production readiness. Keep these artifacts as you proceed:

1. Candidate recall on a stated eligible catalog, with new-user/new-item cohorts separated.
2. Final ranking metric and tie/empty-relevance conventions; record policy violations separately.
3. A time-based data/label split and a point-in-time feature example.
4. One cold-start strategy and its failure case, such as popularity hiding new items.
5. A proposed online experiment with stable assignment, guardrails and delayed-outcome handling.

Worked study decision

A new item has no interactions. Popularity retrieval never includes it, while the ranker is perfect on candidates it does see. Would changing the ranker's architecture solve the problem?

Answer before selecting a larger model

No: the item is excluded upstream. A content-based/fresh-item candidate source or a deliberately controlled exploration policy can make it eligible to compete. Measure retrieval coverage, then rank within the allowed set. Missing clicks do not prove dislike when the old policy never showed the item. Exploration trades information and discovery against immediate utility and must respect eligibility, privacy and user controls; it is not a license to optimize clicks alone.

What this series does not cover

Matrix factorization optimization, approximate-nearest-neighbor index construction and a complete causal-inference course are separate deep topics, not replaced by three short articles. The design lesson explains the mechanism and nDCG fixture; historical features provides a temporal leakage counterexample. This route does not claim a live experiment, production model training, paper benchmark reproduction or operation of the entire TikTok system.

Sources and review boundary

Primary passages were checked on 2026-10-07. Versioned sources below delimit the claims; mutable pages are captured in the corrective evidence report. No cluster, cloud service, load benchmark, provider send or model inference was executed for this review.

• Google recommendation systems overview — Recommendation architectures separate candidate generation, scoring and reranking.
• OWASP authorization cheat sheet — Authorization should deny access by default and be enforced consistently.