Design a Newsfeed / Recommender — Pull vs Push, Ranking

Why this session exists

A feed design combines delivery, retrieval, ranking and product policy. A fast timeline can still be irrelevant; a strong ranker cannot select a relevant item that was never retrieved. Neither makes an unauthorized post safe to show. We will derive push/pull costs, trace the ranking funnel, implement a bounded hybrid fixture and explain feedback-loop evaluation. The original universal revenue percentages, highest-QPS claim and guaranteed sub-200 ms timings were not evidence and are not design requirements.

You will be able to: draw candidate generation, scoring and reranking; compare fanout using follower and reader activity distributions; distinguish recency/affinity/predicted utility from hard safety rules; test cold-start and exposure bias; and reason about feature freshness and rollout. These are assessment objectives, not an employer-specific interview rubric.

Prerequisites

Know caching, queues, sharding, a basic recommender taxonomy and a requirements-first design process. Decide whether the product is a chronological follow feed or a discovery feed: they have different candidate sources, user expectations and evaluation labels.

For the preparation route, revisit S126: design framework, S063: caching and S074: sharding. The original S070 sharding label was a numbering mismatch; S070 addresses CAP/PACELC. For collaborative/content-based recommendation background use the recommendation series, not the old S091 label (that session is gradient boosting).

Optional preparation resources

These are retained discovery pointers, not required viewing or endorsements. Video titles and creator labels below are historical finding labels; this review did not watch the recordings or verify their current attribution, runtime or availability. Choose a short preparation slot rather than interpreting the old “30 min total” as the sum of every resource.

• Newsfeed push/pull video pointer: originally labelled “Designing INSTAGRAM: System Design of News Feed” / Gaurav Sen. Use it to compare delivery vocabulary, not to establish a production threshold.
• Newsfeed architecture walkthrough pointer: originally labelled “Design Facebook News Feed” / ByteByteGo. Compare its fanout assumptions with the activity-based calculation here; “45 minutes” was a planning label, not verified viewing time.
• Python recommender tutorial search: a search, not an identified recording. Optional follow-on topics are matrix factorisation and two-tower retrieval; no training or model download is needed for this lesson.
• ByteByteGo newsfeed reading pointer: optional design comparison, not a prerequisite supposedly expected by every interviewer.
• Meta Newsroom feed-prediction explanation: optional public-facing ranking context, distinct from a verified implementation contract.
• Meta Engineering Explore article: the source-reviewed four-stage architecture used below, not evidence for universal stage counts or latency targets.

(a) Intuition · 5 min

A chronological feed resembles a newsstand arranged by date. A ranked feed selects among eligible articles for a reader, but it is an uncertain estimate of usefulness—not a butler who knows what the reader wants. Candidate delivery determines which posts are cheaply available; ranking orders them; reranking enforces diversity and policy within the authorized universe.

Push materializes post IDs into follower inboxes at write time. Pull reads authors' recent posts at request time. Hybrid uses both, often pulling high-fanout authors. All still hydrate content, check access and rank; precomputed IDs do not make the entire response O(1). A chronological product may not need a learned ranker at all. A discovery product must explain how candidates beyond the follow graph are found.

The two-stage funnel

The two-stage abstraction is retrieval then scoring; real systems often add lightweight ranking and final reranking. Meta's August 2023 Explore description explicitly has four stages: retrieval, first-stage ranking, second-stage ranking and final reranking. Google's course describes candidate generation, scoring and reranking. Neither prescribes exactly 5,000→500→30 items for every request. Those can be scenario budgets, subject to recall and latency evaluation.

Fanout strategy trade-off table (know this cold)

All costs include constants, replication and product policy. There is no fixed 10 ms/2 s split by strategy, no universal one-million-follower threshold, and no proof that only hybrid can survive a public social graph. Caching, queueing, bounded candidates and workload constraints matter.

Hybrid feed for a normal user — step by step

1. Load the bounded materialized inbox (for example 500 IDs as a scenario).
2. Fetch recent posts from followed pull-path authors, batching requests and enforcing a fan-in deadline.
3. Union/deduplicate by stable post identity and retain generator provenance. A classifier transition may temporarily nominate the same post from both paths.
4. Hydrate from canonical post state; apply current follows, blocks, tenant, deletion and audience policy before returning content. A cache hit is not an authorization result.
5. Fetch features using a schema/version and freshness policy. Score with a baseline or chosen model; stale/missing features trigger a tested fallback, not arbitrary default zeros.
6. Rerank for diversity/freshness and enforce hard policy. Integrate separately eligible ads with an explicit exposure/frequency budget.
7. Return a bounded page and a session/snapshot-aware cursor; retain seen IDs or stable ranking state so changing scores do not silently duplicate/skip entries across pages.

A worked budget might allocate 20 ms to candidate cache, 40 ms to parallel pulls, 30 ms to hydration/features, 50 ms to scoring and 10 ms to reranking plus network. This totals 150 ms only if those are serial stage budgets; actual overlap, queueing and tails require measurement. Percentiles do not generally add like deterministic times.

Ranker model families (design review vocabulary)

• Wide & Deep: the 2016 Google paper combines a linear wide part that memorizes selected feature crosses with a deep embedding network that generalizes. Its reported production application is Google Play. It is not a blanket statement about YouTube's current ranker.
• DLRM: the 2019 paper combines dense features, sparse embeddings, feature interaction and MLPs. It supplies a model/implementation design, not a universal 10 ms per 1,000 GPU candidates benchmark.
• Two-tower: user and item representations can be computed independently; precomputed item vectors enable efficient retrieval by dot-product/similarity. Features that depend jointly on the exact pair usually belong in a later joint scorer or change that separability.
• Sequential models: order recent interactions and model changing interests. Attention can represent history dependencies but does not guarantee state-of-the-art engagement; label/exposure bias, serving budget and training data still dominate some outcomes.

A multi-task scorer might estimate click, dwell, hide and long-term satisfaction separately, then combine them under a documented policy. Multiplying arbitrary probabilities without calibration or an agreed utility model does not automatically optimize the product.

Storage layout

Canonical post content and audience state belong in a durable store with workload-appropriate keys. A Cassandra partition by author and time bucket can support ordered ranges; saying Cassandra cannot do that was wrong. Redis sorted sets can hold a bounded inbox of (post_id, score) pointers; Redis persistence modes exist, so “Redis is never durable” is also wrong. Here it is deliberately a rebuildable cache, with explicit eviction and origin recovery.

Online features need low-latency access; offline features need reproducible point-in-time training joins. Both user and item signals can change rapidly: a user's current session and an item's sudden popularity may be streamed, while stable metadata/embeddings may be batch-updated. Kafka exposure/interaction logs supply training evidence, and model registry/serving versions make rollback inspectable.

Misconception: precompute every user's feed

Follower distributions can be heavy-tailed. Consider 9,999 authors with 100 followers and one author with 1,000,000 followers: one post per author creates 1,999,900 pointer writes. The largest author's share is about 50.0025%, despite a median of 100 followers. This is a synthetic distribution, not a measured social graph. It shows why a mean alone hides tail capacity.

First principles: the economic threshold

For one author let p be posts/day, F eligible active followers, r total feed reads/day among those followers, and c_w/c_r the incremental write/read-merge cost. Approximate push work as p×F×c_w; pull work as r×c_r, with caching and candidate depth included in the cost measurements. At p=10, F=100,000, c_w=1, r=50,000 and c_r=4, push costs 1,000,000 units versus 200,000 pull units. If r becomes 1,000,000, pull costs 4,000,000 and push may win. These arbitrary units compare a model, not benchmark services.

Rare celebrities do not imply every reader follows only a few. Bound followee fan-in explicitly, cache shared author logs and plan degraded responses. Add hysteresis to the push/pull classifier and a replay/cutover watermark so switching modes does not lose posts. Product caps bound both sides; “unbounded” is an asymptotic warning, not literal infinity in a finite user population.

Mental model and tradeoff verdict

Store pointers rather than duplicating mutable bodies where that helps update cost, but still reauthorize hydration. Cap hot inboxes based on a working-set budget and provide a cold-history route rather than assuming nobody scrolls deeper. Fanout is asynchronous after durable post+outbox acceptance, with deduplication on retry. Start from the simplest strategy justified by the workload, not “always push first” or “hybrid is mandatory.”

Follow/unfollow, edits and deletion are not trivial even for pull: caches, replicas and pagination can retain old state. Use versioned audience state and current policy checks, then asynchronously clean redundant pointers. High fanout also needs fairness so one author cannot monopolize all workers.

(c) Hands-on · 25 min

This small CPU fixture preserves push/pull/rank/diversity mechanics without opening Redis or deleting any database. It models current follow authorization, tombstones, deterministic tie order and an author-run cap. All time is supplied explicitly; it is not a recommender-training benchmark.

from math import log1p
 
class Feed:
    def __init__(self, follows, pull_authors, cap=500):
        if type(cap) is not int or cap < 1:
            raise ValueError("positive cap required")
        self.follows = {u: set(v) for u,v in follows.items()}
        self.pull = set(pull_authors)
        self.cap, self.posts, self.inbox = cap, {}, {u: [] for u in follows}
 
    def post(self, pid, author, text, ts):
        record = dict(id=pid, author=author, text=text, ts=ts, deleted=False)
        if pid in self.posts:
            if self.posts[pid] != record:
                raise ValueError("conflicting post identity")
            return
        self.posts[pid] = record
        if author not in self.pull:
            for user, authors in self.follows.items():
                if author in authors:
                    ids = self.inbox[user]
                    ids.append(pid)
                    ids.sort(key=lambda p: (self.posts[p]["ts"], p), reverse=True)
                    del ids[self.cap:]
 
    def feed(self, user, now, limit=10, affinity=None):
        if type(limit) is not int or limit < 0:
            raise ValueError("nonnegative limit required")
        if limit == 0:
            return []
        authors = self.follows.get(user, set())
        ids = set(self.inbox.get(user, []))
        ids.update(pid for pid,p in self.posts.items() if p["author"] in (authors & self.pull))
        candidates = [self.posts[pid] for pid in ids
                      if not self.posts[pid]["deleted"] and self.posts[pid]["author"] in authors]
        affinity = affinity or {}
        def score(p):
            age_min = max(0, (now-p["ts"])/60)
            return affinity.get(p["author"],1.0)/(1+log1p(age_min))
        ranked = sorted(candidates, key=lambda p: (-score(p), p["id"]))
        picked, last, run = [], None, 0
        for p in ranked:
            if p["author"] == last and run == 2:
                continue
            picked.append(p["id"])
            run = run+1 if p["author"] == last else 1
            last = p["author"]
            if len(picked) == limit:
                break
        return picked
 
f = Feed({"eve":{"alice","bob","star"}}, {"star"}, cap=3)
f.post("a", "alice", "hello", 0)
f.post("b", "bob", "systems", 60)
f.post("s", "star", "music", 120)
assert "s" not in f.inbox["eve"]
assert f.feed("eve",120) == ["s","b","a"]
f.follows["eve"].remove("star")
assert "s" not in f.feed("eve",120)
f.posts["b"]["deleted"] = True
assert f.feed("eve",120) == ["a"]
assert f.feed("eve",120,0) == []
print("hybrid candidates, ranked order, unfollow and deletion PASS")

What each block is doing

The author-log dictionary is the source fixture; the inbox contains pushed IDs only; pull authors are merged on read. Hydration checks the current follow set and tombstone, so a stale pushed pointer cannot return an unfollowed/deleted post. Production also checks tenant, blocks and audience permissions, which this single-tenant fixture does not model. The ranker uses age and optional per-author affinity; uniformly multiplying every eligible post by the same affinity factor cannot change order. The original exercise's 2→10 uniform weight claim was mathematically false.

The inbox cap bounds pushed pointers, not all work in this fixture: posts grows with every post and the pull path scans it. If P is stored posts, K is gathered IDs and C is eligible candidates, this read scans all P posts and sorts C candidates in O(C log C). The displayed generator also recomputes the follow/pull set intersection for each post; that cost depends on both set sizes. Temporary storage includes the K-ID union, C candidates and intersection sets, not merely the requested page size. Production needs bounded per-author ranges, a candidate budget and a deadline as described above. Post IDs here are comparable strings and timestamps/affinities are finite numeric inputs; identity retry is exact-record equality. A replay after setting a tombstone conflicts rather than resurrecting content. It is not a durable idempotency protocol or a block/audience implementation.

Deletion must update authoritative state before acknowledging the chosen visibility guarantee. Check that state at the exposure boundary; if authorization or deletion cannot be established, fail closed rather than serve an old cache hit. Revoke/invalidate derived inboxes, cached pages, search/vector candidates and retained evaluation copies under the retention policy; asynchronous cleanup does not authorize stale content. A check followed by a concurrent revocation still has a race: define the consistency boundary (for example a version-fenced read/release), and test it. Already delivered bytes cannot be withdrawn from a reader's device.

The greedy diversity loop skips a third consecutive author; it can return fewer than requested even when skipped posts exist. Decide whether to return a shorter list, search a larger pool, or relax a soft diversity rule. Never relax privacy to fill the page. This loop is not a globally optimal constrained ranking algorithm.

Complete Redis reference procedure — explicitly unexecuted

Use a disposable, namespaced test deployment with externally supplied credentials; never FLUSHDB a default connection. Canonical durable post/outbox acceptance occurs before cache materialization. Idempotent workers ZADD a post identity into each eligible user's namespaced timeline and trim with ZREMRANGEBYRANK key 0 -(cap+1); its stop rank is inclusive, leaving the newest cap members. Empty/smaller sets need no reset. Sorted-set scores are binary64: avoid storing full 64-bit Snowflake identity as an exact numeric score; use a safe timestamp/rank plus lossless string member and deterministic ties.

Batch reads with a pipeline to reduce round trips; a pipeline is not a multi-store transaction. Materialize iterable IDs before using them for both request generation and response zipping. Retrieve author-log pointers for pull authors, merge/deduplicate, hydrate current content/authorization, rank, and paginate with a stable session contract. Recover a cache miss from durable author logs and follow state. Validate worker retry after partial fanout, duplicate ZADD, trim at cap±1, mode-switch overlap, deletion during hydration, and a slow pull author before measuring latency. No Redis service or CLI was invoked here.

Feel the celebrity problem — worked answer

At 100,000 eligible followers, push creates 100,000 pointer updates per new post (plus trim work); pull creates no per-follower inbox update but adds author-log merge work to subsequent reads. If only 1,000 followers open the feed and the shared log is cached, most push work was unused. If all followers repeatedly read, precomputation can be worthwhile. Measure accepted-post latency separately from fanout completion and feed latency; asynchronous push can acknowledge quickly while feeds lag. A bounded CPU count comparison suffices to understand the mechanism without creating 100,000 Redis keys.

(d) Production reality · 15 min

Reported systems versus synthetic incidents

Meta's 2023 Explore report supports multi-stage ranking, two-tower retrieval, caching and infrastructure for recommendations. It does not substantiate the original story of a 15% Instagram engagement drop for 48 hours in 2019. Use a labeled hypothetical instead: a video-biased update reduces image exposure; fewer image labels then reinforce the bias. Detect it through candidate coverage, exposure by media type, feature freshness and cohort quality, then roll back while preserving the incident dataset.

The high-fanout author is a capacity scenario, not proof that Twitter and Instagram both use a one-million threshold. Monolith's paper supplies a concrete feedback/join/synchronization architecture, but is not evidence that one product's dwell time always improves the very next video. Version availability times; a late label or delayed sync can cross many turns.

The original timeline used Facebook feed launch, Twitter fanout changes, Instagram chronology/ranking changes, DLRM, TikTok sequence feedback and LLM reranking as historical signposts. Treat those as questions for source-led comparison, not proof of first-in-industry dates, a 30% engagement gain, universal adoption or a fixed migration threshold. A switch from chronological to ranked delivery changes the ordering policy; a switch from push to pull changes candidate delivery. They are independent decisions. An LLM reranker would add inference cost, untrusted-content and evaluation constraints to the same eligible-candidate funnel; naming the model does not establish a production result.

Common failure modes

Exposure bias: missing interactions from unshown items are not negative preference labels. Log policy/version, eligibility and propensities where an estimator needs them. Randomized exploration can reveal underexposed candidates but its rate must meet risk and sample-size needs; 5–10% is not universal. Feedback loops: monitor diversity/coverage and long-term outcomes, not only CTR. Cold start: use eligible content/metadata/popularity plus bounded exploration. Freshness: define recency decay and feature-age budgets for both users and items. Ads: separate auction eligibility, attribution and frequency windows from organic utility. Storage: cap hot pointers and account for replicas/object overhead. Revocation: enforce at hydration even while cleanup lags.

For 3 billion users × 800 pointers × 100 bytes, raw payload is 240 trillion bytes = 240 TB decimal. It excludes scores/keys/allocator overhead, replication and inactive-user policy. For 10 million active users × 500 × 8-byte IDs, raw ID payload is 40 GB. These estimates describe different inputs, not contradictory measured footprints.

Where this shows up in the rest of the plan

Requirements and estimation determine fanout budgets. Contrast the read-heavy URL shortener with a stateful feed cache. Chat shares delivery/replay patterns but has stronger conversational ordering needs; recommendation fundamentals explain feature/label choices; AI evaluation contributes regression/holdout methodology; AI chat similarly separates retrieval from expensive generation. These are routes into companion lessons, not certification of their contents. Optional historical pre-read videos and product anecdotes are not primary evidence.

(e) Recall + stretch · 10 min

Explain-out-loud test

The celebrity scenario motivates, but does not force, hybrid fanout. Reconstruct the candidate→score→rerank pipeline and explain how logged exposure, guarded exploration and controlled evaluation counter feedback loops. Then change the activity distribution and recompute the threshold.

What comes next

AI chat capstone applies candidate retrieval, expensive scoring and quality/cost gates to a different output contract. Learning hub.

Sources and review boundary

Checked 8 October 2026: Google recommendation stages, Meta Explore, 2023, Wide & Deep, 2016, DLRM, 2019, Redis trimming, OWASP authorization, Feast point-in-time retrieval. Only the CPU fixture and arithmetic are reproduced, not Redis, model training, publisher benchmarks or an online social experiment.

Study pacing: S129 in the original 130-session route; plan about 90 minutes across preparation, intuition, visual walkthrough, hands-on, production scenarios and recall. The visible phase labels are flexible study budgets, not a completion-time guarantee or a claim that hypothetical incidents were witnessed.