Designing for Scale · Newsfeed and Fan-out
Hybrid fan-out is a workload decision, not a law. A source-bounded lesson with worked reasoning, failure analysis and explicit runtime limitations.
Original publication date unrecordedUpdated
Lesson preparation & details
Level: advanced
By the end, you should be able to
- Hybrid fan-out is a workload decision, not a law
- A private post ID remains in a follower's cached timeline after access is revoked. Is asynchronous cache deletion enough?
Bring with you
Sources checked: . A source check does not change the publication or update date.
Editorial review: · What review means
In this article · 11 sections
The problem this post solves
A chronological feed asks for recent eligible posts from followed authors; a ranked feed adds a user/context-dependent ordering decision. Both need privacy, deletion and pagination semantics. Start with the product's candidate universe, freshness goal, read/write rates, follower distribution and response size. A 200 ms target is a requirement to measure, not proof that a relational join is impossible.
There are two places to assemble candidates: when authors write and when readers request a feed. Moving work earlier can reduce request-time work, but creates materialized state that must catch up, expire and recover. The objective is predictable useful results within a budget—not claiming every read is an O(1) operation independent of output size.
First principles
An ordered index can retrieve a bounded recent range without sorting an author's entire
history. PostgreSQL 16 explicitly describes the ORDER BY ... LIMIT n case with a matching
index. Merging several sorted author streams is still work, but a heap merge of f streams
can produce k candidates in roughly O(f + k log f) heap operations after the source
iterators are initialized. Fetching those streams, filtering and storage I/O are additional
costs. This is a bounded algorithmic option, not a latency guarantee for arbitrary joins.
High fan-in can make pull expensive. High fan-out can make push expensive. A follower's own large follower count does not directly increase the number of posts that this reader must merge; the reader's follow set and those authors' activity do. Use the actual dependency graph rather than conflating both degrees.
Fan-out on Read vs Fan-out on Write
Fan-out on Read (Pull)
Persist each post to its author's durable timeline. At feed time, read bounded recent ranges from followed authors, merge/deduplicate candidate IDs, then hydrate, authorize and rank. Writes avoid copying into every follower inbox, but still need durable persistence and any indexes/outbox work; they are not “instant.” Pull can be appropriate for a small service, infrequent readers, fast-changing rankings or a few high-fan-out authors. It is not restricted to celebrities, and its request-time cost must have explicit candidate/time limits.
Fan-out on Write (Push)
Persist the post and enqueue recoverable fan-out work. Workers insert its ID into bounded
follower candidate lists. A request reads an already assembled list but still fetches k
IDs, hydrates posts, enforces permissions and returns bytes. Background lag means the newest
post may not yet be present. Push trades repeated read-time assembly for write amplification
and materialized-state recovery; it does not guarantee low latency or require that 99% of
users follow one universal policy.
Define idempotent delivery as one (reader, post_id) candidate entry even if a fan-out task
retries. An atomic set-like insertion or a sink uniqueness constraint can enforce that local
effect. Plainly prepending on every delivery can create duplicates. Record fan-out progress
at a resumable boundary so a worker crash neither silently abandons followers nor requires
starting a huge batch from scratch without deduplication.
The Hybrid Architecture
Figure text equivalent — labels and relationships
- Node “Author creates post” (square).
- Node “Durable post plus outbox” (cylinder).
- Node “Bounded fan-out workers” (square).
- Node “Active-reader candidate cache” (cylinder).
- Node “Reader requests feed” (square).
- Node “Read service” (square).
- Node “Bounded recent author ranges” (square).
- Node “Merge and deduplicate IDs” (square).
- Node “Hydrate and enforce current access” (square).
- Node “Score or order with stable cursor” (square).
- “Author creates post” leads to “Durable post plus outbox”. Line: normal; ends: arrow point.
- “Durable post plus outbox” leads to “Bounded fan-out workers”. Line: normal; ends: arrow point.
- “Bounded fan-out workers” leads to “Active-reader candidate cache”. Line: normal; ends: arrow point.
- “Reader requests feed” leads to “Read service”. Line: normal; ends: arrow point.
- “Read service” leads to “Active-reader candidate cache”. Line: normal; ends: arrow point.
- “Read service” leads to “Bounded recent author ranges”. Line: normal; ends: arrow point.
- “Active-reader candidate cache” leads to “Merge and deduplicate IDs”. Line: normal; ends: arrow point.
- “Bounded recent author ranges” leads to “Merge and deduplicate IDs”. Line: normal; ends: arrow point.
- “Merge and deduplicate IDs” leads to “Hydrate and enforce current access”. Line: normal; ends: arrow point.
- “Hydrate and enforce current access” leads to “Score or order with stable cursor”. Line: normal; ends: arrow point.
- “Score or order with stable cursor” leads to “Reader requests feed”. Line: normal; ends: arrow point.
Classify sources by cost and freshness, not by a fixed “100,000 followers means celebrity” rule. A scenario author posts twice a day to 50,000 eligible active followers: push entails 100,000 candidate insertions/day before retries. If only 500 followers open the feed once that day, many insertions were never used. If all 50,000 open twenty times, precomputation may amortize assembly better. These operation counts are not interchangeable CPU costs; measure insertion, merge, hydration and recovery costs on the same workload.
A read combines cached pushed candidates with bounded pulled ranges for selected sources. Deduplicate by post ID, apply the eligibility policy, then rank. Read-time scoring can reuse precomputed candidates: relevance ranking does not require throwing away the push pipeline. Google's recommendation overview separates candidate generation, scoring and re-ranking; feed candidate caches occupy the first stage, not the entire recommendation system.
Changing an author from push to pull needs a transition boundary. For example, store a mode version and source sequence cutoff; while transitioning, merge both paths and deduplicate. Switching abruptly without backfilling or overlap can omit recent posts. Define following and unfollowing semantics similarly: should a new follower see older posts, and should an unfollow remove already cached candidates? Permissions remain a separate mandatory filter.
Sizing the Feed Cache
The original scenario of 100 million active readers, 500 IDs per reader and 8 bytes per
ID gives 100,000,000 × 500 × 8 = 400,000,000,000 bytes, or 400 GB decimal (about
372.5 GiB). For ten million readers the analogous payload is 40 GB. Neither calculation
includes keys, list/sorted-set nodes, scores, object/allocator overhead, replicas, network
buffers or failover headroom. Therefore it cannot establish that the cache fits a “small”
cluster. Measure representative encoded entries and memory after realistic churn.
Cache IDs rather than copying every post body into each list when that suits the access pattern. Editing the post then updates its authoritative/object-cache version instead of rewriting millions of copied bodies. It still requires invalidation/version handling; an ID reference does not automatically make cached content fresh. Content hydration can batch lookups and maintain a bounded object cache, but the result must enforce current access. Keep large media behind their own authorization and delivery policy.
Bound both list length and user activity horizon. If a user has not read for months, push may waste resources; store a rebuild cursor or use pull on return. “Active within 14 days” is one possible policy to evaluate, not an intrinsic correctness threshold. Track eviction rate, hit rate, candidate age, fan-out lag and memory per useful read together.
Failure modes
A cached ID outlives permission. A post becomes private or a user is blocked after fan-out. Recheck authorization before returning the post body/media. Asynchronous cache deletion is cleanup, not the sole security boundary. A stale candidate can be skipped and replaced with another allowed candidate subject to a bounded work budget.
Fan-out falls behind. Monitor the oldest unfinished work and freshness distribution. Separate hot authors/partitions, cap retries and optionally pull a recent tail while workers catch up. An empty cache should not be indistinguishable from a proven empty feed.
Cache miss causes a stampede. Coalesce per-reader rebuilds and bound authors, time range, candidate count and concurrency. Do not reconstruct all history on a login request. If the budget is exceeded, return a documented partial/retry result rather than overload the store.
Ranking changes between pages. A cursor based only on offset can repeat or omit posts
when new items arrive or scores change. A chronological cursor can use (created_at, post_id)
with a fixed upper boundary; a ranked feed may freeze a candidate/session version. State the
freshness-versus-stability choice. Deletions still require revalidation on every page.
A queue message is redelivered. A durable outbox closes the post-write/publication gap,
and idempotent (reader, post_id) insertion closes duplicate candidate effects. SQS standard
queues explicitly allow repeated delivery. The cache remains derived: losing it must not
lose the authoritative posts or permission state.
Worked merge exercise
A cached list contains IDs [p9, p7, p5]. A pulled author range contains [p10, p9, p6].
Assume larger numbers are later in this synthetic chronological order. p7 is no longer
visible to the reader. Return the first four visible unique IDs, then explain what changes
if a relevance scorer prefers p5 over p10.
Candidate versus ranking answer
The merged chronological result is [p10, p9, p6, p5]; p9 appears once and p7 is removed
by authorization. If ranking changes, these same candidate sources can feed a scorer that
puts p5 earlier. Define a stable ranking/session cursor if subsequent pages must remain
consistent. No ranking score may bring p7 back into the allowed set. Candidate generation
can omit useful posts before ranking, so evaluate candidate coverage independently of the
scorer. A cache hit alone proves neither privacy nor recommendation quality.
What to carry forward
Push and pull move candidate-assembly work across time; neither is automatically instant, mandatory or sufficient. Size payload and overhead separately, bound recovery, and preserve current authorization during hydration. Candidate materialization and read-time ranking can coexist. No feed deployment, SQL latency or cache memory benchmark is claimed here.
Sources and review boundary
Checked 2026-10-07: PostgreSQL 16 indexes and ordering for limited ordered access; Google recommendation overview for candidate/scoring/re-ranking separation; SQS standard at-least-once delivery for duplicate handling; OWASP authorization guidance for checks on every request. The latter three are access-dated primary snapshots. Workload numbers and merge traces are synthetic calculations, not measurements from these sources.
Pause / Recall / Apply
Can you explain it without the page?
Close the example. Reconstruct the core idea, then change one assumption. Mark complete when you’re ready; you can always undo it.
Stored in this browser only. No account, no sync. Clearing browser data removes your record.