Design a News Feed (Twitter / Facebook Timeline)
The canonical fan-out problem. Why neither pure push nor pure pull works at scale, and how the hybrid model that every major platform actually uses is derived.
Covers: Fan-out on write vs read, the celebrity problem, hybrid fan-out, feed ranking, pagination, storage sizing
The news feed is the most instructive case study in the catalogue because the naive designs fail in opposite directions, and the correct answer is a hybrid you can derive from first principles rather than memorise.
Requirements and numbers
- Functional: post; follow; view a ranked feed of posts from people you follow; infinite scroll.
- Scale: 300 M DAU, average 200 follows, 2 posts/user/day, 20 feed loads/user/day.
- Writes: 600 M posts/day ≈ 7,000/s. Reads: 6 B feed loads/day ≈ 70,000/s, ~200,000/s peak.
- Latency: feed loads in under 200 ms p99. New posts appear within seconds, not necessarily instantly.
- Skew: the most-followed accounts have 100 M+ followers. This single fact breaks the obvious design.
30-second answer
Fan-out on write pushes each new post into every follower's precomputed feed, so reads are a single fast lookup - but a celebrity with 100 million followers generates 100 million writes for one post, which is unacceptable. Fan-out on read builds the feed at query time by fetching recent posts from everyone you follow, so writes are cheap but a read touches hundreds of timelines and cannot meet a 200 ms budget. The answer is hybrid: fan-out on write for ordinary users, and fan-out on read for the small number of accounts above a follower threshold, merging both at read time.
| Fan-out on write (push) | Fan-out on read (pull) | |
|---|---|---|
| Write cost | O(followers) - brutal for celebrities | O(1) |
| Read cost | O(1) - one list lookup | O(followees) - hundreds of queries |
| Read latency | ~10 ms | 200 ms – seconds |
| Storage | Post duplicated per follower | Stored once |
| Freshness | Slight delay while fan-out completes | Always current |
| Breaks on | Celebrities | Users who follow many accounts |
For a 100 M-follower account posting 10 times a day, that is 1 billion feed writes per day from one user.
Hybrid fan-out
Two paths that converge at read time. The threshold is a tunable business decision, not a constant.
Flow
- New postFan-out router
- Fan-out routerFan-out workers(< 10k followers)
- Fan-out routerCelebrity posts(≥ 10k followers)
- Fan-out workersPrecomputed feeds
- Precomputed feedsFeed service
- Celebrity postsFeed service
Three optimisations that matter
Only fan out to active users
Most registered accounts have not opened the app in months. Maintain an active-user set (say, active in the last 30 days) and fan out only to those. This routinely cuts fan-out volume by 70–90%, and inactive users get a pull-built feed when they return.Store IDs, not content
The feed list holds post IDs. Content is hydrated from a cache at read time. This keeps feed storage tiny, and - more importantly - means an edited or deleted post is handled correctly everywhere instead of leaving stale copies in millions of lists.Cap the list length
Nobody scrolls past ~800 posts. Trim each feed list to a fixed length; anything deeper falls back to a pull query. This bounds storage and keeps Redis memory predictable.
Say these points
- Push for normal users (cheap writes, O(1) reads); pull for celebrities (avoid write amplification).
- Merge both at read time - this is what every major platform does.
- Fan out only to active users: a 70–90% reduction in work.
- Store post IDs, not content, so edits and deletions propagate correctly.
- Cap feed length; deeper pagination falls back to a pull query.
Avoid these mistakes
- Pure fan-out on write, ignoring celebrities entirely.
- Pure fan-out on read, then failing the latency budget.
- Fanning out to every follower including long-inactive accounts.
- Storing full post content in every follower's list.
- No answer for users crossing the celebrity threshold.
Expect these follow-ups
- How do you delete a post that has been fanned out to 5 million feeds?
- A user follows 50,000 accounts. Does your design still work?
- How would you add 'posts your friends liked' to the feed?
30-second answer
Rank in two stages: cheap candidate generation to get a few hundred posts, then a scoring pass combining recency, affinity with the author, engagement signals and content type. Serve with cursor pagination, never offset - with a constantly changing feed, offset causes users to see duplicates and miss posts, because rows shift between requests. The cursor should encode the ranking position and a snapshot identifier so a scrolling session stays stable even as new posts arrive.
Two-stage ranking
1 · Candidate generation (cheap, high recall)
Take the last ~800 IDs from the precomputed feed plus recent posts from followed celebrities. Filter out already-seen posts, blocked authors and deleted content. Target a few hundred candidates in under 10 ms.2 · Scoring (expensive, high precision)
Score each candidate on recency decay, author affinity, predicted engagement, content type and diversity. A few hundred candidates is small enough to score with a real model within the latency budget - which is exactly why the first stage exists.
A simple, explainable baseline. Exponential recency decay is what stops a feed filling with yesterday's content.
Why offset pagination fails here
The user loads page 1 (posts 1–20). While they read, twelve new posts arrive at the top. They request OFFSET 20, which now points twelve positions further into content they have already seen. They see twelve duplicates and miss nothing - or, if posts are deleted, they skip content entirely. On a constantly changing ranked list this is guaranteed, not occasional.
{
"data": [ /* 20 ranked posts */ ],
"page": {
"next_cursor": "eyJzIjoicV84OGYyIiwibCI6MjAsInQiOjE3ODUwNjE1MDB9",
"has_more": true
}
}
// Decoded:
// {
// "s": "q_88f2", // snapshot/session id: the candidate set is pinned,
// // so new posts do not shift this scroll session
// "l": 20, // position within that pinned, ranked set
// "t": 1785061500 // issued-at, so the snapshot can expire
// }
//
// New posts appear via an explicit "12 new posts" pill at the top rather than
// silently reshuffling the list under the user's thumb.Storage sizing
- Feed lists: 300 M users × 800 post IDs × 8 bytes ≈ 1.9 TB in Redis - but only for active users, so realistically a few hundred gigabytes.
- Posts: 600 M/day × 500 B ≈ 300 GB/day ≈ 110 TB/year, sharded by post ID with recent posts cache-hot.
- Social graph: 300 M × 200 follows × 16 B ≈ 1 TB, sharded by user ID, heavily cached - the graph is read on nearly every operation.
Say these points
- Two-stage ranking: cheap recall, then expensive precision on a few hundred candidates.
- Exponential recency decay plus affinity and predicted engagement.
- Add a diversity penalty or the feed fills with one author.
- Offset pagination guarantees duplicates on a changing feed - use cursors.
- Pin the candidate set per scroll session and surface new posts explicitly.
Avoid these mistakes
- Offset pagination on a live feed.
- Ranking every candidate with an expensive model instead of filtering first.
- No diversity control.
- Re-ranking on every page request, causing content to jump.
Expect these follow-ups
- How do you avoid showing the same post twice across devices?
- How would you A/B test a new ranking model safely?
- What is your fallback when the ranking service is down?
Showing 2 of 2 questions for design-a-news-feed-system.
Check your understanding
4 questions · no sign-up, nothing stored
Hands-on challenge
Build it - this is what you talk about in a deep-dive round.
Design and size a news feed for 300M users
Produce the full design with numbers. The deliverable is a document a staff engineer would accept.
Requirements
- Derive the celebrity threshold from measured cost rather than asserting a number.
- Specify the write path for both strategies and the read-time merge.
- Handle the two boundary cases: a user crossing the threshold, and newly followed celebrities.
- Design cursor pagination with a pinned candidate set, including cursor contents and TTL.
- Size Redis, post storage and the social graph, showing the arithmetic.
Stretch goals
- Design post deletion propagation across 5 million precomputed feeds.
- Add a ranking A/B framework with a safe fallback when the model service fails.