Design a News Feed

A news feed (Twitter's home timeline) shows you recent posts from everyone you follow, newest-ish first. Opening the app is the single most common action, so the whole design is about making that read cheap — even though each user follows a different set of accounts.

v1 and why it fails

The obvious design: store each post once, and at read time run SELECT ... WHERE author IN (people I follow) ORDER BY time. Always fresh, dead simple — this is fan-out on read. But it runs on *every app open*. Twitter documented >300K timeline requests/sec against ~150M active users; doing a big merge-and-sort per request does not hold. Feeds are read far more than posts are written.

Fan-out on write

Flip the work to write time. When you post, a Fanout Service looks up your followers in the follow-graph store (Twitter's was FlockDB) and pushes the post's ID into each follower's home timeline — a precomputed list kept in Redis, an in-memory store. Reading a feed is then just 'grab my ~800 IDs from Redis' in about a millisecond.

1. 1User posts → store the post once in the tweet store.
2. 2Fanout Service asks the graph: who follows this user?
3. 3Push the post ID into each follower's Redis timeline (RPUSHX).
4. 4On read: fetch the ~800 IDs, then hydrate them — turn each ID into the full post + author via services with their own caches.

The celebrity / hot-key problem

Fan-out on write breaks for the famous. An account with tens of millions of followers turns one post into tens of millions of timeline writes — this blow-up is called write amplification. It overwhelms the fanout service and delays delivery for everyone. This is a hot-key problem: one write key (the celebrity) is enormously hotter than the rest.

The hybrid fix

Use each strategy where it is cheap: fan-out on write for normal accounts, but for very-high-follower accounts don't — instead pull their recent posts at read time and merge them into the mostly-precomputed feed. Normal accounts have few followers (write is cheap); celebrities are followed by many but are few in number (read-time pull is cheap). Kleppmann's *Designing Data-Intensive Applications* canonicalized this hybrid.

From recency to ranking

Modern feeds are ranked, not just reverse-chronological. Twitter's 2023 open-sourced 'For You' pipeline (Home Mixer) gathers candidates — roughly half in-network (people you follow, via the Earlybird search index) and half out-of-network — narrows hundreds of millions of posts to ~1,500, then scores them with a ~48M-parameter neural ranker before filtering and mixing. The precomputed Redis fan-out is no longer the in-network source.