Walkthrough: News Feed

"Design Twitter/Instagram's feed" is the canonical fanout question: one write (a post) must eventually reach many readers (followers). The whole interview hinges on when you do that fanout work — at write time or at read time.

Step 1: Requirements

Functional — prioritize the top three and confirm with the interviewer:

1. Users can create posts.
2. Users can follow other users.
3. Users see a chronological feed of posts from people they follow.

Non-functional, quantified:

• Availability over consistency: a feed that is up to ~1 minute stale is fine. Say this explicitly — it unlocks aggressive caching.
• Feed loads in under 500ms.
• Scale: 100M daily active users (DAU).

Step 2: Estimation (only where it changes a decision)

100M DAU × 1 feed load ÷ ~100,000 seconds/day ≈ 1,200 QPS average, maybe 5,000 at peak — a cached read path handles that comfortably. Writes: if 1 in 10 users posts daily, that's 10M posts/day ≈ ~115 writes/sec — tiny. This is an extremely read-heavy system, which tells you where the effort goes.

Step 3: Core entities

User, Follow (follower_id, followee_id), Post (post_id, author_id, content, created_at). Add columns as the design firms up.

Step 4: API

POST /v1/posts            { content }         -> 201 { post_id }
GET  /v1/feed?cursor=...                      -> { posts[], next_cursor }
POST /v1/users/:id/follow                     -> 204

The user ID comes from the auth token, never the request body. Pagination is cursor-based (the cursor encodes "posts older than X") because new posts arrive constantly — offset pagination would show duplicates as the list shifts.

Step 5: High-level design — start naive

Client → LB → app servers → Postgres. The naive feed read (fanout-on-read, or "pull"): look up who I follow, fetch each of their recent posts, merge and sort. A compound index on (author_id, created_at) makes each per-author lookup fast.

This works and satisfies every functional requirement — say so. Then flag the problem: a user following 1,000 accounts triggers 1,000 index lookups per feed load, on every refresh. Flag it verbally ("I'll fix this in the deep dives") and keep moving.

Deep dive 1: Fanout-on-write

Invert the work: when a post is created, precompute each follower's feed. The write path becomes: store the post, then drop a job on a queue (a buffer that lets slow work happen asynchronously — e.g., SQS). Workers read the job, look up the author's followers, and insert the post ID into each follower's feed cache — a Redis list of the ~200 most recent post IDs per user.

Now the read path is one cache hit: fetch my precomputed list of post IDs, hydrate the posts (also cached), return. That is how you hit 500ms — actually ~10s of ms.

Cost check: an average author has ~200 followers, so one post = ~200 cheap async list-inserts. At 115 posts/sec that's ~23k inserts/sec — fine for Redis.

Deep dive 2: The celebrity problem

Now the trade-off bites: a celebrity with 100M followers would trigger 100M writes for one post. That fanout takes hours and hammers the cluster. The standard answer is a hybrid:

• Normal accounts (say, under 100k followers): fanout-on-write as above.
• Celebrity accounts: skip precompute entirely. At read time, fetch your precomputed feed and the recent posts of the few celebrities you follow, then merge the two lists.

Each user follows only a handful of celebrities, so the read-time merge is a few extra cached lookups — cheap. This hybrid is the expected mid-level answer; being able to walk the math on both extremes is the signal.

Deep dive 3: Storage and hot keys

Size the feed cache when asked: 2B total users × 200 post IDs × ~10 bytes ≈ 4TB — that needs a sharded Redis cluster, and you should add TTLs so inactive users' feeds expire and get rebuilt on demand.

A viral post creates a hot key: every feed hydration wants the same post, melting the one cache shard that owns it. Fix: redundant caching — copy the hot post to every cache node so reads spread out.

Ranking: we scoped chronological ordering. Note verbally where an ML ranker would slot in (a scoring service between fetch and return) — naming the extension point is enough at mid-level.

Wrap-up

Recap: posting works via POST /posts plus async fanout; the feed meets 500ms via precomputed Redis feeds; celebrities are handled by the hybrid merge; and we prioritized availability throughout — a stale-by-a-minute feed was allowed by our requirements. With more time: ranking, ads insertion, and multi-region reads.