SYSTEM DESIGN:Lesson 19: Why Your Feed Feels Instant

IPL final night, open X, scroll forever

CSK wins the IPL final. You open X (still Twitter in your muscle memory) at 11:47 PM. First three posts are Kohli reaction GIFs. Fourth is a meme about the last over. Fifth is a Swiggy ad somehow. You did not follow any of these accounts — your feed assembled them from hundreds of people you follow, ranked in roughly two seconds.

That scroll is a news feed. Not a database query. Not a single table join. A pre-computed, ranked list of posts from people in your social graph, served fast enough that you keep scrolling instead of closing the app.

This chapter covers the core design: post a tweet, build a timeline, manage the follow graph. No Redis clusters, no hybrid fan-out, no ranking ML. That is Lesson 20.

IPL night infinite feed scroll

Three Kohli GIFs, one Swiggy ad, zero self-control.

Fan-out on write vs fan-out on read

Every IPL night, millions of feeds get rebuilt. The question is when.

Requirements — functional and non-functional

Functional

• Post — user publishes a tweet (text, optional media, timestamp)
• Timeline / feed — user sees recent posts from accounts they follow, newest-ish first
• Follow graph — user A follows user B; B's posts appear in A's feed
• Optional: like, retweet, reply, delete (mention in interview; do not over-build)

Non-functional (state these early)

• Feed read latency — < 200 ms for first page; users abandon slow feeds
• Post write latency — < 500 ms; poster expects instant confirmation
• Availability — stale feed beats error page; eventual consistency is acceptable on reads
• Scale hint: read-heavy — users open feed 20–50× more often than they post

Clarify scope: are we Twitter-scale or a campus social app? For 45 minutes, I commit to text tweets, chronological feed, follow/unfollow. Skip DMs, trending, ads unless the interviewer asks.

Clarifying with the interviewer (say this out loud)

You: "Walk me through the happy path — I post a tweet, my follower
     opens their feed and sees it?"
     -> Interviewer: "Yes."

You: "Chronological feed or ranked by engagement?"
     -> Interviewer: "Start chronological; mention ranking if we have time."

You: "How many users, and what is the read-to-write ratio?"
     -> Interviewer: "50 million DAU, feeds opened way more than posts."

You: "Celebrity accounts with millions of followers — in scope?"
     -> Interviewer: "Yes, that is the hard part."

You: "I will cover post + feed APIs, follow graph, fan-out trade-offs,
     and flag the celebrity problem for the scale chapter."

That last line — naming the celebrity problem before they ask — signals you have read past the happy path.

High-level components

Post service (tweets), Graph service (follow/unfollow), Feed service (timeline). One monolith at MVP; split when teams and traffic force it.

The follow graph

Social feeds are a graph problem wearing a CRUD costume.

User A follows [B, C, D]
User B follows [A, E]
When B posts → A sees it (A follows B)
When A posts → B sees it (B follows A)
When E posts → A does NOT see it (no follow edge)

Store directed edges: (follower_id, followee_id). Query "who does user X follow?" for feed assembly.

Fan-in = how many followers you have (your audience size).

Fan-out = how many feeds your post must land in.

A random user with 200 followers has low fan-out. Virat Kohli with 60 million followers has fan-out that can melt your database. Remember that asymmetry — it drives everything in Lesson 20.

Fan-out on write vs fan-out on read

The central design fork for news feeds. Pick one for normal users; you will hybridize later.

Fan-out on write (push model)

When user B posts tweet T:

1. Insert tweet into tweets table
2. Look up all followers of B
3. For each follower F, prepend T to F's feed cache (Redis list or feed table row)

Read feed: O(1) — fetch pre-built list from cache. Fast scroll.

Write post: O(followers) — Kohli posting touches 60M cache rows. Slow write, possible timeout.

Fan-out on read (pull model)

When user B posts tweet T:

1. Insert tweet into tweets table (done)

When user A opens feed:

1. Get list of accounts A follows
2. Fetch recent tweets from each (or one query: WHERE author_id IN (...))
3. Merge, sort by timestamp, return top N

Read feed: O(following) — user following 500 accounts means 500 lookups or one heavy IN query.

Write post: O(1) — insert one row, done.

Comparison table

Twitter historically used push for normal users and pull for celebrities — hybrid fan-out. For this lesson, understand both extremes. Lesson 20 wires them together.

Database schema (minimal)

Users

CREATE TABLE users (
  id            BIGSERIAL PRIMARY KEY,
  username      VARCHAR(50) NOT NULL UNIQUE,
  display_name  VARCHAR(100),
  created_at    TIMESTAMP DEFAULT NOW()
);

Follows (directed graph)

CREATE TABLE follows (
  follower_id   BIGINT NOT NULL REFERENCES users(id),
  followee_id   BIGINT NOT NULL REFERENCES users(id),
  created_at    TIMESTAMP DEFAULT NOW(),
  PRIMARY KEY (follower_id, followee_id)
);

CREATE INDEX idx_follows_followee ON follows(followee_id);

Index on followee_id is critical for fan-out on write — "give me all followers of Kohli."

Tweets

CREATE TABLE tweets (
  id            BIGSERIAL PRIMARY KEY,
  author_id     BIGINT NOT NULL REFERENCES users(id),
  content       VARCHAR(280) NOT NULL,
  created_at    TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_tweets_author_created ON tweets(author_id, created_at DESC);

Feed cache (push model only)

CREATE TABLE feed_cache (
  user_id       BIGINT NOT NULL REFERENCES users(id),
  tweet_id      BIGINT NOT NULL REFERENCES tweets(id),
  rank_score    BIGINT NOT NULL,  -- for ordering; use tweet_id or timestamp
  PRIMARY KEY (user_id, tweet_id)
);

CREATE INDEX idx_feed_cache_user_rank ON feed_cache(user_id, rank_score DESC);

At scale this table explodes — every tweet × every follower. That is why Redis sorted sets replace Postgres for hot feeds. Schema above is interview-clear; production moves feed cache to memory (Lesson 2).

API design

Post a tweet

POST /api/v1/tweets
Authorization: Bearer <token>
Content-Type: application/json

{ "content": "What a finish. CSK!!!" }

HTTP/1.1 201 Created

{
  "id": 918273645,
  "author_id": 42,
  "content": "What a finish. CSK!!!",
  "created_at": "2026-05-28T23:47:12Z"
}

Server flow (push model):

def post_tweet(author_id: int, content: str) -> Tweet:
    tweet = db.insert_tweet(author_id, content)
    follower_ids = db.get_followers(author_id)
    for fid in follower_ids:
        feed_cache.prepend(fid, tweet.id, tweet.created_at)
    return tweet

Return 201 before fan-out completes if fan-out is async (queue + worker). User sees their tweet instantly; followers see it within seconds.

Get home feed

GET /api/v1/feed?limit=20&cursor=918273600
Authorization: Bearer <token>

HTTP/1.1 200 OK

{
  "tweets": [
    { "id": 918273645, "author": { "id": 42, "username": "rohit" },
      "content": "What a finish. CSK!!!", "created_at": "..." },
    ...
  ],
  "next_cursor": "918273500"
}

Cursor-based pagination — not offset. OFFSET 10000 on a feed table kills Postgres.

def get_feed(user_id: int, limit: int, cursor: int | None) -> list[Tweet]:
    # Push model: read pre-built cache
    tweet_ids = feed_cache.get_range(user_id, limit, cursor)
    return hydrate_tweets(tweet_ids)  # batch fetch tweet + author metadata

Fan-out on read — pull model snippet

def get_feed_pull(user_id, limit):
    following = db.get_following(user_id)
    return db.recent_tweets(author_ids=following, limit=limit)

Works under ~500 following. Power users who follow thousands need push or a hybrid cap.

The celebrity problem (preview)

IPL night. Kohli tweets "Thank you fans." 60 million followers.

Push model: 60 million Redis writes in seconds. Post API times out. Fan-out worker queue backs up for hours.

Pull model: One insert. Fine. But 60 million users have Kohli in their following list — every feed read that merges Kohli's tweets adds load.

Neither pure approach works at both ends of the follower distribution. Real systems use hybrid fan-out:

• Normal user (< 10k followers): push on write
• Celebrity (> 10k followers): pull on read — merge their tweets when follower opens feed

Set the threshold based on napkin math, not dogma. Lesson 20 builds the full hybrid with Redis and ranking.

Async fan-out — do not block POST

POST /tweets → insert → Kafka/SQS → return 201
Worker → fan-out to follower feed caches (1–3 sec lag, acceptable)

Data layer note

Feed cache is a read-heavy, key-per-user workload — exactly what Redis and read replicas were built for. When your feed_cache table grows past a few million rows, revisit Lesson 2: Scaling the Data Layer for cache-aside and replication patterns.

Preview: what Lesson 20 adds

Core design works until:

• 50M DAU open feed 30× per day
• Celebrities post during IPL and fan-out workers catch fire
• Product wants ranked feed, not chronological
• Media tweets need CDN, not just text rows

Continue here: News Feed at Scale — Hybrid Fan-out and Ranking.