Design: News Feed (Facebook/Twitter)

April 2026 · 30 min read

Why News Feed Is THE Classic Interview Question

If you could only prepare one system design problem, it should be the news feed. Every major tech company — Facebook, Twitter/X, Instagram, LinkedIn, TikTok — has a feed at its core. The problem is elegant in its simplicity ("show users a list of posts from people they follow") yet devastatingly complex at scale.

The news feed sits at the intersection of nearly every major distributed systems concept: fan-out strategies, caching at massive scale, real-time delivery, ranking algorithms, social graphs, content delivery networks, and data partitioning. Master this design and you've implicitly mastered half the building blocks of large-scale system design.

What makes this problem particularly interesting is that the "obvious" solution (just query all posts from friends when the user opens the app) completely falls apart at scale. The core challenge is a fundamental trade-off between write amplification and read latency — and the optimal solution is neither extreme but a carefully engineered hybrid.

Requirements

Functional Requirements

1. Publish posts: A user can create a post (text, images, video, links) that is stored and made visible to their followers.
2. Generate feed: A user sees a personalized feed of posts from people they follow, ranked by relevance.
3. Follow/unfollow: Users can follow and unfollow other users, which dynamically changes their feed content.
4. Feed interactions: Users can like, comment, share, and react to posts in their feed.
5. Real-time updates: New posts from followed users should appear in the feed with minimal delay (seconds, not minutes).
6. Multi-media support: Posts can contain text, images, videos, and links with rich previews.

Non-Functional Requirements

• Scale: 500 million daily active users (DAU), each following ~200 users on average.
• Feed latency: Feed loads in <500ms (p99), ideally <200ms (p50).
• Availability: 99.99% uptime — the feed is the core product; if it's down, the platform is down.
• Throughput: ~10 billion feed reads per day (each user opens feed ~20 times/day). ~500 million new posts per day.
• Consistency: Eventual consistency is acceptable — a new post appearing in a follower's feed within 5 seconds is fine.
• Storage: Years of historical posts, but the feed only shows recent content (typically last 7 days, ranked).

Back-of-the-Envelope Estimation

DAU:             500M users
Avg follows:     200 users per user
Posts per day:   500M (1 post per active user on avg)
Feed reads/day:  10B (20 reads per user per day)

Feed read QPS:   10B / 86400 ≈ 115,000 QPS (peak ~300K QPS)
Post write QPS:  500M / 86400 ≈ 5,800 QPS (peak ~15K QPS)

Read:Write ratio ≈ 20:1 (heavily read-heavy system)

Fan-out on write for a post:
  Average user with 200 followers → 200 cache writes per post
  Celebrity with 10M followers → 10,000,000 cache writes per post!
  
Feed cache size per user:
  Store top 500 post IDs = 500 × 8 bytes (postId) = 4 KB per user
  500M users × 4 KB = 2 TB of feed cache (fits in a Redis cluster)

Post storage:
  500M posts/day × 1 KB avg = 500 GB/day ≈ 180 TB/year

The Key Decision: Fan-Out Strategies

The single most important architectural decision in news feed design is: when do we compute the feed? There are exactly three approaches, and understanding the trade-offs between them is what separates a good answer from a great one in interviews.

Approach 1: Fan-Out on Write (Push Model)

When a user publishes a post, the system immediately pushes the post into every follower's pre-computed feed cache. When a follower opens their feed, the cache is already populated — just read and return.

User Alice publishes a post (postId: 12345)
  │
  ├─→ Write post to Posts DB
  │
  └─→ Fan-out Service:
       │  Look up Alice's followers: [Bob, Carol, Dave, ... 500 users]
       │
       ├─→ ZADD feed:bob 1714500000 "12345"     (Redis sorted set)
       ├─→ ZADD feed:carol 1714500000 "12345"
       ├─→ ZADD feed:dave 1714500000 "12345"
       └─→ ... (500 writes total)

Bob opens his feed:
  │
  └─→ ZREVRANGE feed:bob 0 49    →  [postId list, already sorted]
       │
       └─→ Hydrate post objects from Post Cache/DB  →  Return feed

Approach 2: Fan-Out on Read (Pull Model)

No pre-computation. When a user opens their feed, the system dynamically pulls recent posts from all followed users, merges them, ranks them, and returns the result.

Bob opens his feed (follows: [Alice, Carol, Eve, ... 200 users])
  │
  └─→ Feed Service:
       │  Look up Bob's following list: [Alice, Carol, Eve, ...]
       │
       ├─→ GET posts:alice (last 50 posts)
       ├─→ GET posts:carol (last 50 posts)
       ├─→ GET posts:eve (last 50 posts)
       └─→ ... (200 parallel fetches)
       │
       └─→ Merge 200 × 50 = 10,000 posts
            │
            └─→ Rank by relevance/time  →  Return top 50

Alice publishes a post:
  │
  └─→ Write post to Posts DB
       (That's it. No fan-out. Zero extra writes.)

Approach 3: Hybrid (The Real-World Solution)

This is what Facebook, Twitter, and Instagram actually use. The insight: most users don't have millions of followers. The fan-out-on-write problem is really the celebrity problem, not a general problem.

Normal user Alice (500 followers) publishes a post:
  └─→ Fan-out on write: write to all 500 followers' caches  ✓

Celebrity Taylor (10M followers) publishes a post:
  └─→ Store post in celebrity_posts table  (no fan-out)

Bob opens his feed (follows: 195 normal users + 5 celebrities):
  │
  ├─→ Step 1: ZREVRANGE feed:bob 0 99      (pre-computed from normal users)
  │     → Returns 100 cached post IDs
  │
  ├─→ Step 2: Fetch recent posts from 5 celebrities Bob follows
  │     → Returns ~25 posts (5 per celebrity, last 24h)
  │
  ├─→ Step 3: Merge the two sets (125 posts total)
  │
  ├─→ Step 4: Rank by relevance score
  │
  └─→ Step 5: Return top 20 posts

System Architecture

The news feed system decomposes into five core services, each with a clear single responsibility. This separation allows independent scaling — the fan-out service needs massive write throughput, while the feed service needs ultra-low read latency.

Post Service

Handles creating, storing, and retrieving individual posts. This is the write path entry point.

POST /api/v1/posts
{
  "user_id": "alice_123",
  "content": "Hello world!",
  "media_ids": ["img_456", "vid_789"],
  "visibility": "public"
}

Responsibilities:
  1. Validate content (text length, media format, spam check)
  2. Store post in Posts DB (Cassandra/MySQL)
  3. Upload media to object storage → CDN
  4. Publish event to message queue: "post.created"
  5. Return postId to client

Storage schema (Cassandra):
  CREATE TABLE posts (
    post_id     UUID,
    user_id     UUID,
    content     TEXT,
    media_urls  LIST<TEXT>,
    created_at  TIMESTAMP,
    like_count  COUNTER,
    share_count COUNTER,
    PRIMARY KEY (post_id)
  );

  -- Denormalized table for user's own posts (used by pull model)
  CREATE TABLE user_posts (
    user_id    UUID,
    created_at TIMESTAMP,
    post_id    UUID,
    PRIMARY KEY (user_id, created_at)
  ) WITH CLUSTERING ORDER BY (created_at DESC);

Fan-Out Service

The most critical service. Consumes post.created events from the message queue and distributes posts to followers' feed caches.

Fan-Out Service (event-driven, async):
  │
  ├─→ Consume "post.created" event from Kafka
  │     { post_id: "12345", user_id: "alice_123", timestamp: 1714500000 }
  │
  ├─→ Check: is alice_123 a celebrity?
  │     │
  │     ├─→ YES (followers ≥ 10K):
  │     │     Store in celebrity_posts table. Done.
  │     │
  │     └─→ NO (followers < 10K):
  │           │
  │           ├─→ Query social graph: GET followers of alice_123
  │           │     → [bob, carol, dave, ... 500 users]
  │           │
  │           ├─→ Filter out inactive users (haven't logged in >30 days)
  │           │     → [bob, carol, dave, ... 350 active users]
  │           │
  │           └─→ For each active follower:
  │                 ZADD feed:{follower_id} {timestamp} {post_id}
  │                 ZREMRANGEBYRANK feed:{follower_id} 0 -501
  │                 (keep only latest 500 entries per feed)
  │
  └─→ Publish "fanout.complete" event (for monitoring)

Scaling strategy:
  - Kafka partitioned by user_id (ensures ordering per user)
  - Multiple consumer groups for parallel processing
  - Batch Redis writes using PIPELINE (100 writes per round trip)
  - Rate limit fan-out to prevent thundering herd

Feed Service

The read path. Assembles and returns a user's personalized feed.

GET /api/v1/feed?cursor=1714500000&limit=20

Feed Service flow:
  │
  ├─→ Step 1: Read pre-computed feed from cache
  │     ZREVRANGEBYSCORE feed:{user_id} {cursor} -inf LIMIT 0 100
  │     → Returns up to 100 post IDs from normal-user fan-out
  │
  ├─→ Step 2: Identify celebrities the user follows
  │     GET celebrity_follows:{user_id}
  │     → [taylorswift, barackobama, cristiano, ... 12 celebrities]
  │
  ├─→ Step 3: Fetch recent celebrity posts (fan-out on read)
  │     For each celebrity:
  │       ZREVRANGEBYSCORE user_posts:{celeb_id} {cursor} {cursor-24h} LIMIT 0 10
  │     → Returns ~60 celebrity post IDs
  │
  ├─→ Step 4: Merge + deduplicate
  │     Union of cached posts + celebrity posts
  │     Remove duplicates (celebrity might also be in cache during transition)
  │     → ~160 candidate posts
  │
  ├─→ Step 5: Hydrate post objects
  │     MGET post:{id1} post:{id2} ... (from Post Cache)
  │     Fallback to Posts DB for cache misses
  │     → Full post objects with content, media URLs, engagement counts
  │
  ├─→ Step 6: Ranking (call Ranking Service)
  │     Send 160 candidate posts + user context to Ranking Service
  │     → Returns ranked list with relevance scores
  │
  ├─→ Step 7: Apply business rules
  │     - Remove posts from muted/blocked users
  │     - Apply content policy filters
  │     - Inject sponsored posts at positions 3, 8, 15
  │     - Deduplicate near-identical content
  │
  └─→ Step 8: Return top 20 posts + next cursor
       {
         "posts": [...],
         "next_cursor": "1714498200",
         "has_more": true
       }

Ranking Service

This is where the "magic" happens — transforming a chronological list of posts into a personalized, engaging feed. Modern feed ranking is a multi-stage ML pipeline.

Ranking Pipeline (3 stages):

Stage 1: Candidate Generation (~1000 → ~300 posts)
  - Pre-computed feed cache (fan-out on write results)
  - Celebrity post retrieval (fan-out on read)
  - "Out-of-network" recommendations (posts liked by friends of friends)
  - Trending/viral content injection

Stage 2: Scoring (~300 → scored list)
  For each candidate post, predict:
    P(like)     = probability user will like the post
    P(comment)  = probability user will comment
    P(share)    = probability user will share
    P(click)    = probability user will click through
    P(hide)     = probability user will hide/report

  Final score = w₁·P(like) + w₂·P(comment) + w₃·P(share)
              + w₄·P(click) - w₅·P(hide) + w₆·recency_boost
              + w₇·relationship_score

Stage 3: Re-ranking (business rules)
  - Diversity: don't show 5 posts from same user in a row
  - Freshness: boost posts from last hour
  - Content type mixing: alternate text/image/video
  - Ad slot insertion at fixed positions
  - Anti-echo-chamber: inject some diverse viewpoints

Ranking Features

Notification Service

Handles push notifications, in-app badges, and real-time feed updates via WebSocket/SSE (Server-Sent Events).

Notification triggers:
  - "post.created" → notify close friends (configurable)
  - "post.liked"   → notify post author
  - "post.commented" → notify post author + other commenters
  - "post.shared"  → notify post author

Real-time feed update via WebSocket:
  1. Fan-out service completes writing to Bob's feed cache
  2. Publishes "feed.updated" event with user_id=bob
  3. WebSocket gateway routes event to Bob's open connection
  4. Client receives event → fetches new posts → prepends to feed
  
  This gives the "new posts available" banner effect:
  ┌──────────────────────────────────┐
  │  ↑  3 new posts — tap to see    │
  └──────────────────────────────────┘

Storage Design

Feed Cache: Redis Sorted Sets

The feed cache is the heart of the read path. Each user has a Redis sorted set where the member is a postId and the score is the post's creation timestamp.

Redis data structure per user:
  Key:    feed:{user_id}
  Type:   Sorted Set (ZSET)
  Member: post_id (8 bytes)
  Score:  Unix timestamp in milliseconds (8 bytes)

Example:
  ZADD feed:bob 1714500000000 "post_abc123"
  ZADD feed:bob 1714499800000 "post_def456"
  ZADD feed:bob 1714499600000 "post_ghi789"

Read latest 20 posts:
  ZREVRANGEBYSCORE feed:bob +inf -inf LIMIT 0 20
  → ["post_abc123", "post_def456", "post_ghi789", ...]

Cursor-based pagination (next page):
  ZREVRANGEBYSCORE feed:bob (1714499600000 -inf LIMIT 0 20
  (The "(" prefix makes it exclusive — don't re-return the last item)

Trim to keep only latest 500:
  ZREMRANGEBYRANK feed:bob 0 -501

Memory calculation:
  Per entry: ~40 bytes (16 member + 8 score + 16 overhead)
  Per user (500 entries): ~20 KB
  500M users: 10 TB
  With 3x replication: 30 TB across Redis cluster

Redis Cluster Topology

Redis Cluster for Feed Cache:
  ┌──────────────────────────────────────────────────────────┐
  │  Shard 0 (users a-d)    │  Shard 1 (users e-h)          │
  │  ┌─────────┐            │  ┌─────────┐                  │
  │  │ Primary │──replicate──│  │ Primary │──replicate──     │
  │  └─────────┘            │  └─────────┘                  │
  │  ┌─────────┐            │  ┌─────────┐                  │
  │  │ Replica │            │  │ Replica │                  │
  │  └─────────┘            │  └─────────┘                  │
  │  ┌─────────┐            │  ┌─────────┐                  │
  │  │ Replica │            │  │ Replica │                  │
  │  └─────────┘            │  └─────────┘                  │
  ├──────────────────────────┼──────────────────────────────-┤
  │  Shard 2 (users i-p)    │  Shard 3 (users q-z)          │
  │  ...                     │  ...                          │
  └──────────────────────────────────────────────────────────┘

  Total: ~100 shards × 3 replicas = 300 Redis instances
  Reads go to replicas (load distribution)
  Writes go to primaries only
  Hash slot: CRC16(user_id) mod 16384

Post Object Cache

Post object cache (separate Redis cluster):
  Key:   post:{post_id}
  Value: Serialized post object (protobuf/JSON)
  TTL:   7 days (posts older than 7 days evicted, fallback to DB)

  Structure:
  {
    "post_id": "abc123",
    "user_id": "alice_123",
    "content": "Hello world!",
    "media_urls": ["https://cdn.example.com/img/456.jpg"],
    "created_at": 1714500000,
    "like_count": 1542,
    "comment_count": 87,
    "share_count": 23,
    "author": {
      "user_id": "alice_123",
      "display_name": "Alice",
      "avatar_url": "https://cdn.example.com/avatars/alice.jpg",
      "is_verified": false
    }
  }

  Cache-aside pattern:
    1. Feed Service requests post:abc123
    2. Cache HIT → return immediately
    3. Cache MISS → query Posts DB → write to cache → return

  Engagement count updates:
    Use Redis HINCRBY for atomic counter updates
    Sync back to Posts DB asynchronously every 30 seconds

Social Graph Storage

The social graph (who follows whom) is queried on every write (fan-out) and every read (celebrity post fetch). It needs to support two critical queries: "get all followers of X" and "get all users X follows".

Option 1: Adjacency list in relational DB
  CREATE TABLE follows (
    follower_id  BIGINT,
    followee_id  BIGINT,
    created_at   TIMESTAMP,
    PRIMARY KEY (follower_id, followee_id)
  );
  CREATE INDEX idx_followee ON follows(followee_id, follower_id);
  
  -- "Who does Bob follow?" (for fan-out on read)
  SELECT followee_id FROM follows WHERE follower_id = 'bob';
  
  -- "Who follows Alice?" (for fan-out on write)
  SELECT follower_id FROM follows WHERE followee_id = 'alice';

Option 2: Graph database (Neo4j/Amazon Neptune)
  (Bob)-[:FOLLOWS]->(Alice)
  
  MATCH (a:User {id: 'alice'})-[:FOLLOWED_BY]->(follower)
  RETURN follower.id

Option 3: Dedicated graph service (Facebook TAO)
  Optimized for high-throughput edge lookups
  Distributed across regions
  Caches hot edges in memory

  At Facebook scale, the social graph is:
    ~2 billion nodes (users)
    ~400 billion edges (follow/friend relationships)
    Sharded by user_id, replicated globally

Media Storage & CDN

Media upload flow:
  1. Client uploads image/video to Upload Service
  2. Upload Service stores original in object storage (S3)
  3. Media Processing Service:
     - Image: generate thumbnails (150px, 300px, 600px, 1200px)
     - Video: transcode to multiple bitrates (240p, 480p, 720p, 1080p)
     - Extract metadata (dimensions, duration, GPS if allowed)
  4. Processed media pushed to CDN edge nodes
  5. Post stores CDN URL, not the raw storage URL

CDN cache strategy:
  - Popular media (viral posts): cached at all edge nodes
  - Recent media: cached at regional edge nodes
  - Old media: origin-pull on demand
  - CDN TTL: 30 days for images, 7 days for video thumbnails

Pagination: Cursor-Based, Not Offset

Pagination is a critical design detail that many candidates overlook. The feed is a live, constantly-changing list. Offset-based pagination breaks catastrophically in this scenario.

The Offset Problem

Offset-based (BAD for feeds):
  Page 1: GET /feed?offset=0&limit=20   → posts [0..19]
  Page 2: GET /feed?offset=20&limit=20  → posts [20..39]
  
  BUT between page 1 and page 2 requests, 5 new posts are inserted!
  
  Page 1: returns posts [A, B, C, D, E, ... T]     (20 posts)
  — 5 new posts inserted: [V, W, X, Y, Z] —
  Page 2: offset=20 now returns posts [P, Q, R, S, T, ...]
  
  Posts P, Q, R, S, T appear on BOTH pages! (duplicates)
  And posts [V, W, X, Y, Z] are never seen! (missed content)

Cursor-Based Pagination (The Solution)

Cursor-based (CORRECT for feeds):
  Page 1: GET /feed?limit=20
    → returns posts [A, B, C, ... T]
    → response includes: next_cursor = "1714499600000" (timestamp of last post T)
  
  Page 2: GET /feed?cursor=1714499600000&limit=20
    → "give me 20 posts OLDER than timestamp 1714499600000"
    → returns posts [U, V, W, X, ... ] (no duplicates, no gaps)

  Even if 100 new posts arrive between requests,
  the cursor anchors to a fixed point in time.

Redis implementation:
  -- Page 1:
  ZREVRANGEBYSCORE feed:bob +inf -inf LIMIT 0 20 WITHSCORES
  → Last entry score = 1714499600000

  -- Page 2:
  ZREVRANGEBYSCORE feed:bob (1714499600000 -inf LIMIT 0 20 WITHSCORES
  (Exclusive lower bound with parenthesis prefix)

Cursor Edge Cases

Problem: Two posts with the exact same timestamp
  Solution: Use composite cursor = "timestamp:post_id"
  
  Page 1 last item: timestamp=1714500000, post_id="abc"
  Cursor: "1714500000:abc"
  
  Page 2 query: all posts where
    (timestamp < 1714500000) OR 
    (timestamp = 1714500000 AND post_id < "abc")

Problem: Feed cache expires between page loads
  Solution: If cache miss, fall back to Posts DB query
  The cursor still works because it's a timestamp, not a cache-specific offset

Problem: User scrolls infinitely (reaches end of cache)
  Solution: Cache stores 500 entries. After that, switch to DB-backed pagination
  ZCARD feed:bob → if near 500, next page falls through to DB

The Celebrity Problem & Solutions

The celebrity problem is the single biggest challenge in news feed design. It deserves its own deep section because understanding it thoroughly demonstrates mastery of the trade-offs.

The Math of Celebrity Fan-Out

Scenario: Taylor Swift (100M followers) posts a photo

Fan-out on write:
  100,000,000 Redis ZADD operations
  At 100K writes/sec per Redis shard: 1,000 seconds = 16.7 minutes
  Even with 100 fan-out workers in parallel: still 10 seconds
  
  During those 10 seconds:
  - 100M write operations consume massive Redis bandwidth
  - Other users' fan-out operations queue up (head-of-line blocking)
  - Redis cluster CPU and memory spike
  - Network bandwidth: 100M × 40 bytes ≈ 4 GB of data
  
  And Taylor posts 3-5 times per day.
  And there are thousands of celebrities.
  The write amplification is catastrophic.

Solutions

100M followers of Taylor Swift:
  - 30M active in last 24 hours
  - 50M active in last 7 days
  - 20M inactive (haven't opened app in 30+ days)

Fan-out only to 30M active users:
  70% reduction in write volume!

When an inactive user returns:
  1. Their cache is empty/stale
  2. Fall back to fan-out on read for their first feed load
  3. Resume fan-out on write going forward

Priority 1 (immediate): Close friends, frequent interactors
  → Fan-out within 1 second

Priority 2 (fast): Active users who engage with this author
  → Fan-out within 10 seconds

Priority 3 (normal): Active users who follow but rarely engage
  → Fan-out within 1 minute

Priority 4 (lazy): Inactive users
  → Skip fan-out entirely, pull on demand

Tier 1 — L1 Cache (per data center):
  Cache celebrity posts in a shared, regional cache
  Key: celebrity_posts:{user_id}:{date}
  One copy per data center, not per follower!
  
Tier 2 — User Feed Cache (per user):
  Normal user posts cached per-follower as before

At read time:
  Feed = Merge(user_feed_cache, regional_celebrity_cache)

Feed Ranking: Deep Dive

Chronological vs. Ranked Feeds

Early social networks showed posts in pure reverse chronological order. This was simple and transparent, but it had a fundamental problem: as users followed more accounts, the signal-to-noise ratio plummeted. Users missed important posts from close friends because they were buried under a flood of low-quality content.

The Ranking Formula

Modern feed ranking score for a candidate post:

score(user, post) =
    α · P(engage | user, post)     // engagement prediction
  + β · affinity(user, author)      // relationship strength
  + γ · recency(post)               // time decay
  + δ · content_quality(post)       // content quality signal
  + ε · diversity_bonus(post, feed) // diversity injection
  - ζ · P(negative | user, post)    // negative signal penalty

Where:
  P(engage) = ML model predicting click/like/comment/share
  affinity  = weighted sum of past interactions:
              affinity(u, a) = Σ wᵢ · interaction_countᵢ · time_decayᵢ
              interactions: likes (w=1), comments (w=3), shares (w=5),
                           DMs (w=7), profile visits (w=2), tags (w=4)
  
  recency   = 1 / (1 + (now - post.created_at) / half_life)
              half_life = 6 hours (post loses 50% of recency score every 6 hours)
  
  content_quality = f(image_quality_score, text_sentiment, 
                      spelling_score, engagement_velocity)
  
  diversity_bonus = bonus if post type or author differs from last N shown
  
  P(negative) = ML model predicting hide/report/unfollow

Ranking Infrastructure

Ranking service architecture:

  ┌─────────────┐     ┌──────────────────┐     ┌────────────────┐
  │ Feed Service │────→│ Feature Store     │────→│ ML Model Server│
  │ (candidates) │     │ (user/post/edge   │     │ (TensorFlow    │
  │              │     │  features, cached) │     │  Serving)      │
  └─────────────┘     └──────────────────┘     └────────────────┘
                                                        │
                                                        ▼
                                                ┌────────────────┐
                                                │ Scored + Ranked │
                                                │ Post List       │
                                                └────────────────┘

Feature Store:
  - Pre-computed features updated hourly
  - User features: engagement_rate, avg_session_time, content_preferences
  - Author features: credibility_score, avg_post_engagement, posting_frequency
  - Edge features: interaction_count, last_interaction_time, affinity_score
  
  Stored in: Redis (hot features) + Cassandra (full history)
  Latency requirement: feature lookup <5ms

Model Serving:
  - TensorFlow Serving / ONNX Runtime
  - Batch inference: score 200 posts in a single GPU call (~10ms)
  - Model size: ~500MB (distilled from larger training model)
  - A/B testing: run multiple model versions simultaneously
  - Canary deployment: roll out new model to 1% → 5% → 25% → 100%

Cache Design: Deep Dive

Multi-Layer Cache Architecture

Layer 1: Client-side cache
  - App caches last loaded feed in local storage
  - On app open: show cached feed immediately, fetch fresh in background
  - Stale-while-revalidate pattern
  - TTL: 5 minutes

Layer 2: CDN / Edge cache
  - Not used for personalized feeds (each user's feed is different)
  - Used for: media (images, videos), user profiles, trending posts

Layer 3: Application-level cache (Redis)
  ┌─────────────────────────────────────────────────────┐
  │  Feed Cache (ZSET)           Post Cache (STRING)     │
  │  feed:{user_id}              post:{post_id}          │
  │  ┌─────────────────┐        ┌─────────────────┐     │
  │  │ postId₁ : score₁│        │ serialized post  │     │
  │  │ postId₂ : score₂│        │ object (protobuf)│     │
  │  │ ...              │        │                  │     │
  │  └─────────────────┘        └─────────────────┘     │
  │                                                      │
  │  User Cache (HASH)           Social Graph Cache       │
  │  user:{user_id}              followers:{user_id}     │
  │  ┌─────────────────┐        following:{user_id}      │
  │  │ name, avatar,    │        celebrity_follows:       │
  │  │ follower_count,  │         {user_id}              │
  │  │ is_celebrity     │                                │
  │  └─────────────────┘                                 │
  └─────────────────────────────────────────────────────┘

Layer 4: Database
  - Posts DB (Cassandra): source of truth for post content
  - Social Graph DB (MySQL/Neo4j): source of truth for relationships
  - User DB (MySQL): source of truth for user profiles

Cache Invalidation Strategies

Feed cache invalidation:
  - Write-through: fan-out service writes to cache AND DB simultaneously
  - No explicit invalidation needed — new entries push old ones out (ZREMRANGEBYRANK)
  - TTL: 7 days for inactive users (save memory)
  - On unfollow: remove unfollowed user's posts from cache
    → ZRANGEBYSCORE to find their posts, ZREM to remove them
    → Or just let them age out naturally (simpler, slightly stale)

Post cache invalidation:
  - Write-through on post update (edit, delete)
  - Event-driven: "post.updated" → invalidate post:{post_id}
  - Engagement counters: updated in-place via HINCRBY (no invalidation)
  - Deleted posts: set a "deleted" flag, TTL 24 hours, then remove
    → All feed reads check the deleted flag and skip

Social graph cache invalidation:
  - Follow/unfollow events trigger cache invalidation
  - followers:{user_id} invalidated on any follow/unfollow
  - Warm cache on invalidation (write-through, not lazy)
  - Celebrity status changes: rare, handled by background job

Cache Warming & Cold Start

Cold start scenarios:
  1. New user signs up → no feed cache exists
     Solution: Generate initial feed from recommended accounts + trending posts
     
  2. User returns after long absence → cache expired
     Solution: Rebuild cache via fan-out on read for first request
     Background job: re-fan-out recent posts to warm the cache
     
  3. Redis node fails → portion of feed caches lost
     Solution: Replica promotion (Redis Sentinel/Cluster)
     If all replicas lost: graceful degradation to fan-out on read
     Background rebuild from Posts DB within minutes

Cache warming strategy for new deployments:
  1. Deploy new Redis cluster with warm replicas
  2. Run "cache warmer" service that pre-loads feeds for daily active users
  3. Active users (logged in last 24h): warm immediately
  4. Regular users (logged in last 7d): warm within 1 hour
  5. Inactive users: warm on demand (lazy)

Putting It All Together

Complete News Feed Architecture:

  ┌─────────────────────────────────────────────────────────────────────┐
  │                           CLIENTS                                   │
  │   Mobile App / Web Browser / API Consumers                         │
  └──────────────────┬─────────────────────┬───────────────────────────┘
                     │ POST /posts          │ GET /feed
                     ▼                      ▼
  ┌──────────────────────────────────────────────────────────────────┐
  │                    API GATEWAY / LOAD BALANCER                    │
  │        (rate limiting, auth, routing, SSL termination)            │
  └──────┬───────────────────────────────────┬──────────────────────-┘
         │                                   │
    ═══ WRITE PATH ═══                  ═══ READ PATH ═══
         │                                   │
         ▼                                   ▼
  ┌──────────────┐                    ┌──────────────┐
  │ Post Service │                    │ Feed Service  │
  │              │                    │               │
  │ 1. Validate  │                    │ 1. Read cache │
  │ 2. Store post│                    │ 2. Fetch celeb│
  │ 3. Upload    │                    │ 3. Merge      │
  │    media     │                    │ 4. Hydrate    │
  │ 4. Publish   │                    │ 5. Rank       │
  │    event     │                    │ 6. Return     │
  └──────┬───────┘                    └──┬───────┬───┘
         │                               │       │
         ▼                               ▼       ▼
  ┌──────────────┐              ┌────────────┐ ┌──────────────┐
  │ Message Queue│              │ Feed Cache  │ │ Ranking      │
  │ (Kafka)      │              │ (Redis ZSET)│ │ Service      │
  └──────┬───────┘              └─────────---┘ │              │
         │                                      │ ML Model    │
         ▼                                      │ Feature Store│
  ┌──────────────┐                              └──────────────┘
  │ Fan-Out      │
  │ Service      │─────→ Feed Cache (Redis)
  │              │
  │ Check celeb? │─────→ Celebrity Posts Table
  │ Get followers│─────→ Social Graph
  └──────────────┘
         │
         ▼
  ┌──────────────┐
  │ Notification │
  │ Service      │─────→ Push Notifications
  │              │─────→ WebSocket Gateway
  └──────────────┘

  ═══ STORAGE LAYER ═══
  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐
  │ Posts DB   │  │ Social     │  │ User DB    │  │ Object     │
  │ (Cassandra)│  │ Graph DB   │  │ (MySQL)    │  │ Storage    │
  │            │  │ (Neo4j/    │  │            │  │ (S3)       │
  │            │  │  MySQL)    │  │            │  │     + CDN  │
  └────────────┘  └────────────┘  └────────────┘  └────────────┘

Interview Tips & Common Follow-ups

Common Follow-Up Questions

Key Takeaways

• Fan-out strategy is THE decision. Fan-out on write for normal users (fast reads), fan-out on read for celebrities (no write amplification), hybrid for the real world.
• The celebrity problem is what makes this design interesting. The threshold-based hybrid approach with selective fan-out is the industry-standard solution.
• Redis sorted sets are the perfect data structure for feed caches: O(log N) writes, O(log N + K) reads, automatic ordering, native cursor support.
• Cursor-based pagination is mandatory for live feeds. Offset-based pagination causes duplicates and missed content.
• Feed ranking has evolved from chronological → heuristic scores → deep ML models. Modern ranking predicts engagement probability and uses multi-task learning.
• Cache design is multi-layered: client cache → feed cache (Redis ZSET) → post cache (Redis STRING) → database. Each layer has its own invalidation strategy.
• The system is heavily read-dominant (20:1 read:write ratio), which favors pre-computation and caching over on-the-fly computation.
• Eventual consistency is perfectly acceptable — a post appearing in a feed 5 seconds late is fine. Prioritize availability and partition tolerance over strict consistency.