Twitter/X

Functional Requirements

• Tweet publishing — a short text post, media, and the basic interaction model around it.
• Home timeline — a personalized feed built from accounts the user follows.
• User timeline — the complete tweet history of one author in reverse chronological order.
• Follow and unfollow — updates to the social graph.
• Likes and retweets — engagement actions and counters.
• Search — search across tweets, accounts, and keywords.
• Trending topics — fast detection of emerging discussion spikes.
• Notifications — mentions, replies, likes, and other activity around the account.

Non-functional Requirements

The user-facing priorities are low timeline latency, a predictable freshness window, and enough resilience to survive partial failure without large visible outages. Perfect consistency is not required here: a small delay before a tweet appears for all followers is usually acceptable.

• Scale: roughly 500M registered users and 200M DAU.
• Write volume: around 500M new tweets per day.
• Traffic shape: reads outnumber writes by about 1000:1.
• Home timeline latency: ideally under 200 ms.
• Freshness window: 5-10 seconds of propagation delay can be acceptable.
• Availability: a 99.99% target is reasonable, but the important part is graceful degradation instead of all-or-nothing failure.
• Celebrity problem: publish flow must survive accounts with tens or hundreds of millions of followers.

Core Problem: Feed Generation

The main architectural problem in Twitter/X is how to assemble the home timeline fast enough and cheaply enough when almost every tweet may need to reach many followers. At a high level, you either push tweets into ready-made timelines or you assemble the timeline on read from the latest tweets of the accounts a user follows.

Hybrid Approach

In practice, Twitter/X uses a hybrid strategy: ordinary accounts use fanout-on-write, while very large accounts lean on fanout-on-read. That keeps the system from exploding on write amplification when one tweet would otherwise need to be copied into a massive number of personal timelines.

Home Timeline Cache Architecture

The timeline cache exists to make the hottest read path cheap. In most designs, publish is acknowledged quickly while the actual fan-out work moves into background processing so the user is not blocked on all downstream writes.

# Each user has a dedicated home timeline in Redis
# Key: timeline:{user_id}
# Value: Sorted Set (score = timestamp, member = tweet_id)

ZADD timeline:12345 1705234567 "tweet_abc123"
ZADD timeline:12345 1705234890 "tweet_def456"

# Read the latest 100 tweets
ZREVRANGE timeline:12345 0 99

# Approximate storage estimate:
# 200M users × 800 tweets × 8 bytes = 1.28 TB
# With replication ×3 = about 4 TB Redis cluster

Trending Topics

Trend detection is a stream-processing problem. The system cares about bursts of discussion rather than absolute tweet count, so it needs approximate counting, time windows, and spam filtering that all move with the stream.

Trend Scoring Formula

# Simplified Twitter trend score

def calculate_trend_score(topic, current_window):
    # Current velocity: tweets per minute over the last 5 minutes
    current_count = count_in_window(topic, minutes=5)

    # Baseline: average number of tweets per 5-minute bucket over 7 days
    baseline_count = get_baseline(topic, days=7)

    # How much faster the topic grows compared with normal
    if baseline_count > 0:
        velocity_ratio = current_count / baseline_count
    else:
        velocity_ratio = current_count * 10  # new topic bonus

    # Recency weight: exponential decay
    recency_weight = sum(
        tweet.weight * exp(-lambda * (now - tweet.timestamp))
        for tweet in current_window.tweets
    )

    # Engagement boost
    engagement_score = (likes + retweets * 2 + replies * 3) / total_tweets

    # Final score
    score = velocity_ratio * recency_weight * (1 + log(engagement_score))

    # Spam / bot filtering
    if unique_users_ratio < 0.3:  # too few unique users
        score *= 0.1  # heavy penalty

    return score

Timeline Ranking

Modern Twitter/X does not rely purely on chronology. It uses ranking to estimate the probability of engagement and move more relevant tweets higher in the home timeline.

Search Architecture

Search lives in a separate path: new tweets first land in a realtime index and later compact into on-disk segments. On the query path, the system parses the request, searches shards in parallel, and merges the results afterward.

Data Model

The primary store keeps normalized entities, while the hot read path relies on denormalized Redis structures and auxiliary indexes to stay fast.

# Core tables

tweets {
    tweet_id: UUID (Snowflake ID)
    user_id: UUID
    content: VARCHAR(280)
    media_urls: JSON
    created_at: TIMESTAMP
    reply_to_tweet_id: UUID (nullable)
    retweet_of_id: UUID (nullable)
    quote_tweet_id: UUID (nullable)
    like_count: INT
    retweet_count: INT
    reply_count: INT
}

users {
    user_id: UUID
    username: VARCHAR(15)
    display_name: VARCHAR(50)
    bio: VARCHAR(160)
    follower_count: INT
    following_count: INT
    is_verified: BOOLEAN
    is_celebrity: BOOLEAN  # follower_count > 10K
    created_at: TIMESTAMP
}

follows {
    follower_id: UUID
    followee_id: UUID
    created_at: TIMESTAMP
    PRIMARY KEY (follower_id, followee_id)
}

# Denormalized structures for reads
user_timelines (Redis Sorted Set)
    Key: timeline:{user_id}
    Score: tweet_timestamp
    Member: tweet_id

# Tweets from very large accounts
celebrity_tweets (Redis Sorted Set)
    Key: celebrity:{user_id}
    Score: tweet_timestamp
    Member: tweet_id

Snowflake ID Generation

Twitter created Snowflake ID to generate unique, roughly time-sortable identifiers without a centralized coordinator.

# Snowflake ID structure (64-bit)
┌────────────────────────────────────────────────────────────────┐
│ 1 bit │   41 bits timestamp   │ 10 bits │    12 bits          │
│unused │   (milliseconds)      │machine  │   sequence          │
│       │   since epoch         │   ID    │   number            │
└────────────────────────────────────────────────────────────────┘

# Properties:
# - Roughly sortable by time
# - No central coordination required
# - 4096 IDs per millisecond per machine
# - Enough range for decades before overflow

# Example ID: 1605978261000000000
# Timestamp: 2020-11-21 12:31:01 UTC
# Machine: 42
# Sequence: 0

def generate_snowflake_id(machine_id, last_timestamp, sequence):
    timestamp = current_time_ms() - TWITTER_EPOCH

    if timestamp == last_timestamp:
        sequence = (sequence + 1) & 0xFFF  # 12 bits
        if sequence == 0:
            timestamp = wait_next_millis(last_timestamp)
    else:
        sequence = 0

    id = (timestamp << 22) | (machine_id << 12) | sequence
    return id, timestamp, sequence

High-Level Architecture

At the top level, the design separates publishing, home-timeline reads, search, and trends into different paths. The important part is not to force them through one universal chain, but to give each path its own cost profile, caches, and failure behavior.

Interview Tips

Additional Resources