Skip to main content

Design a Social Network

Social networks like Facebook, LinkedIn, Threads, Mastodon, and Weibo connect billions of people through relationships, content sharing, and real-time interactions. Building such a platform involves modeling a massive, constantly evolving graph of users and content, serving personalized feeds at low latency, and handling enormous read-to-write ratios. This article presents a comprehensive system design for a scalable, real-world social networking platform.

Step 1: Requirements Clarification​

Functional Requirements​

  • User registration & profile management – Create accounts, update personal info, upload avatars.
  • Friend / Follow relationships – Establish bidirectional (friends) or unidirectional (follow) connections.
  • Publish posts – Text, images, videos.
  • Engagement actions – Likes, comments, shares/reposts.
  • News Feed – A personalized, ranked stream of content from friends and followed entities.
  • User Timeline – A chronological view of a specific user's posts.
  • User search – Find people by name or other attributes.
  • Notifications – Real-time alerts for relevant activities.
  • Privacy & blocking – Control who sees content, block unwanted interactions.

Non-Functional Requirements​

  • High availability – 99.95%+ uptime. A down social network loses engagement rapidly.
  • Low latency – Feed loading < 200ms, post publishing < 100ms.
  • Massive read throughput – Feeds are read orders of magnitude more than they are written.
  • Horizontal scalability – Must scale out to accommodate growth in users, posts, and connections.
  • Eventual consistency – Tolerate slight delays in feed updates for better performance.
  • Fault tolerance – No single point of failure; degrade gracefully.
  • Durability – Posts and media must not be lost.
  • Security & privacy – Authentication, authorization, data encryption, and abuse prevention.

Step 2: Capacity Estimation​

Assume a large-scale network:

  • Registered users: 2 billion
  • Daily active users (DAU): 500 million
  • Peak concurrent users: 50 million
  • Posts per second: 10,000 average, peaks at 50,000
  • Likes per second: 100,000 average, peaks at 500,000
  • Feed reads per second: ~5 million (10 reads per active user session)
  • Media: 100 million new images daily, 10 million videos.
  • Storage: ~100 TB of new textual data per day, several PB of media.

The read-to-write ratio is extreme: feeds can be read millions of times per second while only thousands of new posts appear. This drives the core architectural decisions.

Step 3: API Design​

REST APIs for client-server communication, with WebSocket for real-time notifications.

  • Create user POST /api/v1/users
  • Follow user POST /api/v1/users/{id}/follow
  • Unfollow user DELETE /api/v1/users/{id}/follow
  • Publish post POST /api/v1/posts (with media uploads handled separately)
  • Fetch News Feed GET /api/v1/feed?page_token={token}&limit=20
  • Fetch Timeline GET /api/v1/users/{id}/timeline?page_token={token}
  • Like post POST /api/v1/posts/{id}/like
  • Comment on post POST /api/v1/posts/{id}/comments
  • Share post POST /api/v1/posts/{id}/share
  • Search users GET /api/v1/search/users?q={query}

Idempotency is achieved via idempotency_key headers to prevent duplicate actions like double-like or duplicate posts.

Step 4: High-Level Architecture​

The system employs a microservices-based architecture with dedicated services for users, graphs, feeds, media, and notifications.

  • API Gateway routes requests and handles authentication.
  • User Service manages profiles.
  • Social Graph Service stores and queries friend/follow relationships.
  • Post Service handles post creation and persistence.
  • Feed Service generates and serves the News Feed.
  • Timeline Service serves user-specific timelines.
  • Media Service manages upload, transcoding, and storage.
  • Notification Service pushes real-time events.
  • Search Service indexes users and posts.
  • Message Queue decouples post publishing from feed generation and notifications.

Step 5: Social Graph Design​

The social graph represents relationships: bidirectional friendships (Facebook) or unidirectional follows (Twitter, Instagram).

Storage Options​

ApproachProsCons
Relational DB (SQL) with indexed adjacency listSimple, ACID, familiarDifficult to scale writes for massive graphs; expensive joins for "friends of friends" queries.
Graph Database (e.g., Neo4j, JanusGraph)Optimized for traversals, native graph modelOperational complexity; scaling horizontally can be challenging.
Key-Value Store (e.g., Redis, DynamoDB) with custom shardingHigh performance, horizontal scaleApplication must manage traversal logic; consistency trade-offs.

At immense scale (billions of edges), many companies use a sharded key-value model. A user's relationship list (friends or followers) is stored as a set in a row keyed by user_id. For mutual friends, a separate table or indexed list can be maintained.

Step 6: Feed Generation​

The News Feed is a personalized list of posts from accounts a user follows. Generating it efficiently is the heart of the system.

Strategies​

  • Fan-out on Write (Push)

    • When a user publishes a post, it is pushed to the feed caches of all their followers.
    • Pros: Feed reads are extremely fast (pre-computed), low read latency.
    • Cons: Write amplification for users with millions of followers (celebrities). Wastes storage for inactive followers.

  • Fan-out on Read (Pull)

    • At read time, query the recent posts of all followees, merge, and rank.
    • Pros: No write amplification, storage efficient.
    • Cons: High read latency, especially when following many users; requires high query capacity.

  • Hybrid Approach

    • For regular users: fan-out on write to the feed caches of their followers.
    • For celebrity users (hundreds of thousands or millions of followers): do not fan-out on write. Instead, during feed read, merge the pre-built feed from regular followees with recent posts pulled from a separate celebrity post cache.
    • This balances latency and resource usage.

Step 7: Content Publishing Pipeline​

Publishing a post triggers several asynchronous processes.

  1. Client uploads media (image/video) to the Media Service, which returns a media ID.
  2. Client sends the post request (text + media IDs) to the Post Service.
  3. Post Service persists the post to a scalable database (sharded by user_id), updates the user's Timeline, and publishes a PostCreated event to a Message Queue (e.g., Kafka).
  4. Feed Worker consumes the event, determines the publisher's follower list, and performs fan-out to the followers' feed caches (excluding celebrities).
  5. Notification Worker generates notifications for tagged users or other triggers.
  6. Search Indexer asynchronously updates the search index.
  7. Media Processing completes asynchronously (transcoding, thumbnail generation).

Step 8: Media Storage​

Photos and videos dominate storage and bandwidth.

  • Upload is handled directly to Object Storage (Amazon S3 / MinIO) via pre-signed URLs or a media service.
  • Processing: Images are resized and optimized into multiple sizes. Videos are transcoded into adaptive bitrate formats (HLS/DASH).
  • Delivery: All media is served via a CDN to minimize latency and offload origin servers.
  • Caching: CDN caches popular media; origin storage stores only the master copies.
  • Database stores only media metadata (URL, dimensions, format), never binary data.

Step 9: Timeline and Feed Storage​

Timeline Storage​

A user's timeline is a chronological list of their own posts, stored in a scalable wide-column database (e.g., Cassandra) with user_id as partition key and post_timestamp as clustering column.

Feed Storage​

The pre-computed feed (for fan-out on write) is stored in a distributed cache (e.g., Redis) as a sorted set or list, keyed by user_id. The value is a list of post_ids, possibly with scores for ranking. This in-memory feed serves reads with sub-millisecond latency. For durability, a backup is maintained in a time-partitioned table.

Pagination​

Feeds are paginated using cursor-based pagination (e.g., last_post_id or timestamp) to support infinite scrolling without duplication.

Hot Content​

Popular posts are cached at multiple levels (application cache, Redis, CDN) and may also be served from a regional read-through cache.

Step 10: Search and Recommendation​

  • User Search: Indexed in Elasticsearch. Stores name, username, location. Supports prefix, fuzzy, and full-text search. Index updated via Change Data Capture (CDC) from the User DB.
  • Hashtag / Post Search: Also in Elasticsearch, fed by the post creation pipeline.
  • Recommendation Engine: Suggests friends (mutual connections, imported contacts) and content (collaborative filtering, trending). Runs as offline batch jobs using Spark or Flink, with pre-computed results stored in a fast KV store for online serving.

Search focuses on exact and near-match retrieval; recommendations uncover latent interests.

Step 11: Scalability Strategies​

  • Horizontal scaling – Stateless services behind load balancers.
  • Database sharding – User DB sharded by user_id; Posts DB sharded by user_id; Graph DB sharded by user_id.
  • Feed Cache – Massive Redis clusters, sharded by user_id, with replication for high availability.
  • CDN – Global edge caching for static assets and popular media.
  • Read replicas – For relational databases handling read-heavy user/profile queries.
  • Event-driven architecture – Kafka decouples writes from feed generation, notifications, and analytics.
  • Geo-distributed deployment – Multi-region active-active for low latency worldwide, with strong consistency requirements relaxed for feeds (eventual consistency).

Step 12: Reliability and Failure Handling​

  • Retry strategies – Message queue with dead-letter queues for failed feed fan-outs or notification deliveries.
  • Queue buffering – Kafka buffers events during traffic spikes, protecting downstream consumers.
  • Cache failures – On Redis outage, feed reads can fall back to a slower reconstruction from the database (pull-based).
  • Feed rebuilding – If a feed cache is corrupted, it can be rebuilt by replaying recent PostCreated events or a dedicated batch job.
  • Service degradation – If the feed ranking service fails, serve a chronological feed. If notifications fail, users can still interact; notifications are delayed.
  • Regional failover – Traffic is rerouted to healthy regions, with DNS-based or BGP anycast failover.

Step 13: Security and Privacy​

  • Authentication – OAuth 2.0 / OIDC with strong password policies and MFA.
  • Authorization – Privacy settings (public, friends-only, custom lists) enforced at the API and data access layer.
  • Rate limiting – Per-user and per-IP to prevent abuse and spam.
  • Content moderation – Automated classifiers (image/text) and human review queues for reported content.
  • Spam detection – Machine learning models and reputation scoring for accounts and IPs.
  • Audit logging – Immutable logs for all administrative access and sensitive actions.
  • Encryption – TLS everywhere; data at rest encrypted (AES-256); PII encrypted at application level.

Real-World Example: Facebook-like Platform​

Consider a user "Alice" publishing a post, which then appears on "Bob's" feed.

  1. Alice posts. The system persists it and asynchronously fans it out to her followers' feed caches.
  2. Bob, a follower, requests his feed. The feed service retrieves the pre-built list of post IDs from his cache and then fetches post details to render.
  3. Because the heavy work was done at write time, Bob's read is extremely fast.

Trade-offs​

  • Fan-out on Write vs Read: Write fan-out makes reads very fast but amplifies storage and I/O. Read fan-out simplifies writes but increases read latency. The hybrid approach balances both.
  • SQL vs Graph Database: SQL is operationally simpler but struggles with deep graph traversals. Graph DBs excel at traversals but are harder to scale. Sharded KV stores offer a middle ground.
  • Cache freshness vs consistency: Pre-built feeds are eventually consistent; a user's feed might not include a very recent post for a few seconds. This is acceptable for social networks.
  • Feed ranking vs latency: Complex ML ranking improves relevance but adds latency. Ranking is often pre-computed or done with lightweight models at read time.
  • Simplicity vs personalization: Adding more personalization features (e.g., feed based on affinity) increases system complexity. Start simple, iterate.

Common Mistakes​

  • Generating feeds synchronously on every read – leads to catastrophic latency.
  • Ignoring celebrity users – Not having a separate strategy for users with millions of followers causes massive write amplification and hot shards.
  • No feed caching – Directly querying the post database for feed generation cannot meet latency requirements.
  • Using relational joins for massive relationship graphs – Doesn't scale; pre-materialize friend lists in a fast cache or KV store.
  • Storing media in the relational database – Terrible for performance and cost; always use object storage and CDN.
  • Ignoring recommendations – Feeds become stale without content discovery; a recommendation service is essential.
  • Poor pagination design – Using offset-based pagination leads to duplication and inconsistency; always use cursor-based.

Interview Perspective​

System design interviews often test your ability to handle data relationships and feed systems. Expect questions like:

  • How do you design Facebook?
  • How do you generate the News Feed?
  • What is fan-out on write? What is fan-out on read?
  • How do you store friendships?
  • How do you scale feed generation for millions of users?
  • How do you handle celebrity users?

Demonstrate understanding of graph modeling, caching, asynchronous processing, and the fundamental read-vs-write trade-off in feeds.

Summary​

A scalable social network combines a highly distributed social graph service, an efficient feed generation pipeline (often hybrid fan-out), massive caching infrastructure, and robust media delivery. It handles billions of interactions daily by decoupling writes from reads using event queues, pre-computing feeds for fast access, and tiering celebrity workloads separately. Privacy and security are layered throughout. The architecture must gracefully handle component failures, ensuring the core experiences—posting and browsing feeds—remain responsive and reliable.

Further Reading​