Design a Timeline System
A Timeline (or Profile Timeline) displays a single user's published content in reverse chronological order. Unlike a News Feed, which aggregates content from many sources and applies complex ranking algorithms, a Timeline is straightforward: it shows what a particular user has posted over time. Platforms like Facebook Profile, Instagram Profile, LinkedIn Activity, X (Twitter) User Timeline, and Threads Profile all rely on a robust Timeline system to serve user-specific content efficiently. Designing a scalable Timeline involves careful choices around data modeling, pagination, media handling, and caching to ensure low-latency access even for users with thousands of posts.
Step 1: Requirements Clarification
Functional Requirements
- Publish posts – Users can create posts containing text, photos, videos, and links.
- Edit and delete posts – Authors can modify or remove their own content.
- View profile timeline – Anyone with permission can see a user's chronological list of posts.
- Pin posts – Users can pin a specific post to the top of their Timeline.
- Infinite scrolling – The Timeline loads more posts as the user scrolls down.
- Photos and videos – Rich media must be uploaded, processed, and displayed optimally.
- Privacy settings – Posts can be public, friends-only, or private, affecting visibility.
- Archived posts – Users can archive posts (soft delete) so they are hidden but not permanently removed.
- Scheduled posts (optional) – Support for publishing at a future date/time.
Non-Functional Requirements
- Low latency – Timeline pages should load in under 200ms for the initial view.
- High availability – 99.95%+ uptime; user profiles must always be accessible.
- Horizontal scalability – The system must serve millions of profile visits per second.
- High durability – Posts must not be lost once published; media should be stored reliably.
- Eventual consistency – Slight delays in seeing a newly published post on one's own Timeline are acceptable.
- Efficient storage – Store and retrieve post metadata and media cost-effectively.
- Fast pagination – Seamless infinite scrolling without duplication or missing posts.
- Fault tolerance – Failures in non-critical components should not make the Timeline inaccessible.
Step 2: Capacity Estimation
Assume a large social platform:
- Registered users: 2 billion
- Daily active users (DAU): 500 million
- Posts created per second: 10,000 average, 50,000 peak.
- Timeline requests per second: 200,000 average, 1 million peak (profile views).
- Average posts per user: 500; heavy users may have tens of thousands.
- Media: 100 million images, 10 million videos daily. Average image 200 KB, average video 10 MB.
- Storage: ~20 TB new post metadata daily, ~2 PB new media daily.
- Read/write ratio: 100:1 or higher; profiles are read much more than written.
Timelines are read-heavy and content is ordered by time, making them simpler than feeds because ranking is not required.
Step 3: API Design
REST APIs with cursor-based pagination for consistent retrieval.
-
Publish Post
POST /api/v1/posts
Body:{ text, media_ids[], visibility }
Returns:{ post_id, created_at } -
Get Timeline (initial load)
GET /api/v1/users/{user_id}/timeline?limit=20
Returns:{ posts: [PostObject...], next_cursor: "encoded_cursor" } -
Load More
GET /api/v1/users/{user_id}/timeline?cursor={next_cursor}&limit=20 -
Update Post
PUT /api/v1/posts/{post_id}(author only) -
Delete / Archive Post
DELETE /api/v1/posts/{post_id}(soft delete, setsis_archived=true) -
Upload Media
POST /api/v1/media(returnsmedia_id) -
Pin / Unpin Post
POST /api/v1/posts/{post_id}/pin
Idempotency is achieved by the client generating a unique idempotency_key for post creation to prevent duplicates on network retries.
Pagination uses an opaque cursor (based on timestamp + unique post ID) rather than OFFSET, ensuring consistent results even when new posts are added.
Step 4: High-Level Architecture
The system is composed of dedicated services that separate concerns and enable independent scaling.
- API Gateway handles authentication and routing.
- User Service manages profiles and privacy settings.
- Timeline Service orchestrates fetching and assembling a user's Timeline.
- Post Service manages post CRUD, validation, and persistence.
- Media Service handles upload, transcoding, and delivery.
- Object Storage (e.g., S3) stores raw media; a CDN accelerates global delivery.
- Timeline Cache (Redis) stores the list of post IDs for each user's Timeline for fast retrieval.
- Search Service indexes posts for full‑text search.
- Notification Service alerts followers of new posts.
Step 5: Timeline Data Model
The data model is centered around posts, which belong to a user and form a time‑ordered list.
Post Table (conceptual, in a sharded database)
| Column | Type | Description |
|---|---|---|
post_id | UUID / bigint | Unique identifier |
user_id | bigint | Author (partition key) |
text | text | Post content |
media_ids | list | References to media |
visibility | enum | public, friends, private |
is_pinned | boolean | Pinned to top |
is_archived | boolean | Soft deleted |
created_at | timestamp | Sorting key |
Timeline Index is a time‑ordered list of post IDs per user. For reads, we fetch this list and then hydrate posts from the Post Table or cache.
SQL vs NoSQL
| Aspect | SQL (e.g., PostgreSQL) | NoSQL (e.g., Cassandra) |
|---|---|---|
| Ordering | ORDER BY created_at DESC | Clustering column on created_at |
| Sharding | Manual by user_id | Built‑in partition by user_id |
| Flexibility | Rich queries for admin | Simple key‑value/list access |
| Scale | Requires careful sharding, read replicas | Naturally scales horizontally |
At large scale, a wide‑column store like Cassandra or DynamoDB is often used for the Post table, with user_id as partition key and created_at as clustering column to efficiently fetch a user's posts in order. A separate Redis sorted set or list caches the recent post IDs per user.
Step 6: Publishing Workflow
When a user creates a post, several actions occur asynchronously to keep the response fast.
- Client sends post metadata; Post Service persists it immediately and returns the
post_id. - Media upload happens in parallel, often via a direct upload to Object Storage using a pre‑signed URL, then the Media Service processes (transcode, thumbnails) and notifies the Post Service.
- Once media is ready, the post becomes visible. The Post Service updates the Timeline Cache and publishes an event.
- Asynchronous workers handle notifications to followers and feed fan‑out.
This decoupling ensures the publish action is fast, even for large files.
Step 7: Pagination Strategy
Timeline pagination must handle continuous writes without gaps or duplicates.
- Cursor‑based pagination uses the last fetched post's timestamp and ID as a cursor. The next query fetches posts older than that cursor.
- Example cursor:
2026-07-22T10:00:00Z_post123. - Stable, even when new posts are inserted.
- Example cursor:
- Timestamp‑based pagination relies on
created_atbut can fail if two posts share the exact timestamp. Combined with a tiebreaker (post ID) is safe. - Offset pagination (
LIMIT 20 OFFSET 40) is problematic because newly added posts shift the entire result set, causing duplicates or missed items.
Best practice is to use an opaque cursor encoding timestamp and post ID, stored client‑side for infinite scrolling.
Step 8: Media Storage
Media is handled separately from post metadata to avoid bloating the transactional database.
- Object Storage (Amazon S3, MinIO): Stores original and processed media. Highly durable and scalable.
- CDN: Caches media at edge locations for fast, global delivery.
- Processing pipeline: Images are resized into multiple sizes (thumbnail, small, original). Videos are transcoded into adaptive bitrate formats (HLS/DASH). Thumbnails and previews are generated.
- Lazy loading: The Timeline loads low‑quality or blurred previews first, then swaps in full resolution as needed.
- Signed URLs: For private media, the API generates time‑limited, signed CDN URLs so only authorized viewers can access.
Step 9: Timeline Caching
Caching is essential for fast Timeline reads.
- Timeline Cache (Redis): For each user, stores a sorted list of recent post IDs (e.g., up to 1000 latest). This list is used to quickly fetch the top of the Timeline.
- User cache: Profile data and privacy settings cached to avoid database hits.
- Post cache: Frequently accessed post objects are cached by
post_idin Redis or a distributed cache. - Edge cache: For public Timelines, a CDN can cache the rendered HTML/JSON for a short period.
- Cache invalidation: When a post is edited, deleted, or its visibility changes, the relevant Timeline caches are invalidated. An event‑driven invalidation worker handles this.
- Cache warming: For popular users, the Timeline can be pre‑warmed in cache.
Step 10: Timeline vs News Feed
It is critical to distinguish a Timeline from a News Feed.
| Aspect | Timeline | News Feed |
|---|---|---|
| Data source | A single user's own posts | Aggregated posts from many followees |
| Personalization | None (chronological order) | Highly personalized (ML ranking) |
| Ranking | No ranking; reverse chronological | Advanced ranking algorithms |
| Storage | Sharded by author user_id | Sharded by reader user_id (in fan‑out) |
| Read pattern | Fetch one user's posts | Fetch many users' recent posts |
| Write pattern | Insert into one Timeline per post | Fan‑out to many feed caches per post |
| Cache strategy | Simple list per user | Complex, multi‑source, ranked |
| Scalability | Easier: horizontal by user shard | Harder: fan‑out, celebrity handling |
| Complexity | Lower | Much higher |
Because of these differences, in large systems, the Timeline service is separate from the Feed service. They share the same Post data but provide completely different read views.
Step 11: Scalability Strategies
- Horizontal scaling: All services are stateless and scale out behind load balancers.
- Database sharding: Posts are sharded by
user_id, so any user's Timeline resides in one shard, allowing efficient sequential reads. - Read replicas: For relational databases, read replicas handle Timeline reads, reducing load on the primary.
- Object Storage and CDN: Offloads media bandwidth and storage from application servers.
- Multi‑region deployment: User data is stored in a primary region; Timeline reads are served from local caches and read replicas for global low latency.
- Event‑driven architecture: Post‑publishing events drive asynchronous processing (media, notifications, feed updates) without slowing the critical write path.
Step 12: Reliability and Failure Recovery
- Retry with backoff: For transient failures in media processing or notification dispatch.
- Cache failures: If the Timeline Cache goes down, the Timeline Service falls back to querying the Post database directly. Latency increases but the Timeline remains available.
- Storage failures: Object storage has built‑in redundancy. Media processing failures are queued for retry.
- Queue buffering: Kafka or equivalent buffers events, so even if downstream consumers are temporarily down, no events are lost.
- Graceful degradation: If the CDN or image processing is slow, the Timeline can serve text‑only posts first.
- Disaster recovery: The Post database is backed up continuously; media in Object Storage is cross‑region replicated.
Step 13: Security and Privacy
- Authentication & Authorization: Only authenticated users can access non‑public Timelines. Post visibility is checked at query time (friends, private).
- Private accounts: If a user's profile is private, only approved followers can view the Timeline.
- Content moderation: Posts are scanned for prohibited content; reported posts are hidden pending review.
- Abuse prevention: Rate limiting on post creation and Timeline reads per IP/user.
- Media protection: Private media is served via signed URLs that expire quickly, preventing hotlinking.
- Audit logging: All administrative access and content deletions are logged immutably.
Real-World Example: Instagram-like Profile Timeline
Let's trace a photo upload and subsequent profile view.
- Alice creates a post; the metadata is stored immediately. The photo upload and processing continue asynchronously.
- Once processed, the post becomes public and appears on her Timeline cache.
- Bob visits Alice's profile; the Timeline Service retrieves the cached list of post IDs, hydrates the post data, and returns it. The experience is fast and smooth.
Trade-offs
- SQL vs NoSQL: SQL provides strong consistency and rich querying for smaller scales; NoSQL offers easier horizontal scaling for billions of posts. A hybrid approach uses SQL for user/profile data and NoSQL for post storage.
- Cursor vs Offset pagination: Cursors avoid duplication and are stable; offset is simpler conceptually but fails in a live Timeline. Always use cursors for production.
- Cache size vs freshness: Caching massive Timelines consumes memory. Evicting old cached posts from the Timeline list (keeping only the last N posts in cache) reduces memory pressure; older posts are fetched from the database on demand.
- Media optimization vs upload latency: Processing media (resizing, transcoding) adds delay before a post appears. Acceptable because the post metadata is available immediately, and media placeholders can be shown.
Common Mistakes
- Using OFFSET pagination – Leads to duplicates and missing posts when content is added.
- Storing media in the relational database – Horrible for performance and cost; always use object storage.
- No cache layer – Direct database queries for every Timeline view overload the database and increase latency.
- No asynchronous media processing – Blocking the upload request until media is fully processed creates a poor user experience.
- No soft deletion – Hard‑deleting posts loses user data permanently; archive first, then purge later.
- Tight coupling with Feed generation – Timeline writes should not block on feed fan‑out; use events to decouple.
Interview Perspective
In a system design interview, the Timeline question tests your understanding of data modeling and caching for chronological data. Interviewers may ask:
- Design the Instagram Profile Timeline.
- What is the difference between a Timeline and a News Feed?
- Why use cursor pagination?
- How do you scale the Timeline for a user with millions of followers?
- How do you store and retrieve posts efficiently?
- How do you handle media uploads?
Show that you can separate the simple chronological Timeline from the complex ranked Feed, and that you can design a lean, cache‑efficient system.
Summary
A Timeline system is a fundamental building block of any social platform. It stores and serves a user's own posts in strict reverse chronological order, requiring careful data modeling, cursor‑based pagination, efficient media handling, and aggressive caching. Unlike a News Feed, a Timeline does not need personalization or ranking, which simplifies scaling. By sharding data by user, employing a dedicated Timeline cache, and decoupling media processing from the write path, the system can achieve low latency and high availability even with billions of posts. Mastering Timeline design provides the foundation upon which feeds, notifications, and search are built.