Design a Real-Time Chat & Instant Messaging System

1. Problem Statement & Scope Clarification

System Mission

Design a globally distributed, highly scalable, real-time instant messaging platform (similar to WhatsApp, Slack, and Discord) capable of supporting ultra-low latency 1-on-1 direct messaging, group conversations (up to 1,000 members), real-time presence tracking (Online/Offline/Last Seen), multi-device synchronization, read receipts, and durable offline catch-up delivery.

Functional Requirements

1. 1-on-1 Direct Messaging: Bi-directional real-time text and media messaging with delivery confirmation ($P99 < 100\text{ ms}$).
2. Group Chats (Fan-Out): Scalable group channels supporting up to 1,000 participants with read-state tracking.
3. Presence & Heartbeat Engine: Real-time user online/offline status detection with debounce mechanisms to suppress flapping.
4. Message Delivery Receipts: Three-stage message lifecycle states: SENT (Server Ack), DELIVERED (Device Ack), and READ (User Seen).
5. Offline Message Synchronization: Durable message storage allowing offline devices to fetch missed messages using sequential sync cursors upon reconnect.
6. Push Notifications: Fallback push notification dispatch (Apple APNs / Google FCM via Amazon SNS) when a recipient is disconnected.

Non-Functional Requirements (SLAs & SLOs)

• High Availability: $99.999\%$ uptime SLA globally across multi-AZ and multi-region AWS deployments.
• Latency:
  • 1-on-1 Message Delivery: $P50 < 30\text{ ms}$, $P99 < 100\text{ ms}$.
  • Presence State Broadcast: $P99 < 500\text{ ms}$.
• Data Durability: Zero message loss ($\text{Durability} \ge 99.999999999\% / \text{11 Nines}$) for all acknowledged messages.
• Connection Scale: Support $20,000,000$ concurrent active persistent WebSocket connections.

2. Capacity & Scale Estimation (Back-of-the-Envelope Math)

User & Traffic Profile

• Daily Active Users (DAU): $100,000,000$ ($100\text{M DAU}$).
• Peak Concurrent WebSocket Connections: $20,000,000$ ($20\text{M concurrent sockets}$).
• Daily Messages Sent: $5.0 \times 10^9\text{ messages/day}$ (5 Billion messages/day).
• Average Message Size: $1\text{ KB}$ (Text + metadata headers).

Throughput Derivations

• Average Ingest QPS: $\text{Average Ingest QPS} = \frac{5 \times 10^9\text{ messages}}{86,400\text{ sec}} \approx \mathbf{57,870\text{ msg/sec}}$
• Peak Ingest QPS ($3\times$ multiplier): $\text{Peak Ingest QPS} = 57,870 \times 3 \approx \mathbf{173,610\text{ msg/sec}}$
• Group Fan-Out Amplification ($3\times$ average group delivery multiplier): $\text{Peak Egress Message Delivery QPS} = 173,610 \times 3 \approx \mathbf{520,830\text{ msgs/sec}}$

Memory & Connection Fleet Sizing

• RAM per Idle WebSocket Connection: $\approx 10\text{ KB}$ (TCP socket buffer + SSL session context in kernel/user space).
• Total In-Memory Socket State: $\text{Memory Required} = 20,000,000 \times 10\text{ KB} = 200,000,000\text{ KB} \approx \mathbf{200\text{ GB RAM}}$
• WebSocket Gateway Cluster Provisioning: Using c6g.2xlarge ECS Fargate tasks (8 vCPU, 16 GB RAM), capped at $50,000$ sockets per task: $\text{Required Gateway Nodes} = \frac{20,000,000\text{ connections}}{50,000\text{ connections/node}} = \mathbf{400\text{ ECS Tasks}}$

Storage Footprint (5-Year Horizon)

• Daily Ingestion Storage: $5 \times 10^9\text{ msg/day} \times 1\text{ KB} = \mathbf{5\text{ TB/day}}$
• 5-Year Storage Footprint (Single Copy): $\text{5-Year Storage} = 5\text{ TB/day} \times 365.25 \times 5 \approx \mathbf{9.13\text{ PB}}$
• Provisioned on Amazon DynamoDB with tiered offload of older messages ($> 90\text{ days}$) to Amazon S3 Standard-IA / Glacier.

3. High-Level Architecture & AWS Component Mapping

Interactive Architecture Diagram
100%Fit100%
Synthesizing vector architecture diagram...

4. API Interface Design & Wire Protocols

1. Client-to-Gateway WebSocket JSON Protocol

jsonCopy
// 1. Client Sending a Message
{
  "action": "SEND_MESSAGE",
  "client_msg_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
  "conversation_id": "conv_991823",
  "recipient_id": "usr_bob_402",
  "content_type": "TEXT",
  "body": "Hey Bob, let's review the architectural spec.",
  "timestamp_client_ms": 1767225600120
}

// 2. Server Acknowledgment to Sender (Status: SENT)
{
  "event": "MESSAGE_SENT_ACK",
  "client_msg_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d",
  "server_msg_id": "718293847561029384",
  "conversation_id": "conv_991823",
  "sequence_number": 10482,
  "timestamp_server_ms": 1767225600135
}

// 3. Server Delivery to Receiver Device
{
  "event": "RECEIVE_MESSAGE",
  "server_msg_id": "718293847561029384",
  "conversation_id": "conv_991823",
  "sender_id": "usr_alice_101",
  "content_type": "TEXT",
  "body": "Hey Bob, let's review the architectural spec.",
  "sequence_number": 10482,
  "timestamp_server_ms": 1767225600135
}

// 4. Client Read Receipt Update
{
  "action": "READ_RECEIPT",
  "conversation_id": "conv_991823",
  "last_read_sequence_number": 10482,
  "timestamp_read_ms": 1767225605000
}

5. Data Models & DynamoDB Single-Table Schema

To achieve sub-millisecond query performance at PB scale, we utilize a single-table DynamoDB design partitioned by conversation_id and indexed by time-sortable Snowflake IDs.

DynamoDB Table: ChatPlatformTable

Range Query for Message History & Offline Catch-Up

jsonCopy
{
  "TableName": "ChatPlatformTable",
  "KeyConditionExpression": "PK = :conv_pk AND SK BETWEEN :last_sync_sk AND :latest_sk",
  "ExpressionAttributeValues": {
    ":conv_pk": {"S": "CONV#conv_991823"},
    ":last_sync_sk": {"S": "MSG#718293800000000000"},
    ":latest_sk": {"S": "MSG#718293999999999999"}
  },
  "Limit": 50,
  "ScanIndexForward": true
}