Design a Distributed Email Service

1. Problem Statement & Scope

System Mission

Design a secure, highly available, fault-tolerant distributed email service capable of managing 100 Million active mailboxes and processing over 1 Billion daily inbound and outbound messages ($34,722\text{ peak msgs/sec}$). The architecture must support standard wire protocols (SMTP, IMAP, POP3, and REST/JMAP), cryptographic sender verification (SPF, DKIM 2048-bit RSA, DMARC), decoupled storage tiering (DynamoDB metadata index, immutable S3 MIME/attachment blobs, OpenSearch full-text search), SQS FIFO worker queues for soft/hard bounce handling, and sub-second JWZ conversation threading.

Synthesizing vector architecture diagram...

Functional Requirements

1. Inbound SMTP Message Ingestion: Accept incoming RFC 5322 MIME messages over SMTP (Port 25 with STARTTLS) via Route 53 MX records, validating SPF, DKIM, and DMARC alignment.
2. Outbound Mail Delivery Engine: Queue, sign with 2048-bit DKIM private keys, resolve recipient DNS MX records, and reliably dispatch emails to external recipient hosts with automatic TLS opportunistic negotiation.
3. Decoupled Storage & Large Attachment Handling: Segregate lightweight mailbox metadata ($1\text{ KB}$) from raw MIME payloads and binary attachments (up to $25\text{ MB}$), streaming attachments directly to Amazon S3.
4. Sub-Second Mailbox Views & Folder Management: Support standard folders (Inbox, Sent, Drafts, Trash, Spam) with atomic unread counters, cursor-based pagination, and JWZ conversation threading.
5. Full-Text Mailbox Search: Sub-300ms full-text search across email subjects, sender/recipient addresses, body contents, and extracted attachment text via Amazon OpenSearch.
6. Bounce, Complaint & Dead-Letter Handling: Enforce exponential retry with jitter over 72 hours for soft bounces (4xx SMTP codes) and immediate dead-letter handling for hard bounces (5xx SMTP codes).

Non-Functional Requirements (SLAs/SLOs)

• High Availability: $99.999\%$ uptime SLA for mail access and delivery ($< 5.26\text{ minutes}$ downtime/year).
• Data Durability: Zero-byte-loss guarantee ($100\%$ durability for committed emails; 11 9s durability via Amazon S3 multi-AZ replication).
• Latency SLOs:
  • Inbound End-to-End Latency: Message visible in user inbox within $< 2\text{ seconds}$ ($P95$) of SMTP handshake completion.
  • Outbound Dispatch Latency: Delivered to recipient MX in $< 5\text{ seconds}$ ($P95$) under normal network conditions.
  • Mailbox Folder Render Latency: First 50 messages retrieved in $< 40\text{ ms}$ ($P99$).
  • Search Query Latency: BM25 query across 100,000 user emails in $< 300\text{ ms}$ ($P95$).
• CAP / PACELC Classification:
  • Mailbox Metadata & Inboxes: CP / PC-EC (Consistent read/unread status, monotonic folder versioning).
  • Search & Analytics: AP / PA-EL (Near real-time search index eventual consistency).

---

2. Capacity & Scale Estimation

Traffic Calculations

• Active Mailboxes: $100\text{ Million}$ accounts.
• Daily Inbound & Outbound Volume: $1\text{ Billion emails/day}$.
• Message Ingestion Throughput: $\text{Avg Ingestion QPS} = \frac{10^9\text{ emails}}{86,400\text{ sec}} \approx \mathbf{11,574\text{ emails/sec}}$ $\text{Peak Ingestion QPS (3}\times\text{ morning surge)} = 11,574 \times 3 \approx \mathbf{34,722\text{ emails/sec}}$
• Outbound Dispatch Throughput:
  • Assuming $30\%$ outbound ratio ($300\text{ Million outbound emails/day}$): $\text{Avg Outbound QPS} = \frac{300 \times 10^6}{86,400\text{ sec}} \approx \mathbf{3,472\text{ msgs/sec (Peak: } 10,416\text{ msgs/sec)}}$

Storage Calculations (3-Year Horizon)

• Average Email Size Profile:
  • Metadata: $1\text{ KB}$
  • HTML & Plain Text Body: $20\text{ KB}$
  • Attachments (averaged over all messages): $79\text{ KB}$
  • Total Average Footprint: $S_{\text{email}} = 1 + 20 + 79 = \mathbf{100\text{ KB}}$.
• Daily Raw Ingestion Volume: $\text{Daily Raw Ingestion} = 10^9 \times 100\text{ KB} = 100\text{ TB/day} \implies \mathbf{36.5\text{ PB/year}}$
• Storage Tier Allocation:
  • DynamoDB Metadata Tier (Hot): $\text{DynamoDB Storage} = 10^9 \times 1\text{ KB} = \mathbf{1\text{ TB/day}} \implies \mathbf{365\text{ TB/year}}$
  • Amazon S3 Raw MIME & Attachments (Immutable CAS): $\text{S3 Ingestion} = 10^9 \times 99\text{ KB} = \mathbf{99\text{ TB/day}} \implies \mathbf{36.135\text{ PB/year}}$ 3-year cumulative S3 storage $= 36.135 \times 3 = \mathbf{108.4\text{ PB}}$. (Automated S3 Lifecycle transitions emails older than 90 days to S3 Infrequent Access, and older than 365 days to S3 Glacier Instant Retrieval, saving $\approx 60\%$ in storage costs).
  • Amazon OpenSearch Search Inverted Index:
    • Indexing sender, recipient, subject, and tokenized plain text body ($\approx 10\text{ KB}$ per document).
    • With 1 primary shard + 1 replica: $\text{Daily Search Index Ingestion} = 10^9 \times 10\text{ KB} \times 2 = \mathbf{20\text{ TB/day}}$ Active 90-day retention window $= 20\text{ TB} \times 90 = \mathbf{1.8\text{ PB}}$ in OpenSearch cluster.

Network Bandwidth

• Ingress / Egress Throughput: $\text{Avg Ingress Throughput} = 11,574\text{ msgs/s} \times 100\text{ KB} = 1.157\text{ GB/s} = \mathbf{9.26\text{ Gbps}}$ $\text{Peak Ingress Bandwidth (34,722 msgs/s)} = 3.472\text{ GB/s} = \mathbf{27.78\text{ Gbps}}$
• Outbound Egress Bandwidth: $\text{Peak Outbound Egress} = 10,416\text{ msgs/s} \times 100\text{ KB} = 1.041\text{ GB/s} = \mathbf{8.33\text{ Gbps}}$

Memory & Cache Sizing (80/20 Pareto Working Set)

• Top $20\%$ of users generate $80\%$ of mailbox access: $20\text{ Million DAU}$.
• Caching the 50 most recent email headers per active user in ElastiCache Redis: $\text{Recent Message Cache} = 20 \times 10^6 \times 50 \times 1\text{ KB} = \mathbf{1\text{ TB RAM (Redis Cluster)}}$
• Folder Unread Counter Cache (100M users $\times 4$ folders $\times 16\text{ bytes}$): $\text{Unread Counter Cache} = 100 \times 10^6 \times 4 \times 16\text{ bytes} \approx \mathbf{6.4\text{ GB RAM}}$

Fleet Sizing & Compute Provisioning

• Streaming MIME Parser Fleet (ECS Fargate):
  • Streaming MIME parser processes 1 email (header demux, body extract, attachment stream) in average $20\text{ ms}$ on 1 vCPU core.
  • Required vCPUs for peak $34,722\text{ msgs/sec}$: $\text{Parser vCPUs} = 34,722\text{ msgs/s} \times 0.020\text{ s} \approx \mathbf{695\text{ vCPUs}}$
  • Sizing: Provision 174 ECS Fargate Tasks (each configured with 4 vCPUs, 8 GB RAM).

---

3. AWS-First High-Level Architecture

Synthesizing vector architecture diagram...

Follow the three flows. Inbound: another mail server finds you through the MX record in Route 53 and delivers to SES, which stores the full raw message in S3 and publishes an event. In the "Streaming Parser & Decoupling Tier" panel, parsers pull the event from SQS, scan it for spam and malware, save the metadata to DynamoDB, index the text in OpenSearch, and push a notification. Reading: in the "Client Webmail & JMAP/REST API" panel, clients list folders from DynamoDB (with unread counts cached in Redis), search in OpenSearch, and fetch bodies from S3. Outbound: in the "Outbound Sending & Reputation Pipeline" panel, sent mail is queued in SQS FIFO, signed with DKIM, and delivered by SES from dedicated IPs, and bounces and complaints are fed back to suppress bad addresses. Keeping large raw messages in S3 and only metadata in DynamoDB keeps the database small and fast.

Data Flow Walkthrough

1. Inbound Handshake: The external SMTP host connects to Amazon SES on port 25 with opportunistic STARTTLS. SES evaluates SPF records, DKIM signatures, and DMARC policies.
2. Durability-First Blob Persist: SES streams the raw RFC 5322 MIME bytes directly into Amazon S3 before acknowledging the SMTP handshake with 250 2.0.0 OK Message Accepted.
3. Asynchronous Streaming Parse: SES publishes an event through SNS to SQS. ECS Fargate Parser tasks pull the task, streaming the MIME payload in $64\text{ KB}$ ring buffers to extract headers, plain/HTML text, and binary attachments.
4. Decoupled Indexing: The parser populates lightweight folder and thread records in DynamoDB (MailboxCoreTable) and tokenizes the text into Amazon OpenSearch. An atomic increment updates the folder's unread counter.
5. Outbound Dispatch & Warm Pools: User send requests land in SQS FIFO queues. Outbound workers attach 2048-bit DKIM signatures and route traffic through Amazon SES Dedicated IP Pools with automated IP warming and 72-hour exponential backoff retry for 4xx soft bounces.

Concrete Step-by-Step Request Walkthrough: Tracing Inbound Mail to Inbox Render

---

4. API Interface Design

1. Send Outbound Email (POST /v1/mail/messages/send)

Queues an outbound email for cryptographic signing and MX dispatch.

httpCopy
POST /v1/mail/messages/send HTTP/1.1
Host: mail.production.aws.internal
Authorization: Bearer <jwt_token>
Content-Type: application/json

{
  "to": ["architect@distributed-systems.org"],
  "cc": ["lead-sre@company.com"],
  "subject": "System Design Review: Decoupled Mailbox Storage Engine",
  "body_html": "<p>Team, attached is the revised architectural blueprint.</p>",
  "body_plain": "Team, attached is the revised architectural blueprint.",
  "in_reply_to": "<msg_88129a@distributed-systems.org>",
  "references": ["<msg_root@distributed-systems.org>", "<msg_88129a@distributed-systems.org>"],
  "attachments": [
    {
      "file_name": "mail_engine_spec.pdf",
      "content_type": "application/pdf",
      "s3_temp_key": "temp-uploads/user_101/tmp_spec_pdf"
    }
  ]
}

Response: 202 Accepted

jsonCopy
{
  "message_id": "<msg_20260916_998124756@mail.aws.internal>",
  "thread_id": "thread_88129a_root",
  "status": "QUEUED_FOR_DELIVERY",
  "queued_at": 1773648050123
}

2. List Folder Messages (GET /v1/mail/folders/{folder}/messages)

Returns paginated message metadata with conversation threading.

httpCopy
GET /v1/mail/folders/INBOX/messages?limit=25&cursor=eyJkYXRlIjoxNzczNjQ4MDAwLCJtc2dfaWQiOiJtc2dfMTAxIn0= HTTP/1.1
Host: mail.production.aws.internal
Authorization: Bearer <jwt_token>

Response: 200 OK

jsonCopy
{
  "folder": "INBOX",
  "unread_count": 4,
  "total_count": 1420,
  "messages": [
    {
      "message_id": "msg_20260916_998124756",
      "thread_id": "thread_88129a_root",
      "from": {"name": "Alice Wang", "email": "alice@cloud.org"},
      "subject": "System Design Review: Decoupled Mailbox Storage Engine",
      "snippet": "Team, attached is the revised architectural blueprint...",
      "has_attachments": true,
      "is_read": false,
      "received_at": 1773648050000,
      "s3_mime_key": "mail-raw-blobs/user_101/2026/09/msg_998124.eml"
    }
  ],
  "next_cursor": "eyJkYXRlIjoxNzczNjQ3ODAwLCJtc2dfaWQiOiJtc2dfMDk5In0="
}

3. Status Codes & Error Contracts

---

5. Data Models & Storage Architecture

Physical Storage Layout: Decoupled Mail Blobs in Amazon S3

DynamoDB Single-Table Schema (MailboxCoreTable)

---