Design S3-Like Distributed Object Storage

1. Problem Statement & Scope

System Mission

Design a massive-scale, highly available, distributed Object Storage system capable of storing exabytes of unstructured binary data with 11 9s of durability, supporting RESTful PUT/GET/DELETE operations, multipart uploads, and lifecycle tiering.

Functional Requirements

1. Object CRUD Operations: Upload, retrieve, and delete immutable objects identified by bucket and key.
2. Multipart Upload: Parallel chunk upload for large files ($> 100\text{ MB}$ up to $5\text{ TB}$).
3. Presigned URLs: Secure time-bounded URL access.
4. Lifecycle Management: Automatic archival to cold storage (Glacier).

Non-Functional Requirements (SLAs/SLOs)

• Durability: $99.999999999\%$ (11 9s) via Reed-Solomon Erasure Coding ($8+4$).
• Availability: $99.99\%$ uptime SLA.

2. Capacity & Scale Estimation

• Total Objects Stored: 100 Billion objects ($10^{11}$).
• Total Raw Storage: $20\text{ PB}$.
• Storage with 8+4 Erasure Coding (1.5x Overhead): $\mathbf{30\text{ PB}}$.
• Throughput: Read QPS: $100,000\text{ RPS}$ (Peak: $250,000\text{ RPS}$); Write QPS: $10,000\text{ WPS}$.
• Bandwidth: Egress Peak: $50\text{ GB/s} = \mathbf{400\text{ Gbps}}$.

3. AWS-First High-Level Architecture

Interactive Architecture Diagram
100%Fit100%
Synthesizing vector architecture diagram...

4. API Interface Design

httpCopy
PUT /v1/buckets/my-photos/objects/vacation.jpg
Host: s3.amazonaws.com
Content-Type: image/jpeg
Content-Length: 204800

<binary payload>

Response: 200 OK
ETag: "9b10e43f05630819324f2563aeabaca0"

5. Data Models & Storage Architecture

DynamoDB Object Metadata Table (ObjectMetadataTable)

• PK = BUCKET#<bucket_name>, SK = KEY#<object_path>
• Attributes: object_size_bytes, etag, storage_class, chunk_manifest (List of 12 chunk locations).

6. Component Deep Dives & Workflows

1. Multi-Part Upload Protocol Execution Flow

For objects larger than $100\text{ MB}$ (and up to $5\text{ TB}$), upload operations execute as parallel chunked parts:

Interactive Architecture Diagram
100%Fit100%
Synthesizing vector architecture diagram...

2. Reed-Solomon Erasure Coding ($8+4$) Mathematics

To maximize durability while minimizing storage cost, raw object data is partitioned into $K=8$ data chunks and encoded into $M=4$ parity chunks using Vandermonde generator matrices over Galois Field $\text{GF}(2^8)$: $\text{Storage Overhead} = \frac{K + M}{K} = \frac{8 + 4}{8} = \mathbf{1.5\times} \quad (\text{vs. } 3.0\times \text{ for 3-way replication})$ $\text{Durability Guarantee} = \text{Can survive simultaneous catastrophic loss of ANY } M=4 \text{ storage racks}$