Design Netflix's Global Video Streaming & Zuul API Architecture

1. Problem Statement & Scope Clarification

System Mission

Design the global video streaming, API gateway, and edge routing infrastructure for Netflix, serving over $260\text{M}+$ paid subscribers across $190+$ countries. The platform must handle continuous high-throughput video delivery ($90+\text{ Tbps}$ peak video egress), millisecond-tier API routing across multi-region AWS infrastructure ($40\text{M}+$ peak API QPS), automated regional evacuation during major AWS infrastructure outages within $< 7\text{ minutes}$, and resilient distributed caching that guarantees zero playback disruption.

Synthesizing vector architecture diagram...

Functional Requirements

1. User Profile & Discovery API: Deliver sub-second personalized homepage carousels, playback bookmarks, and search queries with $P99 < 20\text{ ms}$.
2. Dynamic Ingress API Gateway (Zuul 2): Terminate TLS, authenticate via cryptographic Passport tokens, execute dynamic routing/canary routing, enforce distributed rate limiting, and multiplex traffic via non-blocking asynchronous Netty event loops.
3. Automated Multi-Region Active-Active Evacuation: Detect catastrophic AWS regional degradations (e.g., transit provider fiber cuts or AZ failures in us-east-1) and seamlessly evacuate $100\%$ of ingress traffic to us-west-2 and eu-west-1 in $< 7\text{ minutes}$ without user logout or playback interruptions.
4. Adaptive Bitrate Streaming Manifest Negotiation: Calculate the optimal Open Connect Appliance (OCA) steering manifest based on client ISP AS-path, IP prefix health, and server load, returning signed DRM license tokens (Widevine, PlayReady, FairPlay).
5. Continuous Resilience via Chaos Automation: Continuously inject host failures, latency degradation, and full regional network partitions (Chaos Monkey & Chaos Kong) during live production traffic to validate self-healing boundaries.

Non-Functional Requirements (SLAs & SLOs)

• High Availability: $99.999\%$ uptime for playback initialization (less than 5.26 minutes of downtime per calendar year).
• Gateway Ingress Latency: $P50 < 3\text{ ms}$, $P95 < 10\text{ ms}$, $P99 < 25\text{ ms}$ processing overhead added by Zuul edge filters.
• Playback Start Latency (Time-to-First-Frame): $P50 < 400\text{ ms}$, $P95 < 1,200\text{ ms}$ globally over broadband and 5G connections.
• Regional Traffic Evacuation SLO: Shift $100\%$ of regional traffic ($\approx 13\text{M QPS}$) to alternate regions within $420\text{ seconds}$ ($7\text{ minutes}$) with zero increase in user error rates.
• Data Durability Guarantee: 11 Nines ($99.999999999\%$) for master media assets and subscriber accounts stored across Amazon S3 and multi-region distributed databases.
• CAP / PACELC Classification:
  • Playback Control Plane: AP system under CAP; PA/EL under PACELC (Partition $\to$ Availability; Else $\to$ Latency over Consistency). Playback bookmarks and viewing history rely on eventual consistency; stale bookmarks are preferred over failing a playback request.
  • Billing & Account Tiering: CP / PC/EC system enforcing strict ACID guarantees via Amazon Aurora PostgreSQL Multi-Region.

Out-of-Scope

• Physical manufacturing and ASIC hardware acceleration design of Open Connect Appliance servers.
• Content studio production workflows, raw film ingestion, and post-production editorial review pipelines.
• Direct credit card payment acquisition gateways (delegated to certified PCI-DSS Level 1 payment processors).

---

2. Capacity & Scale Estimation (Back-of-the-Envelope Math)

Traffic Scale & Concurrency Calculations

• Global Paid Subscribers: $260,000,000$ active accounts ($400,000,000$ active viewer profiles).
• Peak Concurrent Video Streams: $20,000,000$ concurrent playback sessions during prime-time evening hours (8:00 PM - 11:00 PM local time across time zones).
• Average Stream Bitrate: $4.5\text{ Mbps}$ (weighted blend of 50% 1080p @ 3.5 Mbps, 30% 4K HDR @ 15 Mbps, and 20% Mobile 720p @ 1.2 Mbps).
• Peak Video Delivery Bandwidth (Data Plane - Offloaded to OCAs): $\text{Peak Video Egress} = 20,000,000\text{ streams} \times 4.5\text{ Mbps} = 90,000,000\text{ Mbps} = \mathbf{90\text{ Tbps (Served by OCAs)}}$ $\text{Daily Video Data Egress} = 20\text{M} \times 3\text{ hours/user} \times 4.5\text{ Mbps} \times 3,600\text{ s} \approx \mathbf{121.5\text{ Petabytes/day}}$

Control Plane API Throughput Calculations (AWS Ingress)

• API Requests per Active Viewer Session: Average $2.0\text{ QPS}$ during browsing/discovery; $0.1\text{ QPS}$ during active video playback (telemetry pings, adaptive bitrate metrics, bookmark heartbeats every 10 seconds).
• Active Browsing Viewers: $10,000,000$ concurrent users navigating discovery carousels.
• Peak Ingress API QPS (Zuul Edge Gateway): $\text{Discovery QPS} = 10,000,000 \times 2.0 = 20,000,000\text{ QPS}$ $\text{Playback Heartbeat QPS} = 20,000,000 \times 0.1 = 2,000,000\text{ QPS}$ $\text{Telemetry, Client Logs \& Background Prefetch QPS} = 18,000,000\text{ QPS}$ $\text{Total Global Peak API QPS} = \mathbf{40,000,000\text{ QPS}}$
• Per-Region Load (Divided across 3 Active-Active AWS Regions: us-east-1, us-west-2, eu-west-1): $\text{Nominal Regional QPS} = \frac{40,000,000}{3} \approx \mathbf{13,333,333\text{ QPS per region}}$ $\text{Failover Capacity Requirement (Survive N-1 regional loss)} = \frac{40,000,000}{2} = \mathbf{20,000,000\text{ QPS per region}}$

Distributed In-Memory Cache Sizing (EVCache / Memcached)

• Active User Session Context (Passport Tokens, Profile Settings): $\text{Session Cache Size} = 260\times 10^6 \text{ accounts} \times 4\text{ KB} = 1.04\text{ TB}$
• Personalized Discovery Graph / Metadata Working Set: Top 20% most active viewer profiles ($80\text{M profiles}$) cached in memory with recommendation arrays ($\approx 50\text{ KB}$ per profile): $\text{Metadata Cache Size} = 80 \times 10^6 \times 50\text{ KB} = 4,000\text{ GB} = \mathbf{4.0\text{ TB}}$
• Video Catalog & OCA Routing Metadata: $50,000\text{ titles} \times 50\text{ encoded renditions} \times 2\text{ KB} \approx 5\text{ GB}$ (negligible).
• Total In-Memory Cache Footprint per Region: $\text{Working Set} = 1.04\text{ TB} + 4.0\text{ TB} = 5.04\text{ TB}$ With an 80/20 Pareto buffer, cache replication across 3 Availability Zones, and regional evacuation headroom ($2.0\times$ multiplier): $\text{Provisioned Regional Cache RAM} = 5.04\text{ TB} \times 2.5 \approx \mathbf{12.6\text{ TB RAM per Region}}$

Persistent Storage Calculations (3-Year Horizon)

• User Viewing History Records: $260\text{M subscribers} \times 5\text{ profiles} \times 500\text{ watched videos} \times 200\text{ bytes} \approx \mathbf{130\text{ TB raw}}$.
• Replication Factor ($RF=3$ per region $\times$ 3 active regions, matching the NetworkTopologyStrategy keyspace in §5): $\text{Total Storage Volume} = 130\text{ TB} \times 3 \times 3 = \mathbf{1.17\text{ PB raw [SSTable](/components/07-write-ahead-log-and-lsm-trees) footprint (Cassandra / Keyspaces)}}$

Fleet Sizing & Compute Infrastructure

• Zuul 2 Gateway Fleet (AWS ECS Fargate / EC2 c6i.8xlarge - 32 vCPU, 64 GB RAM): Each optimized Netty async gateway node handles $50,000$ active concurrent HTTP/2 connections and processes $25,000\text{ req/sec}$. $\text{Zuul Nodes per Region (Normal)} = \frac{13,333,333\text{ QPS}}{25,000\text{ QPS/node}} \approx \mathbf{534\text{ nodes}}$ $\text{Zuul Nodes per Region (Peak Failover)} = \frac{20,000,000\text{ QPS}}{25,000\text{ QPS/node}} = \mathbf{800\text{ nodes per Region}}$

---

3. AWS-First High-Level Architecture

Synthesizing vector architecture diagram...

Split the picture into two planes. Data plane: in the "Global Open Connect Edge Delivery" panel (dotted arrows), video bytes go straight from Open Connect appliances inside the viewer's ISP, or at an internet exchange as a fallback, to the TV; they never pass through AWS. Control plane: everything else goes through Route 53 and the NLB to Zuul in AWS, where the "Zuul Netty Ingress Filter Pipeline" panel authenticates the request, applies the rate limit and routes canary traffic, and Eureka provides the current service instances. Services read from EVCache first and Cassandra or DynamoDB behind it, while playback events flow through Kinesis and Flink into the S3 analytics lake. The "AWS Region: us-west-2" panel is kept in sync so traffic can fail over within minutes. Putting video on ISP appliances is what lets AWS handle only small API calls while tens of terabits per second of video go elsewhere.

Data Flow Walkthrough

1. Client Boot & Discovery Request: The subscriber device sends an HTTP/2 GET request for personalized carousels. Amazon Route 53 performs Anycast latency routing, resolving the DNS record to the nearest healthy AWS region (us-east-1).
2. Edge Ingress & Netty Non-Blocking Pipeline: AWS NLB delivers the raw TCP streams to the Zuul 2 gateway fleet. A Netty event loop thread receives the request, parses the HTTP headers, and passes the payload through Zuul inbound filters:
   • Passport Auth Filter: Validates the cryptographic HMAC signature of the client's session token without making an external database call.
   • Adaptive Concurrency Limiter: Evaluates CPU and downstream latency. If the system is saturated, non-critical background telemetry is shed immediately.
   • Canary Filter: Inspects client device metadata, user ID hash, and experiment cohorts to route traffic to the appropriate backend service version.
3. Client-Side Discovery & Routing (Eureka): Zuul consults its local in-memory Eureka registry cache, selects a healthy target instance for DiscoveryService, and routes via internal gRPC.
4. Cache-Aside Read (EVCache): DiscoveryService queries the multi-AZ EVCache cluster. On a cache hit ($99.2\%$ of requests), the personalized carousel payload is returned in $< 2\text{ ms}$.
5. Playback Manifest Generation & OCA Steering: When the user clicks "Play", a request hits SvcPlayback and SvcSteering. The steering engine inspects the user's IP prefix, autonomous system number (ASN), and the real-time load/health reports from Open Connect Appliances deployed inside the user's ISP. It constructs a dynamic HLS/DASH manifest containing ranked URLs pointing directly to local ISP OCAs.
6. Video Streaming Delivery (Off-Cloud Data Plane): The client media player fetches encrypted video/audio chunks directly from the assigned OCA via TLS 1.3 over HTTPS/QUIC. Video traffic never touches the AWS transit backbone.
7. Telemetry Streaming: During playback, client players transmit periodic telemetry pings (bitrate switches, buffer health, frame drops) back through Zuul into Amazon Kinesis for real-time aggregation by Apache Flink.

---

End-to-End Request Tracing Walkthrough

---

4. API Interface Design & Wire Protocol

1. Playback Manifest & Steering Negotiation (POST /v1/playback/manifest)

Initiates video playback, acquires DRM licensing tokens, and retrieves ranked Open Connect Appliance endpoints.

httpCopy
POST /v1/playback/manifest HTTP/2
Host: api.netflix.com
Content-Type: application/json
X-Netflix-Passport: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
X-Device-Type: SMART_TV_TIZEN
X-Client-BGP-ASN: AS7922
Idempotency-Key: 7f8a9b2c-3d4e-5f6a-7b8c-9d0e1f2a3b4c

{
  "subscriber_id": "sub_84920192",
  "profile_id": "prof_3019",
  "video_id": "vid_81049281",
  "requested_audio_track": "en-US",
  "requested_subtitle_track": "es-419",
  "supported_drm_schemes": ["WIDEVINE_L1", "PLAYREADY_SL3000"],
  "display_capabilities": {
    "max_resolution": "3840x2160",
    "hdr_format": "DOLBY_VISION",
    "audio_codec": "ATMOS"
  },
  "client_ip": "73.189.42.15"
}

Response: 200 OK

jsonCopy
{
  "playback_session_id": "sess_01HZX894KMNPQ",
  "video_id": "vid_81049281",
  "resume_position_ms": 5052000,
  "drm_license": {
    "scheme": "WIDEVINE_L1",
    "license_server_url": "https://drm.netflix.com/widevine/v1/license",
    "token": "dGVzdC1saWNlbnNlLXRva2Vu..."
  },
  "manifest": {
    "type": "DASH",
    "master_url": "https://manifest.netflix.com/vid_81049281/master.mpd",
    "preferred_oca_endpoints": [
      {
        "server_id": "oca-sea01-isp-comcast",
        "base_url": "https://ipv4-c001-sea001-comcast-isp.oca.nflxvideo.net",
        "priority": 1,
        "location": "Seattle, WA, US"
      },
      {
        "server_id": "oca-sea02-ixp-equinix",
        "base_url": "https://ipv4-c002-sea002-ixp.oca.nflxvideo.net",
        "priority": 2,
        "location": "Seattle IXP, WA, US"
      }
    ],
    "segment_duration_ms": 2000
  },
  "telemetry_interval_ms": 10000
}

---

2. Playback State Heartbeat Protocol (POST /v1/playback/heartbeat)

Transmitted by clients every 10 seconds to update playback bookmarks, report quality of experience (QoE), and maintain active concurrent stream counts.

protobufCopy
syntax = "proto3";

package netflix.telemetry.v1;

message PlaybackHeartbeatRequest {
  string playback_session_id = 1;
  string subscriber_id = 2;
  string profile_id = 3;
  string video_id = 4;
  int64 current_playback_position_ms = 5;
  int64 buffer_health_ms = 6;
  int32 current_bitrate_kbps = 7;
  int32 dropped_frames_count = 8;
  string active_oca_server_id = 9;
  int64 timestamp_epoch_ms = 10;
}

message PlaybackHeartbeatResponse {
  bool continue_playback = 1;
  string steering_override_oca_url = 2; // Non-empty if client must failover to alternate OCA
  int32 updated_heartbeat_interval_sec = 3;
}

---

3. Status Codes & Error Contracts

---

5. Data Models & Storage Architecture

Storage Tier Selection Justification

1. Amazon Keyspaces / Apache Cassandra: Selected for subscriber viewing history, bookmarks, and playback sessions. Wide-column distributed architecture provides linear horizontal write scaling, multi-region asynchronous active-active WAN replication, and deterministic latency without lock contention.
2. EVCache (Distributed Memcached): Sharded multi-AZ in-memory caching tier fronting Cassandra. Absorbs $99\%+$ of read QPS, decoupling database clusters from prime-time browsing spikes.
3. Amazon DynamoDB Global Tables: Selected for real-time concurrent stream tracking counters. Strongly consistent reads are available only within the writing region; cross-region replication is asynchronous (typically $< 1\text{ s}$), so a household could briefly exceed its stream limit by starting streams in two regions during an evacuation window. That over-admission is accepted (fail-open) rather than blocking playback on a cross-region round-trip.

Synthesizing vector architecture diagram...

Read the relationships from SUBSCRIBER. One subscriber has many PROFILEs, and each profile has its own VIEWING_BOOKMARKs (keyed by profile + video, so "resume where I stopped" is one lookup) and WATCH_HISTORY (keyed by profile + time, so history comes back newest first in one range query). CONCURRENT_STREAM_SESSION is keyed by subscriber, with one row per active stream and a heartbeat expiry; the service counts live rows against max_allowed_streams to enforce the plan's limit, and a crashed player's row simply expires. Every key starts with the ID the app already has (profile or subscriber), so each screen loads from a single partition, which is how the data model fits Cassandra and DynamoDB.

Cassandra CQL Schema: Viewing Bookmarks & History

sqlCopy
CREATE KEYSPACE netflix_playback
WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'us-east-1': 3,
    'us-west-2': 3,
    'eu-west-1': 3
};

-- Real-Time Viewing Bookmark (Upserted on every 10-second heartbeat)
CREATE TABLE netflix_playback.viewing_bookmarks (
    profile_id uuid,
    video_id text,
    bookmark_offset_ms bigint,
    duration_ms bigint,
    completion_pct int,
    device_type text,
    updated_at timestamp,
    PRIMARY KEY ((profile_id), video_id)
) WITH CLUSTERING ORDER BY (video_id ASC)
AND compaction = {
    'class': 'SizeTieredCompactionStrategy',
    'max_threshold': 32
}
AND default_time_to_live = 7776000; -- 90-day auto retention

-- Append-Only Detailed Viewing History (for Recommendation Engines)
CREATE TABLE netflix_playback.viewing_history (
    profile_id uuid,
    watched_year_month int, -- Partition bucket e.g., 202609 to bound partition size
    event_timestamp timestamp,
    video_id text,
    total_seconds_viewed int,
    completed boolean,
    PRIMARY KEY ((profile_id, watched_year_month), event_timestamp, video_id)
) WITH CLUSTERING ORDER BY (event_timestamp DESC, video_id ASC)
AND compaction = {
    'class': 'TimeWindowCompactionStrategy',
    'compaction_window_unit': 'DAYS',
    'compaction_window_size': 1
};

---