Design Mobile News Feed with Offline-First Architecture

1. Problem Statement & Scope

System Mission

Design a battle-tested, offline-first mobile news feed client and backend cloud synchronization architecture delivering instantaneous ($< 16\text{ ms}$) feed rendering from local persistent cache, robust background delta synchronization via GraphQL and Protocol Buffers, optimistic mutation pipelines with deterministic rollback, and intelligent battery/network-aware media prefetching.

Synthesizing vector architecture diagram...

Functional Requirements

1. Zero-Latency Feed Rendering: Cold launches render cached feed items from local SQLite storage in Write-Ahead Logging (WAL) mode with zero main-thread disk I/O ($< 16\text{ ms}$).
2. Bidirectional Delta Synchronization: Client presents a server-assigned monotonic synchronization cursor since_seq_id (never a device wall-clock timestamp, see §8.2); server returns a compressed changelog containing new, modified, and soft-deleted tombstone records.
3. Optimistic Local Mutations & Durable Offline Queue: Feed actions (posts, likes, comments, bookmarks) immediately commit to local SQLite and update the UI optimistically. Mutations queue in a durable local SQLite outbox and replay via Android WorkManager or iOS BGAppRefreshTask with exponential backoff and jitter upon network recovery.
4. Deterministic Conflict Resolution & Rollback: Automatic rollback on HTTP 4xx validation errors or 409 Conflict version mismatches, reverting the optimistic UI state and alerting the user non-destructively.
5. Adaptive Network & Battery Media Prefetching: Prefetch high-resolution WebP/AVIF images on unmetered high-speed WiFi; restrict prefetching to low-resolution thumbnails or halt prefetching entirely on metered cellular data or when device battery falls below $20\%$.

Non-Functional Requirements (SLAs/SLOs)

• Time-To-Content (TTC) Cold Start: P95 $< 20\text{ ms}$, P99 $< 50\text{ ms}$ reading from local SQLite WAL on mobile device flash storage.
• UI Smoothness & Frame Budget: 120 FPS ($8.33\text{ ms}$ per frame budget) on modern 120Hz ProMotion / LTPO displays; strict zero disk or network I/O on the UI thread.
• Network Bandwidth Efficiency: Delta sync payload $\le 4\text{ KB}$ per refresh ($> 90\%$ bandwidth reduction versus 60 KB full JSON payload).
• Battery & Thermal Budget: $< 1.5\%$ daily device battery drain attributable to background synchronization; zero background CPU execution while app is in suspended state.
• Data Durability Guarantee: Backend achieves $99.999999999\%$ (11 Nines) durability via Amazon DynamoDB and Amazon S3; mobile mutation queue provides zero data loss across app crashes, OS process terminations, and device reboots.
• CAP / PACELC Classification: Mobile client is an AP node operating in PA/EL mode (Partition $\to$ Availability; Else $\to$ Latency over Consistency). The client guarantees instant local read/write availability, converging to eventual consistency via monotonic delta reconciliation.

Out-of-Scope

• Peer-to-peer (P2P) mesh feed distribution (e.g., Bluetooth LE mesh feed sharing).
• Live real-time video stream broadcasting (covered in dedicated Live Video Streaming blueprints).

---

2. Capacity & Scale Estimation

Traffic & Sync Calculations

• Active Mobile Installs: 200 Million registered devices; 50 Million Daily Active Users (DAU).
• Daily Feed Refreshes: Average 20 feed delta syncs per user per day. $\text{Daily Delta Sync Operations} = 50 \times 10^6 \times 20 = \mathbf{1,000,000,000\text{ syncs/day}}$
• Average & Peak Sync QPS: $\text{Avg Sync QPS} = \frac{10^9\text{ syncs}}{86,400\text{ s}} \approx 11,574\text{ syncs/sec}$ $\text{Peak Sync QPS} = 11,574 \times 3.0 \approx \mathbf{34,722\text{ syncs/sec (Provision for 35,000 QPS)}}$
• Daily Mutation Volume (Writes): Average 5 mutations (likes, comments, posts) per user per day. $\text{Daily Mutations} = 50 \times 10^6 \times 5 = \mathbf{250,000,000\text{ mutations/day}}$ $\text{Avg Mutation QPS} = \frac{250 \times 10^6}{86,400\text{ s}} \approx 2,894\text{ QPS} \quad (\text{Peak: } 2,894 \times 3.0 \approx \mathbf{8,682\text{ QPS}})$

Bandwidth & Payload Sizing

• Full Feed Payload: 20 posts with author metadata, thumbnails, engagement counters $\approx 60\text{ KB}$ JSON.
• Delta Sync Payload (Protobuf / GraphQL Compressed): Average 5 changed items $\approx 4\text{ KB}$ ($93.3\%$ cellular data savings).
• Network Bandwidth (Egress & Ingress): $\text{Peak Delta Egress} = 35,000\text{ syncs/sec} \times 4\text{ KB} = 140\text{ MB/sec} = \mathbf{1.12\text{ Gbps}}$ $\text{Peak Mutation Ingress} = 8,700\text{ writes/sec} \times 1\text{ KB} = 8.7\text{ MB/sec} = \mathbf{69.6\text{ Mbps}}$

Cloud Storage Calculations (3-Year Horizon)

• User Feed Record: 1.5 KB average item size.
• 3-Year Post Metadata Growth: $\text{New Posts/Year} = 50 \times 10^6 \times 0.2\text{ posts/day} \times 365 \approx 3.65\text{ Billion posts/year}$ $\text{3-Year Raw Post Storage} = 3.65 \times 10^9 \times 3 \times 1.5\text{ KB} \approx \mathbf{16.4\text{ TB}}$
• Replication Factor ($N=3$) Across Multi-AZ DynamoDB (physical footprint; DynamoDB bills the logical 16.4 TB and replicates internally): $\text{Replicated Storage} = 16.4\text{ TB} \times 3 = \mathbf{49.2\text{ TB}}$
• Change Data Capture (CDC) Delta Outbox Table: Maintains a rolling 14-day changelog buffer for delta queries ($TTL = 14\text{ days}$). Each mutation fans out to the delta outbox of every follower whose feed it touches, so the item count below is a lower bound (write fan-out multiplies it): $\text{14-Day Delta Outbox Items} = 14 \times 250 \times 10^6 \text{ mutations} = 3.5\text{ Billion items}$ $\text{Delta Outbox Volume} = 3.5 \times 10^9 \times 300\text{ Bytes} \approx \mathbf{1.05\text{ TB}}$

Mobile Device Local Storage Budget

• Local SQLite Cache: Max 500 cached posts @ 2 KB per record $= 1.0\text{ MB}$.
• SQLite WAL Journal File: Capped at $4.0\text{ MB}$ via periodic auto-checkpointing.
• Local Media LRU Disk Cache: Capped strictly at $\mathbf{150\text{ MB}}$ on flash storage.
• Heap RAM Bounding: $< 35\text{ MB}$ active memory allocation for feed view models and decoded bitmaps.

Cloud Fleet Sizing

• Backend ECS Fargate Sync Resolvers: Each Fargate task ($2\text{ vCPU}, 4\text{ GB RAM}$) processes up to 500 delta sync requests/sec. $\text{Fargate Task Count} = \frac{35,000\text{ Peak QPS}}{500\text{ req/task}} = 70 \Rightarrow \mathbf{72\text{ tasks (24 per AZ across 3 AZs)}}$

---

3. AWS-First High-Level Architecture

Synthesizing vector architecture diagram...

Start on the device, in the "Mobile Client Architecture Deep Dive" panel: the UI only ever reads from the local SQLite database through a reactive stream, so it works the same online or offline. A like or post is written to the local database at once (optimistically) and queued in a local mutation table; when the connectivity monitor sees a network, the background scheduler sends the queue. On the server, AppSync receives both the queued mutations and delta-sync requests. In the "Primary Storage & Change Capture Tier" panel, workers write the feed table, whose stream copies each change into a delta table kept for 14 days, so a phone returning after a week downloads only what changed. In the "Dead Letter & Async Worker Tier" panel, writes that fail go to a DLQ, where a worker reconciles them. The local database is the UI's single source of truth; the network only keeps it in sync.

Data Flow Architecture

1. Local Render Path (Offline / Online): UI components bind to reactive query observables (Kotlin Flow / Swift Combine) emitted by local SQLite. Cached posts load into memory in $< 16\text{ ms}$ with zero network blocking.
2. Delta Sync Ingress Path: Upon confirmed network reachability, the sync engine issues a delta query over HTTP/2 presenting the client's last acknowledged monotonic sequence token since_seq_id. CloudFront routes to AWS AppSync, which queries the DynamoDB Delta Outbox for entries strictly newer than since_seq_id.
3. Local Ingestion & Tombstone Purge: The server returns an atomic change batch containing newly published, modified, and soft-deleted records. In a single local SQLite transaction, mutations are upserted, tombstoned records (_deleted == true) are purged from local_feed_posts, and last_sync_seq_id advances.
4. Optimistic Mutation Outbox Path: User interactions (likes, comments, posts) execute an atomic local SQLite transaction that updates local_feed_posts with sync_status = DIRTY and stages the payload into local_mutation_queue. Background workers drain the outbox sequentially with exponential backoff.

End-to-End Request Tracing Walkthrough

---

4. API Interface Design

1. GraphQL Delta Synchronization Query

graphqlCopy
query SyncFeedTimeline($sinceSeqId: Long!, $limit: Int!) {
  syncFeed(sinceSeqId: $sinceSeqId, limit: $limit) {
    items {
      postId
      authorId
      authorName
      content
      mediaUrl
      thumbnailUrl
      likesCount
      commentsCount
      userHasLiked
      serverVersion
      serverSeqId
      _deleted
      tombstoneTimestampMs
      updatedAtMs
    }
    nextToken
    newSyncSeqId
    serverTimeMs
  }
}

2. Protocol Buffers Wire Schema (feed_sync.proto)

For extreme bandwidth conservation on metered cellular networks, clients can negotiate binary Protobuf:

protobufCopy
syntax = "proto3";
package feed.mobile.v1;

message FeedPostDelta {
  string post_id = 1;
  string author_id = 2;
  string author_name = 3;
  string content = 4;
  string media_url = 5;
  string thumbnail_url = 6;
  int32 likes_count = 7;
  int32 comments_count = 8;
  bool user_has_liked = 9;
  int64 server_version = 10;
  bool is_deleted = 11;
  int64 tombstone_timestamp_ms = 12;
  int64 updated_at_ms = 13;
  int64 server_seq_id = 14;
}

message DeltaSyncRequest {
  int64 since_seq_id = 1;          // server-assigned monotonic cursor, never device wall-clock
  int32 page_limit = 2;
  string pagination_token = 3;
}

message DeltaSyncResponse {
  repeated FeedPostDelta items = 1;
  string next_pagination_token = 2;
  int64 new_sync_seq_id = 3;       // max server_seq_id in this batch; client stores it only after the local commit
  int64 server_epoch_ms = 4;       // informational only (clock-skew telemetry), not a sync cursor
}

3. Batch Mutation Replay Endpoint (POST /v1/feed/mutations/batch)

Replays queued offline mutations in strict chronological order with idempotency protection.

httpCopy
POST /v1/feed/mutations/batch HTTP/1.1
Host: sync.production.aws.internal
Content-Type: application/json
X-Client-Sync-Epoch: 1773648000120
X-Client-Platform: android-arm64
Authorization: Bearer <jwt_access_token>

{
  "mutations": [
    {
      "client_mutation_id": "018e3d2a-7f12-7000-8000-123456789abc",
      "action_type": "LIKE",
      "post_id": "post_4091",
      "expected_version": 4,
      "payload": {
        "user_has_liked": true
      },
      "created_at_epoch_ms": 1773648000050
    },
    {
      "client_mutation_id": "018e3d2a-7f12-7000-8000-123456789abd",
      "action_type": "COMMENT",
      "post_id": "post_4091",
      "expected_version": 4,
      "payload": {
        "comment_text": "Groundbreaking offline architecture!"
      },
      "created_at_epoch_ms": 1773648000080
    }
  ]
}

Response: 200 OK

jsonCopy
{
  "results": [
    {
      "client_mutation_id": "018e3d2a-7f12-7000-8000-123456789abc",
      "status": "SUCCESS",
      "server_version": 5,
      "updated_at_epoch_ms": 1773648001010
    },
    {
      "client_mutation_id": "018e3d2a-7f12-7000-8000-123456789abd",
      "status": "SUCCESS",
      "server_version": 6,
      "updated_at_epoch_ms": 1773648001015
    }
  ],
  "server_sync_cursor_ms": 1773648001015
}

4. Status Codes & Error Contracts

---

5. Data Models & Storage Architecture

Local SQLite Schema with WAL Mode (mobile_news_feed.db)

Configured explicitly with PRAGMA journal_mode = WAL; and PRAGMA synchronous = NORMAL; to decouple concurrent readers from writers.

sqlCopy
-- Core Cached Feed Posts Table
CREATE TABLE local_feed_posts (
    post_id TEXT PRIMARY KEY NOT NULL,
    author_id TEXT NOT NULL,
    author_name TEXT NOT NULL,
    content TEXT NOT NULL,
    media_url TEXT,
    thumbnail_url TEXT,
    likes_count INTEGER NOT NULL DEFAULT 0,
    comments_count INTEGER NOT NULL DEFAULT 0,
    user_has_liked INTEGER NOT NULL DEFAULT 0, -- 0 = false, 1 = true
    server_version INTEGER NOT NULL DEFAULT 1,
    sync_status INTEGER NOT NULL DEFAULT 1,     -- 1 = SYNCED, 2 = OPTIMISTIC_DIRTY, 3 = FAILED
    is_deleted INTEGER NOT NULL DEFAULT 0,
    tombstone_timestamp INTEGER,
    created_at INTEGER NOT NULL,
    updated_at INTEGER NOT NULL
);

-- Index optimizing instantaneous feed chronological scrolling
CREATE INDEX idx_feed_timeline ON local_feed_posts(is_deleted, updated_at DESC);

-- Durable Mutation Queue Table (Client Transactional Outbox)
CREATE TABLE local_mutation_queue (
    mutation_id TEXT PRIMARY KEY NOT NULL,     -- Client-generated UUIDv7
    post_id TEXT NOT NULL,
    action_type TEXT NOT NULL CHECK (action_type IN ('LIKE', 'UNLIKE', 'CREATE_POST', 'COMMENT', 'DELETE')),
    payload_json TEXT NOT NULL,
    expected_version INTEGER NOT NULL,
    state TEXT NOT NULL CHECK (state IN ('QUEUED', 'IN_FLIGHT', 'FAILED_RETRY')),
    retry_count INTEGER NOT NULL DEFAULT 0,
    created_at INTEGER NOT NULL
);

-- Index optimizing chronological outbox drain
CREATE INDEX idx_mutation_queue_drain ON local_mutation_queue(state, created_at ASC);

-- Quarantine Dead-Letter Outbox Table for Poison Pill Mutations (Permanent HTTP 400/422 Failures)
CREATE TABLE local_mutation_quarantine (
    mutation_id TEXT PRIMARY KEY NOT NULL,
    post_id TEXT NOT NULL,
    action_type TEXT NOT NULL,
    payload_json TEXT NOT NULL,
    expected_version INTEGER NOT NULL,
    http_status INTEGER NOT NULL,
    error_code TEXT NOT NULL,
    error_message TEXT NOT NULL,
    quarantined_at INTEGER NOT NULL
);

-- Sync Metadata Table (Monotonic Sequence Tracking)
CREATE TABLE local_sync_metadata (
    feed_key TEXT PRIMARY KEY NOT NULL,
    last_sync_seq_id INTEGER NOT NULL DEFAULT 0,    -- Authoritative server-assigned 64-bit monotonic sequence counter
    last_sync_epoch_ms INTEGER NOT NULL,            -- Informational server epoch (telemetry only, never used as the sync cursor)
    last_sync_success_at INTEGER NOT NULL
);

-- SQLite Concurrency & Maintenance Configuration:
-- PRAGMA journal_mode = WAL;
-- PRAGMA synchronous = NORMAL;
-- PRAGMA wal_autocheckpoint = 1000;              -- Trigger passive checkpoint every 1,000 pages (~4MB)
-- Background Idle Maintenance (Zero Active Readers):
-- PRAGMA wal_checkpoint(TRUNCATE);                -- Checkpoints and truncates WAL file to 0 bytes

Backend Amazon DynamoDB Single-Table Design

textCopy
Table Name: ProductionNewsFeedStore
Partition Key (PK): String | Sort Key (SK): String
Global Secondary Index 1 (GSI1): GSI1PK (String) | GSI1SK (String)

---