Design a Digital Wallet System

1. Problem Statement & Scope Clarification

System Mission

Design a bank-grade, horizontally scalable, globally distributed digital wallet backend platform (equivalent to PayPal, Cash App, or Venmo). The system must orchestrate high-throughput peer-to-peer (P2P) balance transfers, fiat deposits and withdrawals, locked multi-currency foreign exchange (FX), and sub-5ms balance queries. The core ledger must mathematically eliminate transaction deadlocks, prevent overdraft race conditions, and guarantee absolute financial balance conservation ($\sum \text{Debits} - \sum \text{Credits} = 0$) across multi-master or sharded database clusters.

Digital Wallet Platform at a Glance

Functional Requirements

1. Instant Peer-to-Peer (P2P) Transfer (ExecuteTransfer): Atomically debit the sender's wallet and credit the receiver's wallet within a strict single ACID transaction boundary, supporting optional memo metadata and idempotent execution tokens.
2. Deposit & Withdrawal (TopUp / Withdraw): Move funds between external banking rails (FedNow, ACH, SEPA, debit card networks) and the internal digital wallet system.
3. Double-Entry Immutable Accounting: Every transfer must generate balanced debit and credit entries; account balance snapshots are strictly reconciled against the append-only ledger transaction history.
4. Guaranteed Multi-Currency FX Conversion: Execute cross-border transfers by locking real-time foreign exchange rates for a guaranteed 60-second window, executing debit in source currency and credit in destination currency.
5. Ultra-Low Latency Balance Inquiry (GetBalance): Provide sub-5 millisecond responses for available, reserved, and locked balances via version-fenced in-memory caches.

Non-Functional Requirements (SLAs & SLOs)

• Consistency: Strict ACID Serializability. Zero lost updates, zero negative balance overdrafts, zero dirty reads, and zero Phantom reads.
• Availability: $99.999\%$ ("five nines") uptime SLA for balance queries and P2P transfers.
• Latency (P99):
  • Balance Inquiry: $P99 < 5\text{ ms}$ (served via Redis cache with monotonic version fencing).
  • Intra-Shard P2P Transfer: $P99 < 20\text{ ms}$ (row lock, balance check, debit/credit append).
  • Cross-Shard P2P Transfer (Two-Phase Saga): $P99 < 60\text{ ms}$.
• Throughput & Scale: Support a baseline peak throughput of $5,000\text{ financial transactions/sec (TPS)}$, engineered to scale to $20,000\text{ TPS}$ during high-volume cultural events (Super Bowl, Lunar New Year Red Envelopes, Black Friday).
• Auditability & Retention: $100\%$ tamper-evident audit logging with a 7-year regulatory retention lifecycle in immutable AWS S3 Object Lock storage.

---

2. Capacity & Scale Estimation (Back-of-the-Envelope Math)

User Scale & Transaction Traffic

• Total Registered Wallets: $50,000,000$ ($50\text{ Million}$).
• Daily Active Wallets (DAU): $10,000,000$ ($10\text{ Million DAU}$).
• Daily P2P Transfers: $20,000,000\text{ transfers/day}$.
• Average Transfer QPS: $\text{TPS}_{\text{avg}} = \frac{20 \times 10^6\text{ transactions}}{86,400\text{ sec}} \approx \mathbf{231.5\text{ TPS}}$
• Peak Transfer QPS ($20\times$ burst during peak hours & sporting events): $\text{TPS}_{\text{peak}} = 231.5 \times 20 \approx \mathbf{4,630\text{ TPS}} \quad (\text{provisioned for } \mathbf{5,000\text{ TPS}})$
• Flash Sale Scale Ceiling: Dimensioned to scale out to $\mathbf{20,000\text{ TPS}}$ across database shards.
• Balance Inquiry Read Traffic ($10:1$ Read/Write Ratio): $\text{QPS}_{\text{read, peak}} = 5,000 \times 10 = \mathbf{50,000\text{ reads/sec}}$

Storage Footprint & Capacity Growth (7-Year Regulatory Retention)

Every P2P transfer creates three physical rows:

1. Transaction Header: ID, idempotency key, source account, dest account, amount, status $\approx 220\text{ bytes}$.
2. Sender Ledger Line (Debit): Entry ID, transaction ID, account ID, direction, amount, balance after $\approx 180\text{ bytes}$.
3. Receiver Ledger Line (Credit): Entry ID, transaction ID, account ID, direction, amount, balance after $\approx 180\text{ bytes}$.

• Index & Metadata Overhead: $\approx 170\text{ bytes}$.
• Total Storage per Transfer: $\approx 750\text{ bytes}$.

$$\text{Daily Ledger Data Volume} = 20 \times 10^6 \times 750\text{ bytes} = 15\text{ GB/day}$$

$\text{Annual Ledger Growth} = 15\text{ GB/day} \times 365.25 \approx \mathbf{5.48\text{ TB/year}}$

$$\text{7-Year Retained Ledger Footprint} = 5.48\text{ TB} \times 7 \approx \mathbf{38.36\text{ TB}}$$

• With B-tree indexes ($1.8\times$) and Aurora 3-AZ physical replication: $\text{Total 7-Year Physical Footprint} = 38.36\text{ TB} \times 1.8 \times 3 \approx \mathbf{207.1\text{ TB}}$

In-Memory Working Set (ElastiCache Redis)

• Active Wallets in Cache: 10M DAU $\times 200\text{ bytes}$ (Account ID, Balance Cents, Locked Cents, Version Number, Timestamp) $\approx 2\text{ GB RAM}$.
• With Redis Hash metadata and cluster replication ($2\times$ factor): $\approx \mathbf{6\text{ GB RAM}}$, easily accommodated by a small multi-node Redis cluster (cache.r6g.xlarge).

---

3. High-Level Architecture & AWS Component Mapping

The architecture implements a decoupled Read/Write Segregated Model: read traffic (balance inquiries) hits an in-memory Redis cache protected by monotonic version fencing, while write traffic (P2P transfers) executes against sharded Amazon Aurora PostgreSQL clusters with row-level pessimistic locking.

Synthesizing vector architecture diagram...

Follow the two kinds of request. Balance checks go from the wallet service to Redis in the "In-Memory Balance Cache" panel; each cached balance carries a version number, and an update is applied only if its version is newer, so a delayed or out-of-order update can never put an old balance back. Transfers go through the shard router to the "ACID Core Ledger Tier" panel, where account IDs decide which Aurora shard owns each wallet; rows are always locked in sorted ID order, so two opposite transfers (Alice to Bob, Bob to Alice) cannot deadlock. After commit, the "Transactional Outbox & Cache Invalidation Pipeline" panel streams each change from the WAL through Debezium into Kafka, where consumers update the Redis balance, archive the ledger in write-once S3, and push a notification. Money moves only inside ACID transactions; everything users see is derived from the committed log.

Data Flow Walkthrough

1. Balance Inquiry (Fast Read Path): The client queries GET /v1/wallets/{account_id}/balance. The API checks ElastiCache Redis. If hit, the cached balance and monotonic version are returned in $< 2\text{ ms}$. If missed, it queries the Aurora read replica, populates Redis, and returns.
2. Transfer Ingress & Routing (Write Path): Alice transfers $\text{\textdollar}25$ to Bob. The request arrives with an Idempotency-Key. The service determines the database shards for Alice and Bob using consistent hashing on account_id:
   • Intra-Shard Transfer (Alice & Bob on Shard 1): Handled in a single, high-speed ACID transaction.
   • Cross-Shard Transfer (Alice on Shard 1, Bob on Shard 2): Coordinated via a Two-Phase Saga with semantic reservations.
3. Deadlock-Free Row Locking: The transaction executes SELECT balance_cents FROM accounts WHERE account_id IN ('acc_alice', 'acc_bob') ORDER BY account_id ASC FOR UPDATE. Sorting account IDs lexicographically guarantees circular lock deadlocks are mathematically impossible.
4. Balance Verification & Double-Entry Commit: The engine verifies Alice has sufficient funds (balance_cents >= 2500), decrements Alice's balance, increments Bob's balance, increments both account version counters by 1, and appends balanced debit/credit lines into ledger_entries.
5. Change Data Capture & Async Invalidation: PostgreSQL WAL emits the commit via Debezium CDC to Amazon MSK. An ECS worker consumes the event and evicts or atomically updates the Redis cache using monotonic version checks (NEW_VERSION > OLD_VERSION). Bob receives an instant push notification via Amazon SNS.

Core Request Tracing Execution Walkthrough

---

4. Double-Entry Accounting Model & Ledger Schema

Relational PostgreSQL DDL Schema (Aurora Multi-AZ)

sqlCopy
-- 1. Accounts Master Table (Stores Current Balance Snapshot, Striping & Monotonic Version)
CREATE TABLE accounts (
    account_id VARCHAR(64) PRIMARY KEY, -- e.g. "acc_usr_998124" or "acc_merchant_nike_stripe_12"
    user_id VARCHAR(64) NOT NULL,
    parent_account_id VARCHAR(64),       -- Set for striped sub-accounts (e.g., hot merchant / payroll wallets)
    stripe_id INT NOT NULL DEFAULT 0,    -- Stripe index [0..N-1] for partitioned hot accounts
    currency CHAR(3) NOT NULL DEFAULT 'USD',
    balance_cents BIGINT NOT NULL DEFAULT 0,
    locked_cents BIGINT NOT NULL DEFAULT 0, -- Funds held for pending cross-shard sagas / card authorizations
    version BIGINT NOT NULL DEFAULT 1,     -- Monotonic fencing version for cache synchronization & read fencing
    status VARCHAR(16) NOT NULL DEFAULT 'ACTIVE' CHECK (status IN ('ACTIVE', 'FROZEN', 'CLOSED')),
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    CONSTRAINT chk_balance_non_negative CHECK (balance_cents >= 0),
    CONSTRAINT chk_locked_non_negative CHECK (locked_cents >= 0),
    CONSTRAINT chk_available_balance CHECK (balance_cents >= locked_cents)
);

CREATE INDEX idx_accounts_parent_stripe ON accounts(parent_account_id, stripe_id) WHERE parent_account_id IS NOT NULL;

-- 2. Transaction Headers Table
CREATE TABLE transactions (
    transaction_id VARCHAR(64) PRIMARY KEY,
    idempotency_key VARCHAR(128) UNIQUE NOT NULL,
    source_account_id VARCHAR(64) NOT NULL REFERENCES accounts(account_id),
    destination_account_id VARCHAR(64) NOT NULL REFERENCES accounts(account_id),
    amount_cents BIGINT NOT NULL CHECK (amount_cents > 0),
    currency CHAR(3) NOT NULL,
    type VARCHAR(32) NOT NULL CHECK (type IN ('P2P_TRANSFER', 'DEPOSIT', 'WITHDRAWAL', 'FX_EXCHANGE', 'COMPENSATION')),
    status VARCHAR(16) NOT NULL CHECK (status IN ('PENDING', 'COMMITTED', 'REJECTED', 'CANCELLED_REFUNDED')),
    memo VARCHAR(255),
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- 3. Immutable Double-Entry Ledger Lines Table (Append-Only Audit Trail)
CREATE TABLE ledger_entries (
    entry_id BIGSERIAL PRIMARY KEY,
    transaction_id VARCHAR(64) NOT NULL REFERENCES transactions(transaction_id),
    account_id VARCHAR(64) NOT NULL REFERENCES accounts(account_id),
    direction VARCHAR(6) NOT NULL CHECK (direction IN ('DEBIT', 'CREDIT')),
    amount_cents BIGINT NOT NULL CHECK (amount_cents > 0),
    currency CHAR(3) NOT NULL,
    balance_after_cents BIGINT NOT NULL, -- Point-in-time balance snapshot for instant auditing
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- 4. Transactional Outbox Events (CDC & Cross-Shard Saga Dispatch)
CREATE TABLE outbox_events (
    event_id BIGSERIAL PRIMARY KEY,
    aggregate_type VARCHAR(32) NOT NULL DEFAULT 'WALLET_TRANSFER',
    aggregate_id VARCHAR(64) NOT NULL, -- transaction_id
    event_type VARCHAR(64) NOT NULL,   -- 'TRANSFER_RESERVED', 'TRANSFER_COMPENSATED'
    payload JSONB NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Compound Indexes for Fast Queries & Settlement Audits
CREATE INDEX idx_ledger_account_history ON ledger_entries(account_id, created_at DESC, entry_id DESC);
CREATE INDEX idx_tx_idempotency ON transactions(idempotency_key);
CREATE INDEX idx_outbox_unprocessed ON outbox_events(event_id ASC);

---

5. API Interface Design & Data Contracts

1. Execute P2P Balance Transfer (POST /v1/wallets/transfers)

Request Headers

httpCopy
POST /v1/wallets/transfers HTTP/1.1
Host: api.wallet.platform.aws.internal
Authorization: Bearer sec_tok_live_09182374
Idempotency-Key: idemp_p2p_99a8b7c6-d5e4-4a3b-b2c1
Content-Type: application/json

Request Payload

jsonCopy
{
  "source_account_id": "acc_alice_101",
  "destination_account_id": "acc_bob_202",
  "amount_cents": 2500,
  "currency": "USD",
  "memo": "Dinner split from Friday tacos"
}

Response: 200 OK

jsonCopy
{
  "transaction_id": "tx_p2p_771829384",
  "status": "COMMITTED",
  "source_account_id": "acc_alice_101",
  "destination_account_id": "acc_bob_202",
  "amount_cents": 2500,
  "currency": "USD",
  "source_remaining_balance_cents": 12500,
  "idempotency_key": "idemp_p2p_99a8b7c6-d5e4-4a3b-b2c1",
  "created_at": "2026-09-16T14:45:10.112Z"
}

2. Balance Inquiry with Monotonic Fencing (GET /v1/wallets/{account_id}/balance)

httpCopy
GET /v1/wallets/acc_alice_101/balance HTTP/1.1
Host: api.wallet.platform.aws.internal
Authorization: Bearer sec_tok_live_09182374

Response: 200 OK

jsonCopy
{
  "account_id": "acc_alice_101",
  "currency": "USD",
  "total_balance_cents": 12500,
  "locked_cents": 0,
  "available_balance_cents": 12500,
  "version": 42,
  "last_updated": "2026-09-16T14:45:10.112Z"
}

---