Design a Stock Exchange Matching Engine

This page is one interview loop in three rounds. All three rounds design the same system. Each round opens with the interviewer raising the scope, and the design from the round before has to evolve to meet it.

You can start at any round. Rounds 2 and 3 open with a "Where we left off" summary that catches you up.

Loop Opener: What Is a Matching Engine?

An Auctioneer Who Never Sleeps

Picture an auction hall for one stock. Buyers shout the highest price they'll pay. Sellers shout the lowest price they'll accept. The auctioneer writes every shout on a board, and whenever a buyer's price meets a seller's price, the auctioneer bangs the hammer and a trade happens. When two buyers shout the same price, the one who shouted first goes first.

A matching engine is that auctioneer as software. It never gets tired, it never plays favorites, and it has to do the whole thing thousands of times a second.

A few words carry this whole loop. Each gets one line now:

Synthesizing vector architecture diagram...

Orders flow in from both sides. Two kinds of news flow out: a private execution report to the firms involved in each trade, and public market data that everyone sees at the same moment. Trades also go to clearing, where the money and shares actually change hands later.

What Makes It Hard

1. Fairness is the product. An exchange sells one thing: a fair, predictable place to trade. If two firms send the same order and the slower one gets filled first, the exchange has failed, however fast it is.
2. The result must be reproducible. Given the same orders in the same sequence, the engine must produce exactly the same trades, today and when a regulator asks about this day years from now.
3. It's fast and bursty. At the opening bell, thousands of orders arrive in a millisecond, and the busiest instrument can't be split across machines (you'll see why in step 1.2).
4. Nothing acknowledged may be lost. If we told a firm "your order is in" or "you bought 800 shares", that must survive any single crash.

So every component in this loop defends one of three rules:

The Question the Whole Loop Answers

How do we match orders fairly, deterministically and fast, and never lose one?

The answer gets sharper every round:

• Round 1: an in-memory order book, one thread per book, a sequencer, and a replicated journal we can replay.
• Round 2: make it microseconds and failure-proof: no allocation, lock-free handoff, a hot standby in lockstep, and market data that participants can recover.
• Round 3: the rules of a market: an audit trail regulators can replay, fairness by design, a disaster-recovery site that loses no trades, and where the machines should live.

The surprising lesson, which we'll earn step by step: the fastest design and the most correct design are the same one. One thread per order book, fed by a sequencer, with state rebuilt by replaying a journal.

---

Round 1 · Mid-level · "A Small Exchange for 100 Instruments"

~35 min · SDE II (L5) · 1 region, 3 AZs · ~1,000 orders/s peak · millisecond latency · no acknowledged order lost

R1.1 Establish Design Scope

The interviewer says: "We're launching a small regional exchange. Design the part that takes orders and matches them." Before we draw anything, we ask questions, and we say out loud what each answer changes.

Out of scope for this round:

• Microsecond latency, hot standby, circuit breakers. Round 2.
• Clearing and settlement. We hand trades to a clearing firm in a file; the clearing house moves money and shares.
• Auctions (opening and closing crosses). The session starts with an empty book and continuous trading.
• Order types beyond limit, market and cancel (stops, icebergs, pegs, modify). A modify is a cancel plus a new order for now.

R1.2 Functional Requirements, Derived Step by Step

Not yet: microsecond latency, a hot standby, price collars and halts. The interviewer may bring these back.

R1.3 Non-Functional Requirements: the Questions

Numbers come in R1.7. For now, the questions, in the order that matters for an exchange:

• Determinism and fairness. The same inputs in the same order must give the same trades. Two orders at the same price must be served in arrival order, and "arrival" must mean one thing everywhere.
• Durability of accepted orders. Once we acknowledge an order, a cancel or a trade, it must survive a crash of any one machine, and even the loss of an Availability Zone (AZ).
• Availability during trading hours. Outside trading hours we can do maintenance. During them, every minute down is a minute nobody can trade.
• Latency. Milliseconds, measured from the order reaching our front door to the execution report leaving it.

R1.4 The API

Real exchanges speak compact binary protocols (Round 2). A small venue can start with JSON over HTTPS for orders and a WebSocket for streams, and that's what we show first, because it makes the fields easy to read.

Submit an order. The member sends its own client order ID: a unique ID the firm generates for each new order. It doubles as an idempotency key: if the firm resends the same ID (after a timeout, say), we return the first result instead of entering a second order.

httpCopy
POST /v1/orders HTTP/1.1
Host: orders.exchange.example.com
Authorization: Bearer <member session token>
Content-Type: application/json

{
  "client_order_id": "GSX-20260927-000417",
  "instrument": "XYZ",
  "side": "BUY",
  "type": "LIMIT",
  "price_ticks": 10005,
  "quantity": 800,
  "time_in_force": "DAY"
}

price_ticks is the price in ticks: with a one-cent tick, 10005 means 100.05. Prices are integers so that "equal price" is exact (a floating-point price like 100.05 can't be stored exactly in binary, and two prices that print the same might not compare equal).

httpCopy
HTTP/1.1 200 OK
Content-Type: application/json

{
  "order_id": "XYZ-5001",
  "client_order_id": "GSX-20260927-000417",
  "sequence": 5001,
  "status": "FILLED",
  "fills": [
    { "trade_id": "T-5001-1", "price_ticks": 10004, "quantity": 200 },
    { "trade_id": "T-5001-2", "price_ticks": 10004, "quantity": 400 },
    { "trade_id": "T-5001-3", "price_ticks": 10005, "quantity": 200 }
  ],
  "leaves_quantity": 0
}

sequence is the order's position in the exchange's one sequence of events (step 1.3). leaves_quantity is what's still resting. The trade IDs are built from the sequence number and a fill counter, so a replay produces the same IDs.

Cancel an order:

httpCopy
DELETE /v1/orders/XYZ-4102 HTTP/1.1
Host: orders.exchange.example.com
Authorization: Bearer <member session token>

It returns 200 with "status": "CANCELLED" and the cancelled quantity, or 409 with "reason": "TOO_LATE_TO_CANCEL" if the order was already filled. A cancel is itself an input with its own sequence number, so "did the cancel or the fill win?" has exactly one answer.

Streams (WebSocket): /v1/stream/executions sends the member's own execution reports (fills can arrive long after the order was entered). /v1/stream/marketdata?instrument=XYZ sends public market data. Every message carries a sequence number so the client can spot a gap (step 1.5).

Recap

• One order call with a client order ID that makes resends safe; one cancel call; two streams.
• Prices are integer ticks.
• Every input gets a sequence number, and so does every output.
• Correctness first: price-time priority, and nothing acknowledged may be lost.

Let's build it, starting with the simplest thing that works.

R1.5 Design Evolution: From a Database Table to a Journaled Engine

Every step below follows the same pattern: a problem, your turn to think, the answer, and what the answer costs us. The cost is always the next problem.

Step 1.0: The Baseline

Orders go into a SQL table. A matching service, on each new order, queries the best opposite order and updates both rows.

Synthesizing vector architecture diagram...

What's good about it: everyone understands it, and the database keeps the data safe.

What it costs us: every match is several round trips to the database, and three API instances run matches for the same instrument at the same time. The next six steps are the ways it breaks.

Step 1.1: Matching by SQL Is Slow and Racy

The book, drawn. Instrument XYZ, one tick = 0.01. Numbers in the level boxes are the total quantity at that price.

Synthesizing vector architecture diagram...

Read each row left to right: a price level, then its orders from oldest to newest. The best bid is 100.02 and the best ask is 100.04, so the spread is two ticks. The dotted arrow is the index: a cancel of B2 jumps straight to its node.

The same book as a table, the way a trader's screen shows it:

Matching, as pseudocode. The trade price is always the resting order's price: the incoming order agreed to pay up to its limit, and the resting order was there first at its own price.

textCopy
on_new_order(o):                               # o.side, o.type, o.price_ticks, o.qty, o.tif
    opp = asks if o.side == BUY else bids
    while o.qty > 0 and opp is not empty:
        level = opp.best()                     # lowest ask or highest bid
        if o.type == LIMIT and not crosses(o.price_ticks, level.price):
            break                              # the best opposite price is worse than our limit
        while o.qty > 0 and level.queue is not empty:
            r = level.queue.head               # oldest resting order at this price
            q = min(o.qty, r.qty)
            emit TRADE(price = level.price, qty = q, maker = r.id, taker = o.id)
            o.qty = o.qty - q
            r.qty = r.qty - q
            level.total = level.total - q      # every fill, partial or full
            if r.qty == 0:
                level.queue.unlink(r); index.remove(r.id); pool.release(r)
        if level.queue is empty:
            opp.remove_level(level)
    if o.qty > 0 and o.type == LIMIT and o.tif == DAY:
        rest(o)                                # append to its level's tail, add to index
        emit ACCEPTED(o.id, leaves = o.qty)
    else if o.qty > 0:
        emit CANCELLED(o.id, qty = o.qty)      # IOC or market remainder never rests

crosses(limit, resting_price):
    BUY:  resting_price <= limit
    SELL: resting_price >= limit

on_cancel(id):
    r = index.get(id)
    if r is missing: emit CANCEL_REJECT(id, TOO_LATE_TO_CANCEL); return
    level = r.level
    level.queue.unlink(r); level.total = level.total - r.qty
    index.remove(id); pool.release(r)
    if level.queue is empty: side_of(r).remove_level(level)
    emit CANCELLED(id, qty = r.qty)

One subtle bug catches many first versions: reducing the level's total only on partial fills, and counting on the unlink to handle full fills. By the time the unlink runs, the order's quantity is already zero, so the total stays too high, the empty level is never removed, and the "best price" points at a ghost level. The pseudocode reduces the total on every fill.

A worked example. Order B9 arrives: buy 800 at 100.05, DAY, sequence 5001.

1. Best ask is 100.04, which is ≤ 100.05, so it crosses. The head is S1 (200): trade 200 at 100.04. S1 is done and unlinked. B9 has 600 left.
2. Next in the queue is S3 (400): trade 400 at 100.04. S3 is done. The 100.04 level is empty and removed. B9 has 200 left.
3. Best ask is now 100.05, still ≤ 100.05. The head is S2 (700): trade 200 at 100.05. S2 has 500 left and keeps its place at the head.
4. B9 is filled. Nothing rests. Three trades, which are exactly the three fills in the API response above.

After B9, the best ask is 100.05 (500, S2) and the spread is three ticks. Then a cancel for B2 arrives: the index finds B2's node, the 100.01 level becomes empty and is removed, and the best bid is unchanged at 100.02.

Step 1.2: Two Threads Matched Against the Same Resting Order

We group the 100 instruments into 4 instrument groups of 25, each owned by one matching thread. Four is a choice: it's plenty for the load and keeps one slow instrument from delaying all 100.

Synthesizing vector architecture diagram...

Each book has exactly one writer. Threads never share a book, so there are no locks, and each thread sees its instruments' inputs in the same order every time.

Step 1.3: Which Order Came First?

The sequenced input is the heart of the whole design. Everything that follows (the journal, replay, the standby, the audit trail) is just a copy of this one stream.

Step 1.4: The Engine Crashed. Where Is the Book?

What's in a journal record. Every input becomes one fixed-size binary record. In Round 1 we reserve 128 bytes per record, which is generous:

Who may write the journal: fencing with an epoch. If the engine instance is slow rather than dead, and we start a replacement, two engines could append to the journal at once. Each engine incarnation has an epoch number. A new engine first asks the journal nodes to move to epoch e + 1; a node that has moved rejects appends from epoch e. The new engine needs two of the three nodes to agree, and the old engine needs two of three to accept its appends. Any two pairs out of three nodes share at least one node, so once the new engine holds two, the old one can never collect two acceptances again, even if it's still running and hasn't heard the news. Then the new engine reads the journal from the nodes it holds, takes the longest copy, and re-replicates any tail that's on only one node. This is the same majority logic consensus protocols use (Distributed Consensus: Raft and Paxos).

Replay to a point in time. A regulator asks: "What did XYZ's book look like at 11:02:17?" We don't replay the whole day. We load the snapshot taken before that moment (at most 10 minutes earlier), replay the journal up to the last input stamped before 11:02:17, and stop. With snapshots every 10 minutes, no question needs more than 10 minutes of inputs replayed. That's the general cure for event-sourced systems with long histories: snapshots bound replay, and the event log keeps every earlier state reachable.

Primitives: Write-Ahead Log and LSM Trees (append, flush, then apply) · Event Sourcing and CQRS (state as a fold over events, snapshots)

Step 1.5: Participants Need to See the Book

Synthesizing vector architecture diagram...

The engine writes each message once. Fan-out, gap-fill and snapshots all happen on the market-data servers, so a slow or recovering subscriber costs the engine nothing.

Step 1.6: An Order for More Money Than the Member Has

Round 1 Step Summary

R1.6 Architecture v1

Now the concepts get AWS names.

Synthesizing vector architecture diagram...

Follow an order: the NLB passes the TLS session to the member's home gateway, which authenticates and runs risk checks, then sends it to the engine. The sequencer numbers it, the journal nodes store it (two of three must flush), a matching thread matches it, and only then do the execution report and market data go out. The spare engine in another AZ does nothing until it's needed.

The pieces:

• Gateways: 3 EC2 instances (c7i.large), one per AZ, behind a Network Load Balancer (a layer-4 load balancer; it terminates TLS here). A member's session is routed to its home gateway; we give each gateway its own listener port so the mapping is explicit.
• Engine: one c7i.2xlarge (8 vCPUs) in AZ a. One sequencer thread, four matching threads, one output thread that talks to the gateways and market-data servers, and one thread writing to the journal nodes.
• Journal nodes: 3 c7i.large, one per AZ, each with an EBS io2 volume (block storage that AWS replicates inside its AZ, with sub-millisecond latency). Each appends, flushes, and acknowledges. They also seal and upload finished journal segments to S3 every few minutes.
• Market-data servers: 2 c7i.large in two AZs, behind their own NLB.
• Spare engine: one c7i.2xlarge in AZ b, running but not processing. On failover it fences the journal (epoch e + 1), loads the latest snapshots from S3, replays, and takes over.
• Post-trade: a small writer that copies trades from the output stream into Aurora PostgreSQL for the end-of-day clearing file. It is off the critical path: if it falls behind, trading doesn't notice.

The trades table, keyed so a replay can't insert a trade twice:

sqlCopy
CREATE TABLE trades (
    trade_id        TEXT PRIMARY KEY,          -- 'T-5001-1': input sequence + fill counter, deterministic
    trade_date      DATE NOT NULL,
    sequence        BIGINT NOT NULL,
    instrument      TEXT NOT NULL,
    price_ticks     BIGINT NOT NULL CHECK (price_ticks > 0),
    quantity        BIGINT NOT NULL CHECK (quantity > 0),
    buy_member      TEXT NOT NULL,
    sell_member     TEXT NOT NULL,
    buy_order_id    TEXT NOT NULL,
    sell_order_id   TEXT NOT NULL,
    aggressor_side  CHAR(1) NOT NULL CHECK (aggressor_side IN ('B','S'))
);
CREATE INDEX ON trades (trade_date, instrument);

The writer inserts with ON CONFLICT (trade_id) DO NOTHING, so replaying the output after a crash is harmless. Nothing references this table by foreign key, so the retention job (delete rows older than the retention period, by trade_date, in batches) is never blocked.

Trace 1: a limit order rests, then a marketable order matches it.

Synthesizing vector architecture diagram...

The engine may compute the match while the journal flush is in flight, but it releases nothing about input 5001 until two journal nodes have it. The member hears "filled" only after the input that caused the fill is safe.

Trace 2: a crash and replay. At 11:14:03 the engine instance stops responding. The spare's health check misses heartbeats for 10 seconds and starts a takeover:

1. The spare asks the journal nodes to move to epoch 8. Nodes b and c agree; node a (same AZ as the dead engine) doesn't answer. Two of three is enough, and the old engine can no longer get two acknowledgments.
2. It reads the journal from b and c. The longest copy ends at sequence 612,877; node b's copy is one record shorter, so it copies record 612,877 to b.
3. It loads the four snapshots from 11:10 (taken at about sequence 592,000) from S3 and replays about 21,000 inputs, well under a second of work.
4. It regenerates the outputs for the replayed inputs and resends the tail to the gateways and market-data servers, which drop sequence numbers they've already seen.
5. Gateways reconnect to the new engine (they learn its address from the journal nodes, which record the current epoch's owner) and resend any input they hadn't had acknowledged. The sequencer recognizes resends by (gateway_id, gateway_seq).

Members saw about 15 to 20 seconds of 503s and no lost acknowledgment.

R1.7 Numbers

Traffic.

$$\text{Average} = \frac{2{,}000{,}000 \text{ inputs}}{23{,}400 \text{ s}} \approx 85.5/\text{s}$$

The open and close are the busy minutes. We assume the peak second is 10 times the average and plan for a round number:

$$85.5 \times 10 = 855/\text{s} \;\Rightarrow\; \text{plan for } 1{,}000 \text{ inputs/s}$$

We assume the busiest instrument carries 10% of the flow: about 100 inputs a second at peak on one book. We assume 10% of inputs produce a trade: about 200,000 trades a day.

Journal.

$$2{,}000{,}000 \times 128 \text{ B} = 256 \text{ MB/day} \qquad 256 \text{ MB} \times 252 \text{ trading days} \approx 64.5 \text{ GB/year}$$

Peak write rate: $1{,}000 \times 128 \text{ B} = 128 \text{ KB/s}$ to each of three journal nodes. Trivial for any disk; what matters is the latency of each flush.

Book memory per instrument. We size each instrument's pool for 5,000 resting orders (a choice with headroom). Each order node is 64 bytes, the index is an open-addressing hash table kept at most half full, and price levels are small:

For 100 instruments: about 64 MB, sized for the worst case; the live books are far smaller. It fits easily in an 8 vCPU instance's 16 GB of memory.

Replay time. A snapshot every 10 minutes means a replay covers at most 10 minutes of inputs. At the peak rate that's $600 \text{ s} \times 1{,}000 = 600{,}000$ inputs. We estimate a replay thread applies at least 500,000 inputs a second (it reads a sequential file and does in-memory work, with no network waits; an estimate to benchmark), so at most about 1.2 seconds. A whole day's 2 million inputs would take about 4 seconds, which is why snapshots matter less at this scale than in Round 2.

Snapshot pause. Each matching thread serializes its 25 books between two inputs. At about 16 MB per group at worst and several GB/s of memory bandwidth, that's a pause of a few milliseconds every 10 minutes per group (an estimate). At millisecond latency targets, that's acceptable; Round 2 removes it.

Latency (P99 estimate, dependent steps add, parallel steps take the max).

The match and the journal flush run in parallel, so they count as max(0.1, 1.5) = 1.5 ms, not their sum.

Availability. Trading hours are $21 \times 6.5 \text{ h} = 136.5$ hours, or 8,190 minutes, a month. At a 99.95% target that's $8{,}190 \times 0.0005 \approx 4.1$ minutes of downtime a month. One engine failover (about 20 seconds) uses about 8% of it.

Monthly cost (us-east-1 on-demand prices, 730 hours a month):

The data-transfer line: we assume 200 subscribers each receive about 1.2 market-data messages per input at about 200 bytes of JSON each, during trading hours only: $85.5 \times 1.2 \times 200 \text{ B} \approx 20.5$ KB/s per subscriber, times 200 subscribers is 4.1 MB/s, times 23,400 s times 21 days ≈ 2.0 TB a month.

The biggest line is the io2 IOPS we provision on the three journal volumes. It buys what we actually need: flushes that complete in well under a millisecond.

R1.8 Trade-Offs

Drill: The Banking Ledger That Took 3 Days to Replay (snapshots bound replay, as in step 1.4; why a replayable history beats state-in-place for audit, as in this table)

R1.9 Failure Modes

R1.10 Pillar Check

R1.11 Round 1 Rubric and Follow-Ups

What a strong mid-level (L5) answer shows

• Draws the order book correctly: sorted levels, FIFO queues, an index for O(1) cancel, and matching at the resting order's price.
• Explains why locks break fairness, and chooses one writer per book.
• Defines "first" with a sequencer, not with timestamps from many machines.
• Journals inputs before releasing outputs, and recovers by replaying from a snapshot.
• Publishes market data as a sequenced stream with gap recovery, and never lets a consumer slow the engine.
• Puts risk checks before the sequencer.

Follow-up questions

1. "Why journal the inputs and not the trades?" Answer: the inputs determine everything; the trades are a consequence. From inputs we can rebuild the book, the trades and every market-data message. From trades alone we couldn't rebuild resting orders or cancels. And inputs are the smaller, simpler record: one per order, whatever it caused.

2. "A member sends a cancel and, a millisecond later, its order is filled. The member says the cancel was first." Answer: both are inputs with sequence numbers, and the journal shows which was sequenced first. If the fill's input (the other member's order) has the lower number, the cancel was too late, and we reply TOO_LATE_TO_CANCEL. The sequence number is the whole argument.

3. "Why not use Kafka as the journal?" Answer: we could: a topic with one partition, replication factor 3 and acks=all gives an ordered, replicated log, and Round 3 uses a Kafka stream for post-trade. For the journal itself we want control over two things: the flush policy (Kafka relies on replication and doesn't flush each write to disk by default) and the epoch fence tied to our engine's takeover. A small purpose-built journal is simple enough to own.

Interview gotchas from this round's wrong answers

---

Round 2 · Senior · "10,000 Symbols, Microseconds, and No Data Loss"

~40 min · Senior SDE (L6) · 1 colocation site, hot standby per engine · 500M orders/day, ~214K/s peak second · matching P99 < 50 µs (target) · RPO 0

R2.0 Where We Left Off

This is what the candidate says aloud in the first 60 seconds of Round 2. If you're starting here, it's everything you need from Round 1.

Round 1 in 60 seconds. "We built a small exchange for 100 instruments: 2 million inputs a day, about 1,000 a second at peak, millisecond latency, in one AWS Region across three AZs. Each instrument has an in-memory order book: sorted price levels, a FIFO queue at each level, and an order-ID index for O(1) cancel; trades happen at the resting order's price. One thread owns each book, so there are no locks and the result doesn't depend on the scheduler. A sequencer gives every input one gap-free number and a timestamp; 'first' means 'lower sequence number'. Every sequenced input is journaled to two of three journal nodes, one per AZ, before any output about it is released, and a new engine fences the old one with an epoch and rebuilds by loading snapshots and replaying. Market data is a sequenced incremental stream fanned out by separate servers with gap-fill and snapshots. Pre-trade risk runs in each member's home gateway, before the sequencer. About $2,400 a month. Open costs: 3 ms is far too slow for the next customers, a failover takes about 20 seconds, and one instance holds every book."

Architecture v1, compact

Synthesizing vector architecture diagram...

Round 1 in one picture: one sequence, one writer per book, and a journal that makes the books rebuildable.

Round 1 step summary

Open costs: milliseconds of latency, mostly the cross-AZ journal; a 20-second failover; one engine instance for everything.

R2.1 The Scope Raise

Interviewer: "The regional exchange is becoming a national equities exchange. Ten thousand symbols, 500 million orders and cancels a day, and the first and last minutes of the day are brutal. Our members are trading firms that colocate their servers in our data center, and they measure us in microseconds. We can't lose an order, and a failover must look like a hiccup, not an outage. Market data goes to about a thousand direct subscribers, who must be able to recover anything they miss. And after a bad afternoon at another venue, we need protection against runaway prices."

Scope change

R2.2 What Breaks in the Round 1 Design

R2.3 New Requirements and API Additions

Binary order entry. Members now send fixed-size binary messages over a persistent TCP session instead of JSON over HTTPS. Parsing is a few loads from known offsets instead of scanning text, and the message is about a fifth of the size. The field layout below is ours; it follows the style of real exchange protocols such as Nasdaq's OUCH (order entry) and ITCH (market data), and of Simple Binary Encoding (SBE), a binary message standard from the FIX Trading Community.

Enter Order, 36 bytes, little-endian:

Cancel Order is 21 bytes: msg_type X, session_seq, client_order_id, and an optional quantity to reduce by (0 = cancel all).

The sequenced input record (the journal record, and what the standby receives), exactly 64 bytes, one cache line:

Round 1's 128-byte record carried a gateway ID and sequence; here session_id plus session_seq identifies every input uniquely, so a resend after a failover is recognized. kind also covers the non-order inputs the sequencer injects: TIMER (step 2.6), BAND_UPDATE (step 2.6), CHECKPOINT (step 2.4) and operator CONTROL messages.

Market data. UDP packets, each with a 20-byte header (channel_session 10 B, first_seq uint64 8 B, msg_count uint16 2 B, in the style of Nasdaq's MoldUDP64) followed by one or more messages. Every message has a sequence number: the header's first_seq plus its position in the packet.

We publish one channel per engine partition (16 channels, step 2.7), each on two independent feeds, A and B.

Recovery requests. Over TCP to a recovery server:

jsonCopy
{ "type": "RETRANSMIT", "channel": 7, "from_seq": 48211903, "count": 212 }

It returns the messages as originally sent, byte for byte. At most 10,000 messages per request and a per-member request rate (our choice), so recovery traffic can't become its own storm.

jsonCopy
{ "type": "SNAPSHOT", "channel": 7 }

It returns every resting order on the channel, stamped with as_of_seq: the last channel sequence number the snapshot includes.

Collar parameters per symbol, in the daily reference file (YAML):

yamlCopy
symbol_id: 4417
ticker: XYZ
tick: 100                      # price units: 0.01 in 4-decimal fixed point
luld_tier: 1
max_order_qty: 250000
max_order_notional_usd: 50000000
entry_collar_pct: 10           # reject limit orders priced more than 10% through the band's reference price

R2.4 Design Evolution: Microseconds and No Loss

Step 2.1: Millisecond Pauses From Memory Allocation

An order node, laid out to fit one cache line:

Step 2.2: Threads Hand Work Over Slowly

Synthesizing vector architecture diagram...

One partition's pipeline on one engine host. The matcher and the journal read the same input ring at the same time; the publisher waits for both before releasing anything.

Step 2.3: Durability Costs a Flush per Order

Primitive: Write-Ahead Log and LSM Trees (group commit: many appends, one flush)

Step 2.4: The Primary Died Mid-Burst

Why three arbiter nodes, and what a partition does to them. If one arbiter node is cut off from the other two, it's a minority: it can't commit anything, so it can't promote anybody. The two connected nodes still form a majority and keep working. Three nodes survive one failure; five would survive two, at the cost of two more servers and a slightly slower commit. The arbiter isn't on the order path and is consulted only at failover, so we choose three.

Synthesizing vector architecture diagram...

An engine's roles. Every arrow into Primary or Solo goes through the arbiter, so at most one engine per partition can release output in any epoch.

Primitive: Distributed Consensus: Raft and Paxos (the arbiter's majority and terms)

Drill: The Network Partition That Elected Two Leaders (why a minority can't promote anyone; three nodes versus five)

Step 2.5: Participants Lost Market-Data Packets

Synthesizing vector architecture diagram...

Arbitration hides most losses for free. Only a loss on both feeds costs a round trip to the retransmission server, and the subscriber keeps buffering the live feed while it waits.

Step 2.6: A Huge Market Order Swept the Book Down to 0.01

The swept book, with and without bands. Bids for a thin stock, reference price 20.00, Tier 2, so a 10% band: 18.00 to 22.00.

Step 2.7: One Symbol's Burst Slows the Others

Primitive: Database Sharding and Partition Keys (partitioning by load, isolating hot keys)

Drill: One Customer, One Shard, One Outage (a hot symbol gets its own partition; cross-partition questions go to the post-trade store)

Round 2 Step Summary

R2.5 Architecture v2

Synthesizing vector architecture diagram...

Follow an order: a gateway checks it and sends it to its symbol's partition on a primary host. The sequencer numbers it; the matcher, the local journal and the standby all get it at once; results are released to the gateway and both feeds only when the journal and the standby have it. The arbiter is consulted only when something fails.

Component counts: 10 gateways (8 needed for about 4,000 sessions at 500 each, plus 2 spare), 8 engine hosts (4 primary, 4 standby; 16 partitions), 4 recovery servers (2 on each feed network), 3 arbiter nodes, 2 post-trade servers: 27 servers.

Latency budget, wire to wire (a target to measure, not a promise any platform makes by itself; dependent stages add, parallel stages take the max):

The journal write and the standby round trip are the two stages to watch: they set the release time. If they ever sat in series instead of in parallel, the budget would be 57 µs and blown.

Trace 1: an order at the open.

Synthesizing vector architecture diagram...

The publisher is the gate: it holds the matcher's outputs until the journal core says the input is durable locally and on the standby. At 09:30:00 the journal batch may carry dozens of inputs, but each is released as soon as its own batch completes.

Trace 2: a failover. The primary host for partitions 5 to 8 loses power at 09:30:02.000000.

About 5 ms end to end (a target, proven in failover drills, never assumed). No released message is lost and none is released twice with different content, because the new primary computed the same outputs for the same inputs.

Trace 3: a packet gap recovered. This is the sequence diagram in step 2.5: a burst drops sequence 100 on both feeds; the subscriber buffers 101, asks a retransmission server for 100, applies it, then applies 101. With a retransmission server in the same site, recovery takes well under a millisecond (an estimate).

R2.6 Numbers and Cost

Traffic.

$$\text{Average} = \frac{500 \times 10^6}{23{,}400 \text{ s}} \approx 21{,}367 \text{ inputs/s exchange-wide}$$ $$\text{Peak second} = 21{,}367 \times 10 \approx 213{,}675 \approx 214\text{K inputs/s exchange-wide}$$

Reconciling "200,000 a second per partition" with that: the 200K figure is not the exchange's average or even its busy-second rate spread over partitions. It's the burst rate one partition must absorb for a few milliseconds at the open (we assume the busiest partition can see 200 inputs within one millisecond), and so it's our capacity target per partition core: 200K inputs/s sustained, or 5 µs of work per input on average.

Can one core do 200K/s? A match that trades is budgeted at 8 µs P99; most inputs (90%) are adds or cancels that don't trade. If a trading input averages 3 µs and a non-trading one 0.5 µs (estimates to benchmark), the average is $0.1 \times 3 + 0.9 \times 0.5 = 0.75$ µs, about 1.3 million inputs a second, so 200K/s leaves more than 6 times headroom.

Book memory, for 50 million resting orders across the exchange (the average of 5,000 per book × 10,000 books):

This fits easily in RAM. It does not fit in a CPU's last-level cache (tens to a few hundred MB), and it doesn't need to: the hot part of each book, the few levels near the best prices, stays in cache because it's touched constantly.

Journal.

$$500 \times 10^6 \times 64 \text{ B} = 32 \text{ GB/day} \qquad \times 252 \text{ days} \approx 8.1 \text{ TB/year}$$

Peak write rate: $214\text{K} \times 64 \text{ B} \approx 13.7$ MB/s exchange-wide; a 1 ms microburst on one partition is $200 \times 64 \text{ B} = 12.8$ KB in that millisecond. Bandwidth is trivial; what matters is each write's latency. Replication to the standbys is the same 13.7 MB/s, about 110 Mbps.

Ring memory: 65,536 slots × 64 B = 4 MB per input ring; 16 partitions × 2 rings (input and output) × 2 hosts ≈ 256 MB in total.

Replay time. A normal day is $500\text{M} \div 16 \approx 31.25$M inputs per partition. At an estimated 2M inputs a second per core (no network, sequential reads), a partition replays a whole day in about 16 seconds, and all partitions replay in parallel. The standby also feeds a snapshotter process on its host, which applies the stream to its own copy of the books and writes a snapshot every 10 minutes, so no engine thread ever pauses for a snapshot.

Market-data bandwidth. We assume 1.3 market-data messages per input (an add, delete or execution, plus the extra executions of trading inputs) at 40 bytes on average, and about 10 messages per packet at peak, so the 62 bytes of packet headers (20 B feed header, 42 B Ethernet, IP and UDP) add about 6.2 bytes per message:

$$214\text{K} \times 1.3 \times 46.2 \text{ B} \approx 12.9 \text{ MB/s} \approx 103 \text{ Mbps per feed at the busiest second}$$

Microbursts over a millisecond can run several times higher, so each feed network runs at 10 Gbps. Multicast versus unicast: with multicast we send that 103 Mbps once per feed. As TCP unicast to 1,000 subscribers it would be $1{,}000 \times 103 \text{ Mbps} = 103$ Gbps per feed, and twice that for A and B. The day's feed is $500\text{M} \times 1.3 \times 40 \text{ B} = 26$ GB of payload, which each retransmission server keeps in memory (so they're 64 GB machines).

Availability. Trading time is $252 \times 23{,}400 \text{ s} = 5{,}896{,}800$ s a year. At 99.999%:

$$5{,}896{,}800 \times 0.00001 \approx 59 \text{ seconds a year}$$

A 5 ms failover is 0.005 s. One Round 1-style 20-second recovery would use a third of the year's budget. Five nines is a target; one bad day (a determinism bug, a botched release) can spend it, which is why R2.8 and Round 3's runbook exist.

Hardware cost. Our servers sit in the colocation site, not in an AWS Region (step 2.1's scope answer). As a verified yardstick, here is what the same machines would cost as EC2 bare-metal instances at us-east-1 on-demand list prices; the owned-hardware line below it is a rough estimate.

Either way, the platform costs about the same as a few engineers. The expensive resources in this round are engineering discipline (determinism, zero allocation) and the network, not servers.

R2.7 Trade-Offs

Four ways to build a matching engine. Figures are rough orders of magnitude for one symbol's book, to show the shape of the trade-off, not benchmarks.

R2.8 Failure Modes

R2.9 Production Gotchas

R2.10 Pillar Check

R2.11 Round 2 Rubric and Follow-Ups

What a senior (L6) answer adds over L5

• Explains where microseconds go (allocation, handoffs, the kernel, flushes) and removes each one, with a budget that uses max() for parallel stages.
• Uses a lock-free ring with busy-polling pinned consumers, and knows about false sharing.
• States exactly what RPO 0 covers: released means on two hosts.
• Builds a lockstep standby, lists the determinism rules, and verifies them with digests.
• Fences failover so it holds while the news is still spreading, and handles the standby dying too.
• Designs market data for loss: multicast, A/B arbitration, retransmission and snapshots, and knows what AWS does and doesn't offer for multicast.
• Describes LULD-style bands accurately and drives time-based rules from sequenced timer inputs.
• Partitions by load, not by hash, and puts backpressure at the gateways.

Follow-up questions

1. "Why not have the standby vote, so the arbiter isn't needed?" Answer: with only two engines, neither can tell "the other died" from "the link between us died". Each would promote itself. A third party that both must ask (the arbiter's majority) is what breaks the tie. The arbiter isn't on the order path, so it costs nothing until something fails.

2. "A member says its order reached our gateway before a competitor's but was sequenced after it." Answer: that can happen: two gateways process at slightly different speeds, and the sequencer orders by arrival at the sequencer. The promise we make is that the path is the same for everyone (same gateway hardware, same cable lengths, R3.4), not that arrival at different gateways is compared. The journal shows the sequence and the gateway timestamps, so we can show exactly what happened.

3. "Could we make matching itself parallel for one very busy symbol?" Answer: not without giving up price-time priority. Each order's result depends on what the previous order did to the same book. What we can do is take everything else off that core (risk in gateways, journaling and publishing on other cores) and give the symbol a partition of its own.

Interview gotchas from this round's wrong answers

---

Round 3 · Architect · "A Regulated Venue: Fairness, DR, and the Cloud"

~45 min · Principal (L7) · primary site + journal bunker + far DR site + an AWS Region · up to 1.5B orders/day, ~641K/s peak second · no released trade lost, even in a site disaster · bit-identical replay for regulators

R3.0 Where We Left Off

This is what the candidate says aloud in the first 60 seconds of Round 3. If you're starting here, it's everything you need from Round 2.

Round 2 in 60 seconds. "We built a national equities exchange in one colocation site: 10,000 symbols, 500 million inputs a day, about 214,000 a second in the busiest second, with members' servers a few racks away. Symbols are split by measured load into 16 partitions; each is a pipeline of four pinned cores connected by lock-free rings: receive and sequence, match, journal and replicate, publish. Nothing allocates during the session. Every input goes to the local NVMe journal in batches and to a hot standby on another host, and its results are released only when both have it, so a released message survives the loss of any one host. The standby runs the same deterministic engine in lockstep with output suppressed; digests every second prove the two agree; a three-node arbiter hands out epochs, and a standby that suspects the primary stops acknowledging first, so the old primary can't release anything while the news spreads. Failover is about 5 ms as a target. Market data is multicast on feeds A and B with retransmission and snapshot servers; LULD bands and pauses run inside the engine on sequenced timer inputs; gateways push back when a partition's ring fills. About 42 µs P99 as a budget. Open costs: one site, audit data that exists but isn't managed as a record, paths that may not be equal, a trade handoff that's only a file, and capacity sized for normal days."

Architecture v2, compact

Synthesizing vector architecture diagram...

Round 2 in one picture: every released message is on two hosts, and one arbiter decides who may publish.

Round 2 step summary

Open costs: everything lives in one building; the journal is on local drives, not in a record store; nothing guarantees equal paths; clearing gets a file; and nothing is sized for an extreme day.

R3.1 The Scope Raise

Interviewer: "We're now a regulated national market, and the rules come with it. Regulators want a complete audit trail and the ability to replay any trading day, years later. Members are complaining about fairness: one firm's rack is closer to our gateways than another's. If our building is lost, a disaster-recovery site must take over, and the board says no trade may be lost. Clearing needs every trade, reliably, for settlement the next business day. The board also asks whether we can move all this to the cloud. And last spring, on a meme-stock day, volume tripled."

Scope change

R3.2 What Breaks in the Round 2 Design

R3.3 New Requirements and API Additions

Audit records. The sequenced input record (R2.3) is already the main audit record. Three more kinds join the journal and the archive:

Drop copies. A drop copy is a second, read-only stream of a member's own execution reports, across all its sessions, sent to the member's risk and back-office systems. Same binary format as execution reports, on a separate connection; it's how a member rebuilds its position after any failure of its own trading systems, and after our DR declaration.

The clearing contract. One message per trade on the post-trade stream (JSON on the stream; the clearing house's own format is produced by our clearing gateway):

jsonCopy
{
  "trade_id": "P12-00000048211903-01",
  "trade_date": "2026-03-03",
  "partition": 12,
  "partition_seq": 48211903,
  "symbol": "XYZ",
  "price": "100.0400",
  "quantity": 200,
  "buy":  { "member": "GSX", "order_id": "O-12-48211903", "clearing_firm": "CF-0417" },
  "sell": { "member": "MSQ", "order_id": "O-12-48190554", "clearing_firm": "CF-0091" },
  "executed_at_ns": 34200000412000
}

trade_id is built from the partition, the input's sequence number and the fill index, so every replay, every standby and the DR site produce the same ID for the same trade. Consumers treat it as an idempotency key.

The DR declaration procedure (a document, rehearsed with members):

1. Confirm the primary site is lost or can't be trusted (site staff, the colocation provider, our own monitoring).
2. The incident commander declares DR. Only a person can do this; no automation moves the market between sites.
3. Fence the primary site at the bunker (step 3.3).
4. DR engines take their missing tails from the bunker and verify the last checkpoint digests.
5. Members log on at the DR site (keeping connectivity there is a membership requirement) in a pre-open state and reconcile using drop copies.
6. Resting orders are cancelled (our policy), then trading reopens with an announcement.

R3.4 Design Evolution: The Rules of a Market

Step 3.1: Regulators Want to Replay March 3rd

Lifecycle, and why it doesn't fight the lock. A lifecycle rule moves segments to S3 Glacier Deep Archive after a year (Object Lock retention stays with the object version when its storage class changes) and a lifecycle rule with Expiration plus NoncurrentVersionExpiration removes versions once their 7-year retention has passed (Expiration alone would only add a delete marker on this versioned bucket). Before the retention date, S3 refuses to delete a locked version. Restores from Deep Archive take up to 12 hours (standard) or 48 hours (bulk), which is fine for a regulator's request that arrives with weeks of notice; the newest year stays in S3 Standard for quick answers.

Primitive: Event Sourcing and CQRS (the event log as the record; projections rebuilt from it)

Step 3.2: One Firm's Shorter Cable Gives It an Edge

Step 3.3: The Primary Site Is Lost

Latency against distance (release time = 10 + 5 + 2 + max(match, journal, standby, remote copy) + 10 µs, from the R2.5 budget):

Synthesizing vector architecture diagram...

Only the bunker is on the order path. The far site stays nearly current on its own and borrows the last few milliseconds from the bunker when it takes over.

Primitive: Cloud Disaster Recovery and Multi-Region Active-Active (RPO and RTO as designed numbers; synchronous only where required)

Step 3.4: Clearing Needs Every Trade Exactly Once

The trade store in the Region (Aurora PostgreSQL):

sqlCopy
CREATE TABLE cleared_trades (
    trade_id        TEXT PRIMARY KEY,               -- 'P12-00000048211903-01'; a duplicate insert does nothing
    trade_date      DATE NOT NULL,
    partition_no    SMALLINT NOT NULL,
    partition_seq   BIGINT NOT NULL,
    payload         JSONB NOT NULL,
    status          TEXT NOT NULL CHECK (status IN ('RECEIVED','SUBMITTED','ACCEPTED','REJECTED')),
    submitted_at    TIMESTAMPTZ,
    accepted_at     TIMESTAMPTZ
) PARTITION BY RANGE (trade_date);

Monthly partitions are detached and exported to S3 after two years (the journal archive can regenerate them anyway). Nothing references this table by foreign key, so retention never blocks.

Primitive: Change Data Capture and the Outbox Pattern (publish from the committed log, not from the application's memory)

Step 3.5: Can We Run in the Cloud?

Step 3.6: Meme-Stock Day, 3× Volume

Round 3 Step Summary

R3.5 Global Architecture

Synthesizing vector architecture diagram...

The order path never leaves the primary site except for one synchronous hop to the bunker. Everything that can tolerate milliseconds (clearing, archive, replay, slow market data) runs in the Region, fed over Direct Connect.

Component counts: primary site 41 servers (16 gateways, 16 engine hosts as 8 primary and 8 standby for 32 partitions, 4 recovery servers, 3 arbiter nodes, 2 relay and post-trade servers); DR site the same 41; bunker 3 journal servers: 85 servers on-site. Direct Connect: two 10 Gbps dedicated connections from each site on separate devices, 4 in total.

Trace 1: a regulator replay.

Synthesizing vector architecture diagram...

The replay proves itself as it goes: every second's digest must equal the one the live engine published that day. A mismatch would stop the report, not produce a wrong one.

Trace 2: a site failover (the primary building loses power at 14:02; timings are drill targets):

About an hour in drills, inside the 2-hour documented target. The 15 minutes before declaration is deliberate: moving a market between sites on a false alarm is worse than a few minutes of certainty.

Trace 3: a trade reaching clearing.

Synthesizing vector architecture diagram...

The trade is only "handed off" at the clearing house's acceptance. Every earlier hop can repeat safely because the trade ID is the same every time.

R3.6 Numbers and Cost

Extreme-day traffic.

$$\text{Average} = \frac{1.5 \times 10^9}{23{,}400 \text{ s}} \approx 64{,}103/\text{s} \qquad \text{Peak second} = 64{,}103 \times 10 \approx 641\text{K/s}$$ $$\text{Plan} = 2 \times 641\text{K} \approx 1.28\text{M/s} \qquad \frac{1.28\text{M}}{32 \text{ partitions}} = 40\text{K/s per partition} = 20\% \text{ of } 200\text{K/s}$$

Hosts. 32 partitions at 4 per host is 8 primary and 8 standby engine hosts: 16 per site. Gateways: we estimate one gateway host handles 100K messages a second (to be load-tested); $1.28\text{M} \div 100\text{K} \approx 13$, plus 3 spare = 16. Recovery servers keep a whole day's feed in memory, 78 GB on an extreme day (below), so they move from 64 GB to 128 GB machines.

Journal and archive. An extreme day is $1.5\text{B} \times 64 \text{ B} = 96$ GB of journal. For a year we assume 240 normal days and 12 extreme ones:

$$240 \times 32 \text{ GB} + 12 \times 96 \text{ GB} = 7{,}680 + 1{,}152 = 8{,}832 \text{ GB} \approx 8.8 \text{ TB/year of journal}$$

The published market data is archived too, at $500\text{M} \times 1.3 \times 40 \text{ B} = 26$ GB on a normal day and 78 GB on an extreme one:

$$240 \times 26 + 12 \times 78 = 6{,}240 + 936 = 7{,}176 \text{ GB} \approx 7.2 \text{ TB/year}$$

About 16 TB a year raw. We assume fixed-size binary records compress about 3× (to be measured): ~5.3 TB a year stored. Kept 7 years, the archive holds $5.3 \times 7 \approx 37$ TB at steady state.

Object Lock adds no storage charge of its own. The audit archive of a national exchange costs less than one server.

Bandwidth to the bunker and the DR site at the extreme peak: $641\text{K} \times 64 \text{ B} \approx 41$ MB/s ≈ 330 Mbps. Market data at that peak: $641\text{K} \times 1.3 \times 46.2 \text{ B} \approx 38.5$ MB/s ≈ 308 Mbps per feed. Both fit 10 Gbps links with room for millisecond bursts.

Replay. An extreme day is $1.5\text{B} \div 32 \approx 47$M inputs per partition: about 23 seconds per partition at 2M inputs/s, all partitions in parallel. The nightly verification replay is minutes of compute.

DR link latency is the table in step 3.3: 10 µs per route kilometre round trip, plus equipment. The bunker's ~110 µs is the single largest item in the new release budget:

Cloud market-data egress. We assume 5,000 non-colocated subscribers at about 20 KB/s each (a conflated feed) during trading hours: 100 MB/s × 23,400 s × 21 days ≈ 49.1 TB a month. At list prices: 10,240 GB × $0.09 + 38,900 GB × $0.085 ≈ $4,230.

Monthly cost in the Region (us-east-1 list prices, 730 hours a month):

On-site, rough and unverified. 85 servers ($2.2M), low-latency networks at two sites plus bunker equipment ($1M): about $3.2M, or ~$66K a month over 4 years. Add colocation space, power and cross-connects at three sites, and wavelength or dark-fibre circuits to the bunker and the DR site: very roughly $60K to $100K a month. About $130K to $170K a month. As a yardstick, just the 64 engine and gateway hosts as c7i.metal-24xl instances would list at about $200K a month, so owning is not the expensive option here; latency is the reason the core stays put, not price.

R3.7 Trade-Offs

Closing the loop. The opening question was: how do we match orders fairly, deterministically and fast, and never lose one? The answer: one order of events (a sequencer per partition, and equal paths to it), same inputs, same outputs (one writer per book, no clocks or randomness inside the engine, verified by digests every second and replayable years later with the same build), and nothing released is lost (release only after the input is on the standby, the local journal and the bunker). Speed came from the same choices: no locks, no allocation, no waiting on anyone who doesn't need to be waited on. The one place we deliberately made it slower, the bunker, is where the board chose safety over microseconds, with the number in front of it.

R3.8 Failure Modes

R3.9 Runbook and Incident Response

Golden signals OPS 8 · REL 6

Opening procedure (every trading day)

1. Load and verify the reference file: symbols, partition map, tick sizes, LULD tiers, collars.
2. Start engines with the day's build on primary, standby, bunker and DR; confirm build hashes match on all.
3. Confirm the arbiter holds a primary for every partition and the bunker acknowledges on all partitions.
4. If a heavy day is expected, the DR engines ran the 2× replay of a recorded extreme open before step 2 and were restarted clean.
5. Open sessions at 07:00 for logon; books start empty; trading starts at 09:30 with the first CONTROL input.

Closing procedure: at 16:00 a CONTROL input ends trading; DAY orders expire (published); final CHECKPOINT; journals sealed, hashed and uploaded to S3; nightly verification replay compares every checkpoint; three-way clearing reconciliation.

Failover within the site is automatic (step 2.4). The runbook covers what follows: confirm the new primary's digest matches the next checkpoint, rebuild the failed host as the standby, and file the incident.

Halting a symbol or the market: issue a CONTROL input (halt, with reason and ticket); it is sequenced, journaled, replayed like any other input, and published as a Trading Status message. Resume the same way. Never halt by stopping processes: an input is the only way that the halt is in the record.

Go deeper: CLI playbook

Plain AWS commands an on-call engineer runs, one at a time. Replace the names with real ones.

textCopy
# 1. Alarms currently firing for the exchange
aws cloudwatch describe-alarms --state-value ALARM --alarm-name-prefix exchange-

# 2. State of the Direct Connect links from both sites
aws directconnect describe-connections

# 3. Confirm the audit bucket's default Object Lock rule
aws s3api get-object-lock-configuration --bucket exchange-journal-archive

# 4. Retention on one journal segment
aws s3api get-object-retention --bucket exchange-journal-archive --key journal/2026-03-03/p12/000417.seg

# 5. Start restoring an archived segment for a regulator replay (Bulk: within 48 hours)
aws s3api restore-object --bucket exchange-journal-archive --key journal/2026-03-03/p12/000417.seg --restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Bulk"}}'

# 6. Check whether the restore has finished (see the Restore field)
aws s3api head-object --bucket exchange-journal-archive --key journal/2026-03-03/p12/000417.seg

# 7. State of the post-trade MSK cluster
aws kafka list-clusters-v2

R3.10 Pillar Check

R3.11 Round 3 Rubric and Follow-Ups

What an architect (L7) answer adds over L6

• Treats the journal as the legal record: immutable, retained, and replayable with the exact engine build, proven by checkpoints.
• Puts a number on RPO 0 across sites (10 µs per route kilometre, round trip) and designs the bunker plus far-DR pattern to get zero loss without a 5 ms penalty.
• Makes the fence hold during a declaration, and refuses automatic degradation that would create a second market.
• Designs fairness into the physical layout, and has a reasoned position on speed bumps.
• Hands trades to clearing exactly once in effect, confirmed by the clearing house's acceptance, and reconciled three ways.
• Answers "can we run in the cloud?" component by component, with accurate AWS facts (multicast, ENA Express, Outposts), and knows a real example.
• Plans capacity from history with headroom, and knows which limits (one symbol, one core) can't be bought away.

Follow-up questions

1. "Why not put the bunker 1 km away and save 90 µs?" Answer: the bunker exists to survive whatever destroys the primary building: fire, flood, a power or network event in that neighbourhood. At 1 km it may share the same substation, flood plain or fibre ducts. We pick the closest site that shares none of those, and ~10 km is our estimate of that; the latency follows from the choice, not the other way round.

2. "Two years from now, a new engine build changes how IOC orders round. Can we still replay today?" Answer: yes, because we replay each day with the build that ran that day, which we keep for the whole retention period. We never "replay old days with new code" for audit; that would answer a different question.

3. "The board asks you to cut the latency back to 50 µs." Answer: then the board is choosing between two losses: it can move the bunker's copy into the same building (and lose released trades if the building is lost) or accept documented async DR. We'd present both with numbers, not decide it quietly.

Interview gotchas from this round's wrong answers

---

Loop Closer: Interview Strategy for All Three Rounds

How to Run Each 60-Minute Round

For how to spend a single 45-minute round, see the 45-minute interview blueprint.

The Two Sentences That Matter Most

• Opening a round: "An exchange sells fairness, so before speed I'll fix what 'first' means: one sequencer gives every input one number, and one thread per book applies them in that order, which also makes the whole thing replayable."
• When anything fails: "We never release anything about an input until it's stored on another machine, so a failover can only lose things nobody was told about, and the new primary recomputes the rest identically."

Well-Architected Review Sheet

Interviewers rarely ask "which pillar is this?". They ask the pillar's question in plain words. Rehearse one sentence per row.

Rubric Across Levels