The Dual-Write That Broke Search Consistency
The Dual-Write That Broke Search Consistency
Your e-commerce platform manages product catalog updates. When a merchant edits a product title, price, or inventory count, the Catalog Service writes the change to a PostgreSQL database, and immediately in the next line of code publishes an update event to a Kafka topic so that the OpenSearch search cluster and Redis cache can update their indexes. At a volume of 8,000 updates/minute, the data science team discovers that over 14,000 products in the search index display stale or incorrect prices compared to PostgreSQL. When analyzing incident logs, they find two root causes: (1) network blips between the application pod and Kafka caused the Kafka publish call to throw an exception after PostgreSQL had already committed the transaction, leaving OpenSearch permanently out-of-sync; and (2) concurrent updates to the same product by two different administrators landed in PostgreSQL in order (A then B), but reached Kafka out-of-order (B then A), overwriting the newer price with the older one. You are tasked with replacing the dual-write pattern with a Transactional Outbox and Change Data Capture (CDC) architecture that guarantees atomic, ordered event delivery.
The Dual-Write That Broke Search Consistency
Your e-commerce platform manages product catalog updates. When a merchant edits a product title, price, or inventory count, the Catalog Service writes the change to a PostgreSQL database, and immediately in the next line of code publishes an update event to a Kafka topic so that the OpenSearch search cluster and Redis cache can update their indexes. At a volume of 8,000 updates/minute, the data science team discovers that over 14,000 products in the search index display stale or incorrect prices compared to PostgreSQL. When analyzing incident logs, they find two root causes: (1) network blips between the application pod and Kafka caused the Kafka publish call to throw an exception after PostgreSQL had already committed the transaction, leaving OpenSearch permanently out-of-sync; and (2) concurrent updates to the same product by two different administrators landed in PostgreSQL in order (A then B), but reached Kafka out-of-order (B then A), overwriting the newer price with the older one. You are tasked with replacing the dual-write pattern with a Transactional Outbox and Change Data Capture (CDC) architecture that guarantees atomic, ordered event delivery.
Provide 1–2 precise sentences for each architectural dimension. Each box guides you on what staff-level interviewers evaluate.
Define SLA targets, hard consistency constraints, and conditions the system must never violate.
Quantify throughput (QPS/RPS), read:write ratios, and peak burst multipliers.
Step-by-step path: client ingress → API gateway → queues → background workers → persistence.
Database engine, table schema, partition keys (PK/SK), and durability strategy.
What resource hits saturation first under 10x traffic? (CPU, disk IOPS, connection pools, network).
Worker crashes, network partitions, split-brain, poison pill DLQ, retries, and idempotency.
What did you sacrifice in exchange and why? (e.g. eventual consistency vs latency, cost vs redundancy).