System Design Situations Vault

Objection 1: If the cache node just restarted and is empty, won't the first wave of requests all miss at once and hit Postgres together?

Objection 1: Your biggest tenant is about to 10x their headcount and move onto the platform. What actually happens to their shard the day that migration lands?

Objection 1: What happens to an order event if the warehouse-allocation consumer throws an exception halfway through processing it?

Objection 1: Your rate limiter runs as local counters on each of the 12 API gateway instances behind the load balancer. Does a single partner's quota actually hold at 1,000 req/min, or could they get more than that?

Objection 1: A customer in Frankfurt books the last seat, then their connecting flight's confirmation page loads against the Virginia region two seconds later and shows the seat as available. What went wrong, concretely?

Objection 1: A photographer uploads a new cover photo to replace a bad one. How long before viewers around the world actually stop seeing the old cached image?

Objection 1: When you add 4 new physical storage nodes to the consistent hashing ring, why won't each existing node shed exactly 25% of its keys without virtual nodes?

Objection 1: What happens when your Bloom filter fills past its designed capacity and the false positive rate climbs to 15%?

Objection 1: Client 1 acquires a 10-second distributed lock in Redis, hits a 15-second Stop-The-World JVM pause, and then executes its write. Why doesn't an expiring TTL lock protect storage from corruption?

Objection 1: Why do writes stall completely when background LSM compaction falls behind the incoming write rate?

Objection 1: If a network partition isolates 2 nodes from 3 nodes in a 5-node Raft cluster, can the minority partition commit any client write?

Objection 1: If the hotel reservation succeeds, the payment is charged, but the flight service fails and the compensating refund fails halfway through, what prevents money from being permanently lost?

Objection 1: Why does a fixed-precision Geohash (e.g. 6-character precision) fail when driver density varies by 1,000x between rural highways and downtown Times Square?

Objection 1: What happens in a dual-write architecture when the primary database commit succeeds but the network call to publish to Kafka drops or times out?

Objection 1: Why does traversing a raw prefix Trie to find top-10 trending queries on every keystroke degrade when the trie holds 50 million search phrases?

Objection 1: What happens in a Twitter Snowflake ID generator if the physical server's system clock jumps backward by 250 milliseconds during an NTP synchronization event?

Objection 1: Why does a binary heartbeat threshold (e.g. mark dead after 3 missing pings) cause catastrophic flapping in large-scale distributed clusters?

Objection 1: Why does an in-memory HNSW (Hierarchical Navigable Small World) vector index consume 4x more RAM than the raw 1536-dimension floating-point vectors themselves?

Objection 1: When 1.5 million WebSocket clients disconnect simultaneously during an edge gateway rollout, why does immediate reconnection crush backend authentication and Redis Pub/Sub?

Objection 1: Why does a single dropped TCP packet on an HTTP/2 connection cause head-of-line blocking across all 500 multiplexed gRPC streams simultaneously?

Objection 1: If stateless JWTs are validated cryptographically without a database lookup, how do you revoke an active token immediately when an employee is terminated?

Objection 1: Why does Snapshot Isolation (or Repeatable Read in PostgreSQL) fail to prevent Write Skew when two concurrent transactions check a global invariant and update different rows?

Objection 1: Why does an upstream gateway run out of worker threads when a non-critical downstream microservice slows down from 30ms to 5,000ms latency?

Objection 1: Why does an event-sourced architecture struggle with point-in-time state queries (e.g. 'What was account balance on June 1st?') when an entity has accumulated 2 million historical events?

Objection 1: Why do standard per-IP rate limiters fail completely against a residential proxy botnet rotating across 500,000 distinct residential IP addresses?

Objection 1: Why do visual honeypot fields (e.g. `display: none` inputs) fail when sophisticated bots use headless Chrome with accessibility tree inspection?

Objection 1: Why does a static hardcoded blocklist of disposable email domains (e.g. `mailinator.com`, `guerrillamail.com`) fail within days of deployment?

Objection 1: Why do aggressive managed challenges (Cloudflare Turnstile / AWS WAF Challenge) falsely challenge or block legitimate users during high-traffic flash sales?

Objection 1: How do automated botnets bypass behavioral risk-scoring (reCAPTCHA v3 / Enterprise score 0.0 to 1.0) and achieve high 'human' trust scores?

Objection 1: Why does a fixed client-side Hashcash difficulty (e.g. 20 leading zero bits in SHA-256) cause severe asymmetry between high-end desktop gaming CPUs and low-end mobile devices?

Objection 1: If we open the journal with `O_DIRECT`, every `write()` goes straight to the disk and skips the page cache, so why is that not enough to make "Saved" survive a power loss?

Objection 1: Leveled compaction already drops tombstones and dead values when it merges files, so why do you need to run a manual compaction at all to meet the 7-day rule?

Objection 1: Why not simply raise the lease to 16 seconds so the 11-batch chunk fits, instead of changing the worker's code?

Objection 1: Why not just raise the queue's visibility timeout from 30 s to 120 s and keep the worker code as it is, since that alone stops the slow-call duplicate?

Objection 1: Why not set the slot cap large enough to hold a whole weekend of WAL, or remove the cap, so the connector simply resumes on Monday with no republish at all?

Objection 1: Why change the election setting at all, when an alarm on under-replicated partitions and a volume replacement within minutes would have kept three copies and lost nothing?

Objection 1: Every write is tagged with its checkpoint id and the job flushes after a restore, so why does the replay still overwrite `block_wh` with 70 Wh?

Objection 1: If S0 is at 240% of its write ceiling and 800% of its read ceiling, why not add more shards so the load on S0 is spread out?

Objection 1: Both vantage points failed and the metric is down, so why not override the lease check and force the failover to Ohio at 12:03:10?

Objection 1: Why not keep only the 2 s timeout and the cap of 54 and leave out the circuit breaker, since the timeout already bounds Skyline Air at 60 threads per instance?

Objection 1: Why not set every TTL to 4 s, in Valkey and in the API servers, and skip the tombstones and the change-stream invalidator?

Objection 1: Why not keep the direct SNS-to-Lambda subscription and add an on-failure destination, instead of putting a new SQS queue in front of the function?

Objection 1: After the takeover, why not just send the registration again, as you would any step whose DONE record is missing?

Objection 1: Why not give each of the 16 nodes one sixteenth of the plan in its own memory, 125 KB a second with a burst of 500 KB, so the nodes together can never admit more than the plan and we need no store?

Objection 1: Why not keep each charger's hourly max in its row and sum the four chargers' maxima to tell the grid company whether cab-5 passed 10,000 W?