The database acknowledged a write the replica cannot read

A hypothetical profile service returns success after changing a display name. The next GET, routed to a replica, shows the old name. The replication dashboard says the write has arrived.

“Arrived” turns out to be the wrong question.

Track three positions

A replica may have stored log position 120 durably but applied only through 114. Reads from its materialized state cannot see position 120 yet. Separately, the replication protocol determines which entries are committed and may become authoritative after failover.

pseudocodeCopy
on write:
    commit under the replication contract
    return position token
on read(token):
    choose a replica that has applied at least token
    if no eligible replica:
        wait within deadline or route to valid leader
    if required position still unavailable:
        return declared retry/staleness result
    return readable state satisfying token

Sticky routing can prevent bouncing between replicas at different positions, but does not by itself make the chosen replica current. Read-your-writes is a session guarantee. Monotonic reads and linearizable reads are different contracts.

Acknowledgement selects the loss window

Asynchronous replication can acknowledge before another failure domain receives the change. At 200 commits/second and 0.8 seconds shipping lag, roughly 160 writes may be exposed in a constant-rate illustration. That is not an exact loss bound when traffic is bursty or the lag metric incomplete. Apply lag instead predicts stale reads; measure both with meaningful positions and heartbeats.

Synchronous replication waits for specified copies or protocol acknowledgements. State whether those acknowledgements mean received, persisted, or applied. In a simplified leader protocol that overlaps local persistence with follower replication, latency is bounded by the slower of local persistence and the required follower response. Some engines order those stages differently; do not reuse one formula across them.

Quorum overlap is only the beginning

With $N$ fixed home replicas, a write quorum $W$ and read quorum $R$ intersect when $W+R>N$. For $N=3$, $W=2$, $R=2$, a read set shares a replica with an acknowledged write set.

But what version does a read select? What about concurrent writes, incomplete writes, changing membership, or sloppy quorums using substitute nodes? The arithmetic alone supplies neither linearizability nor a safe compare-and-set operation. Leaderless designs need conflict/version resolution, repair rules, and explicit read/write semantics.

Hints can defer delivery to an unavailable replica; read repair can reconcile differences observed on reads; Merkle trees can narrow anti-entropy comparisons to divergent ranges. None makes an unavailable acknowledgement magically durable. Reconciliation must preserve deletion evidence: a tombstone removed before a long-absent replica is repaired can let old data return. Bound absence plus repair duration, or rebuild the replica rather than admitting unbounded old state.

A copy also copies mistakes

Replication improves availability for specified failures. It can quickly replicate accidental deletion or corruption at the logical layer. Keep point-in-time recovery or other independent recovery material and practice restoration.

For replica catch-up, retain the needed log suffix; if unavailable, transfer a consistent snapshot at a known position and apply the subsequent suffix without gaps. Throttle transfer so recovery does not destroy the serving capacity it is meant to restore.

Mental model: received, durable, committed, and applied are different milestones. Every guarantee should name which one the caller waits for.

Transfer: the fastest replica

---

Source trail: 01-consistent-hashing, 09-distributed-consensus-raft-and-paxos, 06-replication-quorums-and-read-your-writes. See the source map for consolidation and reference-retained material.