The client gave up; the server kept working

A hypothetical checkout times out after 200 ms. The server finishes at 250 ms. The client retries, and the retry also finishes. The customer sees one success; the payment provider sees two attempts.

Reducing the timeout made the error faster. It did not make the first operation disappear.

Unknown is a real outcome

A timeout says you stopped waiting. It does not say where the remote operation stopped. Cancellation can reduce wasted work, but must be propagated and honored; a side effect may already have committed.

A timeout caps one wait. An end-to-end deadline caps useful work for the entire operation. Pass a remaining budget or a well-defined propagated deadline, include connection establishment, and use monotonic local timing for elapsed duration. At each dequeue, discard work whose deadline already passed.

pseudocodeCopy
while attempts remain and retry budget permits:
    remaining = deadline - monotonic_now()
    if remaining <= 0:
        return unknown_or_timeout
    attempt_limit = min(configured_attempt_timeout, remaining)
    result = call_with_cancellation(attempt_limit, same_operation_key)
    if final(result):
        return result
    if not transient_or_safe_to_repeat(result):
        return result
    wait bounded full-jitter backoff, only within remaining budget

This is pseudocode. “Safe to repeat” requires an operation contract, not an HTTP-method guess. A read may be safe, an idempotent keyed mutation may be safe, and an unkeyed charge is not made safe by exponential backoff.

Backoff spreads work; it does not remove it

Full jitter chooses a delay uniformly between zero and min(cap, base * 2**attempt). It reduces synchronized retry waves. Respect a meaningful server Retry-After and the caller's remaining deadline; do not turn it into a promise to retry forever.

A retry budget caps additional traffic across requests. A per-client budget scales with the client fleet, so account for fleet size. Prefer retries for transient availability failures and explicit throttling when time remains; authentication failures and malformed requests ordinarily require a changed request, not another identical attempt.

Periodic jobs need a different rhythm: a stable offset derived from client identity spreads scheduled work predictably. Fresh randomness on every run can create hard-to-debug timing drift.

The fastest timeout can be the slowest recovery

Choose an attempt timeout from observed latency and a tolerated false-timeout fraction, with network/setup margins. An arbitrary short timeout abandons healthy work and manufactures retries. An arbitrary long one pins resources during failure.

Little's law gives a useful average relationship for a stable system: $L=\lambda W$. At 1,000 requests/second and 20 ms average residence, about 20 requests are in flight. At two seconds, about 2,000 are. A percentile-based concurrency estimate is a conservative planning heuristic, not Little's law itself.

Hedged reads start a second attempt after a delay to reduce tail latency. They spend capacity even when nothing is broken, need a budget, and should cancel losers. Correlated slow replicas can make both attempts expensive. Do not hedge side effects without a proven deduplication contract.

Observe useful completions within deadline—goodput—alongside total requests, retry fraction, cancellations honored, queue age, and unknown outcomes. Rising throughput with falling goodput is often a failure, not a success.

Mental model: a retry is new load attached to an old uncertainty. Spend it only when the operation and remaining time justify it.

Transfer: the overloaded service

---

Source trail: 22-circuit-breaker-bulkhead-and-fault-tolerance-patterns, 10-retries-timeouts-backpressure-and-load-shedding. See the source map for consolidation and reference-retained material.