Twenty correct limiters enforce the wrong limit

Twenty gateways each pass the token-bucket tests. One tenant now gets twenty times the advertised burst. The release was locally correct and globally wrong.

This hypothetical API has one customer contract but twenty independent memories of it.

Put a number on the compromise

A central atomic bucket avoids that stranded allocation at the cost of a request hop, store load, and a shared dependency. Sticky routing reduces sharing but ownership moves must not create a fresh full bucket. Local shares sacrifice utilization; centrally leased blocks reduce calls but require a careful grant protocol.

For example, ten gateways each holding up to twenty unused leased tokens can strand up to 200 tokens. Those are already allocated, not automatically “overshoot.” They become overshoot if a coordinator reissues them while the old holders can still spend them.

Budget grants need ownership, too

A grant should identify the limit, holder, generation, amount, and validity rules. Do not reclaim a silent gateway's grant merely because a heartbeat is late. It may still serve isolated clients. Wait until old permission cannot be exercised under the protocol, or use a new fenced ownership regime at the resource.

The same problem appears across regions. Shares can be reassigned according to measured demand, but decreases must take effect before increases elsewhere spend the same capacity. Freeze or conservatively reclaim an unreachable region's allocation. Exact global admission under a partition may require rejecting requests; availability is a chosen trade-off, not a hidden implementation detail.

One request has several budgets

A request may consume a user limit, tenant limit, route limit, and global safety budget. Sequentially debiting each and stopping at the first denial can leak permission from earlier buckets. If the contract requires all-or-nothing, evaluate and debit atomically together. In a clustered store, that may require co-location, which can create a hot key or cross-slot limitation.

pseudocodeCopy
load rules and all relevant bucket states under one atomic boundary
refill each using the chosen authority clock
if every bucket can pay its cost:
    debit all
    persist rule versions and timestamps
    admit
else:
    debit none
    report the binding reason and appropriate wait

This is pseudocode, not an assertion that a script can span arbitrary shards. When co-location is too expensive, choose and document an approximate hierarchical policy rather than calling it exact.

Failure policy is part of the product contract

If the shared store times out, its debit may have happened. Decide per operation whether to fail closed, permit a bounded local fallback, or fail open. Record every bypass so an outage does not silently erase enforcement. Never treat a store error as a newly empty bucket.

Change rules with a monotonic version and explicit state migration. Lowering capacity may clamp stored tokens; changing rates requires a defined effective time. Giving each rule version an unrelated bucket can hand out another full burst. Test shadow decisions before enforcement, including eviction, restart, and stale clients.

A billing quota is different again. Admission counters may expire, approximate, or bypass. A usage ledger needs durable metering, stable operation IDs, reconciliation, and a defined accounting period. Running absolute totals with sequence numbers can avoid double-adding retransmitted meter batches, but delayed metering still creates a quota-enforcement lag.

Mental model: local algorithms enforce local state. A global promise needs a global allocation protocol, including how permission dies.

Transfer: low-rate abuse

---

Source trail: 02-distributed-rate-limiting, 14-rate-limiting-algorithms. See the source map for consolidation and reference-retained material.