The payment succeeded twice after one timeout
A hypothetical payment request reaches its provider, but the response never reaches your service. Your client retries with a new request ID. Both payments succeed.
The second request was not recognized as another attempt at the first intent.
Name the intent before sending it
Generate a stable operation key once, persist it before the first send, and reuse it across retries. Scope it to the authenticated caller and operation. Store a fingerprint of the request's meaning, so the same key cannot silently change from ācharge 10ā to ācharge 100.ā Avoid personal data in the key.
Can you implement deduplication by checking a table, performing the effect, then inserting the key?
For a local database effect, insert a unique inbox/operation record and mutate the business state in the same transaction. On conflict, verify the fingerprint and return the durable outcome. In a real implementation, concurrent conflict handling must respect the database's transaction state and visibility rules.
pseudocodetransaction: claim unique (caller, operation, key) if already completed: validate fingerprint return stored result perform local business change store final result / stable resource pointer commit
The result cache can expire for efficiency, but deleting every trace of a durable effect can make an old retry look new. Define an acceptance horizon, retain durable uniqueness as long as required, or explicitly reject expired keys. TTL cleanup is not an exact deadline check.
An external charge will not join your transaction
Persist an intent before calling the provider. Use a stable provider idempotency key derived from or durably attached to that intent. Store states such as PENDING, SUCCEEDED, REJECTED, and UNKNOWN; keep final rejections distinct from failures you can retry.
After a timeout, reconcile using the same provider key or status lookup. Do not mark the operation failed merely because the HTTP response failed. A takeover worker must read durable state and respect an ownership/version condition before finalizing. Reissuing a fresh provider key on takeover creates the very duplicate the table was meant to prevent.
If the provider has neither deduplication nor a reliable lookup by operation identity, you cannot generally promise exactly one external effect after an ambiguous response. Choose a reconciliation/manual process or accept a stated duplicate/loss trade-off.
Repetition is not always harmless
SET status='paid' looks idempotent until a later refund sets refunded and a stale retry restores paid. Use versioned state transitions. āAbsolute valueā is safe only with the right authority and ordering; blindly writing an old total can lose newer work.
A useful operation table records scope, key, fingerprint, state, owner/generation, timestamps, provider key, and outcome. Limit retention growth with a policy linked to effect lifetime and retry admission. Monitor aged pending/unknown operations and reconciliation failures, not just duplicate hits.
Notification intent can be transactional with local state, but delivery remains a separate effect. Some channels support replacement or collapse identifiers with specific semantics; those are not universal exactly-once delivery. Verify the provider contract before promising that the user sees one notification.
Mental model: deduplication remembers attempts; idempotency preserves one intent's effect across those attempts and failures.
Transfer: the expired key
An operation key expires after a day. A mobile device reconnects two days later and retries an already completed purchase. What policy prevents a second purchase?
Source trail: 10-two-phase-commit-and-saga-orchestration, 04-idempotency-and-effectively-once-processing. See the source map for consolidation and reference-retained material.