Why a retry charges the customer twice

Order 88213 needs one payment of ₹4,998. The client calls the payment service with a 2,000 ms timeout and retries once on timeout, a policy many of us have written, and one that SDKs, service meshes and some HTTP clients apply without being asked.

Five ordinary steps

1. The client sends the charge.
2. The payment service commits charge ch_0001, then replies. That order is correct: replying before committing would confirm charges that never happened.
3. The reply is lost on the way back.
4. The client waits 2,000 ms and times out.
5. The client retries, and the service, seeing what looks like a new instruction, commits ch_0002.

The customer has paid ₹9,996 for a ₹4,998 order. Nothing exotic happened: no corrupted packet, no clock skew, no misbehaving node. A correct service, a correct client, a default retry policy and one lost response.

The client cannot fix this

After a timeout, the client does not know whether the first charge committed. It cannot find out by waiting longer or reasoning harder, because the information it needs never arrived. Removing the retry does not help either: then a request that really was lost leaves an order unpaid.

So the repair has to happen at the other end. The service must be able to recognize the second request as a repeat of the first.

What an idempotency key changes

The client creates one key for the payment, before the first attempt, and sends the same key with every retry. The service stores the key together with the charge it created, in the same transaction, with a uniqueness constraint on the key. When a request arrives with a key it has seen, the service returns the stored charge instead of creating a new one.

charge(request):
  if a charge exists for request.idempotencyKey:
    return that charge          # a repeat: charge nothing
  in one transaction:
    create the charge
    record idempotencyKey -> charge
  return the charge

The uniqueness constraint matters when two copies of the same request arrive at once: only one of them can commit.

Run the same five steps again and only step 5 differs: the service replays ch_0001, and the customer pays ₹4,998.

Where else this shows up

Any operation with an effect outside the request is exposed to the same lost reply: sending an email, reserving stock, starting a job, calling another company's API. Before shipping a retry policy, it is worth asking of every endpoint it covers what happens when the work succeeds and only the answer disappears.