Skip to content
sysgeeks

Design a Payment System, stage 8 of 13: break it

Webhooks arrive twice, and out of order

The provider's documentation says it all plainly: events can be delivered more than once, in any order, and your endpoint must respond within 10 seconds or the delivery is retried.

System so far· 4 parts
1234CLIENTBuyer's browserSERVICECheckout APIDATABASEPostgresEXTERNALPayment provider

Select a component to see what it is responsible for and which state it owns.

  1. 1Buyer's browser → Payment provider: Card entry and 3-D Secure in hosted fields
  2. 2Buyer's browser → Checkout API: Pay (Idempotency-Key header); poll status
  3. 3Checkout API → Payment provider: Create payment with the attempt's key
  4. 4Checkout API → Postgres: Claim attempt; conditional transitions

What you need to know

0 of 2 checks done

  1. A webhook is a hint that something may have changed, not an instruction. Handlers that treat it as an instruction ("set status to X, then send the receipt") break in two ways:

    • A duplicate delivery repeats the instruction: two receipts.
    • A late delivery applies an old status over a newer one.

  2. Two ways to make a webhook handler safe:

    • Deduplicate and move forward only: record each event id under a unique constraint, skip ones already seen, and apply only forward transitions.
    • Fetch current state: treat the webhook as a nudge and ask the provider for the payment's latest state. Ordering stops mattering, at the cost of one API call per event.

    Either way, verify the signature first, so nobody can forge "payment succeeded".

  3. Check

    Where should the 'send receipt' action be triggered?