How does an idempotency key prevent double charges?

Each payment attempt carries a unique key that's stored with a unique constraint before the provider is called (and also sent to the provider). A retry with the same key finds the existing payment and returns its result instead of creating a new charge.

What is a double-entry ledger and why use it for payments?

Every money movement is recorded as entries whose debits equal credits across accounts (e.g. clearing, fees, revenue). Entries are append-only, balances are computed from them, and the totals must always balance, which gives an audit trail and makes errors detectable.

Design a Payment System — System Design Interviews, Step by Step

The question

"Design the payment system for an online store: customers pay at checkout with a card, and we must record every payment correctly."

Clarify the requirements

Functional: charge a card at checkout through an external payment provider; refunds; payment status visible to the order system; a record of every money movement.
Scale: 1 million orders per day, peaks at 10× during sales.
Non-functional: never charge a customer twice; never lose a successful payment; strong consistency for balances; full audit trail; card numbers never touch our servers (the provider's hosted fields or SDK collect them, keeping us out of most card-security compliance scope).
Out of scope: building a card network connection, fraud models (mention), payouts to sellers (mention).

Estimate the scale

1M/day ≈ 12 payments/s, peak ~120/s. Low volume: the challenge is correctness, not throughput.
Ledger entries: ~4 per payment (two per side, plus fees) ≈ 4M rows/day ≈ 1.5 billion/year; a relational database with partitioning by month handles this.

API and data model

POST /payments   Idempotency-Key: order-8812-attempt-1
                 { order_id, amount_cents, currency, payment_method_token }
                 -> 201 { payment_id, status: "succeeded" | "requires_action" | "failed" }
POST /payments/{id}/refunds  Idempotency-Key: ...   { amount_cents }
POST /webhooks/provider      (signed events from the payment provider)

payments:  payment_id | order_id | idempotency_key (unique) | amount | currency | status | provider_ref
ledger:    entry_id | payment_id | account | debit | credit | created_at   (append-only)
accounts:  customer_receivables, provider_clearing, revenue, fees, refunds ...

High-level design

checkout ─► payment service ─► payments DB (idempotency key unique)
               │      │
               │      └─► payment provider API ─► card networks / banks
               │                 │
               │      webhooks ◄─┘  (async confirmation, signed)
               │
               ├─► ledger (double-entry, append-only, same DB transaction as status change)
               └─► events ─► order service, email, analytics

nightly: reconciliation job compares our ledger with the provider's settlement report

Deep dives

1. Idempotency: never charge twice. The client (or the order service) sends an idempotency key with each payment attempt. The payment service stores the key with a unique constraint before calling the provider and passes the same key to the provider. A retry (after a timeout, a double click or a crash) finds the existing record and returns its result instead of charging again.

2. A payment state machine. Statuses move only forward along allowed transitions: created → processing → succeeded | failed | requires_action, succeeded → refunded | partially_refunded. Unknown outcomes (a timeout from the provider) stay processing until a webhook or a status query resolves them; never guess.

3. Double-entry ledger. Every money movement is recorded as entries whose debits equal credits, in the same transaction as the status change. Balances are sums of entries, never a mutable number someone can overwrite, and the books always balance.

Idempotent charges and a ledger that always balancesTry it yourself

class Payments:
    def __init__(self):
        self.by_key = {}            # idempotency key -> payment (unique constraint in real life)
        self.ledger = []            # append-only (account, debit, credit)
        self.provider_calls = 0

    def charge(self, key, order, cents, provider_result="ok"):
        if key in self.by_key:                   # retry or double click: no second charge
            return self.by_key[key]
        payment = {"order": order, "cents": cents, "status": "processing"}
        self.by_key[key] = payment               # saved BEFORE calling the provider
        self.provider_calls += 1
        if provider_result == "ok":
            payment["status"] = "succeeded"
            fee = round(cents * 0.029) + 30
            self.post([("provider_clearing", cents - fee, 0), ("fees", fee, 0), ("revenue", 0, cents)])
        else:
            payment["status"] = "failed"
        return payment

    def refund(self, key, cents):
        payment = self.by_key[key]
        assert payment["status"] == "succeeded" and cents <= payment["cents"]
        payment["status"] = "refunded" if cents == payment["cents"] else "partially_refunded"
        self.post([("refunds", cents, 0), ("provider_clearing", 0, cents)])

    def post(self, entries):
        assert sum(d for _, d, _ in entries) == sum(c for _, _, c in entries), "unbalanced!"
        self.ledger.extend(entries)

    def balance(self, account):
        return sum(d - c for a, d, c in self.ledger if a == account)


p = Payments()
print(p.charge("order-1-a", "order-1", 5000)["status"])
print(p.charge("order-1-a", "order-1", 5000)["status"], "(retry)")
print(p.charge("order-2-a", "order-2", 2000, provider_result="declined")["status"])
p.refund("order-1-a", 1000)
print("provider calls:", p.provider_calls)
print("revenue:", -p.balance("revenue"), "| fees:", p.balance("fees"), "| refunds:", p.balance("refunds"),
      "| clearing:", p.balance("provider_clearing"))
print("total debits == total credits:", sum(d for _, d, _ in p.ledger) == sum(c for _, _, c in p.ledger))

class Payments:
    def __init__(self):
        self.by_key = {}            # idempotency key -> payment (unique constraint in real life)
        self.ledger = []            # append-only (account, debit, credit)
        self.provider_calls = 0

def charge(self, key, order, cents, provider_result="ok"):
        if key in self.by_key:                   # retry or double click: no second charge
            return self.by_key[key]
        payment = {"order": order, "cents": cents, "status": "processing"}
        self.by_key[key] = payment               # saved BEFORE calling the provider
        self.provider_calls += 1
        if provider_result == "ok":
            payment["status"] = "succeeded"
            fee = round(cents * 0.029) + 30
            self.post([("provider_clearing", cents - fee, 0), ("fees", fee, 0), ("revenue", 0, cents)])
        else:
            payment["status"] = "failed"
        return payment

def refund(self, key, cents):
        payment = self.by_key[key]
        assert payment["status"] == "succeeded" and cents <= payment["cents"]
        payment["status"] = "refunded" if cents == payment["cents"] else "partially_refunded"
        self.post([("refunds", cents, 0), ("provider_clearing", 0, cents)])

def post(self, entries):
        assert sum(d for _, d, _ in entries) == sum(c for _, _, c in entries), "unbalanced!"
        self.ledger.extend(entries)

def balance(self, account):
        return sum(d - c for a, d, c in self.ledger if a == account)

p = Payments()
print(p.charge("order-1-a", "order-1", 5000)["status"])
print(p.charge("order-1-a", "order-1", 5000)["status"], "(retry)")
print(p.charge("order-2-a", "order-2", 2000, provider_result="declined")["status"])
p.refund("order-1-a", 1000)
print("provider calls:", p.provider_calls)
print("revenue:", -p.balance("revenue"), "| fees:", p.balance("fees"), "| refunds:", p.balance("refunds"),
      "| clearing:", p.balance("provider_clearing"))
print("total debits == total credits:", sum(d for _, d, _ in p.ledger) == sum(c for _, _, c in p.ledger))

Output

succeeded
succeeded (retry)
failed
provider calls: 2
revenue: 5000 | fees: 175 | refunds: 1000 | clearing: 3825
total debits == total credits: True

4. Webhooks and reconciliation. Providers confirm asynchronously with signed webhooks, which can arrive late, twice or out of order: verify the signature, dedupe by event ID, and apply only valid state transitions. Every night, a reconciliation job compares our ledger with the provider's settlement file; any mismatch (a payment they have and we don't, a different amount) raises an alert for a human.

Trade-offs

Synchronous vs asynchronous confirmation: waiting for the provider gives the customer an instant answer; webhooks are the source of truth for edge cases (3-D Secure, delayed bank responses).
Strong consistency (one relational database with transactions) over raw scale: payments are low volume and correctness matters more.
Build vs buy: payment providers handle card storage and compliance; building it yourself means a huge compliance burden.
Retries: retry only with the same idempotency key; a new key means a new charge.

What interviewers look for

Idempotency keys stored before the external call, passed to the provider.
A clear state machine, including the "unknown outcome" case.
A double-entry, append-only ledger.
Webhook handling and reconciliation.
Keeping card data out of your systems.

Key points

Idempotency key + unique constraint = no double charges.
Status changes follow a state machine; timeouts stay "processing" until confirmed.
Double-entry ledger, appended in the same transaction as the status.
Verify and dedupe webhooks; reconcile daily against the provider.