How Stripe Handles Millions in Concurrent Idempotent Payments (System Architecture Deep Dive)

In distributed payment processing, the most dangerous scenario is an HTTP timeout. When an e-commerce backend issues a POST /v1/charges request to Stripe, the request traverses multiple hops: reverse proxies, TLS termination, API gateways, core payment orchestrators, and finally upstream card networks (Visa, Mastercard, or banking rails).

If an HTTP connection drops at second 11 of a 12-second timeout, the client has no way of knowing whether the request never reached the server, failed during validation, or successfully debited $5,000 from the customer's credit card right before the connection severed. A naive client retry without idempotency causes the customer to be debited twice.

To enforce exactly-once execution semantics across retry storms, Stripe requires clients to transmit an 'Idempotency-Key: <UUIDv4>' HTTP header with every mutating POST request. This key acts as a transactional identifier across the entire lifecycle of the payment.

When a request arrives at Stripe's edge, the API Gateway combines the client's API Key (or Merchant ID), the HTTP method/path, and the Idempotency Key into a composite distributed cache key: 'idempotency:merch_123:charges:uuid_456'.

A classic security vulnerability in naive idempotency implementations is key reuse with conflicting arguments: a client sends a $10 charge with Key-A, and then accidentally (or maliciously) sends a $10,000 charge using the exact same Key-A.

To guarantee cryptographic safety, Stripe hashes the HTTP verb, URL path, and serialized request body using SHA-256 upon first receipt. This hash is stored alongside the idempotency record in the database.

If a subsequent request arrives with matching Key-A but a mismatched SHA-256 hash, the engine immediately aborts and returns an HTTP 400 Bad Request: 'Keys cannot be reused with different request parameters'.

CREATE TABLE idempotency_keys (
    id VARCHAR(64) PRIMARY KEY, -- 'merch_abc:v1/charges:uuid_123'
    merchant_id VARCHAR(64) NOT NULL,
    request_hash CHAR(64) NOT NULL, -- SHA-256 hex digest of request body
    status VARCHAR(20) NOT NULL, -- 'PENDING', 'RESOLVED', 'FAILED'
    response_code SMALLINT NULL,
    response_headers JSONB NULL,
    response_body JSONB NULL,
    locked_by VARCHAR(64) NULL, -- Worker / Pod instance ID
    locked_at TIMESTAMP WITH TIME ZONE NULL,
    lock_timeout_at TIMESTAMP WITH TIME ZONE NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    expires_at TIMESTAMP WITH TIME ZONE NOT NULL
);

CREATE INDEX idx_idempotency_lookup 
ON idempotency_keys(merchant_id, id, expires_at);

Imagine a network retry or double-clicking user firing two identical POST /v1/charges requests simultaneously. If both threads pass cache lookup before either finishes, both might attempt to charge the credit card.

To neutralize this race condition, Stripe leverages atomic distributed locks (via Redis Redlock or database row locks). When Request 1 begins, it atomically inserts or sets the status to PENDING with a lease timeout (e.g., 60 seconds).

When Request 2 arrives and finds the state is already PENDING, it does not execute payment logic. Stripe's edge worker enters a short exponential polling loop (backoff between 50ms and 500ms) waiting for Request 1 to transition to RESOLVED. If Request 1 completes within the SLA, Request 2 fetches the cached result and returns it immediately. If it times out, Request 2 yields an HTTP 409 Conflict instructing the client to retry.

When designing payment engines, order processing systems, or ticket booking platforms in a system design interview, follow this 4-step blueprint:

1. Always define the idempotency boundary at the API Gateway before invoking downstream microservices.

2. Store request hashes alongside idempotency keys to defeat parameter mismatch tampering.

3. Separate the read-path cache (Redis, fast 24h TTL) from the persistent financial ledger (PostgreSQL, ACID transactional guarantees).

4. Explain crash recovery: What happens when the worker executing the charge crashes while the key is PENDING? Discuss lease TTLs and asynchronous payment reconciliation queues.

• TTL Expiration: Stripe caches idempotency responses for 24 hours; after that, expired keys are garbage collected.
• Atomic DB Upsert: Use 'INSERT INTO ... ON CONFLICT DO NOTHING' or Redis 'SET NX EX' to prevent distributed race conditions.
• Downstream Idempotency: Always propagate the idempotency key to downstream PSPs (Adyen, Chase Paymentech) so they also reject duplicates.