A payment system starts with money as an invariant: mistakes around retries, partial failures, and event ordering are more dangerous here than pure performance issues.
The case ties together payment intent creation, charge orchestration, the ledger, status reconciliation, fraud controls, and audit into one architecture.
For interviews and architecture reviews, it quickly shows whether you protect financial correctness first and talk about scaling second.
Idempotency
A repeated request must not turn into a second charge, even when the client, network, or queue replays the same operation.
Ledger
Payment status alone is not enough: the system needs a separate immutable record of financial effects that can be verified and rebuilt.
Status Reconciliation
The external PSP and the internal database drift over time, which is why regular reconciliation is mandatory rather than optional.
PCI Scope
Tokenization, tight card-data scope, and strict operational boundaries matter as much as the speed of the user-facing path.
Source
System Design Interview
A classic payment-architecture walkthrough: orchestration, ledger design, and PSP integration.
Payment systems look like a simple API that charges money — right up to the first timeout halfway through a transaction. After that you are running a distributed system where the cost of a mistake is measured in real money: state correctness, behavior under partial failure, and careful handling of sensitive data matter more than raw throughput. That is why a strict exactly-once guarantee is rarely built head-on; in practice it is assembled from at-least-once delivery, idempotency, and regular status reconciliation.
Requirements
Functional
- Create a payment intent for an order and support the authorization-then-capture flow.
- Support cards and alternative payment methods through a PSP.
- Provide idempotent charge and refund operations, including partial refunds.
- Accept PSP webhooks for statuses such as authorized, captured, failed, and chargeback.
- Store payment history and a transparent audit trail for support and finance teams.
Non-functional
Availability: 99.99%
Payments sit on a direct revenue path, so downtime turns into immediate business loss.
Latency: p95 < 300ms
Checkout should stay fast and predictable even when external dependencies are slow.
Correctness: No double charge
Retries, timeouts, and network failures must not result in a second charge.
Security: Minimal PCI DSS scope
The less sensitive card data the platform handles, the lower the operational risk.
High-level architecture
Payment Platform: High-Level View
synchronous payment path and asynchronous settlement/reconciliation pathSynchronous Path
Asynchronous Path
The payment platform is split between a fast user-facing payment path and a slower asynchronous settlement and reconciliation path.
The key decision here is to split the fast user-facing payment path from the asynchronous settlement and reconciliation path. Then a slow PSP hits the latency of background jobs rather than checkout, and financial consistency is restored by reconciliation instead of by blocking the user.
Critical flows
Payment flow explorer
Synchronous payment path and asynchronous status-reconciliation path.
Synchronous payment path: what matters
- The payment API creates an intent and stores an idempotency key so a repeated request does not create a second charge.
- The orchestrator drives the payment through valid state transitions such as INITIATED -> AUTHORIZED -> CAPTURED.
- The PSP adapter isolates provider-specific behavior from the core payment domain.
- The synchronous checkout response does not wait for the full settlement and status-reconciliation cycle.
- Intermediate states are persisted in the internal database so the system can recover safely after failure.
Asynchronous reconciliation path: what matters
- Webhooks are processed as at-least-once events with deduplication.
- The ledger and outbox record financial effects in immutable form.
- Background jobs compare internal statuses with PSP reports and close mismatches.
- Cases such as a missing capture or a mismatched refund go into a remediation queue.
- Finance, support, and fraud-detection services receive events through the outbox without coupling directly to the PSP.
Data model (simplified)
payment_transactions
- payment_id (UUID), order_id, customer_id
- amount, currency, status, psp_reference
- idempotency_key, created_at, updated_at
ledger_entries
- entry_id, payment_id, account_id
- direction (debit/credit), amount, currency
- entry_type (auth/capture/refund/chargeback)
Reliability and consistency
Required patterns
- Use an idempotency key for every mutating operation: authorization, capture, and refund.
- Use a transactional outbox to publish events without loss: delivery is at-least-once, and duplicates are absorbed by idempotent deduplication on the consumer side.
- Use retries with exponential backoff, jitter, and a circuit breaker around PSP calls.
- Model payment status as a strict finite state machine: INITIATED -> AUTHORIZED -> CAPTURED/FAILED/REFUNDED.
- Regularly reconcile internal states, ledger entries, and PSP reports.
Dangerous anti-patterns
- Treating the webhook as the only source of truth: a lost or delayed event silently drifts from the real money state, and without reconciliation the gap only surfaces in a complaint.
- Expiring the idempotency key too early, or not storing it at all — then the first retry after a timeout charges the customer twice.
- Keeping financial status in one mutable table without a separate immutable change log: after a customer dispute there is nothing left to reconstruct what happened and when.
- Mixing checkout business logic and provider-specific gateway code in one module — swapping or adding a PSP then turns into surgery on the payment core.
- Routing PAN/CVV through your own system where it can be avoided: every such point widens PCI DSS scope and operational risk.
Minimum security and compliance requirements
- Use tokenization so PAN/CVV never enter the core platform service.
- Use mTLS and service identity between internal services on the payment path.
- Apply strict RBAC to refunds and manual capture operations.
- Keep a continuous audit trail for financial and operational actions.
- Keep PCI DSS scope as small as possible to reduce operational risk.
Reference
Payment Intents
How the same flow looks at a real provider: explicit authorization and capture states instead of one opaque “pay” call.
Related chapters
- Rate Limiter - The payment API sits on the revenue path — how to hold it together under traffic spikes and card-testing abuse without dropping honest users.
- API Gateway - Where to push authentication, access policies, and routing so the payment path does not have to own them itself.
- Identification/AuthN/AuthZ - Refunds and manual capture are the most dangerous operations; this is about who may reach them and on what grounds.
- Encryption, keys and TLS - What protects traffic between services on the payment path: mutual authentication, key management, and secure transport.