A payment system starts with money as an invariant: mistakes around retries, partial failures, and event ordering are more dangerous here than pure performance issues.
The case ties together payment intent creation, charge orchestration, the ledger, status reconciliation, fraud controls, and audit into one architecture.
For interviews and architecture reviews, it quickly shows whether you protect financial correctness first and talk about scaling second.
Idempotency
A repeated request must not turn into a second charge, even when the client, network, or queue replays the same operation.
Ledger
Payment status alone is not enough: the system needs a separate immutable record of financial effects that can be verified and rebuilt.
Status Reconciliation
The external PSP and the internal database drift over time, which is why regular reconciliation is mandatory rather than optional.
PCI Scope
Tokenization, tight card-data scope, and strict operational boundaries matter as much as the speed of the user-facing path.
Source
System Design Interview
A classic payment-architecture walkthrough: orchestration, ledger design, and PSP integration.
Payment systems are not just APIs that charge money. They are distributed systems where state correctness, resilience to partial failure, and careful handling of sensitive data matter more than raw throughput. In practice, an exactly-once guarantee is usually replaced by at-least-once delivery, idempotency, and regular status reconciliation.
Requirements
Functional
- Create a payment intent for an order and support the authorization-then-capture flow.
- Support cards and alternative payment methods through a PSP.
- Provide idempotent charge and refund operations, including partial refunds.
- Accept PSP webhooks for statuses such as authorized, captured, failed, and chargeback.
- Store payment history and a transparent audit trail for support and finance teams.
Non-functional
Availability: 99.99%
Payments sit on a direct revenue path, so downtime turns into immediate business loss.
Latency: p95 < 300ms
Checkout should stay fast and predictable even when external dependencies are slow.
Correctness: No double charge
Retries, timeouts, and network failures must not result in a second charge.
Security: Minimal PCI DSS scope
The less sensitive card data the platform handles, the lower the operational risk.
High-level architecture
Payment Platform: High-Level View
synchronous payment path and asynchronous settlement/reconciliation pathSynchronous Path
Asynchronous Path
The payment platform is split between a fast user-facing payment path and a slower asynchronous settlement and reconciliation path.
The central pattern is to separate the fast user-facing payment path from the asynchronous settlement and reconciliation path. That keeps checkout latency low while preserving verifiable financial state.
Critical flows
Payment flow explorer
Synchronous payment path and asynchronous status-reconciliation path.
Synchronous payment path: what matters
- The payment API creates an intent and stores an idempotency key so a repeated request does not create a second charge.
- The orchestrator drives the payment through valid state transitions such as INITIATED -> AUTHORIZED -> CAPTURED.
- The PSP adapter isolates provider-specific behavior from the core payment domain.
- The synchronous checkout response does not wait for the full settlement and status-reconciliation cycle.
- Intermediate states are persisted in the internal database so the system can recover safely after failure.
Asynchronous reconciliation path: what matters
- Webhooks are processed as at-least-once events with deduplication.
- The ledger and outbox record financial effects in immutable form.
- Background jobs compare internal statuses with PSP reports and close mismatches.
- Cases such as a missing capture or a mismatched refund go into a remediation queue.
- Finance, support, and fraud-detection services receive events through the outbox without coupling directly to the PSP.
Data model (simplified)
payment_transactions
- payment_id (UUID), order_id, customer_id
- amount, currency, status, psp_reference
- idempotency_key, created_at, updated_at
ledger_entries
- entry_id, payment_id, account_id
- direction (debit/credit), amount, currency
- entry_type (auth/capture/refund/chargeback)
Reliability and consistency
Required patterns
- Use an idempotency key for every mutating operation: authorization, capture, and refund.
- Use a transactional outbox to publish events without loss or duplication.
- Use retries with exponential backoff, jitter, and a circuit breaker around PSP calls.
- Model payment status as a strict finite state machine: INITIATED -> AUTHORIZED -> CAPTURED/FAILED/REFUNDED.
- Regularly reconcile internal states, ledger entries, and PSP reports.
Dangerous anti-patterns
- Treating the webhook as the only source of truth and skipping regular reconciliation.
- Not storing the idempotency key or expiring it too early.
- Keeping financial status in one mutable table without a separate immutable change log.
- Mixing checkout business logic and provider-specific gateway code in one module.
- Handling PAN/CVV inside your own system when it can be avoided.
Minimum security and compliance requirements
- Use tokenization so PAN/CVV never enter the core platform service.
- Use mTLS and service identity between internal services on the payment path.
- Apply strict RBAC to refunds and manual capture operations.
- Keep a continuous audit trail for financial and operational actions.
- Keep PCI DSS scope as small as possible to reduce operational risk.
Reference
Payment Intents
A practical example of a stateful payment flow with authorization and capture stages.
Related chapters
- Rate Limiter - How to protect payment APIs from traffic spikes, abuse patterns, and checkout-path overload.
- API Gateway - A common entry point, access policies, and routing in front of the payment path.
- Identification/AuthN/AuthZ - Access control for charges, refunds, and sensitive financial operations.
- Encryption, keys and TLS - Cryptographic protection of the payment path with mTLS, key management, and secure transport.