Designing a Payment Transfer System for High Throughput
Building an instant transfer platform sounds simple until you meet the banks. Each one has its own quirks, availability windows, and definitions of "success." This is a look at how I think about designing a transfer system that stays correct and fast at scale.
The real problem
Moving money is not a throughput problem first — it is a correctness problem that happens to need throughput. The hard requirement is simple to state and brutal to guarantee: every transfer completes exactly once, or not at all, and we always know which.
Downstream banks make this hard. A request can time out after the bank already processed it. A "failed" response can turn out to be a success on reconciliation. Some banks are fast at 2am and unusable at 9am.
Architecture
The shape that has worked for me is a stateless orchestration service in front of per-bank adapters, backed by a durable transaction ledger.
- A transfer becomes a row in the ledger before anything leaves the building.
- Each bank is its own adapter with its own timeouts and circuit breaker.
- A reconciliation worker resolves anything left in an ambiguous state.
type Transfer struct {
ID string
IdempotencyKey string
State State // pending, submitted, settled, failed
BankCode string
}
func (s *Service) Submit(ctx context.Context, t Transfer) error {
if existing, ok := s.ledger.FindByKey(t.IdempotencyKey); ok {
return s.resume(ctx, existing) // never double-submit
}
// ...persist as pending, then hand to the bank adapter
return nil
}
Handling failure as a first-class citizen
Every bank integration is treated as an independent failure domain. If one bank degrades, its circuit breaker opens and the rest of the platform is unaffected. The orchestration path never blocks on a slow bank — status is resolved asynchronously.
Reconciliation
Reconciliation is not a cleanup job you add later. It is the source of truth. The bank's statement of record wins, and the reconciliation worker's job is to make our ledger agree with it.
Scale
Because the orchestration layer is stateless, the system scales horizontally — all state lives in the ledger and the queue. Throughput becomes a matter of adding workers, not redesigning the core.
Lessons learned
Idempotency and reconciliation are the whole game. If you get those two right, throughput is an operational concern. If you get them wrong, no amount of scaling saves you.