Batch to API: why I argued against reusing a proven pattern

We were integrating KCP, a Korean payment processor, to handle captures and refunds. Our existing processor, Chase Spectrum, used a file-based interface: we submitted a batch file with all the transactions for a period, and Chase responded with a result file. KCP had an API-based interface that accepted a single capture or refund per call.

The principal engineer on the team proposed keeping the batch model. Collect captures and refunds into hourly batches, then iterate through each transaction and call the KCP API. Two arguments supported this: Chase’s batch model had a built-in optimization window where refunds arriving within the same hour as a capture could be netted out, and the batch code path was battle-tested.

Why the abstraction was wrong

The batch model worked for Chase because Chase was a batch system. KCP was not. Forcing a batch abstraction onto an API-based processor created problems that did not need to exist.

If a batch contained both failed and successful transactions, the state of the batch as a whole was ambiguous. Chase handled idempotency at the file level and retried only failed transactions on re-upload. We would need to recreate that logic ourselves for KCP. And if we needed to retry specific failed transactions, we would have to create a new batch, introducing more state management.

Finding the angle that worked

Arguing against a working pattern on principle was not going to move the room. The refund optimization was the real concern. If we moved to per-transaction API calls, we would lose the one-hour window where captures and refunds could be netted.

I proposed using the delayed execution capability of our async execution framework. When a capture arrived, delay its execution by one hour. If a refund came in during that window, perform the optimization. This preserved the refund netting behavior while eliminating the artificial batch.

The per-transaction model also enabled something the batch model could not: publishing transaction data in real-time to our Payment Transaction Repository, making it immediately available in Payment Search. With batches, that data was always at least an hour stale.

The broader pattern

The instinct to reuse proven patterns is usually right. But it requires checking whether the pattern’s assumptions still hold. The batch model assumed the downstream system was batch-oriented. When that assumption changed, the pattern became overhead rather than safety.