← projects

High-Volume Transaction Exports

Go, AWS S3 — 2026

Audits and compliance reviews need full transaction exports, sometimes covering hundreds of thousands of rows. Building that as a single batch in memory doesn’t scale — it’s slow, and it risks exhausting memory on large accounts.

This export streams instead:

  • Rows are fetched and written incrementally rather than loaded all at once
  • Output streams out as it’s generated, rather than being assembled on disk first
  • The result: 500,000+ records exported in under 30 seconds, with memory usage that stays flat regardless of export size

Exports run as background jobs with a status endpoint, so the request that kicks one off returns immediately.