fallintoplace opened a new pull request, #2373: URL: https://github.com/apache/iceberg-go/pull/2373
## What Skip sorting and gathering when compaction input is already ordered. ## Why Ordered rows can be written without rebuilding every Arrow column. ## Implementation - Check row order across batch boundaries with the existing comparator. - Reuse whole batches or slices. Keep stable sorting for unordered input. ## Benchmark Sort and Arrow gathering only, 32,768 rows and 16 columns. Apple M1 Pro, Go 1.26.3, median of 5 runs: - **Ordered:** 2.17 ms → 0.223 ms (**9.7x faster**); 6.55 MB → 480 B allocated. - **Nearly ordered:** 3.47 ms → 3.53 ms (**1.8% slower**). These are not end-to-end compaction timings. `GOMAXPROCS=1 go test ./table -run '^$' -bench '^BenchmarkYieldSortedPartitionFastPath/(globally_sorted|nearly_sorted)$' -benchmem -benchtime=500ms -count=5 -cpu=1` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
