anoopj opened a new pull request, #3080: URL: https://github.com/apache/iceberg-rust/pull/3080
## Which issue does this PR close? Fixes #3079 ## What changes are included in this PR? `create_primitive_array_repeated` built a throwaway `vec![value.clone(); num_rows]` for the Utf8/Binary/LargeBinary/FixedSizeBinary arms before handing it to the array constructor. For strings that clones the value into num_rows separate heap allocations per batch; the binary arms allocate a throwaway intermediate Vec. Stream the single value straight into the Arrow buffer via `from_iter_values` with `std::iter::repeat_n` instead. Benchmarks (release, throwaway harness, old vs new toggled on this file only): - leaf create_primitive_array_repeated (Utf8, 8192-row batch): 23.0 -> 4.1 ns/row (5.6x) - full scan select(x, _partition) over a real manifest + 262k-row parquet file, string-partitioned: 20.9 -> 5.7 ns/row (3.7x) Note that the gain is allocation-counts and scales with how much of the projection is the string partition column. ## Are these changes tested? New + old tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
