andygrove commented on PR #5003: URL: https://github.com/apache/datafusion-comet/pull/5003#issuecomment-5064240578
> True for multi-partition and spill. For single-partition, a `>= batch_size` batch on an empty coalescer buffer now yields one oversized block instead of `batch_size`-row blocks Confirmed, `single_partition.rs` routes `num_rows >= batch_size` straight to the writer, so those batches now pass through as a single block. I have updated the description to scope the unchanged-boundaries claim to the `PartitionedBatchIterator` paths and to spell out the single-partition behavior. The description also still described the earlier hand-rolled bypass rather than `biggest_coalesce_batch_size`, so that section is refreshed as well. The code comments in `buf_batch_writer.rs` carried the same overstated claim and are fixed in fb86cd613. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
