peterxcli opened a new pull request, #4932: URL: https://github.com/apache/datafusion-comet/pull/4932
## Which issue does this PR close? Closes https://github.com/apache/datafusion-comet/issues/4874. ## Rationale for this change The native QuantileSummaries port behind approx_percentile / percentile_approx allocates more than necessary on its hot paths: - `QuantileSummaries::merge` takes `&self` / `&other` and allocates a fresh summary, so `merge_batch` reallocates on every incoming digest. - `with_head_buffer_inserted` rebuilds the whole `sampled` vector on every flush. ## What changes are included in this PR? - Reuse scratch buffers in QuantileSummaries during flush, compression, and merge. - Change summary merging to update in place while preserving the first-digest move fast path. - Update memory accounting and add a regression test for buffer reuse. ## How are these changes tested? benchmark - flamegraph: - source: - [baseline-nativ-bytes.html](https://github.com/user-attachments/files/30037547/baseline-nativ-bytes.html) - [patch-native-bytes.html](https://github.com/user-attachments/files/30037520/patch-native-bytes.html) result: - `merge_batch`: **(15GB -> 11GB)** - upstream: <img width="3338" height="496" alt="image" src="https://github.com/user-attachments/assets/f34d622c-a315-4036-9c32-e657cac9ad0c"; /> - pr: <img width="3354" height="472" alt="image" src="https://github.com/user-attachments/assets/0d1a8e30-9c8f-4191-b3f8-5ecac36ef324"; /> - `with_head_buffer_inserted`: **(10GB -> 1GB)** - upstream: <img width="3178" height="770" alt="image" src="https://github.com/user-attachments/assets/45bcc331-537e-4170-864d-f377e322973c"; /> - pr: <img width="3178" height="856" alt="image" src="https://github.com/user-attachments/assets/e3fd05a0-416d-4280-b3da-af2db09b9ad4"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
