mkleen opened a new pull request, #26956: URL: https://github.com/apache/datafusion/pull/26956
## Which issue does this PR close? - Follow-up #26595 - Benchmarks are available #26954 and must be merged first ## Rationale for this change `normalize_floats` runs on every float key column before it is compared, hashed or grouped. It first scans the column for values that need rewriting. When there are none, which is the common case, it returns the input unchanged, so that scan is the whole cost. The scan used an early-exit `iter().any()`. Because the loop can stop at any value, the compiler checks one value at a time and doesn't vectorize it. There was also no benchmark for `normalize_floats`. This was suggested by @jayzhan211 in the review of #26595. ## What changes are included in this PR? the scan in `normalize_primitive` folds over every value instead of using `any()`. The fold has no early exit, so the compiler turns it into a SIMD OR-reduction. It always scans the whole column, but that only costs extra when the column has a value to rewrite, and then the column is copied anyway. ## What is the testing strategy for this PR? The behavior is unchanged. The existing `normalize_floats` unit tests in `float_semantics.rs` and the float sqllogictests cover both the case with nothing to normalize and the rewrite path. ## Are there any user-facing changes? No. Queries with float join, group or sort keys spend less time checking for values to normalize. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
