andygrove opened a new pull request, #4941: URL: https://github.com/apache/datafusion-comet/pull/4941
## Which issue does this PR close? Part of #4936. ## Rationale for this change The float-to-int and decimal-to-int narrowing casts built their output with a per-element `iter().map(...).collect::<Result<...>>()` over `Option`/`Result`, the same slow pattern that `spark_cast_int_to_int` moved off of (that change was up to 100x faster). ## What changes are included in this PR? Rewrites the four macros (`cast_float_to_int16_down`, `cast_float_to_int32_up`, `cast_decimal_to_int16_down`, `cast_decimal_to_int32_up`) to use Arrow's `unary` (legacy/try) and `try_unary` (ANSI) kernels, which map the values buffer in one pass and carry the null buffer over, following the existing `cast_int_to_int_macro`. Each macro gains a destination `ArrowPrimitiveType` parameter, threaded through the 12 call sites. The decimal macros also hoist the constant `10^scale` divisor out of the per-element loop. Overflow, NaN, saturation, and the cast-through-Int wrap semantics are preserved exactly; only the iteration mechanism changes. ## How are these changes tested? Added Rust unit tests (these paths previously had only Scala coverage): float64-to-Byte legacy wrap, float64-to-Int ANSI ok + overflow error, decimal-to-Int legacy, decimal-to-Byte legacy wrap, and decimal-to-Int ANSI overflow error. Output is bit-identical to `main`. Benchmark (criterion), baseline `main` vs this branch, 8192-row columns: ``` cast_narrowing: f64 -> i8: 21.7 µs -> 1.9 µs (-91%) cast_narrowing: f64 -> i32: 21.4 µs -> 1.8 µs (-92%) cast_narrowing: f64 -> i32 ansi: 23.4 µs -> 11.9 µs (-49%) cast_narrowing: dec -> i8: 41.6 µs -> 17.1 µs (-59%) cast_narrowing: dec -> i32: 43.7 µs -> 16.6 µs (-62%) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
