andygrove opened a new pull request, #4940: URL: https://github.com/apache/datafusion-comet/pull/4940
## Which issue does this PR close? Part of #4936. ## Rationale for this change `CAST(<float|double> AS DECIMAL(p, s))` is used in TPC-DS. The native path, `cast_floating_point_to_decimal128`, used a per-element `Decimal128Builder` loop with a null check and branch on every row. ## What changes are included in this PR? Replaces the element-wise loop with a single vectorized `unary_opt` pass: a value with no in-range integer form (NaN / infinity) or that does not fit the output precision maps to null, and the input null buffer is carried over. `unary_opt` applies the closure only to non-null slots, so it reproduces the loop's behavior in one pass with no sentinel and no second masking pass. ANSI mode must raise on out-of-range values instead of nulling them. `unary_opt` only nulls non-null inputs that overflow, so a null count beyond the input's signals an overflow. That check is O(1) and gates a rare element-wise rescan that reports the first offending value with Spark's exact `NumericValueOutOfRange` error. ## How are these changes tested? The existing `test_cast_float_to_decimal` (which covers NaN / infinity / precision-overflow to null in legacy mode) still passes unchanged. Added tests for the no-overflow fast path (values and nulls preserved), ANSI no-overflow, and ANSI overflow-error. Output is bit-identical to `main`. Because it is a single pass for every eval mode and shape, there is no regression on any shape: ``` cast_float_to_decimal: f64 -> dec(15,4): 20.28 µs -> 16.4 µs (-19%) cast_float_to_decimal: f64 -> dec(15,4), nulls: 24.10 µs -> 20.5 µs (-15%) cast_float_to_decimal: f32 -> dec(15,4): 20.45 µs -> 17.4 µs (-15%) cast_float_to_decimal: f64 -> dec(15,4) ansi: 20.28 µs -> 16.4 µs (-19%) cast_float_to_decimal: f64 -> dec(15,4) overflow: 26.69 µs -> 17.0 µs (-36%) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
