andygrove opened a new pull request, #4939: URL: https://github.com/apache/datafusion-comet/pull/4939
## Which issue does this PR close? Part of #4936. ## Rationale for this change `CAST(<int> AS DECIMAL(p, s))` is common in TPC-DS (decimal casts are the second most frequent cast after `AS DATE`). The native path, `cast_int_to_decimal128_internal`, used a per-element `Decimal128Builder` loop with a null check and branch on every row. ## What changes are included in this PR? Replaces the element-wise loop with a single vectorized `unary` pass that rescales each value and marks anything overflowing the multiply or the output precision with an `i128::MAX` sentinel, carrying the input null buffer over. When no sentinel is present (the common case for real data, where int values comfortably fit the target precision) the result is final for every eval mode and is returned directly. Work is only added when a value actually overflows: - Legacy / Try run the vectorized null-masking pass (`null_if_overflow_precision`). - ANSI rescans element-wise to raise Spark's exact `NumericValueOutOfRange` error on the first offending value. (A sentinel can also come from a value stored beneath a null slot; the element-wise scan skips nulls, so ANSI does not raise on it.) ## How are these changes tested? Added unit tests for the legacy null-on-overflow (with nulls preserved), ANSI no-overflow, and ANSI overflow-error paths; the existing no-overflow test still passes. Output is bit-identical to `main`. Benchmark (criterion), baseline `main` vs this branch, 8192-row columns, two samples: ``` cast_int_to_decimal: i32 -> dec(15,4): 12.97 µs -> 8.8 µs (-31 to -32%) cast_int_to_decimal: i32 -> dec(15,4), nulls: 19.22 µs -> 8.9 µs (-53%) cast_int_to_decimal: i64 -> dec(38,4): 13.04 µs -> 8.9 µs (-32%) cast_int_to_decimal: i32 -> dec(15,4) ansi: 13.02 µs -> 9.0 µs (-31%) cast_int_to_decimal: i64 -> dec(15,4) overflow: 19.44 µs -> 20.3 µs (+4%) ``` The four no-overflow shapes are the realistic case and are 31-53% faster. The `overflow` shape forces every value to exceed the target precision, which cannot happen for valid data (int32 x 10^4 is far below the 10^15 bound); it does the vectorized pass plus the masking pass and is ~4% slower. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
