andygrove opened a new issue, #4943:
URL: https://github.com/apache/datafusion-comet/issues/4943
Part of #4936.
## Problem
`WideDecimalBinaryExpr` implements wide (128-bit intermediate) decimal `+ -
*`, which runs on the hot path of most TPC-DS queries. It computes the result
in a single fused `try_binary` pass
(`native/spark-expr/src/math_funcs/wide_decimal_binary_expr.rs`), writing
`i128::MAX` as an overflow sentinel in non-ANSI mode. It then unconditionally
runs `null_if_overflow_precision` to convert sentinels into nulls:
```rust
let result = if eval_mode != EvalMode::Ansi {
result.null_if_overflow_precision(p_out)
} else {
result
};
```
`null_if_overflow_precision` is a second full pass that allocates a new
array, and it runs even when no value overflowed, which is the common case.
## Proposed change
Run the masking pass only when a sentinel is actually present, mirroring
what was done for `DecimalRescaleCheckOverflow` in #4938:
```rust
let result = if eval_mode != EvalMode::Ansi &&
result.values().contains(&i128::MAX) {
result.null_if_overflow_precision(p_out)
} else {
result
};
```
`contains` short-circuits at the first sentinel, so the overflow path is
unaffected while the common no-overflow path skips the extra allocation. ANSI
mode never produces a sentinel (it errors during the `try_binary` pass) and is
unchanged.
Output stays bit-identical: when no sentinel is present,
`null_if_overflow_precision` would have nulled nothing, so skipping it yields
the same array.
## Notes
This is the non-ANSI counterpart to the `CheckOverflow` ANSI fast path in
#4937. A review of the other decimal expressions found no remaining
ANSI-detection asymmetry: `DecimalRescaleCheckOverflow` (#4938),
`WideDecimalBinaryExpr`, and the decimal casts already do their ANSI overflow
check inside a single cheap pass.
A criterion benchmark (no-overflow, no-overflow-with-nulls, sparse-overflow,
dense-overflow, and an ANSI shape) should accompany the change to confirm the
win and that overflow shapes do not regress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
