Smallfu666 opened a new issue, #5356:
URL: https://github.com/apache/datafusion-comet/issues/5356
### Describe the bug
Under ANSI mode, `abs` on an integer minimum raises an overflow error whose
type name comes from
Arrow rather than from Spark. Comet reports `Int64 overflow` where Spark
reports `long overflow`.
`native/spark-expr/src/math_funcs/abs.rs` builds the error from the Arrow
`DataType`'s `Display`, so
the rendered message carries `Int8`, `Int16`, `Int32` and `Int64`. Spark's
`ARITHMETIC_OVERFLOW`
template renders `<message> overflow`, where the message for these types is
`byte`, `short`,
`integer` and `long`.
This is user-visible: the message is what a Spark user sees in the
exception, and it is what a test
asserting on the error class parameters would compare against.
### Steps to reproduce
```sql
SET spark.sql.ansi.enabled=true;
SELECT abs(col) FROM (SELECT CAST(-9223372036854775808 AS BIGINT) AS col);
```
Spark raises `[ARITHMETIC_OVERFLOW] long overflow.`
Comet raises the same error class with `Int64 overflow`.
The same divergence applies to `INT` (`integer` vs `Int32`).
### Expected behavior
The error carries Spark's type name, so `long overflow` and `integer
overflow`.
### Additional context
This is the "wrong type names" part of #5071, which was closed after #5162
landed the byte/short
error-class item. The type-name divergence is still present on `main`, so
this issue carves it out
as a self-contained fix rather than reopening the larger one. Happy to move
it back under #5071 if
you would rather track it there.
The mapping is not uniform across supported Spark versions, and that
constrains the fix:
- `integer` and `long` match Spark on 3.4, 3.5 and 4.x.
- `byte` and `short` match 4.x only. On 3.4 and 3.5 there is no byte or
short `negateExact`
overload, and those widths raise `_LEGACY_ERROR_TEMP_2043` with `-
<sqlValue> caused overflow.`
instead of `ARITHMETIC_OVERFLOW`, so no single string satisfies every
version.
I verified this by decompiling the relevant methods in the 3.4.3, 3.5.8,
4.0.2 and 4.1.2 jars rather
than reading one version and generalizing. Any fix should say in a comment
which versions each
chosen string matches and which it cannot.
Worth noting that the existing `abs_ansi.sql` fixture cannot catch this:
every assertion is
`expect_error(overflow)`, which `Int64 overflow` satisfies just as happily
as `long overflow`.
Decimal types are left alone here. `Decimal128` and `Decimal256` overflow
goes through a different
Spark path and needs its own analysis.
I have a fix and tests ready and can send a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
