theirix opened a new pull request, #24703:
URL: https://github.com/apache/datafusion/pull/24703
## Which issue does this PR close?
- Closes #22511
- Closes #20640
## Rationale for this change
In edge cases, an extra decimal point is required to accommodate a result.
Currently, ceil/floor just fail with an overflow.
For example, `ceil(-999)` is `-1000`, and it cannot fit into `Decimal(4,1)`
with 3 digits, but only into a decimal type with a lower scale and/or different
precision (e.g. `Decimal(4,0)`).
The proposed experimental fix is to widen the input decimal type to zero
scale.
How it's done in other engines:
1. It matches Spark ceil/floor behaviour (limited support in the Spark UDF),
when the scale drops to zero, precision is recalculated via `p-s+1`.
2. DuckDB performs slightly differently, just dropping the scale to zero and
keeping precision as input:
`select floor('-999.9'::DECIMAL(4,1));` -> `-1000::DECIMAL(4,0)`
`select ceil(9.9::DECIMAL(2,1));` -> `10::DECIMAL(2,0)`
3. ClickHouse surprisingly keeps the resulting type as is
`SELECT floor(CAST('-999.9', 'DECIMAL(4, 1)'))` -> `-1000::DECIMAL(4,1)`
4. Postgres has its own big integer types, not applicable
From three possible behaviours, we can go with either Spark's or DuckDB's
behaviour.
I didn't investigate ClickHouse behaviour yet. Since we already have
existing Spark logic in place and it uses precision sparingly, I lean towards
it.
## What changes are included in this PR?
- Change output type of floor/ceil to `Decimal(p-s+1, 0)` - could be a
breaking change
- Change the logic to match Spark's floor/ceil
- Extend `apply_decimal_op` to specify output scale (could be different from
input scale)
- Change preimage logic to consider both argument and literal precision and
scale
## Are these changes tested?
- Added examples from two linked issues
- A few more SLTs to cover clamping
## Are there any user-facing changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
