benbellick opened a new issue, #25129:
URL: https://github.com/apache/datafusion/issues/25129
### Describe the bug
`date_trunc` can underflow when its array fast path truncates an extreme
timestamp. In debug builds, the unchecked subtraction can panic. In release
builds, it can wrap and return an incorrect timestamp.
The scalar path handles out-of-range arithmetic by returning a DataFusion
error, but `general_date_trunc_array_fine_granularity` performs the truncation
with unchecked arithmetic:
```rust
*v - i64::rem_euclid(*v, unit)
```
This was identified while reviewing #24501. The optimizer proposed there
evaluates scalar boundary values, while normal query execution uses the array
path. These paths should have consistent overflow behavior before `date_trunc`
is used to establish partitioning guarantees.
### To Reproduce
```sql
CREATE TABLE extreme_times AS
SELECT arrow_cast(value, 'Timestamp(Nanosecond, None)') AS ts
FROM (VALUES (-9223372036854775807 - 1)) AS t(value);
SELECT ts, date_trunc('microsecond', ts) AS truncated
FROM extreme_times;
```
[DataFusion
fiddle](https://datafusion-fiddle.vercel.app?q=eyJkZGwiOiJDUkVBVEUgVEFCTEUgZXh0cmVtZV90aW1lcyBBU1xuU0VMRUNUIGFycm93X2Nhc3QodmFsdWUsICdUaW1lc3RhbXAoTmFub3NlY29uZCwgTm9uZSknKSBBUyB0c1xuRlJPTSAoVkFMVUVTICgtOTIyMzM3MjAzNjg1NDc3NTgwNyAtIDEpKSBBUyB0KHZhbHVlKTsiLCJzZWxlY3QiOiJTRUxFQ1QgdHMsIGRhdGVfdHJ1bmMoJ21pY3Jvc2Vjb25kJywgdHMpIEFTIHRydW5jYXRlZFxuRlJPTSBleHRyZW1lX3RpbWVzOyJ9)
On DataFusion CLI 54.1.0 in a release build, this returns:
```text
+-------------------------------+-------------------------------+
| ts | truncated |
+-------------------------------+-------------------------------+
| 1677-09-21T00:12:43.145224192 | 2262-04-11T23:47:16.854775616 |
+-------------------------------+-------------------------------+
```
Truncating the year 1677 timestamp should not produce a timestamp in 2262.
### Expected behavior
Return a DataFusion error when the truncated timestamp cannot be represented
by the timestamp's underlying `i64`, rather than panicking or wrapping.
The array path should behave consistently with the checked scalar path.
### Additional context
- Identified during review of #24501.
- Related to #22209, which fixed a different `date_trunc` overflow when
scaling extreme non-nanosecond scalar timestamps to nanoseconds.
- #14738 concerns overflow in `date_part` and is a separate code path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
