viirya opened a new pull request, #25815: URL: https://github.com/apache/datafusion/pull/25815
## Which issue does this PR close? - Closes #25812. ## Rationale for this change #25692 stopped `date_bin` from propagating the ordering of second, millisecond and microsecond timestamps (and of sources whose range type is unknown), because scaling them to i64 nanoseconds can overflow and the overflow became a per-row `NULL`. As discussed on that PR, this forces a full re-sort for `ORDER BY date_bin(..., time)` and prevents streaming aggregation for `GROUP BY date_bin(..., time)` over sorted input, for data that never gets near the overflow range. ## What changes are included in this PR? - The existing i64 nanosecond computation stays as the fast path. When it overflows (scaling, `source - origin`, or the bin arithmetic), the bin is recomputed in i128 in a `#[cold]` path, for both fixed-duration and month strides. - A non-null input now becomes `NULL` only when its bin cannot be represented: a bin starting before the minimum value of the type, or a month bin outside the range of `DateTime<Utc>`. For example, `Timestamp(Second)` values in 1653 or 2286 now bin to real values instead of `NULL`. - `date_bin` propagates the ordering of its source for every precision again, reverting the guard from #25692. The remaining unrepresentable extremes are documented in `output_ordering`. ## What is the testing strategy for this PR? - Unit tests check the results for s/ms/us values outside the nanosecond range, month strides, `source - origin` overflow for nanoseconds, and the cases that still return `NULL`, in both the scalar and array paths. - `timestamps.slt`: the #25690 queries now return `1653-02-10T06:13:20`, `1970-01-01T00:00:00`, `2286-11-20T17:46:40` in order, and a new `EXPLAIN` shows the final sort is removed again. - `date_bin_errors.slt`: three overflow cases that returned `NULL` now return their correct bins. - The `date_bin_1000` benchmark (`Timestamp(Second)`, in range) goes from 2.78 µs on `main` to 2.50 µs. ## Are there any user-facing changes? Coarse-precision `date_bin` results outside the nanosecond range are now correct values instead of `NULL`, and queries over them no longer add sorts. There are no public API changes. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
