sunchao commented on PR #24668: URL: https://github.com/apache/datafusion/pull/24668#issuecomment-5427763767
### [P2] Temporal double negation still stalls ordered streams @Amogh-2404, flagging the [existing P2 report](https://github.com/apache/datafusion/pull/24668#discussion_r3864338094) here in the main PR discussion: it still reproduces on current head `69af50622`. The integer case is fixed, but the retained timestamp/interval expressions still lose the ordering of subsequent keys. The reported `GROUP BY -(-i), j LIMIT 1` case still stalls. The latest base/head execution also confirmed another manifestation of the same issue in `LEAD` windows: ```sql SELECT i, j, LEAD(j, 1) OVER (PARTITION BY -(-i), j) AS n FROM s LIMIT 1; ``` Reproduction setup: a single-partition, unbounded `StreamingTable` ordered by `(i ASC NULLS LAST, j ASC NULLS LAST)`, with `target_partitions=1` and `batch_size=1`. It emits two rows with `i = TimestampNanosecond(1)` and `j = [0, 1]`, then remains pending without end-of-stream. - Base `63f5b55f` chooses `BoundedWindowAggExec` with `mode=[Sorted]` and immediately returns `(i=1ns, j=0, n=NULL)`. - Head `69af50622` chooses `mode=[PartiallySorted([0])]` and produces no row before the 500 ms timeout. The direct `PARTITION BY i, j` control returns immediately on both revisions. I reproduced the same difference with `IntervalYearMonth(1)` and nullable timestamps containing NULL. These cases do not involve overflow or the pre-existing signed single-negation issue tracked in #24683. `find_longest_permutation` does not establish that equal `-(-i)` values pin `i`, so it cannot recognize the ordered suffix `j`. The partial-order window path then waits for the leading key to change before closing the completed `j=0` partition. With a permanently fixed leading key, the query never emits its requested row. Could you address the existing P2 before merge by preserving the full ordering relationship while retaining checked temporal negation, with streaming execution regressions for both grouping and `LEAD`? This is one ordering issue with multiple affected operators, not a second independent finding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
