somandal opened a new pull request, #10286: URL: https://github.com/apache/pinot/pull/10286
This PR introduces the multi-stage runtime changes to support Phase 1 of Window Functions, specifically targeting empty `OVER()` and `OVER(PARTITION BY)`. Runtime support has been added for the following classes of window queries: - Empty OVER() - OVER(PARTITION BY) This PR also ads/fixes some bugs in the window function planning, specifically: - Create Empty `OVER()` and `OVER(ORDER BY)` as singleton stages which need a single node assignment (similar to global aggregation without group by and global sort) - Add some logic for `OVER(PARTITION BY key1 ORDER BY key1)` queries which get marked as `OVER(PARTITION BY key1)` to ensure that the `ORDER BY` key1's direction and null direction are the defaults, otherwise treat these as `OVER(PARTITION BY key1 ORDER BY key2)` type queries during planning. The window functions supported as part of Phase 1 are: SUM, AVG, MIN, MAX, and COUNT. This PR also adds a new JSON file for window function query runtime tests via the output mechanism (H2 doesn't support window functions and got parse errors when trying to use it). (I still need to add some more tests, working on them now) Design document: https://docs.google.com/document/d/13CmFm4djI09JKF_Xty5acoXxJoxC9CLXAsmikgzgtIs/edit?usp=sharing OSS issue: https://github.com/apache/pinot/issues/7213 Window Function Planner PR: https://github.com/apache/pinot/pull/10228 This PR does not include support for: - Execution engine changes for Phase 1 for OVER(ORDER BY) and OVER (PARTITION BY ORDER BY) (will be the next PR for this feature) - Custom frames - Other window functions related to rank and values - Multiple window groups (basically multiple OVER clauses with different PARTITION BY, ORDER BY and/or FRAME specifications, if these specifications are the same they get grouped into a single window group) The above will be part of future changes for window function support cc @siddharthteotia @walterddr @vvivekiyer @Jackie-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
