Jackie-Jiang opened a new pull request, #11234: URL: https://github.com/apache/pinot/pull/11234
Add supports for the new `tableOptions` with `partition_column` and `partition_size`. When this table option (hint) is attached to the leaf stage, we will honor it, and process the leaf stage for each partition with a separate thread. In order to do so, the table should be partitioned, and all the segments for any partition must be served by the same server. It will throw exception if table fails to reach this condition. Without the hint, the leaf stage will always be processed as a whole part, which is less efficient. More importantly, when the leaf stage result is partitioned, the intermediate stage can also benefit from it by increasing the parallelism and avoid shuffling the data. It can benefit JOIN (achieve colocated join) and GROUP BY (higher parallelism) the most. The old `joinOptions` `is_colocated_by_join_keys` is removed because it can be achieved with the new hint on both left and right table. See some example queries in `QueryHints.json` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
