songwdfu opened a new pull request, #16123: URL: https://github.com/apache/pinot/pull/16123
Staging the enriched join operator that fuses ``` Project | Filter | HashJoin ``` into a single operator. It uses a "view" to perform filtering and projection, thus avoids one redundant copy of the join result. The goal is to match and fuse in more combination / ordering of filter, project, and other non-pipeline-breaker nodes. Whether Sort should be included in this fusion is debatable, currently it is took out. My reasoning is that: 1. fusing in a sort node would require us to make the probe side of the join a pipeline breaker as well (we had to collect all joined rows before emitting for sort). This makes it no different than having the sort as a seperate operator above. If we consider the sort to be after a project, we couldn't really save any copying as well since we had to copy the tuple after project. 2. If there's no collation, just offset and limit, we have `earlyTerminate` that handles this, fusing the sort in this case saves at-most one blocks' of work. 3. In case that the sort condition could be pushed / duplicated through join, we might want to use calcite rules to handle this at the planner level rather than hardcoding this logic in the operator -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
