walterddr opened a new issue #8260: URL: https://github.com/apache/pinot/issues/8260
We have come up with a design doc to support complex multi-stage query execution. Motivation === Today's Pinot query engine only supports a single scatter-gather model. This design runs very well if most of the heavy-lifting computation can be push down to segment servers, however it also have some limitations we would like to address. Namely, * Remove bottleneck in Pinot Broker when heavy reduce is needed. * Support additional SQL features such as distributed join. * Support standardized full SQL semantics Proposal === We propose to build a new multi-stage query engine to address these, specifically * a new SQL planner that can support full-SQL semantic planning. * a new execution engine that can support multi-stage, DAG-based execution. Design Doc === Please see https://docs.google.com/document/d/10-vL_bUrI-Pi2oYudWyUlQl9Kf0cLrW-Z8hGczkCPik/edit# for more details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
