gortiz opened a new pull request, #14495: URL: https://github.com/apache/pinot/pull/14495
This PR is a continuation of #14296 and the next step on #14196. Here we define how to modify a query to replace stages that have been found equivalent. Not all equivalent stages can be substitute. Specifically, two stages that send to the same stage cannot be substituted by a spool. The reason for that is that the mailbox service assumes there the tuple (receiveStage, sendStage) is unique in the query. Probably this can be changed, but it doesn't seem to be worth it in this initial phase. This PR also changes the equivalence function to consider not equivalent two senders with different distribution. The design algorithm includes some ways to also support this, but again, it adds a complexity that doesn't seem to be required in the first spool version. Several tests have been added and also a new builder API to create spools in tests. Although the spool builder code is a bit ugly, it simplifies the tests that use it. There are some examples in `EquivalentStagesReplacerTest`. The next PR may be the final one, where the worker protocol and operators are changed to support these new senders that can send everywhere. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
