ankitsultana opened a new pull request, #10349: URL: https://github.com/apache/pinot/pull/10349
v1 engine uses two types of thread-pools: runners and workers. Workers run the per-segment operators and runners run the BaseCombineOperator. Workers provide data to the blockingQueue in the combine operators and the runners poll the same to merge the final results. In v2 engine at present we use the same thread-pool for both of the above which can lead to a dead-lock in cases when the thread-pool is fully occupied with threads blocked on the queue. In this PR, I have tried to align v2 engine with v1 engine: 1. query_worker_* threads will run the per-segment operators (same as worker-threads in v1) and the v2 operators. 2. query_runner_* threads will run the BaseCombineOperator#mergeResults for leaf-stage. For intermediate stages they'll be responsible for creating the physical plan and submitting the OpChain to the scheduler. @walterddr lmk if you also had this in mind. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
