walterddr commented on code in PR #10135:
URL: https://github.com/apache/pinot/pull/10135#discussion_r1073713700
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/service/QueryServer.java:
##########
@@ -89,25 +88,12 @@ public void submit(Worker.QueryRequest request,
StreamObserver<Worker.QueryRespo
return;
}
- // return dispatch successful.
- // TODO: return meaningful value here.
+ // TODO: break this into parsing and execution, so that responseObserver
can return upon compilation complete.
+ // compilation complete indicates dispatch successful.
+ _executorService.submit(() ->
_queryRunner.processQuery(distributedStagePlan, requestMetadataMap));
Review Comment:
> However we are getting this advantage implicitly (it relies on the fact
that we are using the same thread pool).
The main advantage is to release the default-grpc-thread as soon as possible
as all the subsequence query execution are not related to the grpc response of
the QueryServer.
> Long term would we want to use the scheduler itself to schedule the leaf
stages too? We could add another API in the scheduler that fast-tracks
execution for leaf stages (by perhaps doing exactly what we are doing here).
IMO, the leaf stage in the long term should also run in the OpChain
scheduler for various reasons (see https://github.com/apache/pinot/pull/9887,
https://github.com/apache/pinot/pull/9753 and the original design doc)
> Right now we are extracting the scheduler's thread-pool out and then using
it to submit threads directly which doesn't seem like the cleanest design,
since the scheduler may also be interested in tracking the leaf stage operators
later.
Correct, we are relying on the same threadpool being used across. I can do
another refactor to make the threadpool available only on QueryServer and
passed in as executor to QueryRunner and ServerQueryExecutorImplV1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
