itschrispeck opened a new pull request, #12274: URL: https://github.com/apache/pinot/pull/12274
https://github.com/apache/pinot/pull/11558 changed the mutable Lucene index to be queried using a separate `ThreadPoolExecutor` to avoid interrupts corrupting the index. `ThreadPoolExecutor` was used improperly which resulted in queries against realtime segments to be executed sequentially. The thread count of the executor generally would never grow beyond 1, as the queue would not reject new tasks. This PR adds a new `ScalingThreadPoolExecutor` which has behavior that is the intended behavior from the previous PR. This is added in `pinot-common`, since it can later be used for other features such as the async deep store upload retry. With `ScalingThreadPoolExecutor` we increase the pool size if all threads are busy by rejecting the additional task. The rejection handler will queue the task anyway after the queue initially rejects it, so it will be executed by the new thread. tags: `bugfix` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
