albertobastos opened a new pull request, #15143: URL: https://github.com/apache/pinot/pull/15143
Fixes https://startree.atlassian.net/browse/STP-3716 This PR adds a new Server configuration parameter (`pinot.server.mse.threads.hardLimit`) with default value `-1`. If the value if set to N>0 then MSE queries will be rejected if they create internally more than N threads. The query will return an error with message `Exceeded maximum number of tasks`. How it has been verified: - Updated the QuickStart launcher to add `pinot.server.mse.threads.hardLimit=2000`. - Change temporarily the `MaxTasksExecutor` so the decorated callable includes a `Thread.setName('DecoratedByMaxTasksExecutor`)` statement before triggering the actual task. - Run a query with a _group by_ clause using the MSE and add the following breakpoints: - `LeafStageTransferableBlockOperator.startExecution`: the current thread is properly named and the executorService used to spawn new tasks is also decorated. - `BaseCombineOperator.startProcess`: the current thread is properly named and the executorService used to spawn processors is also decorated. - `GroupByOperator.getNextBlock`: the current thread is properly named. If we run the same query but using the SSE, no changes are observed (no decorated thread is detected). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
