sohami commented on issue #12347: URL: https://github.com/apache/lucene/issues/12347#issuecomment-1589876811
> Could you share what you'd want to customize around the slice execution? I've also been looking at it and there's a number of things that I'd want to adjust. I am open to making things pluggable but I think there should be a good reason compared to changing the default behaviour. In default, [SliceExecutor (i.e. QueueSizeBasedExecutor)](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/QueueSizeBasedExecutor.java#L28) it applies the backpressure in concurrent execution based on a limiting factor of 1.5 times the passed in threadpool maxPoolSize. If the queue size goes beyond that it ends up executing all the tasks in caller thread. In OpenSearch, there is a search threadpool which serves the search request to all the lucene indices (or OpenSearch shards) assigned to a node. Each node can get the requests to some or all the indices on that node. With custom slice computation to control the max slices per request/index the limiting factor in SliceExecutor will not be needed. For example: if all the indices on a node is limited to have 2/4 slices each then the threadpool shared to execute these slices does not need to have any limiting factor. In this model the top level search threadpool will limit the number of active search requests on the node which will limit the number of work units in the SliceExecutor threadpool and will keep the behavior uniform across the requests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
