msokolov commented on issue #12740: URL: https://github.com/apache/lucene/issues/12740#issuecomment-1963100010
Thinking about holistically as someone trying to tune the performance of the indexer I want as few knobs as possible that enable me to get the best performance for my particular workload. This is kind of tricky since some folks will be indexing and searching on the same node, whileothers will have dedicated a host entirely to indexing. In any case indexing clients can currently control the number of threads dedicated to index by: (1) setting the number of threads given to CMS and (2) controlling the number of indexing threads they create. Adding a third pool seems to make this a tricky problem. I wonder if we could leverage the same thread pool that is used by CMS? Also, I agree that if CMS is already using all the threads, it won't help to have another thread pool for KNN merging. So these things have to be configured together somehow. At the end of the day I think I would want to prioritize supplying CPU cores to the bottleneck. If KNN merging consumes threads that would otherwise be used for scheduling merges (by sharing its thread pool) then I think that would fall out naturally? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
