original-brownbear commented on PR #13622: URL: https://github.com/apache/lucene/pull/13622#issuecomment-2631740301
I agree @dsmiley , I actually did continue to work on this on the ES side lately in https://github.com/elastic/elasticsearch/pull/120024. What I did there was introduce logarithmic activation into this, making each thread start one other thread and then have the tasks started that way all loop on top of the same set of tasks. In that scheme the level of parallelism becomes a natural function of what the pool can deal with. The main issue I had with this approach in Lucene core is the `Executor` API. I can't really apply the approach neatly to it because I don't know if a call to `execute` will fork or not. Starting to think it might be worthwhile to special case `ThreadPoolExecutor` here like we do in ES to get around this issue in the short-term. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
