jpountz commented on PR #14335: URL: https://github.com/apache/lucene/pull/14335#issuecomment-2724923272
Apologies I had missed your reply. > should this be a shared global pool across all IndexWriters, or should each writer have its own pool? It should be shared, we don't want the total number of threads to scale with the number of index writers. The reasoning for the numProcessors/2 number is that merging generally should not be more expensive than indexing, so by reserving only half the CPU capacity for merging, it should still be possible to max out hardware while indexing, while also having a peak number of threads running merges under numProcessors/2. > If it's a shared pool, how should we handle cases where a few writers are highly active while others are idle? Should we allow active writers to take more resources dynamically, or keep a strict fixed allocation? Idle writers would naturally submit fewer tasks than highly active writers. IMO the fixed allocation is key here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
