mccullocht commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1999677796
##########
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##########
@@ -0,0 +1,72 @@
+package org.apache.lucene.index;
+
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.io.IOException;
+
+/**
+ * A multi-tenant merge scheduler that shares a global thread pool across
multiple IndexWriters.
+ */
+public class MultiTenantMergeScheduler extends MergeScheduler {
+
+ // Shared global thread pool with lazy initialization
+ private static class LazyHolder {
+ static final ExecutorService MERGE_THREAD_POOL =
+
Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() / 2);
+ }
+
+ private static ExecutorService getMergeThreadPool() {
+ return LazyHolder.MERGE_THREAD_POOL;
+ }
+
+ // Use getMergeThreadPool() instead of direct access
+
+ @Override
+ public void merge(MergeScheduler.MergeSource mergeSource, MergeTrigger
trigger) throws IOException {
+ while (mergeSource.hasPendingMerges()) { // Use hasPendingMerges()
instead of relying on null check
+ MergePolicy.OneMerge merge = mergeSource.getNextMerge();
+ if (merge == null) {
+ break; // Explicitly exit if no merge is available
+ }
+
+ // Submit merge task to the shared thread pool
+ MERGE_THREAD_POOL.submit(() -> {
+ try {
+ mergeSource.merge(merge);
+ } catch (IOException e) {
+ throw new RuntimeException("Merge operation failed", e);
+ }
+ });
+
+ // Cleanup completed merges
+ activeMerges.removeIf(Future::isDone);
+ }
+ }
+
+ @Override
+ public void close() throws IOException {
Review Comment:
IIRC `IndexWriter.close()` will call `MergeScheduler.close()` which means
that you would wait for all merges across all indexes using this scheduler to
complete. The `IndexWriter` that is currently calling `close()` will stop
scheduling new merges, but the other writers may not, which means this may not
terminate. To fix this you may need to map active merges back to the
`IndexWriter` or `Directory` they belong to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
