github-actions[bot] commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2774035527
This PR has not had activity in the past 2 weeks, labeling it as stale. If
the PR is waiting for review, notify the d...@lucene.apache.org list. Thank you
for your contributi
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2735879968
Thankyou for your help @vigyasharma !
I would love to explore small and more focused issues to start with !
--
This is an automated message from the Apache Git Service.
To respo
vigyasharma commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r2000345662
##
lucene/core/src/test/org/apache/lucene/index/TestMultiTenantMergeScheduler.java:
##
@@ -0,0 +1,73 @@
+package org.apache.lucene.index;
+
+import org.apache.luce
vigyasharma commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r2001826532
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,70 @@
+package org.apache.lucene.index;
+
+import java.util.concurren
mccullocht commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1999677796
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,72 @@
+package org.apache.lucene.index;
+
+import java.util.concurrent
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2729358357
I have a request to you. Kindly ignore the following two deleted files in
the "Files Changed" section :
"KeepOnlyLastCommitDeletionPolicy.java"
"ConcurrentMergeScheduler.java"
DivyanshIITB commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1998648834
##
lucene/core/src/test/org/apache/lucene/index/TestMultiTenantMergeScheduler.java:
##
@@ -0,0 +1,30 @@
+package org.apache.lucene.index;
+
+import org.apache.luc
DivyanshIITB commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1998646829
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,44 @@
+package org.apache.lucene.index;
+
+import java.util.concurre
DivyanshIITB commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1998645593
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,44 @@
+package org.apache.lucene.index;
+
+import java.util.concurre
DivyanshIITB commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1998646386
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,44 @@
+package org.apache.lucene.index;
+
+import java.util.concurre
DivyanshIITB commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1998645110
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,44 @@
+package org.apache.lucene.index;
+
+import java.util.concurre
vigyasharma commented on code in PR #14335:
URL: https://github.com/apache/lucene/pull/14335#discussion_r1998041333
##
lucene/core/src/java/org/apache/lucene/index/MultiTenantMergeScheduler.java:
##
@@ -0,0 +1,44 @@
+package org.apache.lucene.index;
+
+import java.util.concurren
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2727212266
Thanks for your feedback, @jpountz!
I have created a new MultiTenantMergeScheduler as suggested, instead of
modifying ConcurrentMergeScheduler.
I have restored ConcurrentMe
jpountz commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2727001008
I think it'll be simpler to create a new merge scheduler rather than modify
ConcurrentMergeScheduler. Also we'll need tests.
--
This is an automated message from the Apache Git Service
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2726271504
Thanks for the detailed clarification, @jpountz!
I have made the necessary changes to the implementation:
- Implemented a shared global thread pool with a fixed size of
jpountz commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2724923272
Apologies I had missed your reply.
> should this be a shared global pool across all IndexWriters, or should
each writer have its own pool?
It should be shared, we don't want
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2724394013
Just a gentle reminder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2710583795
Thank you for the review, @jpountz!
I see your concern regarding equal resource distribution across IndexWriter
instances potentially leading to inefficiencies when some write
DivyanshIITB commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2715055808
Thank you for the clarification, @jpountz!
I'll drop the merge throttling aspect from the changes since it's disabled
by default.
Regarding the fixed thread pool approach (
jpountz commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2714988316
Merge throttling is now disabled by default, IMO it's fine to ignore merge
throttling for now. Regarding thread creation, I'm thinking of a shared fixed
(e.g. number of processors / 2) t
jpountz commented on PR #14335:
URL: https://github.com/apache/lucene/pull/14335#issuecomment-2708071909
This looks too naive to me, we don't want each index writer to have 1/N of
the resources, which would prevent one writer from maxing out resources (e.g.
if one index has a heavy write lo
DivyanshIITB opened a new pull request, #14335:
URL: https://github.com/apache/lucene/pull/14335
This PR enhances the ConcurrentMergeScheduler by introducing dynamic
resource allocation for multi-tenant indexing scenarios.
Key Improvements:
1) Global Counter for Active IndexWriters
22 matches
Mail list logo
