msokolov commented on a change in pull request #1155: LUCENE-8962: Add ability
to selectively merge on commit
URL: https://github.com/apache/lucene-solr/pull/1155#discussion_r365837334
##########
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##########
@@ -3147,6 +3149,38 @@ public final boolean flushNextBuffer() throws
IOException {
}
}
+ private MergePolicy waitForMergeOnCommitPolicy(MergePolicy source, final
SegmentInfos toCommit,
+
AtomicReference<CountDownLatch> mergeLatchRef) {
+ return new OneMergeWrappingMergePolicy(source, (toWrap) -> new
MergePolicy.OneMerge(toWrap.segments) {
+ @Override
+ public void mergeFinished() throws IOException {
+ super.mergeFinished();
+ CountDownLatch mergeAwaitLatch = mergeLatchRef.get();
+ if (mergeAwaitLatch == null) {
+ // Commit thread timed out waiting for this merge and moved on. No
need to manipulate toCommit.
+ return;
+ }
+ if (isAborted() == false) {
+ deleter.incRef(this.info.files());
+ toCommit.add(this.info.clone());
+ long segmentCounter =
Long.parseLong(this.info.info.name.substring(1), Character.MAX_RADIX);
+ toCommit.counter = Math.max(toCommit.counter, segmentCounter + 1);
+ Set<String> segmentNamesToRemove = new HashSet<>();
+ for (SegmentCommitInfo sci : this.segments) {
+ deleter.decRef(sci.files());
+ segmentNamesToRemove.add(sci.info.name);
+ }
+ for (int i = toCommit.size() - 1; i >= 0; i--) {
+ if (segmentNamesToRemove.contains(toCommit.info(i).info.name)) {
+ toCommit.remove(i);
Review comment:
This relies on some peculiar semantics of SegmentInfos, namely that it
manages Segment internally as a List. I guess that's true now, but I wonder if
we shouldn't make SegmentInfos actually implement List, have asList return a
modifiable List, or at least document and/or add a unit test to
TestSegmentInfos enforcing that you can remove while iterating and expect a
certain ordering of the elements? It seems to be implicit in the contract given
that segments are ordered, but it's not explicitly stated in the javadocs at
least.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
