benwtrent commented on code in PR #13124:
URL: https://github.com/apache/lucene/pull/13124#discussion_r1525266457
##########
lucene/core/src/java/org/apache/lucene/index/SegmentMerger.java:
##########
@@ -130,19 +135,31 @@ MergeState merge() throws IOException {
IOContext.READ,
segmentWriteState.segmentSuffix);
+ TaskExecutor taskExecutor = new
TaskExecutor(mergeState.intraMergeTaskExecutor);
+ List<Callable<Void>> mergingTasks = new ArrayList<>();
+
if (mergeState.mergeFieldInfos.hasNorms()) {
mergeWithLogging(this::mergeNorms, segmentWriteState, segmentReadState,
"norms", numMerged);
}
mergeWithLogging(this::mergeTerms, segmentWriteState, segmentReadState,
"postings", numMerged);
Review Comment:
OK, I added a commit that does this. The candidate merge time is now:
`forceMerge took 4965 ms`
vs. baseline (i re-ran to make sure)
`forceMerge took 15783 ms`
So, 3x faster? Seems pretty good. The major downside is that now memory
usage on merge might increase as we are potentially doing all this merge
activity at the same time. If we are cool with that, this seems like a nice
speed improvement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
