zhaih commented on code in PR #12660:
URL: https://github.com/apache/lucene/pull/12660#discussion_r1367953194
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene95/Lucene95HnswVectorsWriter.java:
##########
@@ -635,17 +667,31 @@ private static DocsWithFieldSet writeVectorData(
return docsWithField;
}
+ private HnswGraphMerger createGraphMerger(
+ FieldInfo fieldInfo, RandomVectorScorerSupplier scorerSupplier) {
+ if (exec != null) {
+ return new ConcurrentHnswMerger(
+ fieldInfo, scorerSupplier, M, beamWidth, exec, numMergeWorkers);
+ }
+ return new IncrementalHnswGraphMerger(fieldInfo, scorerSupplier, M,
beamWidth);
+ }
+
@Override
public void close() throws IOException {
IOUtils.close(meta, vectorData, vectorIndex);
+ if (exec != null) {
+ exec.shutdownNow();
+ }
+ System.out.println(
+ "Total contention time: " + NeighborArray.contentionTime.get() /
1000000 + " ms");
Review Comment:
Let me post one to the description. In general it's normally 5000ms for a
100k docs merge using 8 threads. (single thread time roughly 95000 ms,
multithread wall time roughly 17000 ms)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
