ethbak commented on code in PR #16627:
URL: https://github.com/apache/lucene/pull/16627#discussion_r4160730383
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswConcurrentMergeBuilder.java:
##########
@@ -42,10 +43,13 @@ public class HnswConcurrentMergeBuilder implements
HnswBuilder {
private static final int DEFAULT_BATCH_SIZE =
2048; // number of vectors the worker handles sequentially at one batch
+ // Number of disconnected nodes a repair worker claims atomically at a time.
+ private static final int REPAIR_BATCH_SIZE = 64;
Review Comment:
64 was chosen because I didn't see much improvement in wall time under it.
When testing on cohere, wall time was flat from batch size 1 through 256 (~15s)
and about ~7% slower at 1024. 64 sits in the middle of the flat region and it
seems like tweaking it doesn't produce any real effect within that bottom
range.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
