jtibshirani commented on a change in pull request #641:
URL: https://github.com/apache/lucene/pull/641#discussion_r799057820
##########
File path:
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphBuilder.java
##########
@@ -93,6 +96,11 @@ public HnswGraphBuilder(
this.random = new SplittableRandom(seed);
int levelOfFirstNode = getRandomGraphLevel(ml, random);
this.hnsw = new HnswGraph(maxConn, levelOfFirstNode);
+ this.graphSearcher =
+ new HnswGraphSearcher(
+ similarityFunction,
+ new NeighborQueue(beamWidth, similarityFunction.reversed == false),
+ new FixedBitSet(vectorValues.size()));
Review comment:
I used a `FixedBitSet` here because the insertion and access operations
are faster. Also, using `SparseFixedBitSet` here doesn't actually really save
memory allocations, since during the graph construction we're expected to visit
each node at least once.
I tried an experiment where we shared the `visited` set but kept
`SparseFixedBitSet` and it didn't help performance. I also tried switching to
`FixedBitSet` during search and it hurt performance, probably because we are
allocating a very large array once per search.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
