Kaival Parikh created LUCENE-10611:
--------------------------------------
Summary: KnnVectorQuery throwing Heap Error for Restrictive Filters
Key: LUCENE-10611
URL: https://issues.apache.org/jira/browse/LUCENE-10611
Project: Lucene - Core
Issue Type: Bug
Reporter: Kaival Parikh
The HNSW graph search does not consider that visitedLimit may be reached in the
upper levels of graph search itself
This occurs when the pre-filter is too restrictive (and its count sets the
visitedLimit). So instead of switching over to exactSearch, it tries to [pop
from an empty
heap|https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/hnsw/HnswGraphSearcher.java#L90]
and throws an error
To reproduce this error, we can +increase the numDocs
[here|https://github.com/apache/lucene/blob/main/lucene/core/src/test/org/apache/lucene/search/TestKnnVectorQuery.java#L500]
to 20,000+ (so that nodes have more neighbors, and visitedLimit is reached
faster)
Stacktrace:
The heap is empty
java.lang.IllegalStateException: The heap is empty
at
__randomizedtesting.SeedInfo.seed([D7BC2F56048D9D1A:A1F576DD0E795BBF]:0)
at org.apache.lucene.util.LongHeap.pop(LongHeap.java:111)
at org.apache.lucene.util.hnsw.NeighborQueue.pop(NeighborQueue.java:98)
at
org.apache.lucene.util.hnsw.HnswGraphSearcher.search(HnswGraphSearcher.java:90)
at
org.apache.lucene.codecs.lucene92.Lucene92HnswVectorsReader.search(Lucene92HnswVectorsReader.java:236)
at
org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsReader.search(PerFieldKnnVectorsFormat.java:272)
at
org.apache.lucene.index.CodecReader.searchNearestVectors(CodecReader.java:235)
at
org.apache.lucene.search.KnnVectorQuery.approximateSearch(KnnVectorQuery.java:159)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
