john-mlika opened a new pull request, #16588: URL: https://github.com/apache/lucene/pull/16588
`DocIdSetIteratorAcceptDocs` hands `createBitSet` an iterator already wrapped for live docs. The wrapper has no `intoBitSet`, so on any segment with deletions the bit set is filled one `nextDoc()` at a time and live docs are applied twice. `createBitSet` already takes `liveDocs` into account, so pass the raw iterator. Details and measurements are in #16586. Adds `TestAcceptDocs#testDenseIteratorIsConsumedInBulkWhenSegmentHasDeletions`, which counts `nextDoc()` calls on the source iterator and fails on main when the segment has deletions and the accept set is dense enough for the bit-set branch, and `testRandomBitsAreMatchesIntersectedWithLiveDocs`, which checks the bits are always the matches intersected with live docs on both sides of the dense/sparse threshold. Also adds `FilteredKnnVectorQueryBenchmark`, the benchmark the numbers in the issue come from. With the benchmark's 95%-selective cached filter and 5% deleted docs: 2.05 -> 1.23 ms/query. The measurements were taken on an earlier revision of this branch that differs only in comments and in the benchmark's structure, not in what it measures. Relates to #16586 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
