kaivalnp commented on PR #14874: URL: https://github.com/apache/lucene/pull/14874#issuecomment-3091408661
Based on the above findings, we think that the JVM is aggressively optimizing the case where both documents are off-heap during indexing (i.e. in [`Lucene99MemorySegmentByteVectorScorerSupplier`](https://github.com/apache/lucene/blob/main/lucene/core/src/java24/org/apache/lucene/internal/vectorization/Lucene99MemorySegmentByteVectorScorerSupplier.java)), and then the query path (i.e. in [`Lucene99MemorySegmentByteVectorScorer`](https://github.com/apache/lucene/blob/main/lucene/core/src/java24/org/apache/lucene/internal/vectorization/Lucene99MemorySegmentByteVectorScorer.java)) with an on-heap query is penalized.. I refactored code around a bit to instantiate `ByteVector` instances more appropriately -- since it looked like wrapping an array with `MemorySegment.ofArray` and then instantiating a `ByteVector` using `ByteVector.fromMemorySegment` was causing some friction^ Benchmarks: `main` with `-reindex`: ``` recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn beamWidth quantized index(s) index_docs/s force_merge(s) num_segments index_size(MB) vec_disk(MB) vec_RAM(MB) indexType 0.962 1.832 1.831 1.000 100000 100 50 64 250 no 11.40 8771.93 24.53 1 77.45 292.969 292.969 HNSW ``` `main` without `-reindex`: ``` recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn beamWidth quantized index(s) index_docs/s force_merge(s) num_segments index_size(MB) vec_disk(MB) vec_RAM(MB) indexType 0.962 1.214 1.213 0.999 100000 100 50 64 250 no 0.00 Infinity 0.12 1 77.45 292.969 292.969 HNSW ``` This PR with `-reindex`: ``` recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn beamWidth quantized index(s) index_docs/s force_merge(s) num_segments index_size(MB) vec_disk(MB) vec_RAM(MB) indexType 0.962 1.140 1.140 0.999 100000 100 50 64 250 no 11.10 9010.63 24.45 1 77.47 292.969 292.969 HNSW ``` This PR without `-reindex`: ``` recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn beamWidth quantized index(s) index_docs/s force_merge(s) num_segments index_size(MB) vec_disk(MB) vec_RAM(MB) indexType 0.962 1.195 1.194 0.999 100000 100 50 64 250 no 0.00 Infinity 0.12 1 77.47 292.969 292.969 HNSW ``` **Note**: I ran the benchmark with `-XX:CompileCommand=inline,*PanamaVectorUtilSupport.*` to avoid the cold-start problem! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
