benwtrent commented on PR #14836: URL: https://github.com/apache/lucene/pull/14836#issuecomment-2997236824
@msokolov [baseline_and_candidate_jfr.zip](https://github.com/user-attachments/files/20869476/baseline_and_candidate_jfr.zip) Additional stats, bit compressed, 50k docs. Its not earth shattering performance changes (wasn't expecting that), but it is measurably nicer. baseline ``` recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn beamWidth quantized index(s) index_docs/s num_segments index_size(MB) overSample vec_disk(MB) vec_RAM(MB) indexType 0.902 0.521 0.514 0.987 50000 100 0 16 100 1 bits 0.00 Infinity 1 153.96 3.000 151.730 5.245 HNSW ``` ``` PERCENT CPU SAMPLES STACK 23.94% 2526 org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsReader$BlockState#doReset() [JIT compiled code] 9.48% 1000 org.apache.lucene.util.LongHeap#downHeap() [Inlined code] 7.54% 796 org.apache.lucene.util.SparseFixedBitSet#getAndSet() [Inlined code] 7.34% 775 org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() [JIT compiled code] 5.13% 541 org.apache.lucene.codecs.lucene99.Lucene99HnswVectorsReader$OffHeapHnswGraph#seek() [JIT compiled code] 4.48% 473 jdk.incubator.vector.IntVector#reduceLanesTemplate() [Inlined code] 3.45% 364 java.util.Arrays#fill() [Inlined code] ``` ``` PERCENT HEAP SAMPLES STACK 23.82% 854M org.apache.lucene.util.ArrayUtil#growExact() [Inlined code] 10.08% 361M org.apache.lucene.util.SparseFixedBitSet#insertLong() [JIT compiled code] 6.43% 230M java.lang.Integer#toString() [Inlined code] 5.63% 202M org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsReader$BlockState#document() [JIT compiled code] 4.93% 176M org.apache.lucene.search.TopKnnCollector#topDocs() [JIT compiled code] 4.62% 165M java.lang.Integer#valueOf() [Inlined code] 4.56% 163M java.util.ArrayList#grow() [Inlined code] 3.47% 124M java.util.HashMap#newNode() [Inlined code] ``` candidate: ``` recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn beamWidth quantized index(s) index_docs/s num_segments index_size(MB) overSample vec_disk(MB) vec_RAM(MB) indexType 0.902 0.489 0.478 0.977 50000 100 0 16 100 1 bits 0.00 Infinity 1 153.96 3.000 151.730 5.245 HNSW ``` ``` PERCENT CPU SAMPLES STACK 25.26% 2551 org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsReader$BlockState#doReset() [JIT compiled code] 10.24% 1034 org.apache.lucene.util.LongHeap#downHeap() [Inlined code] 5.56% 561 org.apache.lucene.codecs.lucene99.Lucene99HnswVectorsReader$OffHeapHnswGraph#seek() [JIT compiled code] 4.76% 481 org.apache.lucene.util.FixedBitSet#getAndSet() [Inlined code] 4.71% 476 jdk.incubator.vector.IntVector#reduceLanesTemplate() [Inlined code] 3.78% 382 java.util.Arrays#fill() [Inlined code] ``` ``` PERCENT HEAP SAMPLES STACK 26.43% 881M org.apache.lucene.util.ArrayUtil#growExact() [Inlined code] 6.41% 213M java.lang.Integer#toString() [Inlined code] 6.23% 207M org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsReader$BlockState#document() [JIT compiled code] 5.28% 176M java.util.ArrayList#grow() [Inlined code] 4.87% 162M java.lang.Integer#valueOf() [Inlined code] 3.99% 132M java.util.HashMap#newNode() [Inlined code] 3.96% 132M org.apache.lucene.search.TopScoreDocCollector#populateResults() [Inlined code] 3.65% 121M org.apache.lucene.util.LongHeap#<init>() [Inlined code] 3.63% 120M org.apache.lucene.search.TopKnnCollector#topDocs() [Inlined code] 3.00% 99M org.apache.lucene.util.FixedBitSet#<init>() [Inlined code] 2.27% 75M org.apache.lucene.document.DocumentStoredFieldVisitor#intField() [Inline ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
