msokolov commented on PR #12311:
URL: https://github.com/apache/lucene/pull/12311#issuecomment-1563317504
I'm seeing very strange results after testing with 768-dim vectors.
```
TaskQPS baseline StdDevQPS candidate
StdDev Pct diff p-value
HighTermVector 139.27 (6.7%) 57.21
(0.8%) -58.9% ( -62% - -55%) 0.000
AndHighHighVector 139.15 (6.4%) 57.15
(0.7%) -58.9% ( -62% - -55%) 0.000
LowTermVector 138.54 (6.2%) 57.00
(0.8%) -58.9% ( -62% - -55%) 0.000
MedTermVector 138.67 (6.9%) 57.23
(0.9%) -58.7% ( -62% - -54%) 0.000
AndHighMedVector 137.95 (6.1%) 57.10
(0.8%) -58.6% ( -61% - -55%) 0.000
AndHighLowVector 137.86 (6.4%) 57.21
(0.8%) -58.5% ( -61% - -54%) 0.000
PKLookup 199.30 (2.3%) 198.44
(2.5%) -0.4% ( -5% - 4%) 0.565
```
I have double-checked that candidate has
609fc9b63f61954a7408faa1669e807a6bbf1da9 and baseline is
c9c49bc5539d83979360e65b39a536c2d452ba2a. Both conditions are run using the
same index, tasks, etc. The JFR output is sort of a head-scratcher too. They
clearly show that the JDK vector API is being called (I think?) but also show a
lot of noisy things (Objects.requireNonNull). Is `
jdk.incubator.vector.FloatVector#reduceLanesTemplate()` really what we want to
see there? Another mystery is why
`org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel() ` would have more
samples in the candidate given that it has nothing to do with the vector api.
In looking at these you should ignore things related to `VectorDictionary` -
these are loading a word->vector dictionary that is used to look up query
vectors; the loading takes a while but happens prior to the index being opened
and the queries being executed.
## candidate
```
PERCENT CPU SAMPLES STACK
9.00% 2721
jdk.incubator.vector.FloatVector#reduceLanesTemplate()
5.44% 1645
org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel()
3.89% 1177
org.apache.lucene.util.packed.DirectReader$DirectPackedReader12#get()
3.03% 917
org.apache.lucene.util.packed.DirectReader$DirectPackedReader16#get()
2.99% 904 java.lang.invoke.VarHandleGuards#guard_LJ_I()
2.70% 816 jdk.internal.misc.Unsafe#copyMemoryChecks()
2.52% 761 perf.VectorDictionary#vectorDiv()
2.45% 741 org.apache.lucene.store.DataInput#readVInt()
2.43% 734
org.apache.lucene.util.packed.DirectMonotonicReader#get()
2.17% 655 java.util.Objects#requireNonNull()
1.92% 580 jdk.jfr.internal.JVM#emitEvent()
1.70% 513 java.util.Arrays#binarySearch0()
1.40% 423
org.apache.lucene.codecs.lucene95.Lucene95HnswVectorsReader$OffHeapHnswGraph#seek()
1.35% 407 java.util.HashMap#resize()
1.29% 391
org.apache.lucene.codecs.lucene95.OffHeapFloatVectorValues#vectorValue()
1.29% 389 sun.nio.ch.UnixFileDispatcherImpl#unmap0()
1.27% 385
org.apache.lucene.util.VectorUtilPanamaProvider#dotProduct()
1.26% 380
jdk.internal.foreign.AbstractMemorySegmentImpl#checkBounds()
1.23% 373 org.apache.lucene.util.LongHeap#downHeap()
1.22% 370
jdk.incubator.vector.FloatVector#fromArray0Template()
1.19% 361 java.util.zip.Inflater#inflateBytesBytes()
1.15% 348
org.apache.lucene.util.hnsw.NeighborQueue#decodeScore()
1.13% 341
org.apache.lucene.util.SparseFixedBitSet#getAndSet()
1.12% 338
org.apache.lucene.util.SparseFixedBitSet#insertLong()
1.02% 308 jdk.internal.misc.Unsafe#copyMemory()
0.96% 289
jdk.internal.foreign.AbstractMemorySegmentImpl#copy()
0.95% 286 perf.VectorDictionary#<init>()
0.94% 283 java.util.Objects#checkIndex()
0.88% 267
jdk.internal.foreign.AbstractMemorySegmentImpl#getBaseAndScale()
0.85% 257
org.apache.lucene.store.MemorySegmentIndexInput#readByte()
```
## baseline
```
PERCENT CPU SAMPLES STACK
24.69% 7386 perf.VectorDictionary#vectorDiv()
20.49% 6127 org.apache.lucene.util.VectorUtil#dotProduct()
3.42% 1022 java.nio.FloatBuffer#getArray()
2.65% 792
org.apache.lucene.util.hnsw.HnswGraphSearcher#searchLevel()
2.50% 748 perf.VectorDictionary#<init>()
2.34% 699 jdk.internal.misc.Unsafe#checkPrimitivePointer()
1.95% 583 jdk.jfr.internal.JVM#emitEvent()
1.62% 485 java.util.HashMap#getNode()
1.61% 481
org.apache.lucene.util.SparseFixedBitSet#insertLong()
1.50% 450 java.nio.Buffer#position()
1.32% 395 java.util.HashMap#resize()
1.03% 309 java.io.BufferedReader#readLine()
0.98% 292 jdk.internal.misc.Unsafe#checkOffset()
0.91% 272 perf.PKLookupTask#go()
0.90% 270 java.util.zip.Inflater#inflateBytesBytes()
0.85% 253 java.util.HashMap#containsKey()
0.79% 236 org.apache.lucene.util.LongHeap#downHeap()
0.67% 200
org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum#seekExact()
0.65% 194 sun.nio.ch.FileChannelImpl#unmap0()
0.57% 170
org.apache.lucene.util.SparseFixedBitSet#getAndSet()
0.53% 159 jdk.internal.util.ArraysSupport#mismatch()
0.51% 154
org.apache.lucene.util.hnsw.HnswGraphSearcher#graphNextNeighbor()
0.48% 145 sun.nio.cs.UTF_8$Decoder#decodeArrayLoop()
0.48% 143 java.io.FileOutputStream#writeBytes()
0.46% 137 org.apache.lucene.store.ByteBufferGuard#getByte()
0.44% 131
org.apache.lucene.codecs.lucene95.Lucene95HnswVectorsReader$OffHeapHnswGraph#seek()
0.41% 123 java.nio.Buffer#checkIndex()
0.40% 120 org.apache.lucene.store.DataInput#readVInt()
0.38% 113 perf.VectorDictionary#vectorNorm()
0.35% 104 org.apache.lucene.util.LongHeap#upHeap()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
