msokolov commented on PR #14874:
URL: https://github.com/apache/lucene/pull/14874#issuecomment-3063327162
This got me wondering -- what is hotspot doing with the code we have on
main? By default, `knnPerfTest.py` runs 1000 iterations, plus we have a warmup
phase that runs the same number of iters as the "main" phase, so a total of
20000 by default).
On Intel CPU with preferredBitSize=256
```
recall latency(ms) netCPU avgCpuCount nDoc nIters
0.982 5.500 5.100 0.927 500000 10
0.991 3.510 3.450 0.983 500000 100
0.979 5.851 5.834 0.997 500000 1000
0.975 5.891 5.881 0.998 500000 2500
```
On ARM with preferredBitSize=128
```
recall latency(ms) netCPU avgCpuCount nDoc nIters
0.986 9.300 9.400 1.011 500000 100 10
0.989 8.010 8.010 1.000 500000 100 100
0.986 25.860 25.858 1.000 500000 100 1000
0.983 7.628 7.628 1.000 500000 100 2500
```
So I think the bad performance we measured may just be a hotspot glitch
around 1-2000 iterations, and in fact this PR is making things worse.
Now I tested this PR + the weird hack (on ARM)
```
Results:
recall latency(ms) netCPU avgCpuCount nDoc nIters
0.986 15.900 15.900 1.000 500000 100 10
0.989 14.270 14.280 1.001 500000 100 100
0.986 13.219 13.220 1.000 500000 100 1000
0.983 13.508 13.508 1.000 500000 100 2500
```
and finally this PR (no weird hotspot hack):
```
recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn
beamWidth quantized visited index(s) num_segments
0.986 16.000 16.000 1.000 500000 100 50 32
250 no 26404 0.00 3
0.989 14.640 14.650 1.001 500000 100 50 32
250 no 26253 0.00 3
0.986 13.381 13.382 1.000 500000 100 50 32
250 no 25702 0.00 3
0.983 13.794 13.794 1.000 500000 100 50 32
250 no 26361 0.00 3
```
net-net I don't think we should merge this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
