benwtrent commented on PR #14085:
URL: https://github.com/apache/lucene/pull/14085#issuecomment-2578519157
Here are some benchmarks (100k float32[1024]).
Baseline:
```
recall latency (ms) nDoc topK fanout maxConn beamWidth visited
selectivity
0.915 0.950 100000 100 0 16 100 2054
0.95
0.918 0.950 100000 100 0 16 100 2128
0.90
0.924 1.090 100000 100 0 16 100 2417
0.75
0.935 1.430 100000 100 0 16 100 3357
0.50
0.962 2.740 100000 100 0 16 100 5846
0.25
1.000 9.530 100000 100 0 16 100 9882
0.10
1.000 4.750 100000 100 0 16 100 4913
0.05
1.000 2.100 100000 100 0 16 100 2507
0.03
1.000 0.970 100000 100 0 16 100 1023
0.01
0.975 1.660 100000 100 100 16 100 3545
0.95
0.977 1.680 100000 100 100 16 100 3682
0.90
0.977 2.120 100000 100 100 16 100 4218
0.75
0.984 2.720 100000 100 100 16 100 5789
0.50
0.990 5.190 100000 100 100 16 100 9889
0.25
1.000 9.900 100000 100 100 16 100 9883
0.10
1.000 4.740 100000 100 100 16 100 4913
0.05
1.000 2.150 100000 100 100 16 100 2507
0.03
1.000 0.970 100000 100 100 16 100 1023
0.01
```
Candidate:
```
recall latency (ms) nDoc topK fanout maxConn beamWidth visited
selectivity
0.852 1.330 100000 100 0 16 100 2723
0.95
0.821 1.510 100000 100 0 16 100 3011
0.90
0.793 1.680 100000 100 0 16 100 3329
0.75
0.795 1.730 100000 100 0 16 100 3357
0.50
0.880 1.820 100000 100 0 16 100 2912
0.25
0.891 1.430 100000 100 0 16 100 1692
0.10
0.823 1.320 100000 100 0 16 100 1070
0.05
0.765 1.170 100000 100 0 16 100 620
0.03
0.630 1.190 100000 100 0 16 100 341
0.01
0.942 2.210 100000 100 100 16 100 4786
0.95
0.934 2.500 100000 100 100 16 100 5360
0.90
0.927 2.950 100000 100 100 16 100 5983
0.75
0.925 3.370 100000 100 100 16 100 6005
0.50
0.956 3.160 100000 100 100 16 100 4711
0.25
0.947 2.380 100000 100 100 16 100 2560
0.10
0.895 2.160 100000 100 100 16 100 1569
0.05
0.842 1.910 100000 100 100 16 100 888
0.03
0.744 1.940 100000 100 100 16 100 482
0.01
```
You can see until about 50% selectivity, latency & recall are worse in
candidate. However, was we select even fewer than 50%, visited gets better, but
recall suffers.
This is likely because in the more restrictive cases we are actually
dropping to brute-force because of excessive exploration (note the 1.0 recall
in baseline).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
