msokolov commented on PR #12255:
URL: https://github.com/apache/lucene/pull/12255#issuecomment-1552173316
Re-running with
comp = competition.Competition(verifyScores=False)
lets the benchmark complete without errors and I get this result:
```
TaskQPS baseline StdDevQPS candidate
StdDev Pct diff p-value
AndHighHighVector 182.88 (15.7%) 150.83
(14.8%) -17.5% ( -41% - 15%) 0.000
LowTermVector 260.54 (16.5%) 215.76
(14.0%) -17.2% ( -40% - 15%) 0.000
AndHighLowVector 201.95 (17.9%) 167.98
(14.3%) -16.8% ( -41% - 18%) 0.001
AndHighMedVector 208.19 (16.8%) 176.03
(14.4%) -15.4% ( -39% - 18%) 0.002
MedTermVector 267.48 (17.3%) 232.24
(16.9%) -13.2% ( -40% - 25%) 0.015
HighTermVector 179.05 (17.4%) 168.01
(16.6%) -6.2% ( -34% - 33%) 0.251
PKLookup 195.93 (3.0%) 198.82
(3.1%) 1.5% ( -4% - 7%) 0.126
```
I don't understand why, but it seems to me we ought to revert this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
