gsmiller edited a comment on pull request #597:
URL: https://github.com/apache/lucene/pull/597#issuecomment-1011212031
>The version of java didn't change. Mike was using java 17 well before
yesterday. Click on a day and you can see the JVM version that ran the
benchmark. It has been 17 for a while.
Thanks @rmuir for clarifying that.
>I suspect it is one of these facet changes:
LUCENE-10350
LUCENE-10250
LUCENE-10356
Yeah, that certainly makes the most sense. And @gf2121's reasoning about
what these different changes touch and how they could relate to these regressed
tasks also make sense, but I can't reason about how there could possibly be a
regression.
I also can't reproduce the regression locally. In fact, I'm seeing quite the
opposite (which makes sense given what @gf2121 observed through benchmarks when
making the change). Here are the results for `wikimedium10m` where the baseline
is the latest `main` and candidate is this proposed revert (rebased on top of
the latest `main`):
```
TaskQPS baseline StdDevQPS candidate
StdDev Pct diff p-value
BrowseDayOfYearTaxoFacets 30.81 (24.0%) 14.56
(4.5%) -52.7% ( -65% - -31%) 0.000
BrowseDateTaxoFacets 30.38 (23.9%) 14.52
(4.5%) -52.2% ( -65% - -31%) 0.000
BrowseMonthTaxoFacets 30.08 (20.9%) 15.73
(3.9%) -47.7% ( -59% - -29%) 0.000
BrowseRandomLabelTaxoFacets 23.21 (24.6%) 12.43
(4.4%) -46.4% ( -60% - -23%) 0.000
MedTermDayTaxoFacets 34.20 (4.5%) 33.65
(4.8%) -1.6% ( -10% - 7%) 0.273
TermDTSort 185.78 (20.4%) 183.85
(20.8%) -1.0% ( -35% - 50%) 0.873
AndHighHigh 53.56 (3.3%) 53.04
(3.3%) -1.0% ( -7% - 5%) 0.361
LowPhrase 97.47 (3.4%) 97.06
(2.1%) -0.4% ( -5% - 5%) 0.634
HighSpanNear 8.59 (4.2%) 8.55
(4.4%) -0.4% ( -8% - 8%) 0.761
OrHighLow 668.71 (1.6%) 666.14
(2.4%) -0.4% ( -4% - 3%) 0.546
AndHighMed 227.67 (1.8%) 226.80
(2.1%) -0.4% ( -4% - 3%) 0.533
OrHighMedDayTaxoFacets 7.26 (5.2%) 7.24
(5.8%) -0.4% ( -10% - 11%) 0.832
OrHighMed 117.01 (3.7%) 116.66
(2.9%) -0.3% ( -6% - 6%) 0.774
MedSpanNear 49.68 (4.0%) 49.55
(4.4%) -0.3% ( -8% - 8%) 0.847
LowSpanNear 45.83 (3.1%) 45.72
(2.7%) -0.2% ( -5% - 5%) 0.796
MedPhrase 95.22 (3.2%) 95.12
(2.2%) -0.1% ( -5% - 5%) 0.897
OrHighHigh 36.26 (3.7%) 36.22
(3.2%) -0.1% ( -6% - 7%) 0.918
IntNRQ 97.48 (1.4%) 97.57
(1.1%) 0.1% ( -2% - 2%) 0.822
MedSloppyPhrase 28.73 (2.4%) 28.76
(2.8%) 0.1% ( -4% - 5%) 0.912
BrowseRandomLabelSSDVFacets 9.48 (3.8%) 9.49
(3.5%) 0.1% ( -6% - 7%) 0.929
OrNotHighHigh 884.42 (3.0%) 885.49
(3.2%) 0.1% ( -5% - 6%) 0.902
LowIntervalsOrdered 90.92 (4.1%) 91.10
(3.9%) 0.2% ( -7% - 8%) 0.878
OrNotHighMed 1089.50 (2.2%) 1091.92
(3.0%) 0.2% ( -4% - 5%) 0.788
OrHighNotHigh 826.99 (3.7%) 829.46
(3.3%) 0.3% ( -6% - 7%) 0.787
HighIntervalsOrdered 7.48 (6.8%) 7.50
(6.5%) 0.3% ( -12% - 14%) 0.885
MedTerm 1890.85 (2.9%) 1896.72
(2.6%) 0.3% ( -5% - 5%) 0.721
MedIntervalsOrdered 6.54 (4.5%) 6.57
(4.2%) 0.4% ( -7% - 9%) 0.770
AndHighHighDayTaxoFacets 15.93 (3.1%) 16.00
(3.0%) 0.4% ( -5% - 6%) 0.662
LowTerm 1977.37 (2.7%) 1986.22
(3.5%) 0.4% ( -5% - 6%) 0.648
LowSloppyPhrase 86.15 (3.9%) 86.56
(4.4%) 0.5% ( -7% - 9%) 0.720
HighTerm 1494.75 (3.1%) 1501.82
(2.9%) 0.5% ( -5% - 6%) 0.622
Fuzzy2 67.47 (2.0%) 67.80
(2.1%) 0.5% ( -3% - 4%) 0.450
OrHighNotLow 1388.49 (3.0%) 1395.34
(2.4%) 0.5% ( -4% - 5%) 0.560
AndHighMedDayTaxoFacets 89.88 (2.3%) 90.38
(2.1%) 0.6% ( -3% - 5%) 0.418
BrowseMonthSSDVFacets 14.54 (20.6%) 14.63
(21.8%) 0.6% ( -34% - 54%) 0.927
HighSloppyPhrase 11.53 (4.3%) 11.60
(5.1%) 0.6% ( -8% - 10%) 0.678
HighTermTitleBDVSort 122.18 (14.6%) 122.97
(17.4%) 0.6% ( -27% - 38%) 0.898
HighPhrase 432.20 (3.3%) 435.35
(2.4%) 0.7% ( -4% - 6%) 0.426
Fuzzy1 81.65 (2.1%) 82.25
(2.2%) 0.7% ( -3% - 5%) 0.280
OrHighNotMed 1060.43 (3.0%) 1068.54
(3.0%) 0.8% ( -5% - 6%) 0.422
Respell 63.22 (2.5%) 63.78
(2.4%) 0.9% ( -3% - 5%) 0.255
AndHighLow 1108.01 (2.6%) 1118.78
(3.1%) 1.0% ( -4% - 6%) 0.287
PKLookup 171.13 (3.0%) 173.05
(4.8%) 1.1% ( -6% - 9%) 0.380
HighTermDayOfYearSort 37.22 (25.8%) 37.84
(22.8%) 1.7% ( -37% - 67%) 0.827
Wildcard 85.54 (5.3%) 87.11
(5.0%) 1.8% ( -8% - 12%) 0.262
OrNotHighLow 938.56 (3.1%) 955.98
(3.0%) 1.9% ( -4% - 8%) 0.054
Prefix3 146.06 (9.7%) 150.33
(8.5%) 2.9% ( -13% - 23%) 0.307
HighTermMonthSort 123.10 (14.9%) 126.94
(16.8%) 3.1% ( -24% - 40%) 0.534
BrowseDayOfYearSSDVFacets 12.11 (9.5%) 12.56
(12.9%) 3.7% ( -17% - 28%) 0.297
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
