gsmiller opened a new pull request, #13657: URL: https://github.com/apache/lucene/pull/13657
After merging #13568 @mikemccand and @jpountz [noticed](https://github.com/apache/lucene/pull/13568#issuecomment-2288666763) some surprising nightly benchmark performance regressions. @epotyom and I were both able to [reproduce](https://github.com/apache/lucene/pull/13568#issuecomment-2289109976) the regressions locally with luceneutil and push a short-term change to unwind an isolated change to `IndexSearcher` that appeared to fix the regression (#13656). Since then, I've spend some time trying to really tightly isolate the change that could cause the performance regression. I've finally been able to reproduce it by just moving the creation of the "collectors list" as demonstrated in this demo PR (not to be merged!). I can't wrap my head around how the diff captured by this PR could possibly lead to the large benchmark impact I'm seeing with luceneutil (posted below). @mikemccand, @jpountz, @epotyom ... others... any thoughts on this? I'm stumped. But, connecting it back to how #13568 caused the regression, I think the explanation is that `CollectorOwner` initializes the collector list early during initialization, effectively having the same effect as this PR by creating the list earlier. Results of running luceneutil with wikimedium10m on this PR: ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value MedTerm 513.21 (4.9%) 369.43 (4.8%) -28.0% ( -35% - -19%) 0.000 HighTerm 523.20 (6.9%) 402.11 (5.0%) -23.1% ( -32% - -12%) 0.000 LowTerm 837.70 (3.9%) 715.94 (3.9%) -14.5% ( -21% - -6%) 0.000 BrowseDayOfYearSSDVFacets 11.97 (18.9%) 11.31 (11.9%) -5.5% ( -30% - 31%) 0.273 MedTermDayTaxoFacets 23.03 (4.9%) 21.95 (6.4%) -4.7% ( -15% - 6%) 0.009 HighPhrase 143.93 (8.3%) 139.35 (4.7%) -3.2% ( -14% - 10%) 0.136 Fuzzy2 53.03 (9.0%) 51.50 (7.3%) -2.9% ( -17% - 14%) 0.265 MedSpanNear 50.70 (5.1%) 49.26 (3.0%) -2.8% ( -10% - 5%) 0.032 LowPhrase 70.38 (4.9%) 68.60 (5.3%) -2.5% ( -12% - 8%) 0.118 MedPhrase 88.15 (5.2%) 86.03 (4.2%) -2.4% ( -11% - 7%) 0.105 OrHighMedDayTaxoFacets 7.01 (5.5%) 6.86 (5.4%) -2.0% ( -12% - 9%) 0.237 HighSpanNear 28.95 (2.7%) 28.42 (2.9%) -1.8% ( -7% - 3%) 0.043 MedSloppyPhrase 201.71 (3.3%) 198.58 (3.1%) -1.6% ( -7% - 4%) 0.124 BrowseDateTaxoFacets 23.97 (28.7%) 23.62 (22.8%) -1.5% ( -41% - 70%) 0.858 AndHighMedDayTaxoFacets 32.81 (5.8%) 32.35 (7.1%) -1.4% ( -13% - 12%) 0.493 AndHighHighDayTaxoFacets 27.86 (6.1%) 27.50 (6.5%) -1.3% ( -13% - 12%) 0.507 LowSloppyPhrase 149.20 (2.9%) 147.50 (3.0%) -1.1% ( -6% - 4%) 0.227 HighTermTitleBDVSort 66.72 (6.6%) 66.04 (5.7%) -1.0% ( -12% - 12%) 0.604 AndHighHigh 187.45 (7.4%) 185.75 (6.7%) -0.9% ( -13% - 14%) 0.684 LowSpanNear 102.21 (2.1%) 101.50 (1.5%) -0.7% ( -4% - 3%) 0.242 OrHighHigh 218.06 (6.3%) 216.74 (4.1%) -0.6% ( -10% - 10%) 0.721 HighTermTitleSort 132.14 (1.5%) 131.93 (1.3%) -0.2% ( -2% - 2%) 0.724 HighSloppyPhrase 31.43 (5.4%) 31.39 (6.6%) -0.1% ( -11% - 12%) 0.949 BrowseRandomLabelSSDVFacets 7.91 (10.2%) 7.91 (11.2%) -0.0% ( -19% - 23%) 0.992 AndHighMed 288.24 (4.9%) 288.33 (4.0%) 0.0% ( -8% - 9%) 0.982 AndHighLow 1339.09 (3.2%) 1345.87 (4.8%) 0.5% ( -7% - 8%) 0.694 OrHighMed 473.22 (3.9%) 476.21 (3.8%) 0.6% ( -6% - 8%) 0.603 BrowseDayOfYearTaxoFacets 23.67 (28.7%) 23.82 (23.5%) 0.6% ( -40% - 74%) 0.938 HighTermDayOfYearSort 415.29 (5.2%) 418.26 (5.9%) 0.7% ( -9% - 12%) 0.684 BrowseDateSSDVFacets 2.14 (21.4%) 2.16 (22.4%) 1.0% ( -35% - 56%) 0.887 Wildcard 489.21 (4.3%) 494.69 (4.5%) 1.1% ( -7% - 10%) 0.420 TermDTSort 216.56 (5.9%) 219.04 (4.8%) 1.1% ( -8% - 12%) 0.499 PKLookup 139.24 (8.7%) 140.89 (10.8%) 1.2% ( -16% - 22%) 0.703 Fuzzy1 74.44 (9.7%) 75.42 (8.3%) 1.3% ( -15% - 21%) 0.643 Respell 48.52 (7.2%) 49.20 (6.6%) 1.4% ( -11% - 16%) 0.519 OrNotHighLow 1260.39 (3.0%) 1279.03 (2.7%) 1.5% ( -4% - 7%) 0.101 MedIntervalsOrdered 132.03 (9.2%) 134.25 (12.6%) 1.7% ( -18% - 25%) 0.630 BrowseMonthTaxoFacets 24.51 (26.9%) 25.02 (25.5%) 2.0% ( -39% - 74%) 0.804 HighTermMonthSort 1117.15 (4.1%) 1143.38 (4.6%) 2.3% ( -6% - 11%) 0.090 BrowseRandomLabelTaxoFacets 15.54 (25.0%) 15.93 (19.7%) 2.5% ( -33% - 62%) 0.724 Prefix3 667.73 (11.1%) 684.51 (11.1%) 2.5% ( -17% - 27%) 0.474 LowIntervalsOrdered 118.38 (14.5%) 121.55 (14.8%) 2.7% ( -23% - 37%) 0.564 HighIntervalsOrdered 30.52 (9.2%) 31.34 (7.0%) 2.7% ( -12% - 20%) 0.298 OrNotHighMed 365.66 (5.9%) 376.73 (6.1%) 3.0% ( -8% - 15%) 0.110 OrHighLow 586.67 (5.7%) 608.48 (5.6%) 3.7% ( -7% - 15%) 0.037 OrNotHighHigh 257.09 (5.8%) 267.66 (6.5%) 4.1% ( -7% - 17%) 0.034 BrowseMonthSSDVFacets 11.21 (9.1%) 11.69 (7.1%) 4.3% ( -11% - 22%) 0.100 OrHighNotLow 446.78 (8.7%) 479.82 (7.1%) 7.4% ( -7% - 25%) 0.003 OrHighNotMed 591.66 (7.6%) 649.35 (4.8%) 9.8% ( -2% - 23%) 0.000 IntNRQ 202.12 (17.5%) 224.77 (28.1%) 11.2% ( -29% - 68%) 0.130 OrHighNotHigh 339.78 (8.0%) 393.02 (6.7%) 15.7% ( 0% - 33%) 0.000 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
