HUSTERGS opened a new pull request, #14834: URL: https://github.com/apache/lucene/pull/14834
### Description This PR propose to implement `nextDocsAndScores` in order to help the auto-vectorize of score calculation, I'v seen similar code under `TermScorer`. Basically this pr add a new `scoreRange` method for `MultiNormsLeafSimScorer` and do `freq()` and `norm()` before the actual score caculation. (I'm not very sure whether I should change the code under `sandbox`, but currently the luceneutil still use the `CombinedFieldQuery` under `sandbox`) https://github.com/mikemccand/luceneutil/blob/2c59f24382da40d980497ddc3ab413fa21b3d1ac/src/main/perf/TaskParser.java#L52 The luceneutil benchmark result (**which use `sandbox` version of `CombinedFieldQuery` by default**) on `wikimediumall` with `searchConcurrency=0`, `taskCountPerCat=5`, `taskRepeatCount=50` after 20 iterations: ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value AndStopWords 9.19 (4.2%) 8.92 (8.4%) -3.0% ( -14% - 9%) 0.151 OrStopWords 9.87 (5.3%) 9.59 (10.0%) -2.8% ( -17% - 13%) 0.266 AndHighHigh 26.30 (3.8%) 25.82 (9.1%) -1.8% ( -14% - 11%) 0.405 And3Terms 86.85 (3.6%) 85.43 (7.2%) -1.6% ( -12% - 9%) 0.366 OrMany 5.53 (4.5%) 5.45 (6.6%) -1.6% ( -12% - 9%) 0.372 OrHighHigh 25.09 (3.1%) 24.73 (8.9%) -1.4% ( -13% - 10%) 0.496 Or3Terms 78.24 (3.2%) 77.13 (7.9%) -1.4% ( -12% - 9%) 0.456 AndHighMed 67.33 (3.1%) 66.76 (8.1%) -0.9% ( -11% - 10%) 0.660 DismaxTerm 653.55 (4.1%) 648.63 (4.7%) -0.8% ( -9% - 8%) 0.587 AndMedOrHighHigh 20.67 (1.5%) 20.53 (3.2%) -0.7% ( -5% - 4%) 0.405 DismaxOrHighHigh 45.07 (2.7%) 44.84 (5.6%) -0.5% ( -8% - 7%) 0.715 OrHighMed 88.76 (3.9%) 88.37 (8.7%) -0.4% ( -12% - 12%) 0.837 Term10K 587.56 (5.1%) 585.84 (6.8%) -0.3% ( -11% - 12%) 0.878 Term 587.09 (5.0%) 586.16 (6.8%) -0.2% ( -11% - 12%) 0.933 TermB1M1P 586.61 (5.0%) 585.76 (6.8%) -0.1% ( -11% - 12%) 0.939 Term100 586.92 (5.1%) 586.16 (6.9%) -0.1% ( -11% - 12%) 0.946 Wildcard 58.83 (3.2%) 58.76 (3.7%) -0.1% ( -6% - 7%) 0.909 Term1M 586.18 (5.2%) 585.54 (6.7%) -0.1% ( -11% - 12%) 0.954 Prefix3 99.19 (2.1%) 99.09 (3.4%) -0.1% ( -5% - 5%) 0.907 AndHighOrMedMed 17.48 (3.4%) 17.47 (2.7%) -0.1% ( -6% - 6%) 0.938 DismaxOrHighMed 64.98 (3.3%) 64.96 (5.4%) -0.0% ( -8% - 8%) 0.986 FilteredPrefix3 92.50 (2.1%) 92.51 (3.2%) 0.0% ( -5% - 5%) 0.998 TermB1M 586.74 (5.0%) 586.88 (7.0%) 0.0% ( -11% - 12%) 0.990 IntNRQ 48.59 (1.5%) 48.62 (2.9%) 0.1% ( -4% - 4%) 0.940 CountAndHighHigh 61.27 (1.6%) 61.36 (1.7%) 0.1% ( -3% - 3%) 0.775 FilteredOrStopWords 10.84 (2.1%) 10.85 (3.0%) 0.2% ( -4% - 5%) 0.836 CountAndHighMed 92.67 (2.5%) 92.85 (3.3%) 0.2% ( -5% - 6%) 0.831 TermTitleSort 65.99 (6.0%) 66.13 (6.7%) 0.2% ( -11% - 13%) 0.911 CountFilteredOrMany 6.04 (1.8%) 6.05 (1.7%) 0.2% ( -3% - 3%) 0.654 FilteredIntNRQ 48.22 (1.5%) 48.35 (2.9%) 0.3% ( -4% - 4%) 0.722 SpanNear 3.04 (5.0%) 3.04 (5.4%) 0.3% ( -9% - 11%) 0.870 CountOrHighHigh 62.55 (2.9%) 62.76 (2.5%) 0.3% ( -4% - 5%) 0.701 CountFilteredIntNRQ 22.23 (1.9%) 22.31 (1.8%) 0.4% ( -3% - 4%) 0.538 FilteredOrMany 5.07 (1.9%) 5.09 (2.7%) 0.4% ( -4% - 5%) 0.620 FilteredAndHighMed 42.83 (2.8%) 43.00 (3.6%) 0.4% ( -5% - 7%) 0.699 TermDayOfYearSort 360.60 (2.0%) 362.11 (2.7%) 0.4% ( -4% - 5%) 0.574 CountFilteredOrHighHigh 25.21 (1.7%) 25.32 (1.4%) 0.4% ( -2% - 3%) 0.398 CombinedTerm 13.62 (3.0%) 13.68 (3.9%) 0.4% ( -6% - 7%) 0.685 CountPhrase 3.24 (3.0%) 3.26 (3.1%) 0.4% ( -5% - 6%) 0.644 OrHighRare 116.87 (6.0%) 117.40 (6.6%) 0.5% ( -11% - 13%) 0.818 Or2Terms2StopWords 75.68 (7.0%) 76.04 (9.3%) 0.5% ( -14% - 17%) 0.853 IntervalsOrdered 2.98 (2.3%) 3.00 (2.5%) 0.5% ( -4% - 5%) 0.517 CountFilteredOrHighMed 29.62 (1.9%) 29.76 (1.6%) 0.5% ( -2% - 4%) 0.380 Respell 45.01 (2.7%) 45.24 (3.4%) 0.5% ( -5% - 6%) 0.610 Fuzzy1 51.38 (4.5%) 51.69 (5.0%) 0.6% ( -8% - 10%) 0.689 CountOrHighMed 94.98 (2.7%) 95.56 (3.3%) 0.6% ( -5% - 6%) 0.519 CountOrMany 6.02 (4.0%) 6.05 (3.5%) 0.6% ( -6% - 8%) 0.601 FilteredAnd2Terms2StopWords 74.63 (6.3%) 75.10 (7.4%) 0.6% ( -12% - 15%) 0.770 FilteredAnd3Terms 131.44 (3.0%) 132.27 (3.5%) 0.6% ( -5% - 7%) 0.534 Fuzzy2 46.40 (4.6%) 46.70 (4.8%) 0.7% ( -8% - 10%) 0.662 And2Terms2StopWords 73.93 (8.1%) 74.43 (9.6%) 0.7% ( -15% - 19%) 0.810 CountTerm 7615.36 (3.9%) 7671.90 (5.3%) 0.7% ( -8% - 10%) 0.615 Phrase 9.62 (4.1%) 9.70 (3.3%) 0.8% ( -6% - 8%) 0.490 SloppyPhrase 1.43 (5.2%) 1.45 (4.7%) 1.0% ( -8% - 11%) 0.538 FilteredPhrase 12.52 (3.2%) 12.65 (3.2%) 1.0% ( -5% - 7%) 0.315 FilteredOrHighHigh 17.74 (3.1%) 17.92 (3.4%) 1.0% ( -5% - 7%) 0.320 FilteredAndStopWords 11.35 (2.7%) 11.48 (3.1%) 1.2% ( -4% - 7%) 0.182 TermMonthSort 2892.61 (3.0%) 2928.24 (4.3%) 1.2% ( -5% - 8%) 0.296 TermDTSort 195.96 (3.9%) 198.47 (4.5%) 1.3% ( -6% - 10%) 0.340 IntSet 409.96 (3.9%) 415.45 (4.7%) 1.3% ( -6% - 10%) 0.326 FilteredTerm 86.09 (4.7%) 87.32 (4.6%) 1.4% ( -7% - 11%) 0.335 FilteredOr3Terms 59.16 (4.6%) 60.00 (5.2%) 1.4% ( -7% - 11%) 0.355 FilteredAndHighHigh 14.21 (3.0%) 14.42 (3.3%) 1.5% ( -4% - 7%) 0.127 FilteredOrHighMed 53.47 (4.7%) 54.36 (5.0%) 1.7% ( -7% - 11%) 0.281 CountFilteredPhrase 11.71 (3.4%) 11.92 (3.2%) 1.9% ( -4% - 8%) 0.080 FilteredOr2Terms2StopWords 66.84 (6.4%) 68.29 (7.0%) 2.2% ( -10% - 16%) 0.308 CombinedOrHighHigh 6.53 (3.4%) 6.74 (5.1%) 3.3% ( -5% - 12%) 0.019 CombinedOrHighMed 25.78 (4.4%) 27.29 (4.6%) 5.9% ( -2% - 15%) 0.000 CombinedAndHighHigh 6.50 (3.4%) 6.93 (3.3%) 6.6% ( 0% - 13%) 0.000 CombinedAndHighMed 25.58 (4.5%) 28.00 (3.6%) 9.5% ( 1% - 18%) 0.000 ``` <!-- If this is your first contribution to Lucene, please make sure you have reviewed the contribution guide. https://github.com/apache/lucene/blob/main/CONTRIBUTING.md --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
