HUSTERGS opened a new pull request, #14896: URL: https://github.com/apache/lucene/pull/14896
### Description This PR is a follow-up of the [comment](https://github.com/apache/lucene/pull/14827#issuecomment-3018852667) from #14827 , trying to vectorize the `filterCompetitiveHits` function by utilizing `(Int|Float)Vector#compress`. I'm still working on it, tests are not added yet, nor is the code stable , comments and suggestions are welcomed ! But I do did a quick run of luceneutil based on `62e0276032189deee9559327cc53ac3f59f354a9` with `wikimediumall` with `searchConcurrency=0, taskCountPerCat=1, taskRepeatCount=20`, here is the result after 20 iterations, which seems to be promising (hope I didn't get anything wrong). Will do another run with different setup ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value TermMonthSort 1715.92 (5.2%) 1677.87 (6.7%) -2.2% ( -13% - 10%) 0.245 DismaxTerm 687.90 (3.6%) 672.66 (4.8%) -2.2% ( -10% - 6%) 0.099 FilteredPrefix3 145.56 (3.6%) 142.51 (5.1%) -2.1% ( -10% - 6%) 0.134 OrHighMed 124.23 (5.3%) 122.03 (12.4%) -1.8% ( -18% - 16%) 0.556 Wildcard 89.60 (2.7%) 88.11 (3.4%) -1.7% ( -7% - 4%) 0.087 Fuzzy2 25.52 (3.2%) 25.24 (4.1%) -1.1% ( -8% - 6%) 0.342 Respell 38.38 (2.6%) 37.99 (2.1%) -1.0% ( -5% - 3%) 0.172 TermDayOfYearSort 252.39 (3.3%) 250.17 (4.3%) -0.9% ( -8% - 6%) 0.463 Phrase 4.58 (2.5%) 4.54 (3.3%) -0.8% ( -6% - 5%) 0.378 CountPhrase 3.21 (2.1%) 3.19 (2.7%) -0.7% ( -5% - 4%) 0.352 DismaxOrHighMed 65.05 (3.6%) 64.59 (7.7%) -0.7% ( -11% - 11%) 0.709 FilteredOrStopWords 8.51 (2.8%) 8.46 (2.6%) -0.6% ( -5% - 4%) 0.507 TermTitleSort 63.32 (5.6%) 63.00 (5.0%) -0.5% ( -10% - 10%) 0.761 FilteredOrHighMed 21.50 (3.7%) 21.41 (3.7%) -0.4% ( -7% - 7%) 0.722 FilteredIntNRQ 292.41 (7.3%) 291.22 (7.9%) -0.4% ( -14% - 15%) 0.866 SpanNear 3.32 (3.5%) 3.31 (2.8%) -0.4% ( -6% - 6%) 0.703 CountOrHighHigh 68.53 (2.5%) 68.32 (2.3%) -0.3% ( -4% - 4%) 0.687 SloppyPhrase 0.61 (6.3%) 0.61 (5.0%) -0.3% ( -10% - 11%) 0.871 Term 575.10 (4.6%) 573.49 (8.2%) -0.3% ( -12% - 13%) 0.895 TermB1M 571.23 (5.4%) 569.72 (7.8%) -0.3% ( -12% - 13%) 0.901 FilteredOrHighHigh 17.20 (3.2%) 17.16 (3.5%) -0.2% ( -6% - 6%) 0.834 IntSet 339.77 (5.5%) 339.44 (6.1%) -0.1% ( -11% - 12%) 0.958 FilteredOr3Terms 43.39 (4.4%) 43.36 (4.2%) -0.1% ( -8% - 8%) 0.961 CountOrHighMed 98.19 (4.6%) 98.13 (4.2%) -0.1% ( -8% - 9%) 0.968 Prefix3 80.76 (5.2%) 80.73 (6.3%) -0.0% ( -10% - 12%) 0.985 CombinedTerm 17.50 (3.6%) 17.50 (4.8%) 0.0% ( -8% - 8%) 0.984 CombinedOrHighMed 48.08 (7.5%) 48.09 (8.3%) 0.0% ( -14% - 17%) 0.990 IntNRQ 28.63 (2.2%) 28.65 (2.1%) 0.1% ( -4% - 4%) 0.930 FilteredTerm 69.87 (5.9%) 69.91 (5.3%) 0.1% ( -10% - 11%) 0.972 FilteredOrMany 7.62 (2.4%) 7.62 (2.3%) 0.1% ( -4% - 4%) 0.927 CountFilteredIntNRQ 26.19 (2.5%) 26.21 (2.7%) 0.1% ( -4% - 5%) 0.921 Term1M 641.82 (5.9%) 642.71 (8.5%) 0.1% ( -13% - 15%) 0.953 TermDTSort 189.34 (3.4%) 189.71 (2.7%) 0.2% ( -5% - 6%) 0.838 Term10K 573.66 (5.0%) 574.80 (7.6%) 0.2% ( -11% - 13%) 0.922 FilteredOr2Terms2StopWords 70.27 (5.8%) 70.41 (5.5%) 0.2% ( -10% - 12%) 0.909 And3Terms 98.82 (3.2%) 99.02 (7.1%) 0.2% ( -9% - 10%) 0.906 TermB1M1P 569.87 (4.9%) 571.05 (7.8%) 0.2% ( -11% - 13%) 0.920 And2Terms2StopWords 162.85 (13.1%) 163.32 (12.2%) 0.3% ( -22% - 29%) 0.942 FilteredPhrase 6.13 (2.8%) 6.15 (3.4%) 0.3% ( -5% - 6%) 0.769 OrHighRare 53.22 (9.8%) 53.38 (9.1%) 0.3% ( -17% - 21%) 0.921 Fuzzy1 34.42 (2.7%) 34.53 (5.8%) 0.3% ( -7% - 9%) 0.828 Term100 634.74 (5.1%) 636.93 (7.7%) 0.3% ( -11% - 13%) 0.867 CountFilteredOrHighMed 30.06 (1.4%) 30.17 (1.4%) 0.4% ( -2% - 3%) 0.401 CountAndHighHigh 57.86 (1.6%) 58.09 (1.6%) 0.4% ( -2% - 3%) 0.445 FilteredAnd3Terms 91.01 (2.5%) 91.37 (2.6%) 0.4% ( -4% - 5%) 0.625 CountFilteredOrHighHigh 25.71 (1.6%) 25.83 (1.4%) 0.5% ( -2% - 3%) 0.307 CountFilteredPhrase 10.11 (3.5%) 10.17 (3.2%) 0.6% ( -5% - 7%) 0.602 IntervalsOrdered 3.73 (3.2%) 3.75 (2.5%) 0.6% ( -4% - 6%) 0.526 CountAndHighMed 111.65 (5.1%) 112.38 (5.2%) 0.6% ( -9% - 11%) 0.692 CountOrMany 8.24 (2.1%) 8.30 (2.1%) 0.7% ( -3% - 5%) 0.285 CountFilteredOrMany 5.95 (2.6%) 6.00 (1.7%) 0.8% ( -3% - 5%) 0.246 DismaxOrHighHigh 67.00 (5.7%) 67.55 (5.5%) 0.8% ( -9% - 12%) 0.645 AndHighOrMedMed 10.11 (2.8%) 10.19 (3.0%) 0.9% ( -4% - 6%) 0.351 FilteredAndHighMed 90.50 (2.2%) 91.38 (4.1%) 1.0% ( -5% - 7%) 0.354 FilteredAndHighHigh 14.62 (3.0%) 14.77 (3.1%) 1.1% ( -4% - 7%) 0.265 FilteredAnd2Terms2StopWords 100.48 (7.1%) 101.62 (7.4%) 1.1% ( -12% - 16%) 0.618 CombinedOrHighHigh 8.54 (4.2%) 8.64 (3.8%) 1.2% ( -6% - 9%) 0.352 CombinedAndHighMed 46.87 (7.6%) 47.55 (9.2%) 1.5% ( -14% - 19%) 0.584 FilteredAndStopWords 14.46 (3.7%) 14.71 (3.8%) 1.7% ( -5% - 9%) 0.138 OrHighHigh 24.60 (3.7%) 25.08 (14.6%) 1.9% ( -15% - 21%) 0.565 Or2Terms2StopWords 165.15 (10.1%) 168.56 (11.6%) 2.1% ( -17% - 26%) 0.550 CombinedAndHighHigh 7.29 (1.4%) 7.44 (2.9%) 2.1% ( -2% - 6%) 0.003 AndMedOrHighHigh 32.71 (2.3%) 33.47 (3.8%) 2.3% ( -3% - 8%) 0.021 CountTerm 3847.80 (6.9%) 3959.24 (11.0%) 2.9% ( -14% - 22%) 0.320 AndHighHigh 28.59 (3.8%) 29.49 (11.0%) 3.2% ( -11% - 18%) 0.225 OrStopWords 6.27 (7.1%) 6.57 (13.2%) 4.7% ( -14% - 26%) 0.165 AndHighMed 64.75 (3.5%) 69.44 (9.1%) 7.2% ( -5% - 20%) 0.001 OrMany 4.60 (4.3%) 5.01 (5.7%) 8.9% ( -1% - 19%) 0.000 Or3Terms 48.08 (4.3%) 52.48 (12.6%) 9.1% ( -7% - 27%) 0.002 AndStopWords 5.72 (5.0%) 6.34 (9.9%) 11.0% ( -3% - 27%) 0.000 ``` (BTW, The lastest luceneutil have some constructor problem since #14873 is introduced, will get error like below)  <!-- If this is your first contribution to Lucene, please make sure you have reviewed the contribution guide. https://github.com/apache/lucene/blob/main/CONTRIBUTING.md --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
