zacharymorn commented on pull request #113: URL: https://github.com/apache/lucene/pull/113#issuecomment-840293637
Ran wikibigall for BMM BulkScorer implementation with window following the suggestions from https://github.com/apache/lucene/pull/101#issuecomment-837909869, and got the following results: wikibigall run 1 ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value Fuzzy2 45.83 (9.9%) 41.93 (19.0%) -8.5% ( -34% - 22%) 0.076 Fuzzy1 56.71 (8.2%) 51.99 (14.4%) -8.3% ( -28% - 15%) 0.025 AndHighOrMedMed 43.61 (3.2%) 43.13 (2.7%) -1.1% ( -6% - 4%) 0.246 BrowseMonthTaxoFacets 8.47 (6.8%) 8.43 (6.8%) -0.4% ( -13% - 14%) 0.841 Wildcard 59.90 (2.7%) 59.66 (3.0%) -0.4% ( -5% - 5%) 0.658 Respell 49.17 (2.6%) 49.05 (3.5%) -0.3% ( -6% - 5%) 0.789 PKLookup 205.25 (5.3%) 204.79 (4.4%) -0.2% ( -9% - 10%) 0.883 BrowseDayOfYearSSDVFacets 17.57 (2.2%) 17.55 (2.3%) -0.2% ( -4% - 4%) 0.816 IntervalsOrdered 1.94 (3.8%) 1.94 (3.9%) -0.1% ( -7% - 7%) 0.913 BrowseMonthSSDVFacets 19.16 (2.2%) 19.14 (2.4%) -0.1% ( -4% - 4%) 0.866 BrowseDayOfYearTaxoFacets 7.32 (6.7%) 7.31 (6.8%) -0.1% ( -12% - 14%) 0.963 TermDateFacets 8.56 (6.5%) 8.56 (6.7%) -0.1% ( -12% - 14%) 0.974 Phrase 24.65 (3.1%) 24.64 (3.3%) -0.1% ( -6% - 6%) 0.948 BrowseDateTaxoFacets 7.64 (6.9%) 7.63 (7.0%) -0.0% ( -13% - 14%) 0.985 AndHighMed 44.58 (4.1%) 44.59 (3.8%) 0.0% ( -7% - 8%) 0.984 AndHighHigh 22.83 (4.3%) 22.84 (3.7%) 0.0% ( -7% - 8%) 0.980 AndMedOrHighHigh 29.32 (3.5%) 29.33 (4.0%) 0.0% ( -7% - 7%) 0.976 IntNRQ 122.51 (17.1%) 122.58 (17.2%) 0.1% ( -29% - 41%) 0.992 SpanNear 4.82 (3.0%) 4.83 (2.9%) 0.2% ( -5% - 6%) 0.818 Prefix3 114.88 (6.0%) 115.25 (6.7%) 0.3% ( -11% - 13%) 0.873 SloppyPhrase 1.19 (9.8%) 1.20 (9.9%) 0.3% ( -17% - 22%) 0.913 TermBGroup1M1P 24.69 (4.2%) 24.80 (4.1%) 0.4% ( -7% - 9%) 0.745 TermGroup1M 15.52 (2.8%) 15.60 (3.3%) 0.5% ( -5% - 6%) 0.587 Term 1018.96 (6.7%) 1026.53 (7.1%) 0.7% ( -12% - 15%) 0.734 TermBGroup1M 25.25 (2.9%) 25.44 (3.5%) 0.8% ( -5% - 7%) 0.454 VectorSearch 1014.64 (6.7%) 1023.16 (6.8%) 0.8% ( -11% - 15%) 0.695 TermGroup100 11.41 (4.0%) 11.51 (4.0%) 0.8% ( -6% - 9%) 0.506 TermMonthSort 63.33 (14.0%) 63.90 (14.0%) 0.9% ( -23% - 33%) 0.838 TermGroup10K 18.49 (3.0%) 18.74 (3.7%) 1.4% ( -5% - 8%) 0.196 TermTitleSort 179.96 (13.4%) 182.73 (14.1%) 1.5% ( -22% - 33%) 0.723 TermDTSort 57.89 (11.1%) 59.22 (14.1%) 2.3% ( -20% - 30%) 0.569 TermDayOfYearSort 55.40 (8.0%) 57.06 (11.4%) 3.0% ( -15% - 24%) 0.335 OrHighHigh 74.09 (3.6%) 79.35 (4.4%) 7.1% ( 0% - 15%) 0.000 OrHighMed 45.50 (3.8%) 56.92 (4.1%) 25.1% ( 16% - 34%) 0.000 ``` wikibigall run 2 ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value Fuzzy2 52.37 (9.0%) 47.63 (11.0%) -9.0% ( -26% - 12%) 0.004 Fuzzy1 60.48 (8.9%) 55.26 (10.1%) -8.6% ( -25% - 11%) 0.004 AndHighOrMedMed 24.96 (2.1%) 24.60 (3.6%) -1.4% ( -7% - 4%) 0.121 VectorSearch 819.06 (4.6%) 812.87 (4.9%) -0.8% ( -9% - 9%) 0.617 Respell 42.97 (3.9%) 42.67 (4.4%) -0.7% ( -8% - 7%) 0.599 BrowseMonthSSDVFacets 19.34 (1.3%) 19.22 (2.3%) -0.6% ( -4% - 2%) 0.290 AndMedOrHighHigh 25.32 (3.5%) 25.16 (3.8%) -0.6% ( -7% - 6%) 0.595 Phrase 21.60 (3.6%) 21.47 (3.8%) -0.6% ( -7% - 7%) 0.603 IntervalsOrdered 0.83 (5.4%) 0.82 (5.0%) -0.5% ( -10% - 10%) 0.760 TermGroup100 23.86 (4.3%) 23.75 (4.4%) -0.5% ( -8% - 8%) 0.724 TermDateFacets 10.44 (10.0%) 10.41 (9.4%) -0.4% ( -17% - 21%) 0.903 PKLookup 210.67 (5.9%) 209.94 (4.9%) -0.3% ( -10% - 11%) 0.841 TermGroup1M 10.60 (4.4%) 10.56 (4.7%) -0.3% ( -9% - 9%) 0.811 TermDayOfYearSort 47.25 (10.1%) 47.16 (10.3%) -0.2% ( -18% - 22%) 0.952 TermBGroup1M1P 15.66 (5.3%) 15.63 (5.1%) -0.2% ( -10% - 10%) 0.920 BrowseMonthTaxoFacets 8.22 (7.9%) 8.20 (7.7%) -0.2% ( -14% - 16%) 0.947 IntNRQ 271.37 (2.8%) 271.06 (3.1%) -0.1% ( -5% - 5%) 0.904 SpanNear 10.34 (2.1%) 10.33 (2.1%) -0.1% ( -4% - 4%) 0.895 TermGroup10K 10.47 (4.3%) 10.47 (4.2%) -0.0% ( -8% - 8%) 0.975 BrowseDayOfYearTaxoFacets 7.10 (7.7%) 7.09 (7.4%) -0.0% ( -14% - 16%) 0.986 BrowseDateTaxoFacets 7.41 (8.1%) 7.40 (7.7%) -0.0% ( -14% - 17%) 0.989 SloppyPhrase 5.58 (8.3%) 5.58 (7.7%) -0.0% ( -14% - 17%) 0.991 TermBGroup1M 25.52 (4.3%) 25.56 (4.5%) 0.1% ( -8% - 9%) 0.921 AndHighMed 44.59 (2.3%) 44.68 (3.2%) 0.2% ( -5% - 5%) 0.812 Wildcard 88.60 (3.9%) 88.83 (3.4%) 0.3% ( -6% - 7%) 0.818 Prefix3 48.71 (5.2%) 48.91 (4.8%) 0.4% ( -9% - 10%) 0.792 BrowseDayOfYearSSDVFacets 17.53 (2.0%) 17.63 (3.4%) 0.6% ( -4% - 6%) 0.490 AndHighHigh 31.50 (3.5%) 31.80 (4.4%) 1.0% ( -6% - 9%) 0.450 TermDTSort 163.80 (14.4%) 165.54 (14.3%) 1.1% ( -24% - 34%) 0.814 TermTitleSort 113.02 (9.9%) 114.44 (13.0%) 1.3% ( -19% - 26%) 0.730 TermMonthSort 63.15 (10.4%) 64.13 (13.3%) 1.5% ( -20% - 28%) 0.683 OrHighHigh 17.12 (2.5%) 17.65 (3.1%) 3.1% ( -2% - 8%) 0.000 Term 1038.79 (8.0%) 1077.30 (7.9%) 3.7% ( -11% - 21%) 0.142 OrHighMed 45.90 (2.4%) 56.69 (3.7%) 23.5% ( 16% - 30%) 0.000 ``` wikibigall run 3 ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value Fuzzy1 59.46 (9.9%) 53.54 (12.3%) -10.0% ( -29% - 13%) 0.005 Fuzzy2 61.37 (11.5%) 60.03 (13.4%) -2.2% ( -24% - 25%) 0.580 AndHighOrMedMed 33.91 (3.7%) 33.59 (3.6%) -1.0% ( -7% - 6%) 0.405 Prefix3 29.87 (12.4%) 29.76 (12.4%) -0.4% ( -22% - 27%) 0.924 AndMedOrHighHigh 28.14 (3.7%) 28.05 (4.2%) -0.3% ( -7% - 7%) 0.817 IntNRQ 272.56 (3.9%) 271.78 (3.1%) -0.3% ( -7% - 7%) 0.800 SpanNear 1.21 (16.7%) 1.21 (16.6%) -0.2% ( -28% - 39%) 0.966 IntervalsOrdered 2.03 (4.5%) 2.03 (4.9%) -0.2% ( -9% - 9%) 0.895 Phrase 55.81 (5.4%) 55.72 (4.9%) -0.2% ( -9% - 10%) 0.920 BrowseDayOfYearSSDVFacets 17.45 (1.8%) 17.43 (1.7%) -0.1% ( -3% - 3%) 0.821 Wildcard 37.74 (9.2%) 37.71 (9.6%) -0.1% ( -17% - 20%) 0.983 BrowseMonthSSDVFacets 19.28 (1.3%) 19.28 (1.3%) 0.0% ( -2% - 2%) 0.997 Respell 40.21 (3.9%) 40.25 (5.0%) 0.1% ( -8% - 9%) 0.940 TermBGroup1M 25.59 (4.1%) 25.65 (3.9%) 0.3% ( -7% - 8%) 0.837 BrowseDayOfYearTaxoFacets 7.06 (6.3%) 7.09 (6.2%) 0.5% ( -11% - 13%) 0.780 BrowseMonthTaxoFacets 8.14 (6.9%) 8.19 (6.7%) 0.6% ( -12% - 15%) 0.796 BrowseDateTaxoFacets 7.36 (6.5%) 7.40 (6.5%) 0.6% ( -11% - 14%) 0.785 SloppyPhrase 5.39 (4.5%) 5.42 (4.7%) 0.6% ( -8% - 10%) 0.693 TermDTSort 80.69 (8.2%) 81.24 (8.2%) 0.7% ( -14% - 18%) 0.794 AndHighHigh 18.01 (4.4%) 18.15 (4.9%) 0.8% ( -8% - 10%) 0.601 AndHighMed 56.72 (4.7%) 57.19 (5.2%) 0.8% ( -8% - 11%) 0.598 TermDateFacets 8.23 (8.0%) 8.30 (8.2%) 0.8% ( -14% - 18%) 0.742 TermMonthSort 94.14 (13.2%) 94.94 (15.8%) 0.9% ( -24% - 34%) 0.852 TermGroup1M 22.76 (3.7%) 22.96 (3.8%) 0.9% ( -6% - 8%) 0.460 TermGroup10K 10.46 (4.9%) 10.60 (5.3%) 1.3% ( -8% - 12%) 0.422 TermTitleSort 117.75 (13.5%) 119.36 (16.3%) 1.4% ( -25% - 35%) 0.772 Term 1079.43 (8.3%) 1094.78 (7.3%) 1.4% ( -13% - 18%) 0.565 TermDayOfYearSort 45.89 (8.1%) 46.58 (8.7%) 1.5% ( -14% - 19%) 0.572 PKLookup 207.04 (5.6%) 210.17 (5.3%) 1.5% ( -8% - 13%) 0.381 TermGroup100 13.92 (5.2%) 14.13 (5.6%) 1.5% ( -8% - 13%) 0.372 VectorSearch 1046.61 (6.7%) 1067.51 (6.9%) 2.0% ( -10% - 16%) 0.351 TermBGroup1M1P 18.24 (6.3%) 18.61 (6.5%) 2.0% ( -10% - 15%) 0.319 OrHighHigh 74.99 (5.3%) 81.16 (5.5%) 8.2% ( -2% - 20%) 0.000 OrHighMed 45.89 (3.2%) 57.40 (5.6%) 25.1% ( 15% - 34%) 0.000 ``` wikibigall run 4 ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value Fuzzy1 57.19 (8.5%) 52.98 (6.5%) -7.4% ( -20% - 8%) 0.002 Fuzzy2 45.44 (11.4%) 42.42 (11.4%) -6.6% ( -26% - 18%) 0.066 AndHighOrMedMed 31.75 (3.8%) 31.40 (3.7%) -1.1% ( -8% - 6%) 0.365 AndMedOrHighHigh 25.00 (3.7%) 24.77 (3.5%) -0.9% ( -7% - 6%) 0.413 SloppyPhrase 5.40 (5.3%) 5.36 (5.4%) -0.7% ( -10% - 10%) 0.674 VectorSearch 807.06 (5.0%) 802.68 (5.6%) -0.5% ( -10% - 10%) 0.747 Prefix3 184.66 (4.6%) 183.98 (4.9%) -0.4% ( -9% - 9%) 0.807 AndHighHigh 65.28 (3.3%) 65.10 (4.0%) -0.3% ( -7% - 7%) 0.815 TermGroup10K 15.61 (3.7%) 15.58 (4.0%) -0.2% ( -7% - 7%) 0.874 TermGroup1M 10.44 (3.9%) 10.42 (4.8%) -0.2% ( -8% - 8%) 0.898 TermBGroup1M 21.55 (4.2%) 21.52 (5.1%) -0.1% ( -9% - 9%) 0.919 TermDateFacets 10.89 (9.2%) 10.87 (9.7%) -0.1% ( -17% - 20%) 0.972 IntervalsOrdered 0.81 (4.6%) 0.81 (4.5%) 0.0% ( -8% - 9%) 0.994 BrowseMonthSSDVFacets 18.99 (1.5%) 18.99 (1.2%) 0.0% ( -2% - 2%) 0.973 BrowseDateTaxoFacets 7.18 (7.2%) 7.18 (7.2%) 0.0% ( -13% - 15%) 0.985 SpanNear 2.08 (3.9%) 2.08 (3.7%) 0.1% ( -7% - 8%) 0.962 TermGroup100 13.72 (4.2%) 13.73 (5.0%) 0.1% ( -8% - 9%) 0.964 BrowseMonthTaxoFacets 8.00 (7.5%) 8.01 (7.7%) 0.1% ( -14% - 16%) 0.971 BrowseDayOfYearSSDVFacets 17.14 (2.5%) 17.16 (2.7%) 0.1% ( -4% - 5%) 0.910 TermDTSort 45.05 (8.1%) 45.10 (7.5%) 0.1% ( -14% - 17%) 0.964 TermTitleSort 88.10 (12.0%) 88.20 (11.2%) 0.1% ( -20% - 26%) 0.973 BrowseDayOfYearTaxoFacets 6.87 (6.7%) 6.88 (6.9%) 0.1% ( -12% - 14%) 0.949 AndHighMed 55.06 (3.4%) 55.15 (3.1%) 0.2% ( -6% - 6%) 0.883 TermMonthSort 61.57 (12.2%) 61.72 (11.3%) 0.3% ( -20% - 26%) 0.946 Phrase 53.70 (4.9%) 53.84 (4.5%) 0.3% ( -8% - 10%) 0.863 IntNRQ 263.07 (5.8%) 263.89 (4.7%) 0.3% ( -9% - 11%) 0.851 TermBGroup1M1P 30.16 (4.3%) 30.34 (4.8%) 0.6% ( -8% - 10%) 0.691 Respell 46.63 (5.3%) 46.91 (5.0%) 0.6% ( -9% - 11%) 0.712 Wildcard 70.44 (4.3%) 70.89 (3.9%) 0.6% ( -7% - 9%) 0.627 Term 950.32 (7.6%) 957.36 (7.9%) 0.7% ( -13% - 17%) 0.763 TermDayOfYearSort 46.36 (14.2%) 46.92 (15.2%) 1.2% ( -24% - 35%) 0.797 PKLookup 200.63 (7.4%) 203.46 (6.6%) 1.4% ( -11% - 16%) 0.524 OrHighHigh 10.27 (2.7%) 10.74 (3.0%) 4.5% ( -1% - 10%) 0.000 OrHighMed 43.96 (3.1%) 54.71 (4.3%) 24.4% ( 16% - 32%) 0.000 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
