slow-J commented on issue #10472: URL: https://github.com/apache/lucene/issues/10472#issuecomment-1825871537
I took a quick look at this 3 years on. I took @mohammadsadiq's patch and applied it to `IDVersionSegmentTermsEnum` and `OrdsSegmentTermsEnum` I then changed the LinkedList to a ArrayDeque. I ran 2 benchmarks, both with wikibigall and JDK19 on an m5.12xlarge EC2 host. Test 1: https://github.com/slow-J/lucene/commit/e2f5e745f6523688f8bdff09e901aa346ac14d57 ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value IntNRQ 411.92 (3.2%) 399.44 (4.1%) -3.0% ( -10% - 4%) 0.009 HighPhrase 39.83 (8.4%) 38.94 (11.2%) -2.2% ( -20% - 18%) 0.473 AndHighMed 123.34 (2.6%) 121.53 (2.1%) -1.5% ( -6% - 3%) 0.051 OrHighHigh 53.33 (5.7%) 52.57 (1.7%) -1.4% ( -8% - 6%) 0.282 AndHighHigh 42.16 (4.3%) 41.69 (1.1%) -1.1% ( -6% - 4%) 0.260 OrNotHighMed 247.13 (3.0%) 244.44 (3.6%) -1.1% ( -7% - 5%) 0.295 MedSloppyPhrase 11.55 (7.5%) 11.44 (6.8%) -0.9% ( -14% - 14%) 0.681 MedPhrase 70.95 (3.5%) 70.35 (3.6%) -0.8% ( -7% - 6%) 0.451 HighSloppyPhrase 30.95 (4.9%) 30.75 (4.2%) -0.6% ( -9% - 8%) 0.653 BrowseMonthSSDVFacets 21.44 (11.2%) 21.34 (13.2%) -0.5% ( -22% - 26%) 0.907 BrowseDayOfYearSSDVFacets 21.19 (15.9%) 21.10 (15.0%) -0.4% ( -27% - 36%) 0.927 LowSloppyPhrase 50.43 (4.3%) 50.21 (4.0%) -0.4% ( -8% - 8%) 0.738 HighTermTitleSort 105.90 (1.6%) 105.44 (2.1%) -0.4% ( -4% - 3%) 0.453 Respell 29.29 (1.3%) 29.17 (1.4%) -0.4% ( -3% - 2%) 0.339 PKLookup 173.09 (2.1%) 172.41 (1.8%) -0.4% ( -4% - 3%) 0.527 OrNotHighLow 882.21 (2.7%) 879.44 (3.3%) -0.3% ( -6% - 5%) 0.744 MedIntervalsOrdered 28.81 (3.4%) 28.72 (3.4%) -0.3% ( -6% - 6%) 0.779 Prefix3 60.35 (3.3%) 60.18 (3.4%) -0.3% ( -6% - 6%) 0.800 BrowseMonthTaxoFacets 13.18 (1.0%) 13.15 (1.5%) -0.2% ( -2% - 2%) 0.578 Fuzzy2 52.06 (1.4%) 51.95 (1.3%) -0.2% ( -2% - 2%) 0.630 Fuzzy1 72.08 (1.1%) 71.93 (1.3%) -0.2% ( -2% - 2%) 0.595 AndHighLow 635.21 (1.9%) 633.97 (2.5%) -0.2% ( -4% - 4%) 0.785 Wildcard 75.55 (2.3%) 75.42 (1.9%) -0.2% ( -4% - 4%) 0.807 OrNotHighHigh 140.51 (4.8%) 140.33 (4.9%) -0.1% ( -9% - 10%) 0.936 AndHighMedDayTaxoFacets 44.82 (1.7%) 44.76 (2.3%) -0.1% ( -4% - 3%) 0.847 HighTermDayOfYearSort 345.49 (1.7%) 345.27 (1.5%) -0.1% ( -3% - 3%) 0.898 BrowseRandomLabelTaxoFacets 11.78 (0.7%) 11.77 (0.8%) -0.0% ( -1% - 1%) 0.872 LowSpanNear 45.65 (1.0%) 45.64 (1.3%) -0.0% ( -2% - 2%) 0.929 AndHighHighDayTaxoFacets 15.07 (2.5%) 15.07 (2.8%) -0.0% ( -5% - 5%) 0.971 HighTermTitleBDVSort 15.95 (5.4%) 15.95 (5.9%) -0.0% ( -10% - 11%) 0.987 MedSpanNear 10.13 (1.2%) 10.13 (1.2%) -0.0% ( -2% - 2%) 0.959 BrowseDateTaxoFacets 13.34 (0.6%) 13.34 (0.9%) -0.0% ( -1% - 1%) 0.970 OrHighMed 220.09 (2.8%) 220.08 (2.2%) -0.0% ( -4% - 5%) 0.999 HighIntervalsOrdered 7.05 (2.5%) 7.05 (2.6%) 0.0% ( -4% - 5%) 1.000 HighTermMonthSort 2701.34 (3.4%) 2701.37 (3.2%) 0.0% ( -6% - 6%) 0.999 LowIntervalsOrdered 40.23 (2.2%) 40.24 (2.3%) 0.0% ( -4% - 4%) 0.979 BrowseDateSSDVFacets 4.68 (4.2%) 4.68 (4.3%) 0.0% ( -8% - 8%) 0.984 OrHighNotHigh 152.28 (4.4%) 152.38 (4.6%) 0.1% ( -8% - 9%) 0.964 OrHighLow 449.79 (2.4%) 450.34 (2.0%) 0.1% ( -4% - 4%) 0.863 HighSpanNear 2.83 (1.1%) 2.83 (1.2%) 0.2% ( -2% - 2%) 0.615 OrHighNotMed 275.77 (4.2%) 276.49 (4.6%) 0.3% ( -8% - 9%) 0.850 LowTerm 489.45 (4.8%) 490.83 (5.5%) 0.3% ( -9% - 11%) 0.863 OrHighNotLow 233.89 (4.5%) 234.58 (5.0%) 0.3% ( -8% - 10%) 0.847 LowPhrase 272.48 (2.8%) 273.41 (2.4%) 0.3% ( -4% - 5%) 0.679 MedTermDayTaxoFacets 23.70 (5.1%) 23.82 (5.6%) 0.5% ( -9% - 11%) 0.776 HighTerm 258.55 (4.7%) 260.45 (5.8%) 0.7% ( -9% - 11%) 0.661 MedTerm 292.19 (4.5%) 294.49 (5.5%) 0.8% ( -8% - 11%) 0.623 BrowseDayOfYearTaxoFacets 14.07 (0.8%) 14.20 (3.0%) 0.9% ( -2% - 4%) 0.185 TermDTSort 119.84 (2.3%) 121.20 (5.8%) 1.1% ( -6% - 9%) 0.418 OrHighMedDayTaxoFacets 10.59 (3.6%) 10.71 (4.4%) 1.2% ( -6% - 9%) 0.357 BrowseRandomLabelSSDVFacets 15.07 (2.1%) 15.26 (7.4%) 1.3% ( -8% - 11%) 0.461 ``` Test 2: I initialised the ArrayDeque with size 256 https://github.com/apache/lucene/commit/9e3aa942ccfd37b06913cc0f63c8bcdb603ad866 ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value MedTermDayTaxoFacets 18.00 (7.4%) 17.41 (7.3%) -3.3% ( -16% - 12%) 0.154 BrowseMonthSSDVFacets 22.51 (18.7%) 21.84 (17.3%) -2.9% ( -32% - 40%) 0.605 HighTermMonthSort 2764.58 (5.3%) 2730.14 (4.0%) -1.2% ( -9% - 8%) 0.400 BrowseDayOfYearTaxoFacets 14.38 (5.1%) 14.22 (3.4%) -1.2% ( -9% - 7%) 0.406 OrHighNotHigh 166.23 (4.9%) 164.37 (6.5%) -1.1% ( -11% - 10%) 0.542 AndHighHighDayTaxoFacets 8.90 (3.6%) 8.82 (4.8%) -0.9% ( -9% - 7%) 0.506 OrHighNotLow 261.81 (4.5%) 259.71 (6.2%) -0.8% ( -10% - 10%) 0.639 OrNotHighHigh 168.90 (4.1%) 167.61 (5.4%) -0.8% ( -9% - 9%) 0.611 OrHighMedDayTaxoFacets 6.78 (4.1%) 6.73 (6.2%) -0.8% ( -10% - 9%) 0.648 OrHighMed 101.63 (3.7%) 100.94 (3.7%) -0.7% ( -7% - 6%) 0.558 HighPhrase 15.37 (5.3%) 15.27 (4.8%) -0.6% ( -10% - 9%) 0.690 AndHighMed 173.02 (2.1%) 171.94 (2.1%) -0.6% ( -4% - 3%) 0.350 OrHighNotMed 200.49 (4.0%) 199.36 (5.5%) -0.6% ( -9% - 9%) 0.714 HighTermTitleBDVSort 14.62 (6.3%) 14.54 (5.6%) -0.6% ( -11% - 12%) 0.767 OrHighHigh 24.46 (4.7%) 24.34 (4.8%) -0.5% ( -9% - 9%) 0.743 Respell 37.98 (3.6%) 37.83 (2.9%) -0.4% ( -6% - 6%) 0.694 OrNotHighMed 229.95 (3.9%) 229.03 (3.9%) -0.4% ( -7% - 7%) 0.746 OrNotHighLow 1055.24 (3.6%) 1051.45 (2.8%) -0.4% ( -6% - 6%) 0.723 AndHighHigh 66.77 (1.7%) 66.53 (1.7%) -0.4% ( -3% - 3%) 0.509 LowSloppyPhrase 43.57 (4.0%) 43.42 (4.4%) -0.3% ( -8% - 8%) 0.792 TermDTSort 244.36 (3.1%) 243.77 (3.8%) -0.2% ( -6% - 6%) 0.824 OrHighLow 227.67 (1.9%) 227.13 (2.6%) -0.2% ( -4% - 4%) 0.739 PKLookup 173.11 (1.6%) 172.74 (2.0%) -0.2% ( -3% - 3%) 0.705 Prefix3 29.77 (3.2%) 29.71 (3.2%) -0.2% ( -6% - 6%) 0.848 BrowseMonthTaxoFacets 13.12 (1.1%) 13.10 (1.3%) -0.2% ( -2% - 2%) 0.617 MedSpanNear 7.43 (4.3%) 7.42 (4.1%) -0.2% ( -8% - 8%) 0.884 LowSpanNear 2.87 (0.8%) 2.87 (0.8%) -0.1% ( -1% - 1%) 0.636 MedSloppyPhrase 16.85 (5.2%) 16.83 (6.2%) -0.1% ( -10% - 11%) 0.947 MedIntervalsOrdered 13.69 (1.8%) 13.67 (1.8%) -0.1% ( -3% - 3%) 0.859 AndHighMedDayTaxoFacets 40.33 (2.4%) 40.30 (2.7%) -0.1% ( -5% - 5%) 0.912 BrowseDateSSDVFacets 4.69 (4.8%) 4.69 (5.0%) -0.1% ( -9% - 10%) 0.961 Fuzzy1 68.89 (3.7%) 68.85 (3.1%) -0.1% ( -6% - 7%) 0.950 LowIntervalsOrdered 59.74 (3.8%) 59.71 (3.6%) -0.0% ( -7% - 7%) 0.968 HighIntervalsOrdered 1.32 (2.1%) 1.32 (2.2%) -0.0% ( -4% - 4%) 0.948 Fuzzy2 61.91 (3.4%) 61.88 (2.7%) -0.0% ( -5% - 6%) 0.966 MedPhrase 36.61 (3.8%) 36.59 (3.6%) -0.0% ( -7% - 7%) 0.978 HighSpanNear 0.92 (2.3%) 0.92 (1.9%) -0.0% ( -4% - 4%) 0.965 HighTermTitleSort 100.80 (2.7%) 100.80 (2.5%) -0.0% ( -5% - 5%) 0.996 HighSloppyPhrase 8.74 (3.9%) 8.75 (3.8%) 0.1% ( -7% - 8%) 0.943 HighTermDayOfYearSort 325.58 (2.0%) 326.25 (2.0%) 0.2% ( -3% - 4%) 0.745 AndHighLow 571.89 (3.4%) 573.47 (2.8%) 0.3% ( -5% - 6%) 0.777 BrowseDayOfYearSSDVFacets 20.39 (13.7%) 20.44 (13.9%) 0.3% ( -24% - 32%) 0.949 BrowseDateTaxoFacets 13.29 (0.8%) 13.33 (0.7%) 0.3% ( -1% - 1%) 0.227 IntNRQ 153.91 (2.4%) 154.36 (2.4%) 0.3% ( -4% - 5%) 0.693 BrowseRandomLabelTaxoFacets 11.77 (1.3%) 11.82 (0.8%) 0.4% ( -1% - 2%) 0.242 MedTerm 331.13 (4.8%) 332.84 (7.5%) 0.5% ( -11% - 13%) 0.794 Wildcard 98.63 (3.5%) 99.52 (2.5%) 0.9% ( -4% - 7%) 0.350 LowPhrase 8.23 (4.5%) 8.30 (3.8%) 0.9% ( -7% - 9%) 0.491 HighTerm 242.66 (4.8%) 245.43 (8.2%) 1.1% ( -11% - 14%) 0.592 BrowseRandomLabelSSDVFacets 15.57 (8.9%) 15.78 (10.5%) 1.3% ( -16% - 22%) 0.664 LowTerm 574.42 (5.2%) 583.94 (6.4%) 1.7% ( -9% - 13%) 0.367 ``` I don't think we're seeing a performance gain, but we do still have the performance impact of dealing with the `.stream().skip(` call to the queue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
