[ 
https://issues.apache.org/jira/browse/LUCENE-10334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466709#comment-17466709
 ] 

Feng Guo edited comment on LUCENE-10334 at 12/30/21, 7:53 AM:
--------------------------------------------------------------

In order to save reading time, I deleted some previous progress comments and 
try to make a final summary here.

??one idea is we could try using the new block compression just for ordinals as 
a start??

Thanks [~rcmuir] for the suggestion! I made some optimizations in this approach 
and browse taxo tasks (Browse*TaxoFacets) are getting speed up too. So the 
benchmark is telling "dense faster sparse slower" instead of "SSDV faster Taxos 
slower" now. I suspect we probably did not see a SSDV regression just because 
we have not added reading sparse SSDV values tasks, e.g. a 
{{{}MedTermDaySSDVFacets{}}}.

I've got two schemes in mind so far:

*ForUtil Approach*

This approach tends to make file format friendly to block decoding and do block 
decoding for each get. As a result we can get a rather delicious (130%) speed 
up in {{Browse*}} tasks. But we also get a slight (10%) regression in tasks 
that reading facets with a query (like MedTermDayTaxoFacets) since we are 
reading sparse values there and we need to decompress the whole 128 values 
block even we only need one value in that block.

Here is the [code|https://github.com/apache/lucene/pull/562] and luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
         AndHighMedDayTaxoFacets       71.49      (2.1%)       64.72      
(2.0%)   -9.5% ( -13% -   -5%) 0.000
            MedTermDayTaxoFacets       25.79      (2.6%)       24.00      
(1.8%)   -6.9% ( -11% -   -2%) 0.000
        AndHighHighDayTaxoFacets       13.13      (3.4%)       12.63      
(3.1%)   -3.9% ( -10% -    2%) 0.000
          OrHighMedDayTaxoFacets       13.71      (4.1%)       13.41      
(4.7%)   -2.2% ( -10% -    6%) 0.118
                        PKLookup      204.87      (3.9%)      203.03      
(3.6%)   -0.9% (  -8% -    6%) 0.450
                         Prefix3      113.85      (3.6%)      113.32      
(4.6%)   -0.5% (  -8% -    8%) 0.724
                    HighSpanNear       25.34      (2.5%)       25.26      
(3.1%)   -0.3% (  -5% -    5%) 0.714
                     LowSpanNear       55.96      (2.0%)       55.80      
(2.1%)   -0.3% (  -4% -    3%) 0.658
                     MedSpanNear       56.84      (2.4%)       56.90      
(2.2%)    0.1% (  -4% -    4%) 0.895
                 MedSloppyPhrase       26.57      (1.8%)       26.60      
(1.9%)    0.1% (  -3% -    3%) 0.831
                HighSloppyPhrase       30.20      (3.7%)       30.24      
(3.6%)    0.2% (  -6% -    7%) 0.890
                       OrHighMed       49.96      (2.1%)       50.06      
(1.7%)    0.2% (  -3% -    4%) 0.742
                      AndHighMed       96.70      (2.9%)       96.95      
(2.6%)    0.3% (  -5% -    5%) 0.772
             LowIntervalsOrdered       23.32      (4.6%)       23.38      
(4.5%)    0.3% (  -8% -    9%) 0.856
                      OrHighHigh       38.09      (1.9%)       38.20      
(1.8%)    0.3% (  -3% -    4%) 0.643
                      TermDTSort      128.55     (14.7%)      128.94     
(11.6%)    0.3% ( -22% -   31%) 0.942
                          Fuzzy1       99.54      (7.1%)       99.86      
(8.0%)    0.3% ( -13% -   16%) 0.893
            HighIntervalsOrdered       15.58      (2.6%)       15.65      
(2.6%)    0.4% (  -4% -    5%) 0.636
                         Respell       63.96      (1.9%)       64.22      
(2.3%)    0.4% (  -3% -    4%) 0.542
                   OrHighNotHigh      611.12      (5.8%)      613.85      
(6.2%)    0.4% ( -10% -   13%) 0.814
             MedIntervalsOrdered       59.48      (5.2%)       59.75      
(5.1%)    0.5% (  -9% -   11%) 0.780
                     AndHighHigh       58.76      (3.0%)       59.16      
(3.0%)    0.7% (  -5% -    6%) 0.478
                   OrNotHighHigh      619.53      (6.0%)      623.79      
(7.1%)    0.7% ( -11% -   14%) 0.740
                      HighPhrase       31.00      (2.5%)       31.26      
(2.7%)    0.8% (  -4% -    6%) 0.307
                      AndHighLow      828.41      (5.9%)      835.65      
(7.1%)    0.9% ( -11% -   14%) 0.672
                    OrNotHighLow      986.46      (6.8%)      995.13     
(10.5%)    0.9% ( -15% -   19%) 0.752
            HighTermTitleBDVSort      110.39     (12.3%)      111.38     
(11.1%)    0.9% ( -20% -   27%) 0.807
                          IntNRQ      151.29      (2.6%)      152.96      
(3.5%)    1.1% (  -4% -    7%) 0.262
                         LowTerm     1876.18      (7.8%)     1897.19      
(8.3%)    1.1% ( -13% -   18%) 0.660
           HighTermDayOfYearSort      108.34     (18.9%)      109.87     
(17.4%)    1.4% ( -29% -   46%) 0.805
               HighTermMonthSort       65.84     (11.0%)       66.78     
(11.7%)    1.4% ( -19% -   27%) 0.689
                    OrHighNotMed      770.05      (5.3%)      782.54      
(8.8%)    1.6% ( -11% -   16%) 0.480
                        Wildcard      182.10      (5.5%)      185.24      
(7.2%)    1.7% ( -10% -   15%) 0.394
                 LowSloppyPhrase       33.75      (6.6%)       34.35      
(8.8%)    1.8% ( -12% -   18%) 0.478
                       MedPhrase      161.57      (3.8%)      164.62      
(6.1%)    1.9% (  -7% -   12%) 0.242
                    OrHighNotLow      679.46      (7.2%)      693.59      
(7.6%)    2.1% ( -11% -   18%) 0.374
                    OrNotHighMed      690.91      (7.4%)      706.15      
(8.8%)    2.2% ( -13% -   19%) 0.390
                        HighTerm     1388.14      (6.3%)     1420.26      
(7.8%)    2.3% ( -11% -   17%) 0.302
                       LowPhrase      410.16      (5.0%)      420.38      
(5.0%)    2.5% (  -7% -   13%) 0.114
                       OrHighLow      479.96      (5.1%)      492.39      
(5.7%)    2.6% (  -7% -   14%) 0.128
                         MedTerm     1575.41      (5.9%)     1618.88      
(8.2%)    2.8% ( -10% -   17%) 0.221
                          Fuzzy2       64.75      (8.3%)       66.76      
(8.3%)    3.1% ( -12% -   21%) 0.237
           BrowseMonthTaxoFacets       14.39     (12.1%)       18.58     
(17.3%)   29.1% (   0% -   66%) 0.000
     BrowseRandomLabelTaxoFacets       12.01      (8.5%)       17.01     
(18.2%)   41.6% (  13% -   74%) 0.000
            BrowseDateTaxoFacets       13.72     (11.2%)       19.83     
(26.5%)   44.5% (   6% -   92%) 0.000
       BrowseDayOfYearTaxoFacets       13.84     (11.5%)       20.03     
(27.4%)   44.8% (   5% -   94%) 0.000
     BrowseRandomLabelSSDVFacets       10.31      (2.6%)       17.72      
(4.2%)   71.9% (  63% -   80%) 0.000
           BrowseMonthSSDVFacets       15.56      (3.3%)       34.58     
(12.3%)  122.3% ( 103% -  142%) 0.000
       BrowseDayOfYearSSDVFacets       14.17      (2.9%)       32.91     
(11.6%)  132.3% ( 114% -  151%) 0.000
{code}
*Detect Warm Approach*

This approach is keeping the origin file format to keep the high efficiency of 
reading sparse values, but try to detect a dense reading and do some block 
decoding for reading dense values situations. To be specific, we assume user is 
reading dense values if more than 80% values were required in the first block. 
And then we can use block decoding for following gets. The approach is limited 
to be used in forward reading cases and won't be as efficient as ForUtil in 
block decoding but it is nearly a net win.

Here is the [code|https://github.com/apache/lucene/pull/570] and luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
            MedTermDayTaxoFacets       55.17      (6.5%)       53.38      
(7.1%)   -3.2% ( -15% -   11%) 0.129
                        Wildcard      309.31     (12.9%)      299.80     
(12.6%)   -3.1% ( -25% -   25%) 0.446
                    OrNotHighLow      696.18      (8.7%)      677.46     
(10.1%)   -2.7% ( -19% -   17%) 0.367
                        HighTerm     1183.05      (9.5%)     1151.67      
(9.2%)   -2.7% ( -19% -   17%) 0.368
                       OrHighMed      120.60      (7.0%)      117.55      
(7.8%)   -2.5% ( -16% -   13%) 0.279
          OrHighMedDayTaxoFacets        9.46      (7.5%)        9.25      
(6.7%)   -2.2% ( -15% -   12%) 0.320
                         Prefix3      177.41      (7.3%)      173.96     
(10.3%)   -1.9% ( -18% -   16%) 0.489
        AndHighHighDayTaxoFacets       28.81      (6.7%)       28.35      
(6.3%)   -1.6% ( -13% -   12%) 0.433
           BrowseMonthTaxoFacets       13.50     (13.4%)       13.30      
(6.6%)   -1.5% ( -18% -   21%) 0.658
         AndHighMedDayTaxoFacets       46.43      (7.8%)       45.75      
(7.3%)   -1.5% ( -15% -   14%) 0.540
                       MedPhrase      360.70      (8.5%)      355.47      
(8.3%)   -1.4% ( -16% -   16%) 0.587
                      AndHighMed      233.52      (6.7%)      230.19      
(7.0%)   -1.4% ( -14% -   13%) 0.510
            HighTermTitleBDVSort       72.17     (16.9%)       71.14     
(15.4%)   -1.4% ( -28% -   37%) 0.780
                    OrHighNotMed      659.68      (9.7%)      650.38     
(12.6%)   -1.4% ( -21% -   23%) 0.691
                      HighPhrase       73.05      (7.6%)       72.25      
(9.3%)   -1.1% ( -16% -   17%) 0.685
                      TermDTSort      123.29     (15.5%)      122.10     
(13.8%)   -1.0% ( -26% -   33%) 0.835
                          IntNRQ      167.75      (7.4%)      166.17      
(8.6%)   -0.9% ( -15% -   16%) 0.710
                   OrHighNotHigh      890.84     (13.2%)      883.31     
(11.5%)   -0.8% ( -22% -   27%) 0.828
                       OrHighLow      279.24      (7.5%)      276.97      
(6.6%)   -0.8% ( -13% -   14%) 0.718
                        PKLookup      198.13      (6.6%)      196.54      
(6.9%)   -0.8% ( -13% -   13%) 0.707
                 MedSloppyPhrase       94.28      (8.0%)       93.55      
(6.5%)   -0.8% ( -14% -   14%) 0.737
                      AndHighLow      574.70      (8.2%)      570.50      
(9.6%)   -0.7% ( -17% -   18%) 0.795
                    OrNotHighMed      717.34     (11.5%)      712.33     
(12.4%)   -0.7% ( -22% -   26%) 0.853
                     AndHighHigh       61.26      (7.2%)       60.84      
(6.3%)   -0.7% ( -13% -   13%) 0.753
                HighSloppyPhrase        6.56      (6.5%)        6.52      
(5.3%)   -0.7% ( -11% -   11%) 0.729
                 LowSloppyPhrase      159.12      (6.9%)      158.23      
(6.5%)   -0.6% ( -13% -   13%) 0.794
                       LowPhrase       88.55      (8.6%)       88.07      
(8.6%)   -0.5% ( -16% -   18%) 0.844
                     MedSpanNear       14.63      (6.1%)       14.55      
(5.3%)   -0.5% ( -11% -   11%) 0.786
                          Fuzzy2       24.31      (9.5%)       24.19      
(7.7%)   -0.5% ( -16% -   18%) 0.858
                         MedTerm     1440.59      (9.8%)     1433.53     
(11.5%)   -0.5% ( -19% -   23%) 0.885
                    HighSpanNear       23.52      (6.1%)       23.40      
(5.9%)   -0.5% ( -11% -   12%) 0.797
                      OrHighHigh       32.01      (8.4%)       31.96      
(5.7%)   -0.1% ( -13% -   15%) 0.948
                          Fuzzy1       82.31     (11.7%)       82.30     
(13.5%)   -0.0% ( -22% -   28%) 0.998
                    OrHighNotLow      724.27      (9.5%)      724.70     
(10.1%)    0.1% ( -17% -   21%) 0.985
           HighTermDayOfYearSort      156.00     (14.4%)      156.22     
(14.6%)    0.1% ( -25% -   33%) 0.975
                         Respell       68.40      (8.2%)       68.49      
(7.6%)    0.1% ( -14% -   17%) 0.955
             MedIntervalsOrdered        9.22      (7.4%)        9.23      
(6.9%)    0.2% ( -13% -   15%) 0.936
                   OrNotHighHigh      571.66      (8.1%)      572.72     
(10.9%)    0.2% ( -17% -   20%) 0.951
                     LowSpanNear       82.39      (7.1%)       82.57      
(4.4%)    0.2% ( -10% -   12%) 0.907
                         LowTerm     1355.15     (10.2%)     1358.89     
(10.0%)    0.3% ( -18% -   22%) 0.931
               HighTermMonthSort       58.72     (20.8%)       58.95     
(20.1%)    0.4% ( -33% -   52%) 0.950
             LowIntervalsOrdered      117.01      (7.2%)      117.50      
(6.2%)    0.4% ( -12% -   14%) 0.844
            HighIntervalsOrdered       13.76      (8.8%)       13.82      
(6.8%)    0.5% ( -13% -   17%) 0.841
     BrowseRandomLabelTaxoFacets       11.40     (10.0%)       13.34     
(15.0%)   17.0% (  -7% -   46%) 0.000
       BrowseDayOfYearTaxoFacets       13.32     (11.5%)       15.71     
(15.0%)   17.9% (  -7% -   50%) 0.000
            BrowseDateTaxoFacets       13.15     (11.3%)       15.64     
(14.6%)   18.9% (  -6% -   50%) 0.000
     BrowseRandomLabelSSDVFacets       10.03      (7.9%)       12.20     
(10.1%)   21.6% (   3% -   43%) 0.000
           BrowseMonthSSDVFacets       15.27      (6.9%)       19.16     
(10.8%)   25.4% (   7% -   46%) 0.000
       BrowseDayOfYearSSDVFacets       13.88      (6.0%)       18.71     
(11.8%)   34.7% (  15% -   55%) 0.000
{code}
I wonder which way you will prefer? or if you want to try some other ideas but 
having no time I'm willing to try too. Any feedback will be welcome :)


was (Author: gf2121):
In order to save reading time, I deleted some previous progress comments and 
try to make a final summary here.

??one idea is we could try using the new block compression just for ordinals as 
a start??

Thanks [~rcmuir] for the suggestion! I made some optimizations in this approach 
and browse taxo tasks (Browse*TaxoFacets) are getting speed up too. So the 
benchmark is telling "dense faster sparse slower" instead of "SSDV faster Taxos 
slower" now. I suspect we probably did not see a SSDV regression just because 
we have not added reading sparse SSDV values tasks, e.g. a 
{{MedTermDaySSDVFacets}}.

 I've got two schemes in mind so far:

*ForUtil Approach*

This approach tends to make file format friendly to block decoding and do block 
decoding for each get. As a result we can get a rather delicious (130%) speed 
up in {{Browse*}} tasks. But we also get a  slight (10%) regression in tasks 
that reading facets with a query (like MedTermDayTaxoFacets) since we are 
reading sparse values there and we need to decompress the whole 128 values 
block even we only need one value in that block.

Here is the [code|https://github.com/apache/lucene/pull/562] luceneutil 
benchmark:
{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
         AndHighMedDayTaxoFacets       71.49      (2.1%)       64.72      
(2.0%)   -9.5% ( -13% -   -5%) 0.000
            MedTermDayTaxoFacets       25.79      (2.6%)       24.00      
(1.8%)   -6.9% ( -11% -   -2%) 0.000
        AndHighHighDayTaxoFacets       13.13      (3.4%)       12.63      
(3.1%)   -3.9% ( -10% -    2%) 0.000
          OrHighMedDayTaxoFacets       13.71      (4.1%)       13.41      
(4.7%)   -2.2% ( -10% -    6%) 0.118
                        PKLookup      204.87      (3.9%)      203.03      
(3.6%)   -0.9% (  -8% -    6%) 0.450
                         Prefix3      113.85      (3.6%)      113.32      
(4.6%)   -0.5% (  -8% -    8%) 0.724
                    HighSpanNear       25.34      (2.5%)       25.26      
(3.1%)   -0.3% (  -5% -    5%) 0.714
                     LowSpanNear       55.96      (2.0%)       55.80      
(2.1%)   -0.3% (  -4% -    3%) 0.658
                     MedSpanNear       56.84      (2.4%)       56.90      
(2.2%)    0.1% (  -4% -    4%) 0.895
                 MedSloppyPhrase       26.57      (1.8%)       26.60      
(1.9%)    0.1% (  -3% -    3%) 0.831
                HighSloppyPhrase       30.20      (3.7%)       30.24      
(3.6%)    0.2% (  -6% -    7%) 0.890
                       OrHighMed       49.96      (2.1%)       50.06      
(1.7%)    0.2% (  -3% -    4%) 0.742
                      AndHighMed       96.70      (2.9%)       96.95      
(2.6%)    0.3% (  -5% -    5%) 0.772
             LowIntervalsOrdered       23.32      (4.6%)       23.38      
(4.5%)    0.3% (  -8% -    9%) 0.856
                      OrHighHigh       38.09      (1.9%)       38.20      
(1.8%)    0.3% (  -3% -    4%) 0.643
                      TermDTSort      128.55     (14.7%)      128.94     
(11.6%)    0.3% ( -22% -   31%) 0.942
                          Fuzzy1       99.54      (7.1%)       99.86      
(8.0%)    0.3% ( -13% -   16%) 0.893
            HighIntervalsOrdered       15.58      (2.6%)       15.65      
(2.6%)    0.4% (  -4% -    5%) 0.636
                         Respell       63.96      (1.9%)       64.22      
(2.3%)    0.4% (  -3% -    4%) 0.542
                   OrHighNotHigh      611.12      (5.8%)      613.85      
(6.2%)    0.4% ( -10% -   13%) 0.814
             MedIntervalsOrdered       59.48      (5.2%)       59.75      
(5.1%)    0.5% (  -9% -   11%) 0.780
                     AndHighHigh       58.76      (3.0%)       59.16      
(3.0%)    0.7% (  -5% -    6%) 0.478
                   OrNotHighHigh      619.53      (6.0%)      623.79      
(7.1%)    0.7% ( -11% -   14%) 0.740
                      HighPhrase       31.00      (2.5%)       31.26      
(2.7%)    0.8% (  -4% -    6%) 0.307
                      AndHighLow      828.41      (5.9%)      835.65      
(7.1%)    0.9% ( -11% -   14%) 0.672
                    OrNotHighLow      986.46      (6.8%)      995.13     
(10.5%)    0.9% ( -15% -   19%) 0.752
            HighTermTitleBDVSort      110.39     (12.3%)      111.38     
(11.1%)    0.9% ( -20% -   27%) 0.807
                          IntNRQ      151.29      (2.6%)      152.96      
(3.5%)    1.1% (  -4% -    7%) 0.262
                         LowTerm     1876.18      (7.8%)     1897.19      
(8.3%)    1.1% ( -13% -   18%) 0.660
           HighTermDayOfYearSort      108.34     (18.9%)      109.87     
(17.4%)    1.4% ( -29% -   46%) 0.805
               HighTermMonthSort       65.84     (11.0%)       66.78     
(11.7%)    1.4% ( -19% -   27%) 0.689
                    OrHighNotMed      770.05      (5.3%)      782.54      
(8.8%)    1.6% ( -11% -   16%) 0.480
                        Wildcard      182.10      (5.5%)      185.24      
(7.2%)    1.7% ( -10% -   15%) 0.394
                 LowSloppyPhrase       33.75      (6.6%)       34.35      
(8.8%)    1.8% ( -12% -   18%) 0.478
                       MedPhrase      161.57      (3.8%)      164.62      
(6.1%)    1.9% (  -7% -   12%) 0.242
                    OrHighNotLow      679.46      (7.2%)      693.59      
(7.6%)    2.1% ( -11% -   18%) 0.374
                    OrNotHighMed      690.91      (7.4%)      706.15      
(8.8%)    2.2% ( -13% -   19%) 0.390
                        HighTerm     1388.14      (6.3%)     1420.26      
(7.8%)    2.3% ( -11% -   17%) 0.302
                       LowPhrase      410.16      (5.0%)      420.38      
(5.0%)    2.5% (  -7% -   13%) 0.114
                       OrHighLow      479.96      (5.1%)      492.39      
(5.7%)    2.6% (  -7% -   14%) 0.128
                         MedTerm     1575.41      (5.9%)     1618.88      
(8.2%)    2.8% ( -10% -   17%) 0.221
                          Fuzzy2       64.75      (8.3%)       66.76      
(8.3%)    3.1% ( -12% -   21%) 0.237
           BrowseMonthTaxoFacets       14.39     (12.1%)       18.58     
(17.3%)   29.1% (   0% -   66%) 0.000
     BrowseRandomLabelTaxoFacets       12.01      (8.5%)       17.01     
(18.2%)   41.6% (  13% -   74%) 0.000
            BrowseDateTaxoFacets       13.72     (11.2%)       19.83     
(26.5%)   44.5% (   6% -   92%) 0.000
       BrowseDayOfYearTaxoFacets       13.84     (11.5%)       20.03     
(27.4%)   44.8% (   5% -   94%) 0.000
     BrowseRandomLabelSSDVFacets       10.31      (2.6%)       17.72      
(4.2%)   71.9% (  63% -   80%) 0.000
           BrowseMonthSSDVFacets       15.56      (3.3%)       34.58     
(12.3%)  122.3% ( 103% -  142%) 0.000
       BrowseDayOfYearSSDVFacets       14.17      (2.9%)       32.91     
(11.6%)  132.3% ( 114% -  151%) 0.000
{code}

*Detect Warm Approach*

This approach is keeping the origin file format to keep the high efficiency of 
reading sparse values, but try to detect a dense reading and do some block 
decoding for reading dense values situations. To be specific, we assume user is 
reading dense values if more than 80% values were required in the first block. 
And then we can use block decoding for following gets. The approach is limited 
to be used in forward reading cases and won't be as efficient as ForUtil in 
block decoding but it is nearly a net win. 

Here is the [code|https://github.com/apache/lucene/pull/570] luceneutil 
benchmark:

{code:java}
                            TaskQPS baseline      StdDevQPS my_modified_version 
     StdDev                Pct diff p-value
            MedTermDayTaxoFacets       55.17      (6.5%)       53.38      
(7.1%)   -3.2% ( -15% -   11%) 0.129
                        Wildcard      309.31     (12.9%)      299.80     
(12.6%)   -3.1% ( -25% -   25%) 0.446
                    OrNotHighLow      696.18      (8.7%)      677.46     
(10.1%)   -2.7% ( -19% -   17%) 0.367
                        HighTerm     1183.05      (9.5%)     1151.67      
(9.2%)   -2.7% ( -19% -   17%) 0.368
                       OrHighMed      120.60      (7.0%)      117.55      
(7.8%)   -2.5% ( -16% -   13%) 0.279
          OrHighMedDayTaxoFacets        9.46      (7.5%)        9.25      
(6.7%)   -2.2% ( -15% -   12%) 0.320
                         Prefix3      177.41      (7.3%)      173.96     
(10.3%)   -1.9% ( -18% -   16%) 0.489
        AndHighHighDayTaxoFacets       28.81      (6.7%)       28.35      
(6.3%)   -1.6% ( -13% -   12%) 0.433
           BrowseMonthTaxoFacets       13.50     (13.4%)       13.30      
(6.6%)   -1.5% ( -18% -   21%) 0.658
         AndHighMedDayTaxoFacets       46.43      (7.8%)       45.75      
(7.3%)   -1.5% ( -15% -   14%) 0.540
                       MedPhrase      360.70      (8.5%)      355.47      
(8.3%)   -1.4% ( -16% -   16%) 0.587
                      AndHighMed      233.52      (6.7%)      230.19      
(7.0%)   -1.4% ( -14% -   13%) 0.510
            HighTermTitleBDVSort       72.17     (16.9%)       71.14     
(15.4%)   -1.4% ( -28% -   37%) 0.780
                    OrHighNotMed      659.68      (9.7%)      650.38     
(12.6%)   -1.4% ( -21% -   23%) 0.691
                      HighPhrase       73.05      (7.6%)       72.25      
(9.3%)   -1.1% ( -16% -   17%) 0.685
                      TermDTSort      123.29     (15.5%)      122.10     
(13.8%)   -1.0% ( -26% -   33%) 0.835
                          IntNRQ      167.75      (7.4%)      166.17      
(8.6%)   -0.9% ( -15% -   16%) 0.710
                   OrHighNotHigh      890.84     (13.2%)      883.31     
(11.5%)   -0.8% ( -22% -   27%) 0.828
                       OrHighLow      279.24      (7.5%)      276.97      
(6.6%)   -0.8% ( -13% -   14%) 0.718
                        PKLookup      198.13      (6.6%)      196.54      
(6.9%)   -0.8% ( -13% -   13%) 0.707
                 MedSloppyPhrase       94.28      (8.0%)       93.55      
(6.5%)   -0.8% ( -14% -   14%) 0.737
                      AndHighLow      574.70      (8.2%)      570.50      
(9.6%)   -0.7% ( -17% -   18%) 0.795
                    OrNotHighMed      717.34     (11.5%)      712.33     
(12.4%)   -0.7% ( -22% -   26%) 0.853
                     AndHighHigh       61.26      (7.2%)       60.84      
(6.3%)   -0.7% ( -13% -   13%) 0.753
                HighSloppyPhrase        6.56      (6.5%)        6.52      
(5.3%)   -0.7% ( -11% -   11%) 0.729
                 LowSloppyPhrase      159.12      (6.9%)      158.23      
(6.5%)   -0.6% ( -13% -   13%) 0.794
                       LowPhrase       88.55      (8.6%)       88.07      
(8.6%)   -0.5% ( -16% -   18%) 0.844
                     MedSpanNear       14.63      (6.1%)       14.55      
(5.3%)   -0.5% ( -11% -   11%) 0.786
                          Fuzzy2       24.31      (9.5%)       24.19      
(7.7%)   -0.5% ( -16% -   18%) 0.858
                         MedTerm     1440.59      (9.8%)     1433.53     
(11.5%)   -0.5% ( -19% -   23%) 0.885
                    HighSpanNear       23.52      (6.1%)       23.40      
(5.9%)   -0.5% ( -11% -   12%) 0.797
                      OrHighHigh       32.01      (8.4%)       31.96      
(5.7%)   -0.1% ( -13% -   15%) 0.948
                          Fuzzy1       82.31     (11.7%)       82.30     
(13.5%)   -0.0% ( -22% -   28%) 0.998
                    OrHighNotLow      724.27      (9.5%)      724.70     
(10.1%)    0.1% ( -17% -   21%) 0.985
           HighTermDayOfYearSort      156.00     (14.4%)      156.22     
(14.6%)    0.1% ( -25% -   33%) 0.975
                         Respell       68.40      (8.2%)       68.49      
(7.6%)    0.1% ( -14% -   17%) 0.955
             MedIntervalsOrdered        9.22      (7.4%)        9.23      
(6.9%)    0.2% ( -13% -   15%) 0.936
                   OrNotHighHigh      571.66      (8.1%)      572.72     
(10.9%)    0.2% ( -17% -   20%) 0.951
                     LowSpanNear       82.39      (7.1%)       82.57      
(4.4%)    0.2% ( -10% -   12%) 0.907
                         LowTerm     1355.15     (10.2%)     1358.89     
(10.0%)    0.3% ( -18% -   22%) 0.931
               HighTermMonthSort       58.72     (20.8%)       58.95     
(20.1%)    0.4% ( -33% -   52%) 0.950
             LowIntervalsOrdered      117.01      (7.2%)      117.50      
(6.2%)    0.4% ( -12% -   14%) 0.844
            HighIntervalsOrdered       13.76      (8.8%)       13.82      
(6.8%)    0.5% ( -13% -   17%) 0.841
     BrowseRandomLabelTaxoFacets       11.40     (10.0%)       13.34     
(15.0%)   17.0% (  -7% -   46%) 0.000
       BrowseDayOfYearTaxoFacets       13.32     (11.5%)       15.71     
(15.0%)   17.9% (  -7% -   50%) 0.000
            BrowseDateTaxoFacets       13.15     (11.3%)       15.64     
(14.6%)   18.9% (  -6% -   50%) 0.000
     BrowseRandomLabelSSDVFacets       10.03      (7.9%)       12.20     
(10.1%)   21.6% (   3% -   43%) 0.000
           BrowseMonthSSDVFacets       15.27      (6.9%)       19.16     
(10.8%)   25.4% (   7% -   46%) 0.000
       BrowseDayOfYearSSDVFacets       13.88      (6.0%)       18.71     
(11.8%)   34.7% (  15% -   55%) 0.000
{code}

I wonder which way you will prefer? or if you want to try some other ideas but 
having no time I'm willing to try too. Any feedback will be welcome :)

> Introduce a BlockReader based on ForUtil and use it for NumericDocValues
> ------------------------------------------------------------------------
>
>                 Key: LUCENE-10334
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10334
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/codecs
>            Reporter: Feng Guo
>            Priority: Major
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> Previous talk is here: [https://github.com/apache/lucene/pull/557]
> This is trying to add a new BlockReader based on ForUtil to replace the 
> DirectReader we are using for NumericDocvalues
> -*Benchmark based on wiki10m*- (Previous benchmark results are wrong so i 
> deleted it to avoid misleading, let's see the benchmark in comments.)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to