javanna commented on PR #13542:
URL: https://github.com/apache/lucene/pull/13542#issuecomment-2332114836

   Hey all, I have done some benchmarking with two main goals: 
   
   1) ensure there are no regressions introduced by the proposed change 
   2) ensure there is some performance gain when intra-segment is activated, as 
basic as its support is in this initial proposed step.
   
   
   I ran `wikimediumall` benchmarks with the default parameters and manually 
added count queries to the tasks executed. The default search concurrency is 
automatic, meaning it will create an executor based on the number of CPUs 
available. The index is not force merged, there are multiple segments.
   
   The first run is main (baseline) against my current branch 
(my_modified_version):
   
                               TaskQPS baseline      StdDevQPS 
my_modified_version      StdDev                Pct diff p-value
                          CountTerm     4252.98      (6.3%)     3928.20      
(5.2%)   -7.6% ( -18% -    4%) 0.000
               HighIntervalsOrdered       18.28      (5.7%)       17.80      
(5.4%)   -2.7% ( -12% -    8%) 0.129
                     CountOrHighMed      319.35      (5.3%)      311.55      
(4.5%)   -2.4% ( -11% -    7%) 0.118
                  HighTermMonthSort     1225.80      (3.8%)     1200.00      
(5.3%)   -2.1% ( -10% -    7%) 0.148
                        LowSpanNear       17.17      (5.3%)       16.89      
(4.4%)   -1.7% ( -10% -    8%) 0.280
                MedIntervalsOrdered       62.72      (4.4%)       61.69      
(5.7%)   -1.6% ( -11% -    8%) 0.310
                             Fuzzy1       65.29      (6.7%)       64.36      
(4.6%)   -1.4% ( -11% -   10%) 0.431
               MedTermDayTaxoFacets       17.35      (6.9%)       17.11      
(7.0%)   -1.4% ( -14% -   13%) 0.537
               HighTermTitleBDVSort       19.73      (5.3%)       19.50      
(4.1%)   -1.2% ( -10% -    8%) 0.441
                         AndHighLow      954.69      (4.2%)      943.83      
(4.1%)   -1.1% (  -9% -    7%) 0.384
          BrowseDayOfYearTaxoFacets        3.87      (8.4%)        3.83      
(4.9%)   -1.1% ( -13% -   13%) 0.603
               BrowseDateTaxoFacets        3.83      (8.8%)        3.79      
(5.9%)   -1.1% ( -14% -   14%) 0.641
           AndHighHighDayTaxoFacets       11.99      (4.9%)       11.86      
(6.5%)   -1.1% ( -11% -   10%) 0.556
                          LowPhrase      176.29      (3.3%)      174.85      
(3.7%)   -0.8% (  -7% -    6%) 0.460
        BrowseRandomLabelTaxoFacets        3.20      (3.9%)        3.18      
(4.3%)   -0.8% (  -8% -    7%) 0.547
                  HighTermTitleSort       97.01      (5.0%)       96.30      
(4.6%)   -0.7% (  -9% -    9%) 0.628
                           HighTerm      410.73      (5.6%)      407.91      
(7.1%)   -0.7% ( -12% -   12%) 0.736
                         HighPhrase       68.29      (4.3%)       67.87      
(4.0%)   -0.6% (  -8% -    7%) 0.641
                    CountOrHighHigh       34.41     (25.3%)       34.22     
(20.2%)   -0.5% ( -36% -   60%) 0.942
                      OrHighNotHigh      326.55      (6.4%)      324.92      
(5.6%)   -0.5% ( -11% -   12%) 0.793
                         AndHighMed      232.64      (3.5%)      231.61      
(4.8%)   -0.4% (  -8% -    8%) 0.739
                           PKLookup      163.46      (8.6%)      162.75      
(6.9%)   -0.4% ( -14% -   16%) 0.860
                       OrNotHighLow     1042.34      (3.9%)     1039.01      
(4.2%)   -0.3% (  -8% -    8%) 0.803
                   HighSloppyPhrase       19.17      (4.4%)       19.12      
(5.7%)   -0.3% (  -9% -   10%) 0.855
              BrowseMonthTaxoFacets        4.08      (5.0%)        4.07      
(7.4%)   -0.2% ( -11% -   12%) 0.908
                       HighSpanNear       17.99      (5.9%)       18.00      
(6.0%)    0.0% ( -11% -   12%) 0.984
             OrHighMedDayTaxoFacets        2.49      (7.3%)        2.50      
(6.7%)    0.2% ( -12% -   15%) 0.936
                       OrNotHighMed      296.13      (5.0%)      297.20      
(5.9%)    0.4% (  -9% -   11%) 0.833
                          OrHighMed      350.64      (4.2%)      352.22      
(4.6%)    0.5% (  -8% -    9%) 0.748
                          MedPhrase       60.88      (3.8%)       61.18      
(4.4%)    0.5% (  -7% -    8%) 0.695
                    CountAndHighMed      272.12      (3.4%)      273.55      
(4.8%)    0.5% (  -7% -    9%) 0.691
                            Respell       35.73      (4.9%)       35.93      
(6.7%)    0.6% ( -10% -   12%) 0.763
                    MedSloppyPhrase       19.80      (6.6%)       19.92      
(7.3%)    0.6% ( -12% -   15%) 0.778
                    LowSloppyPhrase       17.75      (4.3%)       17.86      
(3.7%)    0.6% (  -7% -    9%) 0.622
                            Prefix3     1050.17      (3.9%)     1057.01      
(4.8%)    0.7% (  -7% -    9%) 0.636
                           Wildcard      143.63      (4.1%)      144.58      
(4.3%)    0.7% (  -7% -    9%) 0.618
                        CountPhrase       13.59      (4.2%)       13.69      
(4.3%)    0.7% (  -7% -    9%) 0.608
                         OrHighHigh       60.33      (8.5%)       60.77     
(10.6%)    0.7% ( -16% -   21%) 0.810
                      OrNotHighHigh      311.92      (5.7%)      314.20      
(5.2%)    0.7% (  -9% -   12%) 0.672
                       OrHighNotMed      389.61      (5.9%)      394.22      
(5.1%)    1.2% (  -9% -   12%) 0.497
                          OrHighLow      640.22      (3.9%)      648.85      
(5.0%)    1.3% (  -7% -   10%) 0.343
                            MedTerm      796.68      (6.3%)      807.58      
(7.8%)    1.4% ( -11% -   16%) 0.542
                            LowTerm      779.26      (4.8%)      790.22      
(4.2%)    1.4% (  -7% -   10%) 0.324
                   CountAndHighHigh       73.71      (5.1%)       74.75      
(6.7%)    1.4% (  -9% -   13%) 0.453
                LowIntervalsOrdered       76.85      (4.9%)       77.98      
(5.9%)    1.5% (  -8% -   12%) 0.395
               BrowseDateSSDVFacets        1.01      (8.6%)        1.03      
(9.7%)    1.6% ( -15% -   21%) 0.584
            AndHighMedDayTaxoFacets       17.64      (5.3%)       17.96      
(4.7%)    1.8% (  -7% -   12%) 0.268
              HighTermDayOfYearSort      474.18      (6.2%)      482.93      
(5.1%)    1.8% (  -8% -   13%) 0.303
                        AndHighHigh      120.18      (5.4%)      122.60      
(5.0%)    2.0% (  -7% -   13%) 0.219
                        MedSpanNear       94.97      (4.1%)       97.02      
(4.6%)    2.2% (  -6% -   11%) 0.117
              BrowseMonthSSDVFacets        5.99     (17.5%)        6.12     
(11.0%)    2.3% ( -22% -   37%) 0.618
                       OrHighNotLow      611.75      (4.3%)      628.18      
(6.9%)    2.7% (  -8% -   14%) 0.143
        BrowseRandomLabelSSDVFacets        4.02      (7.4%)        4.17      
(7.4%)    3.6% ( -10% -   19%) 0.129
                         TermDTSort      173.73      (6.3%)      180.06      
(7.2%)    3.6% (  -9% -   18%) 0.087
          BrowseDayOfYearSSDVFacets        5.83     (14.1%)        6.07     
(15.6%)    4.1% ( -22% -   39%) 0.382
                             IntNRQ      595.22     (10.5%)      621.42      
(7.2%)    4.4% ( -12% -   24%) 0.122
                             Fuzzy2       50.46      (6.7%)       52.94      
(9.2%)    4.9% ( -10% -   22%) 0.054
   
   
   
   This makes sense as the only place where we can expect a bit of overhead is 
the additional `TotalHitCountCollectorManager` overhead associated with the 
customized `TotalHitCountCollector` that the manager returns to support 
intra-segment concurrency.
   
   
   
   
   The second run is main (baseline) against my current branch 
(my_modified_version) with the overhead in `TotalHitCountCollectorManager` 
removed:
   
   ```
   diff --git 
a/lucene/core/src/java/org/apache/lucene/search/TotalHitCountCollectorManager.java
 
b/lucene/core/src/java/org/apache/lucene/search/TotalHitCountCollectorManager.java
   index 50956725cb1..d95cef8c05d 100644
   --- 
a/lucene/core/src/java/org/apache/lucene/search/TotalHitCountCollectorManager.java
   +++ 
b/lucene/core/src/java/org/apache/lucene/search/TotalHitCountCollectorManager.java
   @@ -51,7 +51,8 @@ public class TotalHitCountCollectorManager
    
      @Override
      public TotalHitCountCollector newCollector() throws IOException {
   -    return new LeafPartitionAwareTotalHitCountCollector(earlyTerminatedMap);
   +    return new TotalHitCountCollector();
   +    //return new 
LeafPartitionAwareTotalHitCountCollector(earlyTerminatedMap);
      }
   ```
   
   
                               TaskQPS baseline      StdDevQPS 
my_modified_version      StdDev                Pct diff p-value
                         AndHighLow     1215.06      (5.0%)     1168.26      
(4.3%)   -3.9% ( -12% -    5%) 0.009
                       OrNotHighLow      934.09      (3.8%)      914.26      
(3.9%)   -2.1% (  -9% -    5%) 0.078
              BrowseMonthTaxoFacets        4.09      (5.0%)        4.04      
(6.8%)   -1.2% ( -12% -   11%) 0.511
                          CountTerm     4521.12      (8.9%)     4467.11      
(7.2%)   -1.2% ( -15% -   16%) 0.642
                         TermDTSort       83.56      (9.3%)       82.71     
(10.7%)   -1.0% ( -19% -   20%) 0.750
              BrowseMonthSSDVFacets        5.89     (12.4%)        5.86     
(13.4%)   -0.5% ( -23% -   28%) 0.907
                           HighTerm      560.23      (7.3%)      558.11      
(5.8%)   -0.4% ( -12% -   13%) 0.856
                           PKLookup      160.43      (6.9%)      159.88      
(6.0%)   -0.3% ( -12% -   13%) 0.868
                  HighTermMonthSort     1183.43      (4.5%)     1180.78      
(3.6%)   -0.2% (  -7% -    8%) 0.862
                         AndHighMed      480.37      (4.8%)      479.74      
(2.8%)   -0.1% (  -7% -    7%) 0.916
                         HighPhrase       93.71      (4.5%)       93.75      
(4.3%)    0.0% (  -8% -    9%) 0.972
                MedIntervalsOrdered       46.65      (5.2%)       46.69      
(5.1%)    0.1% (  -9% -   10%) 0.968
              HighTermDayOfYearSort      309.90      (5.0%)      310.31      
(4.3%)    0.1% (  -8% -    9%) 0.928
          BrowseDayOfYearSSDVFacets        5.37     (10.4%)        5.38     
(10.8%)    0.2% ( -19% -   23%) 0.957
               BrowseDateSSDVFacets        1.01      (8.7%)        1.01      
(9.3%)    0.3% ( -16% -   20%) 0.919
                           Wildcard      153.80      (5.2%)      154.33      
(5.0%)    0.3% (  -9% -   11%) 0.833
             OrHighMedDayTaxoFacets        4.12      (6.8%)        4.14      
(6.7%)    0.4% ( -12% -   14%) 0.865
                          OrHighMed      299.87      (3.8%)      301.12      
(3.3%)    0.4% (  -6% -    7%) 0.711
                      OrNotHighHigh      392.91      (5.3%)      394.61      
(5.2%)    0.4% (  -9% -   11%) 0.793
                    LowSloppyPhrase      109.01      (4.1%)      109.53      
(4.6%)    0.5% (  -7% -    9%) 0.731
                    CountAndHighMed       94.97      (4.0%)       95.42      
(6.4%)    0.5% (  -9% -   11%) 0.777
                          LowPhrase      437.29      (3.4%)      439.41      
(5.3%)    0.5% (  -8% -    9%) 0.733
                             Fuzzy1       69.22      (4.5%)       69.59      
(6.1%)    0.5% (  -9% -   11%) 0.755
                            LowTerm      691.97      (4.4%)      695.79      
(3.2%)    0.6% (  -6% -    8%) 0.649
                        MedSpanNear        9.42      (3.3%)        9.48      
(2.8%)    0.6% (  -5% -    7%) 0.514
                          MedPhrase       91.74      (4.8%)       92.34      
(5.4%)    0.7% (  -9% -   11%) 0.687
                        LowSpanNear       15.79      (4.6%)       15.91      
(4.8%)    0.8% (  -8% -   10%) 0.612
                            Respell       38.77      (8.4%)       39.08      
(7.3%)    0.8% ( -13% -   17%) 0.754
                   CountAndHighHigh       60.15      (4.8%)       60.64      
(4.6%)    0.8% (  -8% -   10%) 0.582
           AndHighHighDayTaxoFacets        7.59      (4.2%)        7.65      
(5.3%)    0.8% (  -8% -   10%) 0.574
                  HighTermTitleSort       95.06      (4.0%)       95.88      
(3.2%)    0.9% (  -6% -    8%) 0.448
                        CountPhrase       12.09      (3.3%)       12.21      
(2.5%)    1.0% (  -4% -    7%) 0.277
                             Fuzzy2       46.03      (5.4%)       46.50      
(5.8%)    1.0% (  -9% -   12%) 0.567
                             IntNRQ       42.07      (3.8%)       42.54      
(3.6%)    1.1% (  -5% -    8%) 0.333
                       HighSpanNear       25.89      (4.0%)       26.23      
(3.8%)    1.3% (  -6% -    9%) 0.289
                          OrHighLow      665.60      (3.8%)      674.34      
(2.8%)    1.3% (  -5% -    8%) 0.218
        BrowseRandomLabelTaxoFacets        3.14      (3.4%)        3.18      
(5.4%)    1.5% (  -6% -   10%) 0.283
        BrowseRandomLabelSSDVFacets        4.00      (8.3%)        4.06      
(9.8%)    1.5% ( -15% -   21%) 0.591
                    MedSloppyPhrase       75.80      (4.8%)       76.96      
(3.8%)    1.5% (  -6% -   10%) 0.262
          BrowseDayOfYearTaxoFacets        3.85      (9.0%)        3.91      
(8.2%)    1.6% ( -14% -   20%) 0.564
               HighIntervalsOrdered        7.43      (6.3%)        7.55      
(4.3%)    1.6% (  -8% -   12%) 0.338
                       OrNotHighMed      451.36      (6.9%)      458.85      
(7.2%)    1.7% ( -11% -   16%) 0.456
                   HighSloppyPhrase       24.07      (7.6%)       24.52      
(5.4%)    1.9% ( -10% -   16%) 0.362
               BrowseDateTaxoFacets        3.88      (8.3%)        3.96      
(8.4%)    2.0% ( -13% -   20%) 0.443
            AndHighMedDayTaxoFacets       32.87      (6.1%)       33.56      
(6.6%)    2.1% (  -9% -   15%) 0.293
                        AndHighHigh       80.31      (7.0%)       82.00      
(6.6%)    2.1% ( -10% -   16%) 0.329
               HighTermTitleBDVSort       32.34      (3.9%)       33.08      
(2.6%)    2.3% (  -4% -    9%) 0.028
                            Prefix3      131.67      (4.4%)      134.74      
(3.9%)    2.3% (  -5% -   11%) 0.075
                       OrHighNotMed      370.91      (7.0%)      380.92      
(5.7%)    2.7% (  -9% -   16%) 0.180
                       OrHighNotLow      445.51      (7.9%)      458.42      
(5.9%)    2.9% ( -10% -   18%) 0.189
                      OrHighNotHigh      396.82      (4.8%)      410.46      
(5.5%)    3.4% (  -6% -   14%) 0.035
                            MedTerm      528.83      (6.4%)      548.26      
(6.2%)    3.7% (  -8% -   17%) 0.066
                         OrHighHigh      103.91      (6.2%)      108.10      
(6.6%)    4.0% (  -8% -   17%) 0.046
               MedTermDayTaxoFacets       11.93      (4.8%)       12.49      
(6.5%)    4.7% (  -6% -   16%) 0.010
                LowIntervalsOrdered       86.39      (8.9%)       90.77      
(7.5%)    5.1% ( -10% -   23%) 0.052
                    CountOrHighHigh       53.24     (25.6%)       60.63     
(28.9%)   13.9% ( -32% -   91%) 0.108
                     CountOrHighMed       46.21     (25.8%)       53.82     
(31.0%)   16.5% ( -32% -   98%) 0.068
   
   
   Conclusion is that the small overhead on count is caused by the 
`TotalHitCountCollectorManager` changes. We have an option to opt-in on the new 
version of the manager, especially as the additional overhead is only needed 
for intra-segment concurrency. That does complicate a bit the API hence it's 
not entirely which direction we should go. In short: add an option to the 
manager to signal that you rely on intra-segment slicing when needed. It would 
be great to make this automatic (can be done in IndexSearcher#count) but there 
is not direct link between a collector manager and the searcher where it is 
used, so whether segments partitions are being searched is not known in the 
manager.
   
   
   The third and last run is main (baseline) against my current branch 
(my_modified_version) but enabling intra-segment concurrency as follows:
   
   ```
   diff --git 
a/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java 
b/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
   index 1c0bff93aa6..5f05177faca 100644
   --- a/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
   +++ b/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java
   @@ -331,7 +331,7 @@ public class IndexSearcher {
       * MAX_DOCS_PER_SLICE will get their own thread
       */
      protected LeafSlice[] slices(List<LeafReaderContext> leaves) {
   -    return slices(leaves, MAX_DOCS_PER_SLICE, MAX_SEGMENTS_PER_SLICE);
   +    return slicesWithPartitions(leaves, MAX_DOCS_PER_SLICE, 
MAX_SEGMENTS_PER_SLICE);
      }
    ```
   
                                TaskQPS baseline      StdDevQPS 
my_modified_version      StdDev                Pct diff p-value
                             IntNRQ       53.67     (30.0%)       22.99     
(13.1%)  -57.2% ( -77% -  -20%) 0.000
                  HighTermMonthSort     1152.98      (4.0%)      521.96      
(2.8%)  -54.7% ( -59% -  -49%) 0.000
                           Wildcard      191.62      (4.0%)       89.97      
(2.8%)  -53.0% ( -57% -  -48%) 0.000
                            Prefix3      583.57      (4.1%)      296.86      
(3.4%)  -49.1% ( -54% -  -43%) 0.000
              HighTermDayOfYearSort      376.24      (6.5%)      198.80      
(2.6%)  -47.2% ( -52% -  -40%) 0.000
                  HighTermTitleSort      106.06      (2.3%)       63.12      
(2.4%)  -40.5% ( -44% -  -36%) 0.000
                           HighTerm      600.13      (4.0%)      428.53      
(3.3%)  -28.6% ( -34% -  -22%) 0.000
                       OrHighNotLow      504.45      (4.8%)      369.99      
(3.7%)  -26.7% ( -33% -  -19%) 0.000
                          CountTerm     4185.03      (8.0%)     3094.54      
(5.7%)  -26.1% ( -36% -  -13%) 0.000
                            LowTerm      738.23      (3.9%)      564.97      
(2.9%)  -23.5% ( -29% -  -17%) 0.000
                       OrNotHighLow      873.49      (4.0%)      686.63      
(2.8%)  -21.4% ( -27% -  -15%) 0.000
                            MedTerm      591.39      (6.0%)      476.60      
(4.6%)  -19.4% ( -28% -   -9%) 0.000
               BrowseDateSSDVFacets        1.01      (9.3%)        0.86      
(6.5%)  -15.0% ( -28% -    0%) 0.000
                         AndHighLow     1216.35      (3.7%)     1036.91      
(4.4%)  -14.8% ( -21% -   -6%) 0.000
        BrowseRandomLabelSSDVFacets        3.99      (9.0%)        3.52      
(9.2%)  -11.7% ( -27% -    7%) 0.000
                       OrNotHighMed      392.27      (5.2%)      347.95      
(2.7%)  -11.3% ( -18% -   -3%) 0.000
          BrowseDayOfYearSSDVFacets        5.36     (16.0%)        4.76      
(8.5%)  -11.2% ( -30% -   15%) 0.006
                          OrHighLow      552.26      (7.5%)      490.96      
(3.1%)  -11.1% ( -20% -    0%) 0.000
              BrowseMonthTaxoFacets        4.01      (6.8%)        3.59      
(4.7%)  -10.6% ( -20% -    1%) 0.000
                       OrHighNotMed      314.53      (6.1%)      285.65      
(4.7%)   -9.2% ( -18% -    1%) 0.000
             OrHighMedDayTaxoFacets        4.91      (5.5%)        4.49      
(6.9%)   -8.6% ( -19% -    3%) 0.000
              BrowseMonthSSDVFacets        5.68     (15.8%)        5.19     
(11.7%)   -8.6% ( -31% -   22%) 0.051
                             Fuzzy1       83.62      (5.1%)       76.44      
(6.5%)   -8.6% ( -19% -    3%) 0.000
               BrowseDateTaxoFacets        3.75      (6.6%)        3.50     
(12.8%)   -6.8% ( -24% -   13%) 0.034
                           PKLookup      163.08      (4.8%)      153.32      
(4.5%)   -6.0% ( -14% -    3%) 0.000
          BrowseDayOfYearTaxoFacets        3.72      (6.0%)        3.51     
(13.6%)   -5.5% ( -23% -   14%) 0.095
        BrowseRandomLabelTaxoFacets        3.17      (3.3%)        3.02     
(13.7%)   -4.9% ( -21% -   12%) 0.118
            AndHighMedDayTaxoFacets       58.87      (6.9%)       56.23      
(7.0%)   -4.5% ( -17% -   10%) 0.041
                            Respell       32.63      (6.5%)       31.20      
(5.0%)   -4.4% ( -14% -    7%) 0.017
           AndHighHighDayTaxoFacets       10.71      (3.2%)       10.26      
(4.2%)   -4.2% ( -11% -    3%) 0.000
                         TermDTSort      139.91      (6.5%)      134.73      
(4.1%)   -3.7% ( -13% -    7%) 0.031
               MedTermDayTaxoFacets       20.89      (5.6%)       20.69      
(5.0%)   -1.0% ( -10% -   10%) 0.566
                             Fuzzy2       49.43      (5.4%)       50.64      
(5.8%)    2.4% (  -8% -   14%) 0.167
                      OrNotHighHigh      256.44      (5.6%)      262.89      
(4.5%)    2.5% (  -7% -   13%) 0.118
                      OrHighNotHigh      299.79      (5.5%)      332.13      
(6.2%)   10.8% (   0% -   23%) 0.000
                          OrHighMed      283.53      (4.2%)      354.04      
(5.1%)   24.9% (  14% -   35%) 0.000
                         HighPhrase      150.49      (3.7%)      198.04      
(8.7%)   31.6% (  18% -   45%) 0.000
                    MedSloppyPhrase      135.03      (4.9%)      182.50      
(6.2%)   35.2% (  23% -   48%) 0.000
                     CountOrHighMed      313.83      (3.8%)      435.20     
(10.5%)   38.7% (  23% -   55%) 0.000
                         AndHighMed      283.90      (2.8%)      422.91      
(4.9%)   49.0% (  40% -   58%) 0.000
                         OrHighHigh      114.64      (5.4%)      178.83     
(10.1%)   56.0% (  38% -   75%) 0.000
                    CountOrHighHigh       74.36     (24.4%)      125.75     
(40.5%)   69.1% (   3% -  177%) 0.000
                    CountAndHighMed      269.50      (5.1%)      472.96      
(7.9%)   75.5% (  59% -   93%) 0.000
                          LowPhrase       34.91      (4.2%)       63.27      
(8.0%)   81.2% (  66% -   97%) 0.000
                        AndHighHigh       34.49      (7.4%)       63.06      
(9.6%)   82.9% (  61% -  107%) 0.000
               HighTermTitleBDVSort       43.49      (5.3%)       79.60      
(6.9%)   83.1% (  67% -  100%) 0.000
                          MedPhrase       88.33      (4.0%)      167.26      
(6.9%)   89.4% (  75% -  104%) 0.000
                        MedSpanNear       24.23      (2.9%)       46.89      
(5.7%)   93.5% (  82% -  105%) 0.000
                   CountAndHighHigh       59.20      (4.2%)      117.07     
(12.4%)   97.8% (  77% -  119%) 0.000
                        CountPhrase       30.21      (4.2%)       60.12      
(9.8%)   99.0% (  81% -  118%) 0.000
                   HighSloppyPhrase       33.61      (5.4%)       67.40     
(10.6%)  100.5% (  80% -  123%) 0.000
                       HighSpanNear       15.22      (3.1%)       30.96      
(9.3%)  103.4% (  88% -  119%) 0.000
               HighIntervalsOrdered       31.78     (10.9%)       66.81     
(15.1%)  110.2% (  75% -  152%) 0.000
                LowIntervalsOrdered       97.00      (7.8%)      209.73     
(10.8%)  116.2% (  90% -  146%) 0.000
                        LowSpanNear       13.38      (3.1%)       29.03      
(8.5%)  117.0% ( 102% -  132%) 0.000
                    LowSloppyPhrase       44.38      (4.8%)      102.93     
(11.7%)  131.9% ( 110% -  155%) 0.000
                MedIntervalsOrdered        5.80      (6.0%)       14.09     
(12.5%)  142.7% ( 117% -  171%) 0.000
   
   
   I think that the regressions make sense in that they come from queries that 
require computation ahead of time at the segment level, which gets duplicated 
across segment partitions. In that case parallelizing makes things worse and 
will need additional work to address that. For other queries, there is quite a 
bit of value already. I'd say this is quite promising, given that the current 
slicing approach is pretty basic and I have not put any effort into optimizing 
it to get better benchmark results, and this index is not force merged. The max 
number of documents per slice is set at 250_000.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

Reply via email to