gf2121 opened a new pull request #2170:
URL: https://github.com/apache/lucene-solr/pull/2170


   # Description
   
   In [LUCENE-6539](https://issues.apache.org/jira/browse/LUCENE-6539), we 
introduced DocValuesNumbersQuery to benefit some special cases (many 
terms/numbers and fewish matching hits). Inspired by 
[LUCENE-7714](https://issues.apache.org/jira/browse/LUCENE-7714). maybe we can 
speed it up when meeting index sort since we can do a binary search to bound 
the docvalues before iter it.
   
   
   # Benchmarks
   
   I constructed some data by the following script:
   ```
   Directory directory = FSDirectory.open(Paths.get("./test"));
   IndexWriter indexWriter = new IndexWriter(directory,
           new IndexWriterConfig()
                   .setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND)
                   .setIndexSort(
                           new Sort(new SortedNumericSortField("a", 
SortField.Type.LONG))
                   ));
   for (int i = 0; i < TOTAL_DOC; i++) {
       int a = random.nextInt(1000000000);
       Document document = new Document();
       document.add(new LongPoint("a", a));
       document.add(new SortedNumericDocValuesField("a", a));
       indexWriter.addDocument(document);
   }
   
   indexWriter.flush();
   indexWriter.commit();
   indexWriter.close();
   ```
   And search them like this pattern:
   ```
   @Benchmark
   @BenchmarkMode({Mode.AverageTime})
   @Fork(1)
   @Measurement(iterations = 30)
   @Warmup(iterations = 10)
   public static void baseline(Blackhole blackhole) throws IOException {
       for (int i = 0; i < termsLen; i++) {
           terms[i] = random.nextInt(1000000000);
       }
       Query query = createQuery("a", terms);
       AtomicInteger res = new AtomicInteger(0);
       indexSearcher.search(query, new SimpleCollector() {
           @Override
           public void collect(int doc) throws IOException {
               res.set(doc ^ res.get());
           }
   
           @Override
           public ScoreMode scoreMode() {
               return ScoreMode.COMPLETE_NO_SCORES;
           }
       });
       blackhole.consume(res.get());
   }
   ```
   **Note**: the random in search use a same seed with writing random, so each 
long in the query can hit at least one doc.
   
   ## Results:
   
   **Based on 50,000,000 doc:**
   LongsCount | PointInSetQuery(s/op) | IndexSortDocValuesNumbersQuery(s/op) | 
DocValuesNumbersQuery(s/op)
   -- | -- | -- | --
   1000 | 0.144 ± 0.001 | 0.428 ± 0.008 | 1.159 ± 0.031
   5000 | 0.404 ± 0.003 | 0.424 ± 0.008 | 1.343 ± 0.022
   10000 | 0.661 ± 0.012 | 0.436 ± 0.008 | 1.372 ± 0.011
   50000 | 1.319 ± 0.011 | 0.450 ± 0.005 | 1.173 ± 0.017
   100000 | 1.328 ± 0.007 | 0.459 ± 0.003 | 1.160 ± 0.098
   
   **Based on 100,000,000 doc:**
   LongsCount | PointInSetQuery(s/op) | IndexSortDocValuesNumbersQuery(s/op) | 
DocValuesNumbersQuery(s/op)
   -- | -- | -- | --
   1000 | 0.189 ± 0.003 | 0.860 ± 0.028 | 2.408 ± 0.087
   5000 | 0.715 ± 0.010 | 0.879 ± 0.010 | 2.642 ± 0.105
   10000 | 1.075 ± 0.034 | 0.859 ± 0.010 | 2.629 ± 0.077
   50000 | 2.652 ± 0.114 | 0.865 ± 0.009 | 2.737 ± 0.099
   100000 | 2.905 ± 0.027 | 0.881 ± 0.005 | 2.820 ± 0.474
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to