gf2121 opened a new pull request #2170: URL: https://github.com/apache/lucene-solr/pull/2170
# Description In [LUCENE-6539](https://issues.apache.org/jira/browse/LUCENE-6539), we introduced DocValuesNumbersQuery to benefit some special cases (many terms/numbers and fewish matching hits). Inspired by [LUCENE-7714](https://issues.apache.org/jira/browse/LUCENE-7714). maybe we can speed it up when meeting index sort since we can do a binary search to bound the docvalues before iter it. # Benchmarks I constructed some data by the following script: ``` Directory directory = FSDirectory.open(Paths.get("./test")); IndexWriter indexWriter = new IndexWriter(directory, new IndexWriterConfig() .setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND) .setIndexSort( new Sort(new SortedNumericSortField("a", SortField.Type.LONG)) )); for (int i = 0; i < TOTAL_DOC; i++) { int a = random.nextInt(1000000000); Document document = new Document(); document.add(new LongPoint("a", a)); document.add(new SortedNumericDocValuesField("a", a)); indexWriter.addDocument(document); } indexWriter.flush(); indexWriter.commit(); indexWriter.close(); ``` And search them like this pattern: ``` @Benchmark @BenchmarkMode({Mode.AverageTime}) @Fork(1) @Measurement(iterations = 30) @Warmup(iterations = 10) public static void baseline(Blackhole blackhole) throws IOException { for (int i = 0; i < termsLen; i++) { terms[i] = random.nextInt(1000000000); } Query query = createQuery("a", terms); AtomicInteger res = new AtomicInteger(0); indexSearcher.search(query, new SimpleCollector() { @Override public void collect(int doc) throws IOException { res.set(doc ^ res.get()); } @Override public ScoreMode scoreMode() { return ScoreMode.COMPLETE_NO_SCORES; } }); blackhole.consume(res.get()); } ``` **Note**: the random in search use a same seed with writing random, so each long in the query can hit at least one doc. ## Results: **Based on 50,000,000 doc:** LongsCount | PointInSetQuery(s/op) | IndexSortDocValuesNumbersQuery(s/op) | DocValuesNumbersQuery(s/op) -- | -- | -- | -- 1000 | 0.144 ± 0.001 | 0.428 ± 0.008 | 1.159 ± 0.031 5000 | 0.404 ± 0.003 | 0.424 ± 0.008 | 1.343 ± 0.022 10000 | 0.661 ± 0.012 | 0.436 ± 0.008 | 1.372 ± 0.011 50000 | 1.319 ± 0.011 | 0.450 ± 0.005 | 1.173 ± 0.017 100000 | 1.328 ± 0.007 | 0.459 ± 0.003 | 1.160 ± 0.098 **Based on 100,000,000 doc:** LongsCount | PointInSetQuery(s/op) | IndexSortDocValuesNumbersQuery(s/op) | DocValuesNumbersQuery(s/op) -- | -- | -- | -- 1000 | 0.189 ± 0.003 | 0.860 ± 0.028 | 2.408 ± 0.087 5000 | 0.715 ± 0.010 | 0.879 ± 0.010 | 2.642 ± 0.105 10000 | 1.075 ± 0.034 | 0.859 ± 0.010 | 2.629 ± 0.077 50000 | 2.652 ± 0.114 | 0.865 ± 0.009 | 2.737 ± 0.099 100000 | 2.905 ± 0.027 | 0.881 ± 0.005 | 2.820 ± 0.474 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
