costin opened a new pull request, #16576:
URL: https://github.com/apache/lucene/pull/16576
Revisit the threshold in IndexOrDocValuesQuery for picking points vs DV
(especially now with DocVlauesSkipper).
With block-level skipping, DV is competitive with points for selective leads
in conjunctions. Reducing to 6x lets the DV path be chosen in the crossover
regime, yielding 1.78x throughput on selective-lead queries without regressing
any other case.
/4 was also tested but caused a 43% regression at 20% lead + 10M docs due
to BKD cost estimation noise at
the threshold boundary
### Benchmark
AMD EPYC 7R32 (c5a.2xlarge), JDK 25, 2 forks, 5 iters × 3s.
Index: monotonic timestamps + 100-bucket keyword field, forceMerge(1).
Queries: TermQuery lead (FILTER) + LongField.newRangeQuery (FILTER).
| method | lead | range | docs | /8 (ops/s) | /6 (ops/s) | ratio |
|-----------------|------|-------|------|-------------|-------------|-------|
| crossover10 | 10% | 80% | 1M | 8,634 ± 66 | 15,362 ± 51 | 1.78x |
| crossover10 | 10% | 80% | 10M | 285 ± 10 | 290 ± 1 | 1.02x |
| crossover20 | 20% | 80% | 1M | 6,948 ± 141 | 6,835 ± 72 | 0.98x |
| crossover20 | 20% | 80% | 10M | 183 ± 2 | 186 ± 0.4 | 1.02x |
| crossover30 | 30% | 80% | 1M | 5,764 ± 47 | 5,832 ± 37 | 1.01x |
| crossover30 | 30% | 80% | 10M | 72 ± 1 | 72 ± 1 | 1.01x |
| dvFavorable | 1% | 80% | 1M | 39,487 ± 161| 39,658 ± 371| 1.00x |
| dvFavorable | 1% | 80% | 10M | 4,508 ± 17 | 4,501 ± 18 | 1.00x |
| pointsFavorable | 50% | 5% | 1M | 4,693 ± 31 | 4,746 ± 18 | 1.01x |
| pointsFavorable | 50% | 5% | 10M | 2,350 ± 16 | 2,390 ± 13 | 1.02x |
The crossover10 case (10% lead + 80% range, 1M docs) shows 1.78x: the reduced
divisor switches from points to DV, and DV with skip blocks is faster for
this
selective-lead, broad-range pattern. All other cases show no regression.
Fix #16425
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
