zacharymorn commented on pull request #101: URL: https://github.com/apache/lucene/pull/101#issuecomment-834114823
> I played with your branch in order to try to speed things up a bit and I'm getting a bit closer to our current WAND implementation: https://github.com/jpountz/lucene/tree/LUCENE-9335-BMMScorer. Hopefully it's not too buggy. On the latest benchmark I ran it was 15% faster than WANScorer on your OrMedMedMedMedMed task. Oh wow thanks Adrien for trying it out! It's quite a speed up from -20 ~ 40% to +15%!! I also run it with the full benchmark queries and see it's making + 20% - 90% speed up for `OrHighHigh`, with the cost of -10 ~ 20% slow down for `OrHighLow` and `OrHighMed`, but it seems like a good net speedup given the latter twos already have much faster QPS compared to `OrHighHigh`? I took a quick look at your implementation and will study it more in the next few days, but right now I can see a few optimizations you applied and suggested earlier that can explain the speed up: 1. The use of faster methods on the priority queue `updateTop`, `topList` etc 2. The use of maxScore instead of `score()` to pick competitive doc from the simplified iterator 3. Pushing more scorers into `nonEssentialScorers` in `repartitionLists` The last two are optimization techniques not mentioned in the paper I think? By the way, please feel free to push directly into this PR branch as well, as I think your changes are in pretty good shape? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
