[GitHub] [lucene] uschindler commented on issue #11838: Adding concurrency to query rewrite?
uschindler commented on issue #11838: URL: https://github.com/apache/lucene/issues/11838#issuecomment-1271537616 For AutomatonQuery it is also required to rewrite, because there are different implementations used. When you expand to a BooleanQuery or TermInSetQuery, you need to do that in the rewrite phase. My question is: What brings moving it to the Weight? It is still not multithreaded! Maybe a good idea would be to allow the heavy calculations in Weight and also Rewrite to execute in an Executor. By default we pass a SameThreadExecutor implementation (basically the executor just calls `run()`), but a query implementation might use multiple threads by splitting the work into smaller pieces. E.g. a BooleanQuery could rewrite all its clauses in a separate task. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] justinrsweeney closed pull request #2672: Justinsweeney/qa node testing
justinrsweeney closed pull request #2672: Justinsweeney/qa node testing URL: https://github.com/apache/lucene-solr/pull/2672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] zhaih commented on issue #11838: Adding concurrency to query rewrite?
zhaih commented on issue #11838: URL: https://github.com/apache/lucene/issues/11838#issuecomment-1271850864 > Maybe a good idea would be to allow the heavy calculations in Weight and also Rewrite to execute in an Executor. +1, although I probably would like to start with rewrite first since I'm not aware of any weight creation that could easily benefit from execute it in parallel. I'm thinking about something similar to what @uschindler suggest, basically create another public method `rewrite(IndexReader, Executor)` on Query and default it to just call `rewrite(IndexReader)` without any action using executor, while in the IndexSearcher we could always pass in whatever executor we have. And then from there we could work on individual query's rewriting implementation. Seems people are not going against this idea I'll get a PR out soon, and we can start from there :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene] zhaih opened a new pull request, #11840: GITHUB-11838 Add api to allow concurrent query rewrite
zhaih opened a new pull request, #11840: URL: https://github.com/apache/lucene/pull/11840 ### Description Issue: #11838 * Added one public method to `Query` to allow passing in an executor, and default to just call single thread version * In `IndexSearcher` we always pass in the executor * I'm not sure whether we should add a parameter to control the behavior, I'm inclined to believe that users who passed in the executor are willing to make use of concurrent rewrite. * Rewritten ConstantScoreQuery so that the query executor will be passed into inner query. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
