epotyom commented on issue #13671: URL: https://github.com/apache/lucene/issues/13671#issuecomment-2320725824
Draft PR for this issue: https://github.com/apache/lucene/pull/13702 , ------ @gsmiller , > I'd like to see if we could bake this into IndexSearcher. > ... > The response types for drill-sideways are a bit of a mess at the moment as well IMO. Maybe your new response type could be general enough that we could move to using it in more places as a follow-up? I see what you mean now. Hmm, so the only difference between `DrillSideways#search(DrillDownQuery, CollectorManager, List<CollectorManager>)` method we are adding/changing https://github.com/apache/lucene/pull/13702 and `IndexSearcher#search(Query, CollectorManager)` that the latter takes only one CollectorManager. Maybe the missing piece here is a `CollectorManager` for drill sideways? Something like: ``` public class DrillSidewaysCollectorManager<C extends Collector, T, K extends Collector, R> implements CollectorManager<DrillSidewaysCollector<C, K>, DrillSidewaysCollectorManager.Result<T, R>> { @Override public DrillSidewaysCollector<C, K> newCollector() throws IOException { return new DrillSidewaysCollector<>(...); } @Override public Result<T, R> reduce(Collection<DrillSidewaysCollector> collectors) throws IOException { // TK: reduce every dimension } public record Result<T, R>(T drillDownResult, List<R> drillSidewaysResults) {} } ``` It will of cause require other changes for example: 1. Merge `DrillDownQuery` and `DrillSidewaysQuery` classes to have `Query` implementation that can be created by user directly and that creates Weight -> Score, etc 1. Alternatively, we can try returning `DrillSidewaysQuery` when `DrillDownQuery#rewrite` is called? 2. Cast leaf collector to the future `DrillSidewaysLeafCollector` in `DrillSidewaysScorer` to handle exact match/near misses. 1. Ideally we should avoid casting, but I'm not sure if there is better way to do it? We could try moving `DrillSidewaysScorer` logic to `DrillSidewaysLeafCollector`, but it seems challenging/not possible to move all scorer optimizations to `collect` method We would also loose parallel drill-sideways `searchConcurrently` , but I'm not sure if anyone uses it, I'd expect that inter-segment parallelism + `DrillSidewaysScorer` is better for both throughput and latency? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
