Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-07-11 Thread via GitHub
javanna commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-3062037144 For reference, this is the ahead of time work that currently gets duplicated in PointRangeQuery across partitions of the same segments: https://github.com/apache/lucene/blob/main/lu

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-07-10 Thread via GitHub
expani commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-3060066462 Will go over all types of queries to check ( other than PointRangeQuery ) that needs special handling by sharing the docId space unless someone has already covered it. -- This is

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-07-10 Thread via GitHub
expani commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-3058268142 I was looking to integrate Intra Segment Concurrent Search and found that this same problem also applies to downstream consumers of Lucene like OpenSearch/ElasticSearch/Solr who use

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-04-29 Thread via GitHub
javanna commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-2838255713 > Does using intra-segment partitioning negatively impact these queries (TermInSet/MultiTerm) or is there still some benefit from using the feature? I guess it depends on dat

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-04-22 Thread via GitHub
Shibi-bala commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-2822481473 @javanna not sure about the underlying implementation of these queries. Does using intra-segment partitioning negatively impact these queries (TermInSet/MultiTerm) or is there st

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-03-04 Thread via GitHub
javanna commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-2699189350 > HNSW vector search heavy lifting is done in rewrite, so out of scope for this, right? I believe so, mostly because query rewrite does not parallelize on slices, but across

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-03-04 Thread via GitHub
msokolov commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-2698550635 HNSW vector search heavy lifting is done in `rewrite`, so out of scope for this, right? Maybe multi-term queries would need to do some work. What about join queries? TermInSet quer

Re: [I] Stop duplicating per-segment work across segment partitions [lucene]

2025-02-25 Thread via GitHub
javanna commented on issue #13745: URL: https://github.com/apache/lucene/issues/13745#issuecomment-2682478759 This is a heads up that I started working on this. My focus is currently on `PointRangeQuery`. The overall goal is to share the bitset computation across scorer suppliers for the sa