dsmiley commented on issue #13387: URL: https://github.com/apache/lucene/issues/13387#issuecomment-2641899421
Addressing this need would be amazing! Many search architectures (including where I work) always filter to a specific field (say a doc type or tenant/user; it depends). That 50-60% performance increase sounds right. I've wanted the ability to segment by DWPT for a long time. The resistance to it then and still now surprises me because (at least in my mind) there's a simple selector mechanism. Just keep it to that MVP; anything else suggested is unnecessary for an expert API. Well of course need that wrapping merge policy but that's an isolated component. There's surely a past thread or two of users asking for this capability. The DWPT approach has a nice characteristic of isolating the complexity (what little there is) to Lucene internals. Slap an expert/experimental label on the API so we might feel free to change it whenever. The alternatives described above are surely a lot more complex and put that complexity at the feet of someone trying to figure this out (few know Lucene as well as us here). RE OpenSearch needing SegmentInfos etc. > This seems to be very specific to how OpenSearch interacts with the IndexWriter. In my view it isn't; Solr does the same and it seems natural to me that another Lucene based search server would similarly have such needs. RE MultiReader: That thing is _so_ simple.... and furthermore is IndexReader based. I suspect the latest discussion/proposal will not lead to enhancing MultiReader tremendously but instead be making a new MultiIndexWriter-ish thing. Anyway; that's just my hot-take on it. We'll see. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
