navneet1v commented on issue #14247: URL: https://github.com/apache/lucene/issues/14247#issuecomment-2664091298
> If the use case is multitenancy, it seems you would never want to search across tenants, so this would apply not only to KNN search but to all kinds of search? I agree the impact on KNN search is outsized, but would it make sense to build a separate index per tenant? @msokolov multi-tenancy is just one use case and I added it as an example. Having separate indices make sense for smaller number of tenants but if the number of tenants goes to millions then multiple indices is not a great solution as a lot of pressure on the operating system in terms of managing files and directories. If as a user I am just doing vector search and not other searches like text then this kind of solution is more intuitive and easy to use. BTW there is another way to solve this problem, if the Lucene Codec Writers can get access to other fields, then we don't need to pass any information with vector. We can put let say a tenant id in BDV and then access it during the HNSWWriter to build the correct graph. WDYT about that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
