jpountz commented on issue #14247: URL: https://github.com/apache/lucene/issues/14247#issuecomment-2666915751
I remember (but I don't remember where) seeing someone doing multi-tenant vector search by using a flat vector index and enabling index sorting on the tenant ID. Then vector search can't take advantage of an advanced structure like HNSW, but the I/O access pattern is disk-friendly, so if each tenant isn't too large on its own the performance may be acceptable. In general, I'm not a fan of this proposal of enabling creating multiple KNN indexes via some user-provided tenant/cluster ID. This looks like working around the fact that vector-search is currently not good at pre-filtering. I'd rather look into how we can make vector search better at pre-filtering (either with HNSW or something else). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
