kaivalnp commented on PR #14131: URL: https://github.com/apache/lucene/pull/14131#issuecomment-2730545360
Exciting change! Since this PR adds a new codec for vector search, I wanted to point to #14178 along similar lines -- adding a new Faiss-based KNN format to index and query vectors Faiss (https://github.com/facebookresearch/faiss) is _"a library for efficient similarity search and clustering of dense vectors"_. It supports various features like vector transforms (eg PCA), indexing algorithms (eg IVF, HNSW, etc), quantization techniques (eg PQ), search strategies (eg 2-step refinement), different hardware (including [GPUs](https://github.com/facebookresearch/faiss/wiki/Faiss-on-the-GPU) -- also has support for cuVS) -- and adding this codec would allow users to make use of (most of) these features! Internally, the format calls the [C API](https://github.com/facebookresearch/faiss/blob/main/c_api/INSTALL.md) of Faiss using Panama (https://openjdk.org/projects/panama) FFI. The codec is present in the sandbox module, and does _not_ add Faiss as a dependency of Lucene -- only relies on the shared library (along with all dependencies) to be present at runtime (on `$LD_LIBRARY_PATH` or `-Djava.library.path`) Would appreciate feedback on the PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
