kaivalnp commented on issue #12615: URL: https://github.com/apache/lucene/issues/12615#issuecomment-2612374897
I'm also tinkering with a Faiss (https://github.com/facebookresearch/faiss) wrapper to perform KNN searches in Lucene (i.e. a standalone [`KnnVectorsFormat`](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsFormat.java)). Faiss has inbuilt support for a variety of vector transforms, indexes, and hardware (mainly GPUs) which could be beneficial for users OpenSearch already has an integration for Faiss, but it uses JNI which has the overhead of separate build systems for Java / CPP and having to maintain "glue" code -- and I'm also not sure if it can be used directly from Lucene. The new wrapper would be a pure Java implementation using Panama FFI (https://openjdk.org/projects/panama) I wanted to check if something like this would be beneficial, and where it could live long-term (i.e. within Lucene, maybe in the sandbox module at first) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
