mikemccand commented on PR #13572: URL: https://github.com/apache/lucene/pull/13572#issuecomment-2332099352
I'm trying to understand the status of this PR... so far it's a standalone JMH benchy that shows that using [FFM](https://openjdk.org/jeps/454) to invoke our own native C implementation of `dotProduct` on ints (e.g. needed for `int7`, `int4` HNSW scalar quantization in Lucene) that's carefully optimized to use the right SIMD instructions (switching depending on the capabilities of the CPU) shows sizable speedups (up to 10X) over our current "pure java" Panama vector API implementation? This is a micro-benchmark of just `dotProduct`, so the overall gains to Lucene's HNSW indexing and searching will be less than 10X. But, `dotProduct` is [very much the hotspot of Lucene's HNSW](https://github.com/mikemccand/luceneutil/issues/256#issuecomment-2159000055) indexing and searching, so the end gains will likely be significant, especially on CPUs that perform poorly today via Panama (e.g. ARM). To make this actually accessible to users I think we would still need to: * Add this `nativeDotProduct` somewhere, likely `lucene/misc` module, and get gradle to compile it all * Make an alternative `NativeFlatVectorScorer` (in `misc`) that uses the `nativeDotProduct` * Make a `NativeLucene99HnswVectorsFormat` that is just like the default for `core` (`Lucene99HnswVectorsFormat` now), but it invokes `NativeFlatVectorScorer` instead. Hopefully this can be done without too much code duplication. * Show examples of how one could use Lucene's default `Codec` but swap in this `NativeLucene99HnswVectorsFormat` for KNN fields I think this all can be done after 10.0 -- there's no particular reason why we need a major release to add this? It's entirely new (optimized) added feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
