goankur commented on PR #13572: URL: https://github.com/apache/lucene/pull/13572#issuecomment-2461155781
#### Quick Update 1. Changes to use native dot-product for both `int7` scalar-quantized and raw vectors have been integrated in `core/src/java21` 2. `Lucene99ScalarQuantizedVectorScorer` was modified to use `Lucene99MemorySegmentFlatVectorScorer` when PanamaVectorizationProvider is enabled. 3. Unit tests in `TestVectorUtilSupport` have been expanded to exercise native dot-product code. 4. JMH benchmarks contain methods to benchmark performance of compiler auto-vectorized implementation and manually vectorized implementation. 5. `test.native.dotProduct` system property has been added to `gradle/testing/randomization.gradle` to enable randomized testing of native code. 6. Running using `KnnPerfTest` tool (in luceneutil) on 768 dimensional mpnet vectors show ~25% reduction in latency compared to baseline. #### What's next A lot of code in `Lucene99MemorySegmentScalarQuantizedVectorScorer` was duplicated from `Lucene99ScalarQuantizedVectorScorer`. This is because the native code only works with off-heap memory and new implementation needs to pass vectors as off-heap MemorySegment to native dotProduct. It is for this (and performance) reasons that a MemorySegment slice containing vector bytes is obtained from underlying MemorySegmentAccessInput and passed to native dotProduct. A flip side is that query vector has to be copied to an off-heap buffer. I don't like the fact that so much code had to be duplicated to make this work, so I am going to take a stab to see if there is an opportunity to minimize the duplication. I will also try to improve the coverage of unit tests although the changes in `Lucene99ScalarQuantizedVectorScorer` are gated behind flag -`Constants.NATIVE_DOT_PRODUCT_ENABLED` which is toggled by randomized testing so it should exercise the existing tests for raw and scalar quantized vectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
