ChrisHegarty opened a new issue, #12482: URL: https://github.com/apache/lucene/issues/12482
### Description Just a crazy idea! Thanks @jpountz ;-) The Panama Vector API supports for loading directly from a memory segment. If we could do this, then for vector similarity purposes we could load the underlying vector data directly from the MemorySegment (rather than the current copy from the segment into a float[] ). A quick experiment, if I did it correctly, and only with float dotProduct, shows approx a 2x performance improvement. ( the experiment builds upon @rmuir 's vectorbench, which we used for the original work that integrated the Panama Vector API ) ``` Linux, AVX 512 Benchmark (size) Mode Cnt Score Error Units FloatDotProductBenchmark.dotProductCopyFromArray 1024 thrpt 5 9.714 ± 0.002 ops/us FloatDotProductBenchmark.dotProductFromMemorySegment 1024 thrpt 5 18.111 ± 0.008 ops/us ``` https://github.com/ChrisHegarty/vectorbench/blob/5380ebf01df77e1c56d9f392a391df780c19d35e/src/main/java/testing/FloatDotProductBenchmark.java The benchmark method, `dotProductCopyFromArray`, is what we have in Lucene today, but I added a copy from the memory segment into the two float[]'s to simulate `MemorySegmentIndexInput::readFloats`. The benchmark method, `dotProductFromMemorySegment`, has the exact same shape of vector operations, just that the values are loaded directly from the memory segment using `FloatVector::fromMemorySegment`. A single segment holds both vectors. I suspect we'd want similar in Lucene too is this were ever to proceed. In practice I don't see any straightforward way that we could approach this, and there is the complication of using preview / Incubating JDK APIs. That aside, I think that the general idea is sound - avoid the extra intermediate copy to the float[]. I filed this issue mostly to raise awareness of `fromMemorySegment`, and also to provoke a discussion and get thoughts from others. Is it worth trying to pursue? Maybe just crudely hack something and get rough luceneutil bench results to see if / how much it could affect things? `MemorySegmentIndexInput` could expose a way to return XXVector's from a given position? Something else? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
