benwtrent commented on PR #12582: URL: https://github.com/apache/lucene/pull/12582#issuecomment-1745753817
I was doing some performance testing and was getting weird results. Quantization search and indexing build were marginally better or exactly the same. Attached a zip of async-profiler outputs for `wall` & `cpu`, the `-95-` files are quantized to the `.95` quantile and are using `dotProduct(byte[], byte[])` [Archive 2.zip](https://github.com/apache/lucene/files/12797059/Archive.2.zip) Digging into why performance wasn't any better, I used Robert's vector bench library to see how my macbook did with our `byte` implementation vs `float` and WOW `byte[]` seems REALLY slow. ``` FloatDotProductBenchmark.dotProductNew 768 thrpt 5 21.781 ± 0.254 ops/us FloatDotProductBenchmark.dotProductNew 1024 thrpt 5 15.091 ± 0.217 ops/us FloatDotProductBenchmark.dotProductNew 2048 thrpt 5 8.233 ± 0.023 ops/us BinaryDotProductBenchmark.dotProductNew 768 thrpt 5 8.041 ± 0.108 ops/us BinaryDotProductBenchmark.dotProductNew 1024 thrpt 5 6.085 ± 0.133 ops/us BinaryDotProductBenchmark.dotProductNew 2048 thrpt 5 3.054 ± 0.123 ops/us ``` 768 dims is not even as fast as 2048 floats! This is counter intuitive to me, SIMD should be able to process MORE information per cycle, even on a MacBook with only 128-bitsize. I am going to have to move my testing off of my machine to something with a different CPU. But in parallel, we should figure out why `byte[]` is so bad when compared to `float[]` (my guess is just because we just don't have the correct intrinsics for now...) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
