tveasey commented on PR #12582: URL: https://github.com/apache/lucene/pull/12582#issuecomment-1745806230
I saw go by since I’m mentioned on the PR. It seems like Java can’t lay out byte vectors properly: https://stackoverflow.com/questions/14531235/in-java-is-it-more-efficient-to-use-byte-or-short-instead-of-int-and-float-inst and so incurs type coercion penalties. But I guess Panama does this properly, so you should have a workaround. > On 3 Oct 2023, at 22:28, Benjamin Trent ***@***.***> wrote: > > > I was doing some performance testing and was getting weird results. Quantization search and indexing build were marginally better or exactly the same. > > Attached a zip of async-profiler outputs for wall & cpu, the -95- files are quantized to the .95 quantile and are using dotProduct(byte[], byte[]) > > Archive 2.zip <https://github.com/apache/lucene/files/12797059/Archive.2.zip> > Digging into why performance wasn't any better, I used Robert's vector bench library to see how my macbook did with our byte implementation vs float and WOW byte[] seems REALLY slow. > > FloatDotProductBenchmark.dotProductNew 768 thrpt 5 21.781 ± 0.254 ops/us > FloatDotProductBenchmark.dotProductNew 1024 thrpt 5 15.091 ± 0.217 ops/us > FloatDotProductBenchmark.dotProductNew 2048 thrpt 5 8.233 ± 0.023 ops/us > BinaryDotProductBenchmark.dotProductNew 768 thrpt 5 8.041 ± 0.108 ops/us > BinaryDotProductBenchmark.dotProductNew 1024 thrpt 5 6.085 ± 0.133 ops/us > BinaryDotProductBenchmark.dotProductNew 2048 thrpt 5 3.054 ± 0.123 ops/us > 768 dims is not even as fast as 2048 floats! This is counter intuitive to me, SIMD should be able to process MORE information per cycle, even on a MacBook with only 128-bitsize. > > I am going to have to move my testing off of my machine to something with a different CPU. But in parallel, we should figure out why byte[] is so bad when compared to float[] (my guess is just because we just don't have the correct intrinsics for now...) > > — > Reply to this email directly, view it on GitHub <https://github.com/apache/lucene/pull/12582#issuecomment-1745753817>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZ5MP65P3QS7ILCV3KHHUDX5R7PRAVCNFSM6AAAAAA5CAEGCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVG42TGOBRG4>. > You are receiving this because you were mentioned. > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
