benwtrent opened a new pull request, #12191:
URL: https://github.com/apache/lucene/pull/12191
This increases the vector size limit only for ByteVectors only.
Byte vectors use much less memory than float (less than half required to
store float vectors). Additionally, dot-products between two byte vectors are
faster than between two float vectors (about 1.5x faster).
Here are some micro-benchmark (the dot-product methods used were the same
from the `VectorSimilarity` class):
```
VectorMultiplicationBenchmark.byteDotProducts 64
avgt 7 119207.749 ± 87.092 ns/op
VectorMultiplicationBenchmark.byteDotProducts 1024
avgt 7 1615737.126 ± 955.261 ns/op
VectorMultiplicationBenchmark.byteDotProducts 2048
avgt 7 3283335.169 ± 1624.888 ns/op
VectorMultiplicationBenchmark.floatDotProductsUnwrapped 64
avgt 7 197209.288 ± 191.285 ns/op
VectorMultiplicationBenchmark.floatDotProductsUnwrapped 1024
avgt 7 2844926.617 ± 10232.337 ns/op
VectorMultiplicationBenchmark.floatDotProductsUnwrapped 2048
avgt 7 5544835.569 ± 2476.568 ns/op
```
Consequently, it makes sense to me to allow larger vectors for `byte`
vectors, but still enforce the current limitation for float. Thus ensuring that
callers with larger vectors will:
- Have to quantize their input
- Do dimensionality reduction
- Some combination of both
At least until we can get hardware accelerated vector multiplication (on its
5th!!!!!! incubator at the time of this PR): https://openjdk.org/jeps/438
This PR is a continuation of various other previous work and discussion:
- https://github.com/apache/lucene/pull/874
- https://github.com/apache/lucene/issues/11507
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
