benwtrent commented on issue #13519:
URL: https://github.com/apache/lucene/issues/13519#issuecomment-2206500270
@mikemccand
> Does the Lucene implementation have per-dimension calculation of
quantiles/min/max? Or is it really a global min/max computed across all
dimensions and all vectors' values in those dimensions?
Indeed, it is the latter, over all dimensions.
> Do ML vectors "typically" have similar distributions of values within each
dimension?
For some vectors, this is true, for others, not so. A good example of
vectors where this is NOT the case is CohereV2. However, the error distribution
per dimension is independent and given the central limit theorem, the errors do
not compound.
> I know we are discussing how to make the conflated quantization and
distance metric combinations math work out (i.e. how to fix the issue), but I'm
just trying to get the big picture of what the issue even is. How would we
explain the buggy behavior to users?
The scalar quantizer currently assumes the destination `byte` values are
unsigned. However, Java cannot have unsigned byte values that are `> 127`.
Consequently, when trying to utilize the full number of bits in `int8` the
scoring falls apart.
There are actually two ways to fix this bug:
- Get unsigned byte dot-product (handling overflow, etc.)
- Get scalar quantizer to accurately represent signed bytes with the
dotProduct score corrections.
Looking back, when I finished up the ScalarQuantization support for int4, I
should have disallowed `8` bits as a parameter to prevent this confusion. I
allowed it and I shouldn't have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
