mccullocht commented on PR #15903: URL: https://github.com/apache/lucene/pull/15903#issuecomment-4171295623
> E.g. does OSQ also not alter the quantization per-segment (merge of flat vectors could optimized copyBytes (the hardest function in the world to implement correctly/performantly!))? Do we get a 3 bit option with OSQ? OSQ centers the vectors -- during a segment build it computes the mean vector then quantizes `v - c`. This adds a data dependency that requires re-encoding vectors as you merge. You could operate OSQ without centering but the resulting quantized vectors would be a less accurate representation on average. Requantizing OSQ is likely cheaper than the transform in TQ, but TQ could more easily discard the original full fidelity vector field. We could support any value in [1,8] for OSQ, but efficiently unpacking for comparisons can be a real challenge. This PR is packing 3 bits as 8 values in 3 consecutive bytes. I can think of an efficient 128 bit implementation of this that would work on x86 and ARM but AVX/AVX512 are not amenable to the approach that I am thinking of. > Isn't it one global transform (not per segment) in this PR? Or would we want to change that to per-segment, to increase randomness/protection against unlucky rotation choice? This PR is doing one global transform. If we use a transform per segment, then we will have to re-quantize vectors during merge so it would be more complicated/expensive than copyBytes. I personally have not examined the effect of the random seed in a rigorous way but it is plausible that some transforms would be "better" than others in some measurable way like minimizing MSE. > If you abandon uniform grid spacing you can no longer implement the dot product via integer arithmetic. This is actually a huge performance hit, IIRC we get 4-8x performance vs float arithmetic for well crafted SIMD variants of low bit integer dot products. The final implementation for TurboQuant is table lookup (centroid positions) followed by floating point arithmetic. For this you'd take a totally different approach -- probably something that looks more like distance computation for product quantization since it uses a codebook in a similar way. This involves generating lookup tables that can be quite large (8KB+) and you would not want to repreat this process on every segment. It can still be very fast but it almost certainly won't be as fast as OSQ's arithmetic comparisons. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
