mccullocht commented on issue #16029: URL: https://github.com/apache/lucene/issues/16029#issuecomment-5781439167
I merged the data blind changes (thanks @kaivalnp) with some expanded scope: you can choose whether or not to keep the underlying float vectors. We disallow dropping the float vectors for asymmetric quantization schemes (1 and 2 bit docs) because graph quality is so dependent on the build using higher fidelity vectors as queries. The next step here is to [optionally quantize residuals](https://arxiv.org/pdf/2304.04759). The idea is to set the lower and upper values to `+/- (primary_delta / 2)`. This doesn't quite work in our case because the coordinate descent is altering the quantization bounds, so we'll either have to take that into account (potentially choosing wider bounds) or run the same interval optimization on the residual vector and store a similarly large header for the residual vector. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
