jmazanec15 commented on issue #13281: URL: https://github.com/apache/lucene/issues/13281#issuecomment-2237366786
> I am not sure what to do for users who quantize their own vectors & rely on cosine. I think I am on same page as @msokolov. Users could "float_vector -> norm_float_vector -> byte_vector" and then apply dot product on byte_vectors. If float_vector->byte_vector leads to approximation on ordering for dot_product for float_vector, then why wouldnt the same logic hold for norm_float_vector and thus an approx on cosine? The case to worry about I think is when they have a data set of byte vectors and need the cosine right (i.e. there is no float normalization possible)? I dont think there is a workaround because of inherent difference in precision used for data type and distance value (i.e. byte vs double). That being said, to support this, I think norms would need to be stored. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
