benwtrent commented on issue #13519: URL: https://github.com/apache/lucene/issues/13519#issuecomment-2203125856
The only way to find out is to test it. I don't see how your suggestion would work without trying it out. Its better to think about what it would be in the unsigned `byte` case. A scale from `0-255` is what `int8` quantization naturally results in. Then to make it signed, you have a change to `-127-127`. I don't immediately see how to adjust the correction to handle the switch from `0 * 255` to `-127 * 127`. @naveentatikonda any solution should support max-inner-product, any testing with normalized vectors & dot-product might result in false assurance. If you are using Lucene util, I would recommend using some older cohere embeddings (with their model from before CohereV3), and using the `-metric` `mip`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
