jmazanec15 opened a new issue, #12342: URL: https://github.com/apache/lucene/issues/12342
Currently, [VectorSimilarityFunction.DOT_PRODUCT](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/VectorSimilarityFunction.java#L54) function can return negative scores if the input vectors are not normalized. For ref, this is the method: ``` public float compare(float[] v1, float[] v2) { return (1 + dotProduct(v1, v2)) / 2; } ``` While in the method javadoc there is a [warning](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/VectorSimilarityFunction.java#L45) to normalize the vectors before using, I am wondering if we can get rid of this by mapping negative scores between 0 and 1 and positive scores between 1 and Float.Max with: ``` dotProd = dotProduct(v1, v2) if (dotProd < 0) { return 1 / (1 + -1*dotProd); } return dotProd + 1; ``` and let the user worry about normalization Related issue: https://github.com/opensearch-project/k-NN/issues/865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
