[
https://issues.apache.org/jira/browse/LUCENE-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Benedetti updated LUCENE-10146:
------------------------------------------
Labels: vector-based-search (was: )
> Add VectorSimilarityFunction.COSINE
> -----------------------------------
>
> Key: LUCENE-10146
> URL: https://issues.apache.org/jira/browse/LUCENE-10146
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Julie Tibshirani
> Priority: Major
> Labels: vector-based-search
> Fix For: 9.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> To perform ANN search with cosine similarity, users are expected to normalize
> the document and query vectors to unit length, then use
> {{VectorSimilarityFunction.DOT_PRODUCT}}. I think it would be good to also
> support cosine similarity directly through
> {{VectorSimilarityFunction.COSINE}}. This would allow users to perform ANN
> based on cosine similarity, while retaining access to the original vectors
> through {{VectorValues}}. That way they can use the original vectors in a
> reranking step or return them to the application for further processing.
> It looks like nmslib and hnswlib support cosine similarity. On the other
> hand, FAISS only supports dot product and suggests users normalize the
> vectors to perform cosine similarity
> (https://github.com/facebookresearch/faiss/issues/95). To me adding this one
> additional similarity is worth it in terms of what it lets users accomplish.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
