[
https://issues.apache.org/jira/browse/LUCENE-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136805#comment-17136805
]
Varun Thacker commented on LUCENE-9322:
---------------------------------------
Hello [~jtibshirani] ! Thanks for tackling this
> Support for storing and retrieving individual float vectors.
How would we feel to break this part and commit it separately ? I believe this
is adding the VectorField field part ? The PR on SOLR-14397 also added a
DenseVectorField ( Solr field ) so maybe we could reuse VectorField ( although
there is some nuance since DenseVectorField currently supports string and
vector encoding and a code comment saying bfloat16 as well )
> Discussing a unified vectors format API
> ---------------------------------------
>
> Key: LUCENE-9322
> URL: https://issues.apache.org/jira/browse/LUCENE-9322
> Project: Lucene - Core
> Issue Type: New Feature
> Reporter: Julie Tibshirani
> Priority: Major
>
> Two different approximate nearest neighbor approaches are currently being
> developed, one based on HNSW ([#LUCENE-9004]) and another based on coarse
> quantization ([#LUCENE-9136]). Each prototype proposes to add a new format to
> handle vectors. In LUCENE-9136 we discussed the possibility of a unified API
> that could support both approaches. The two ANN strategies give different
> trade-offs in terms of speed, memory, and complexity, and it’s likely that
> we’ll want to support both. Vector search is also an active research area,
> and it would be great to be able to prototype and incorporate new approaches
> without introducing more formats.
> To me it seems like a good time to begin discussing a unified API. The
> prototype for coarse quantization
> ([https://github.com/apache/lucene-solr/pull/1314]) could be ready to commit
> soon (this depends on everyone's feedback of course). The approach is simple
> and shows solid search performance, as seen
> [here|https://github.com/apache/lucene-solr/pull/1314#issuecomment-608645326].
> I think this API discussion is an important step in moving that
> implementation forward.
> The goals of the API would be
> # Support for storing and retrieving individual float vectors.
> # Support for approximate nearest neighbor search -- given a query vector,
> return the indexed vectors that are closest to it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
