mayya-sharipova commented on code in PR #16734:
URL: https://github.com/apache/lucene/pull/16734#discussion_r4169797941
##########
lucene/core/src/java/org/apache/lucene/codecs/KnnFieldVectorsWriter.java:
##########
@@ -36,6 +37,28 @@ protected KnnFieldVectorsWriter() {}
*/
public abstract void addValue(int docID, T vectorValue) throws IOException;
+ /**
+ * Add {@code values.size()} vectors for the consecutive doc IDs {@code
[firstDocID, firstDocID +
+ * values.size())}. {@code firstDocID} must be greater than every doc ID
added so far, and every
+ * vector has exactly {@code values.dimension()} elements, which matches the
field's dimension.
+ * Implementations must consume exactly {@code values.size()} vectors.
+ *
+ * <p>The cursor may throw while it is being consumed, for example when a
vector fails validation.
+ * In that case the documents of the whole batch are marked as deleted, but
the writer must remain
+ * in a consistent state: every doc ID it has recorded must have its vector.
+ *
+ * <p>The default implementation calls {@link #addValue} once per vector.
Override for a more
+ * efficient bulk path.
+ *
+ * @lucene.experimental
+ */
+ public void addDenseValues(int firstDocID, VectorValuesCursor<T> values)
throws IOException {
Review Comment:
I added the default flat writer as a first implementer, and the HNSW and SQ
field writers now forward to their flat delegate.
But as you see there is no much optimization done yet: filling one vector
at a time. The plan is to move the flat writer to paged storage in a follow-up,
where addDenseValues fills a whole page per fill call. This will be in the
follow up PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
