thecoop commented on code in PR #16696:
URL: https://github.com/apache/lucene/pull/16696#discussion_r4061743427
##########
lucene/core/src/java/org/apache/lucene/codecs/hnsw/FlatVectorsWriter.java:
##########
@@ -63,4 +66,65 @@ public final IORunnable mergeOneField(FieldInfo fieldInfo,
MergeState mergeState
public abstract void mergeOneFlatVectorField(FieldInfo fieldInfo, MergeState
mergeState)
throws IOException;
+
+ /**
+ * Merges one field and, while writing the merged vectors, optionally
prepares what a merge scorer
+ * needs, so that the merged float vectors need not be read back.
+ *
+ * <p>A wrapping writer should forward this method along with {@link
#mergeOneFlatVectorField}.
+ * Forwarding only the latter stays correct, but the wrapped writer then
prepares nothing.
+ * Forwarding helps only when the wrapping format opens the wrapped writer's
own reader over the
+ * merged segment. Otherwise {@link MergeScorerData#scorerSupplier} rejects
the prepared data and
+ * the merge uses its fallback.
+ *
+ * @param fieldInfo field to merge
+ * @param mergeState merge state
+ * @param needsMergeScorer whether to prepare scorer data for a vector
count. The writer tests
+ * this predicate against the count before deletions, which can exceed
the number of vectors
+ * written.
+ * @return prepared data that the caller must close, or {@code null} if none
was prepared, which
+ * is what the default implementation returns and what any writer
returns for a field it
+ * cannot prepare for. The caller then builds the scorer itself.
+ * @throws IOException if an I/O error occurs
+ */
+ public MergeScorerData mergeOneFlatVectorFieldForMergeScorer(
+ FieldInfo fieldInfo, MergeState mergeState, IntPredicate
needsMergeScorer)
+ throws IOException {
+ mergeOneFlatVectorField(fieldInfo, mergeState);
+ return null;
+ }
+
+ /**
+ * Data prepared while merging a field for scoring its merged vectors. The
caller must close it
+ * whether or not it obtains a supplier. A single merge thread uses each
instance, so
+ * implementations need not be thread-safe.
+ */
+ public interface MergeScorerData extends Closeable {
+
+ /**
+ * Creates a scorer supplier for {@code mergedReader}. The caller must
close the supplier to
+ * release the prepared data.
+ *
+ * <p>The reader is matched to the preparing writer's own reader by type.
A reader that wraps
+ * that reader is rejected even when it reads the correct segment.
+ *
+ * <p>This method may be called at most once, after the preparing writer
is finished and closed
+ * and {@code mergedReader} is open on the segment it wrote. A second
call, or a call after
+ * {@link #close}, throws {@link IllegalStateException}.
+ *
+ * @param mergedReader a reader over the segment the writer wrote
+ * @return a scorer supplier over the merged vectors, or {@code null} if
this data cannot serve
+ * that reader
+ * @throws IOException if an I/O error occurs
+ */
+ CloseableRandomVectorScorerSupplier scorerSupplier(FlatVectorsReader
mergedReader)
+ throws IOException;
+
+ /**
+ * Releases data not transferred by {@link #scorerSupplier}. This method
is idempotent and must
+ * not throw, so cleanup during a failed merge does not replace the
original exception.
Review Comment:
This is what suppressed exceptions is for, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
