jimczi commented on code in PR #16682:
URL: https://github.com/apache/lucene/pull/16682#discussion_r4074378975
##########
lucene/backward-codecs/src/java/org/apache/lucene/backward_codecs/lucene102/Lucene102BinaryQuantizedVectorsFormat.java:
##########
@@ -141,4 +144,13 @@ public String toString() {
+ rawVectorFormat
+ ")";
}
+
+ /** The raw vectors are only read back to rescore. */
+ private static SegmentReadState rescoreOnly(SegmentReadState state) {
+ IOContext.FileOpenHint[] hints =
+ Stream.concat(state.context.hints().stream(),
Stream.of(NoReuseHint.INSTANCE))
+ .distinct()
+ .toArray(IOContext.FileOpenHint[]::new);
+ return new SegmentReadState(state, state.context.withHints(hints));
Review Comment:
Done, with a `.distinct()` added since `Set.of` rejects duplicates.
##########
lucene/backward-codecs/src/java/org/apache/lucene/backward_codecs/lucene102/Lucene102BinaryQuantizedVectorsReader.java:
##########
@@ -128,10 +130,15 @@ public Lucene102BinaryQuantizedVectorsReader(
versionMeta,
VECTOR_DATA_EXTENSION,
Lucene102BinaryQuantizedVectorsFormat.VECTOR_DATA_CODEC_NAME,
- // Quantized vectors are accessed randomly from their node ID
stored in the HNSW
- // graph.
+ // how these are read is up to whoever wraps this format
state.context.withHints(
- FileTypeHint.DATA, FileDataHint.KNN_VECTORS,
DataAccessHint.RANDOM));
+ Stream.of(
+ FileTypeHint.DATA,
+ FileDataHint.KNN_VECTORS,
+
state.context.hints(DataAccessHint.class).findFirst().orElse(null),
+
state.context.hints(NoReuseHint.class).findFirst().orElse(null))
+ .filter(Objects::nonNull)
+ .toArray(IOContext.FileOpenHint[]::new)));
Review Comment:
I added this and then took it out again. Every call site passed a null
access hint, so it returned the context it was given. A format that can be
wrapped has no fallback to apply: whoever wraps it puts the hint on the state,
and a standalone one says nothing about access. The constructors that existed
to carry the hint went with it.
`union` stayed and has ten callers.
##########
lucene/core/src/java/org/apache/lucene/index/SegmentWriteState.java:
##########
@@ -113,6 +113,19 @@ public SegmentWriteState(
this.context = context;
}
+ /** Create a shallow copy of {@link SegmentWriteState} with a new {@link
IOContext}. */
+ public SegmentWriteState(SegmentWriteState state, IOContext context) {
+ infoStream = state.infoStream;
+ directory = state.directory;
+ segmentInfo = state.segmentInfo;
+ fieldInfos = state.fieldInfos;
+ this.context = context;
+ segmentSuffix = state.segmentSuffix;
+ segUpdates = state.segUpdates;
+ delCountOnFlush = state.delCountOnFlush;
+ liveDocs = state.liveDocs;
+ }
+
Review Comment:
Done, on both states.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
