mikemccand commented on code in PR #13359:
URL: https://github.com/apache/lucene/pull/13359#discussion_r1613219485
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene90/blocktree/SegmentTermsEnum.java:
##########
@@ -307,6 +309,30 @@ private boolean setEOF() {
return true;
}
+ @Override
+ public void prepareSeekExact(BytesRef target) throws IOException {
+ if (fr.index == null) {
+ throw new IllegalStateException("terms index was not loaded");
+ }
+
+ if (fr.size() == 0 || target.compareTo(fr.getMin()) < 0 ||
target.compareTo(fr.getMax()) > 0) {
+ return;
+ }
+
+ // TODO: should we try to reuse the current state of this terms enum when
applicable?
+ BytesRefFSTEnum<BytesRef> indexEnum = new BytesRefFSTEnum<>(fr.index);
+ InputOutput<BytesRef> output = indexEnum.seekFloor(target);
+ if (output != null) { // should never be null since we already checked
against fr.getMin()?
+ final long code =
+ fr.readVLongOutput(
+ new ByteArrayDataInput(
+ output.output.bytes, output.output.offset,
output.output.length));
+ final long fpSeek = code >>>
Lucene90BlockTreeTermsReader.OUTPUT_FLAGS_NUM_BITS;
+ initIndexInput();
+ in.prefetch(fpSeek, 1); // TODO: could we know the length of the block?
Review Comment:
OK! This was nagging at me so I dug into it, printing the `FSTEnum`
iteration output on a nightly Lucene benchy index ... I now understand why the
pointers indeed go backwards. I think this was the point you were making above
@vsop-479 -- sorry I misunderstood at first ;)
It's because when writing the blocks we write "bottoms up" on depth first
traversal through the terms, and only write a node when it is finished /
returned from. Leaf blocks will be written immediately / in order since they
are started, terms come out, finished. But for a non-leaf blocks, first all
leaf blocks under them are written (in order), and THEN the non-leaf block is
written only when we are done with all those recursions and writing any
straggler terms that live in the non-leaf block.
But the prefixes are added to the index FST in the correct (term sorted)
order. So this means the file pointer can indeed go backwards when iterating
the terms index. I'll mull some more about whether we could (efficiently) know
the term block length ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
