jpountz commented on code in PR #13596:
URL: https://github.com/apache/lucene/pull/13596#discussion_r1753388629
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/PerThreadPKLookup.java:
##########
@@ -97,5 +174,28 @@ public int lookup(BytesRef id) throws IOException {
return -1;
}
- // TODO: add reopen method to carry over re-used enums...?
+ /**
+ * Reuse loaded segments' termsEnum and postingsEnum.
+ *
+ * @return null if there are no changes; else, a new PerThreadPKLookup
instance which you must
+ * eventually close its reader by {@link #closeReader}.
+ * @throws IOException if there is a low-level IO error.
+ */
+ public PerThreadPKLookup reopen() throws IOException {
+ DirectoryReader newReader =
DirectoryReader.openIfChanged((DirectoryReader) reader);
+ if (newReader == null) {
+ return null;
+ }
+
+ return new PerThreadPKLookup(this, newReader);
+ }
+
+ /**
+ * Close reader.
+ *
+ * @throws IOException if there is a low-level IO error
+ */
+ public void closeReader() throws IOException {
+ reader.close();
+ }
Review Comment:
I would remove this method and make it a responsibility of the caller.
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/PerThreadPKLookup.java:
##########
@@ -37,16 +43,21 @@
*/
public class PerThreadPKLookup {
- protected final TermsEnum[] termsEnums;
- protected final PostingsEnum[] postingsEnums;
- protected final Bits[] liveDocs;
- protected final int[] docBases;
- protected final int numSegs;
- protected final boolean hasDeletions;
+ private IndexReader reader;
+ private final String idFieldName;
+ protected TermsEnum[] termsEnums;
+ protected PostingsEnum[] postingsEnums;
+ protected Bits[] liveDocs;
+ protected int[] docBases;
+ protected int numSegs;
+ protected boolean hasDeletions;
+ protected Map<SegmentInfo, Integer> segmentInfoMap = new HashMap<>();
Review Comment:
Can you make all these fields final again?
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/PerThreadPKLookup.java:
##########
@@ -97,5 +174,28 @@ public int lookup(BytesRef id) throws IOException {
return -1;
}
- // TODO: add reopen method to carry over re-used enums...?
+ /**
+ * Reuse loaded segments' termsEnum and postingsEnum.
+ *
+ * @return null if there are no changes; else, a new PerThreadPKLookup
instance which you must
+ * eventually close its reader by {@link #closeReader}.
+ * @throws IOException if there is a low-level IO error.
+ */
+ public PerThreadPKLookup reopen() throws IOException {
+ DirectoryReader newReader =
DirectoryReader.openIfChanged((DirectoryReader) reader);
Review Comment:
After calling `openIfChanged`, the previous reader should get closed. IMO,
the API should take a new reader and leave calling `openIfChanged` and `close`
to the caller?
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/PerThreadPKLookup.java:
##########
@@ -97,5 +174,28 @@ public int lookup(BytesRef id) throws IOException {
return -1;
}
- // TODO: add reopen method to carry over re-used enums...?
+ /**
+ * Reuse loaded segments' termsEnum and postingsEnum.
+ *
+ * @return null if there are no changes; else, a new PerThreadPKLookup
instance which you must
+ * eventually close its reader by {@link #closeReader}.
+ * @throws IOException if there is a low-level IO error.
+ */
+ public PerThreadPKLookup reopen() throws IOException {
+ DirectoryReader newReader =
DirectoryReader.openIfChanged((DirectoryReader) reader);
+ if (newReader == null) {
+ return null;
+ }
+
+ return new PerThreadPKLookup(this, newReader);
Review Comment:
This creates a new `PerThreadPKLookup` from scratch. The new reader will
often have many leaves in common with the old reader (which can be identified
through
https://lucene.apache.org/core/9_8_0/core/org/apache/lucene/index/LeafReader.html#getCoreCacheHelper()).
We should reuse TermsEnums on these common leaves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
