diwu-sf commented on code in PR #16923:
URL: https://github.com/apache/lucene/pull/16923#discussion_r4471245673
##########
lucene/core/src/java/org/apache/lucene/index/StandardDirectoryReader.java:
##########
@@ -463,10 +463,12 @@ private DirectoryReader doOpenFromWriter(IndexCommit
commit, ExecutorService exe
return null;
}
- DirectoryReader reader = writer.getReader(applyAllDeletes,
writeAllDeletes);
+ StandardDirectoryReader reader = writer.getReader(applyAllDeletes,
writeAllDeletes);
- // If in fact no changes took place, return null:
- if (reader.getVersion() == segmentInfos.getVersion()) {
+ // If in fact no changes took place, return null. A commit that was
syncing files when this
+ // reader was opened changes the referenced segments_N once it completes,
but not the version:
+ if (reader.getVersion() == segmentInfos.getVersion()
+ && reader.segmentInfos.getLastGeneration() ==
segmentInfos.getLastGeneration()) {
Review Comment:
You're right. I reproduced it with a test: w2's reader got the same version
as r1, with different documents.
Fix: closing the writer now commits if a reader got the bumped version. The
next writer then starts after that version. If no reader got it, closing still
writes nothing.
I didn't count the bump as a regular change, because then
`hasUncommittedChanges()` would be true after those commits. Code that commits
whenever there are uncommitted changes would then keep committing an idle index
(`testNoSpuriousChanges` fails that way).
New tests:
- The next writer doesn't reuse or lower a version, for each kind of change
and in `CREATE` mode.
- Readers opened while a commit waits for merges, or during a commit that
only saves user data.
- `TestIndexVersionInvariants`: randomized and concurrent tests that check
version rules across writers, allowing reuse only after a rollback. The
concurrent test fails on the code before this PR, on the PR's first version,
and on its previous version.
These tests also found an older bug: `rollback()` could throw
`NoSuchFileException` when a writer was opened on an older commit or from an
NRT reader and then merged away its starting segments. I fixed it in a separate
commit. I can move it to its own PR if you prefer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
