diwu-sf opened a new pull request, #16923: URL: https://github.com/apache/lucene/pull/16923
### Description Fixes https://github.com/apache/lucene/issues/16900, and a second bug with the same symptom: an NRT reader misses changes, but `isCurrent()` returns true and `openIfChanged` returns null. Lucene checks whether a reader is up to date by comparing version numbers, and in both cases the version doesn't change when it should. There is one commit per bug. **1. Doc values updates written during a flush (#16900).** Writing pending doc values updates to disk doesn't bump the version. This happens on flush, on commit, and when updates use too much memory. Fix: bump the version when they are written. **2. A reader opened during a commit.** A commit bumps the version first, then syncs files to disk, and only then switches to the new `segments_N` file. A reader opened during the sync gets the new version but still points at the old `segments_N`. Because the versions match, it is treated as up to date even after that file is deleted. Fix: also check that the reader points at the latest commit. **Why it matters.** Bug 2 causes https://github.com/opensearch-project/OpenSearch/issues/11025: after a restart, some OpenSearch shards keep trying to upload a `segments_N` file that no longer exists. On stock OpenSearch 3.9.0 with 300 indices, a restart left 3 to 6 shards stuck, and 121 to 128 when disk syncs were slowed by 20 ms. A workaround inside OpenSearch is at https://github.com/diwu-sf/OpenSearch/pull/2. **Tests.** Four new tests, three for bug 1 and one for bug 2. Each fails without its fix and passes with it. `./gradlew :lucene:core:check` and the nightly NRT replication tests pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
