[
https://issues.apache.org/jira/browse/LUCENE-8962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17141809#comment-17141809
]
Simon Willnauer commented on LUCENE-8962:
-----------------------------------------
It seems like this caused this test failure:
{noformat}
18:06:29 [junit4] Suite: org.apache.lucene.index.TestIndexWriter
18:06:29 [junit4] 2> Mod 21, 2020 3:06:19 EBONGI
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
18:06:29 [junit4] 2> WARNING: Uncaught exception in thread:
Thread[Thread-1905,5,TGRP-TestIndexWriter]
18:06:29 [junit4] 2> java.lang.AssertionError:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] 2> at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3886)
18:06:29 [junit4] 2> at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] 2> Caused by:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:739)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:753)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1330)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1619)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1262)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3873)
18:06:29 [junit4] 2> ... 1 more
18:06:29 [junit4] 2> Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] 2> at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] 2> at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] 2> at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] 2> ... 1 more
18:06:29 [junit4] 2>
18:06:29 [junit4] 2> Mod 21, 2020 3:06:19 EBONGI
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
18:06:29 [junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene
Merge Thread #3,5,TGRP-TestIndexWriter]
18:06:29 [junit4] 2> org.apache.lucene.index.MergePolicy$MergeException:
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot
merge
18:06:29 [junit4] 2> at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:703)
18:06:29 [junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:694)
18:06:29 [junit4] 2> Caused by: java.lang.IllegalStateException: this
writer hit an unrecoverable error; cannot merge
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4250)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4230)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4083)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5650)
18:06:29 [junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624)
18:06:29 [junit4] 2> at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:682)
18:06:29 [junit4] 2> Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] 2> at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] 2> at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] 2> at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] 2> at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] 2>
18:06:29 [junit4] 2> Mod 21, 2020 3:06:19 EBONGI
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
18:06:29 [junit4] 2> WARNING: Uncaught exception in thread:
Thread[Thread-1904,5,TGRP-TestIndexWriter]
18:06:29 [junit4] 2> java.lang.AssertionError:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] 2> at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3886)
18:06:29 [junit4] 2> at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] 2> Caused by:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:739)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:753)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4979)
18:06:29 [junit4] 2> at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
18:06:29 [junit4] 2> at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
18:06:29 [junit4] 2> at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
18:06:29 [junit4] 2> at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
18:06:29 [junit4] 2> at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:156)
18:06:29 [junit4] 2> at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
18:06:29 [junit4] 2> at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:176)
18:06:29 [junit4] 2> at
org.apache.lucene.search.ReferenceManager.maybeRefreshBlocking(ReferenceManager.java:253)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3876)
18:06:29 [junit4] 2> ... 1 more
18:06:29 [junit4] 2> Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] 2> at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] 2> at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] 2> at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] 2> ... 1 more
18:06:29 [junit4] 2>
18:06:29 [junit4] 2> Mod 21, 2020 3:06:19 EBONGI
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
18:06:29 [junit4] 2> WARNING: Uncaught exception in thread:
Thread[Thread-1903,5,TGRP-TestIndexWriter]
18:06:29 [junit4] 2> java.lang.AssertionError:
java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] 2> at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3886)
18:06:29 [junit4] 2> at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] 2> Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] 2> at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] 2> at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] 2> at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] 2> at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] 2> at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] 2> ... 1 more
18:06:29 [junit4] 2>
18:06:29 [junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestIndexWriter -Dtests.method=testRandomOperations
-Dtests.seed=83A78A6E34594E08 -Dtests.nightly=true -Dtests.slow=true
-Dtests.badapples=true -Dtests.locale=teo-KE
-Dtests.timezone=Antarctica/Rothera -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
18:06:29 [junit4] ERROR 0.05s J3 | TestIndexWriter.testRandomOperations <<<
18:06:29 [junit4] > Throwable #1:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=2009, name=Thread-1905, state=RUNNABLE,
group=TGRP-TestIndexWriter]
18:06:29 [junit4] > Caused by: java.lang.AssertionError:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] > at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3886)
18:06:29 [junit4] > at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] > Caused by:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:739)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:753)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1330)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1619)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1262)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3873)
18:06:29 [junit4] > ... 1 more
18:06:29 [junit4] > Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] > at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] > at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] > at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] > ... 1 moreThrowable #2:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=2008, name=Thread-1904, state=RUNNABLE,
group=TGRP-TestIndexWriter]
18:06:29 [junit4] > Caused by: java.lang.AssertionError:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] > at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3886)
18:06:29 [junit4] > at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] > Caused by:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:739)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:753)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4979)
18:06:29 [junit4] > at
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
18:06:29 [junit4] > at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
18:06:29 [junit4] > at
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
18:06:29 [junit4] > at
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
18:06:29 [junit4] > at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:156)
18:06:29 [junit4] > at
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
18:06:29 [junit4] > at
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:176)
18:06:29 [junit4] > at
org.apache.lucene.search.ReferenceManager.maybeRefreshBlocking(ReferenceManager.java:253)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3876)
18:06:29 [junit4] > ... 1 more
18:06:29 [junit4] > Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] > at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] > at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] > at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] > ... 1 moreThrowable #3:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=2013, name=Lucene Merge Thread #3,
state=RUNNABLE, group=TGRP-TestIndexWriter]
18:06:29 [junit4] > Caused by:
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot
merge
18:06:29 [junit4] > at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:703)
18:06:29 [junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:694)
18:06:29 [junit4] > Caused by: java.lang.IllegalStateException: this
writer hit an unrecoverable error; cannot merge
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4250)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4230)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4083)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5650)
18:06:29 [junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624)
18:06:29 [junit4] > at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:682)
18:06:29 [junit4] > Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] > at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] > at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] > at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] > at
java.base/java.lang.Thread.run(Thread.java:834)Throwable #4:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=2007, name=Thread-1903, state=RUNNABLE,
group=TGRP-TestIndexWriter]
18:06:29 [junit4] > Caused by: java.lang.AssertionError:
java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] > at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3886)
18:06:29 [junit4] > at
java.base/java.lang.Thread.run(Thread.java:834)
18:06:29 [junit4] > Caused by: java.nio.file.NoSuchFileException: _7.cfs
18:06:29 [junit4] > at
org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
18:06:29 [junit4] > at
org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
18:06:29 [junit4] > at
org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.deleteCommits(IndexFileDeleter.java:382)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:527)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:3546)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3502)
18:06:29 [junit4] > at
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3452)
18:06:29 [junit4] > at
org.apache.lucene.index.TestIndexWriter.lambda$testRandomOperations$48(TestIndexWriter.java:3879)
18:06:29 [junit4] > ... 1 more
18:06:29 [junit4] 2> NOTE: test params are: codec=Asserting(Lucene86):
{=PostingsFormat(name=MockRandom),
a=PostingsFormat(name=LuceneVarGapFixedInterval),
c=PostingsFormat(name=LuceneVarGapDocFreqInterval),
string=PostingsFormat(name=LuceneVarGapDocFreqInterval),
foo=PostingsFormat(name=Direct), body=PostingsFormat(name=Direct),
version=PostingsFormat(name=MockRandom),
content=PostingsFormat(name=LuceneVarGapFixedInterval),
str=PostingsFormat(name=LuceneVarGapFixedInterval),
tvtest=PostingsFormat(name=MockRandom), field=PostingsFormat(name=Direct),
str3=PostingsFormat(name=Direct),
content4=PostingsFormat(name=LuceneVarGapFixedInterval),
binary=PostingsFormat(name=LuceneVarGapDocFreqInterval),
str2=PostingsFormat(name=LuceneVarGapFixedInterval),
content1=PostingsFormat(name=MockRandom),
id=PostingsFormat(name=LuceneVarGapDocFreqInterval),
myfield=PostingsFormat(name=Direct)},
docValues:{numval=DocValuesFormat(name=Asserting),
my_deletes=DocValuesFormat(name=Asserting),
sortedsetdv=DocValuesFormat(name=Asserting),
soft_deletes=DocValuesFormat(name=Asserting),
sortednumericdv=DocValuesFormat(name=Lucene80),
num=DocValuesFormat(name=Lucene80), binarydv=DocValuesFormat(name=Asserting),
dv=DocValuesFormat(name=Lucene80), dv_field=DocValuesFormat(name=Asserting),
numericdv=DocValuesFormat(name=Asserting),
soft_delete=DocValuesFormat(name=Lucene80),
sorteddv=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Asserting),
soft_deletes_1=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=315,
maxMBSortInHeap=6.068347930823877,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@39db836f),
locale=teo-KE, timezone=Antarctica/Rothera
18:06:29 [junit4] 2> NOTE: Linux 3.10.0-1127.10.1.el7.x86_64 amd64/Oracle
Corporation 11.0.2 (64-bit)/cpus=32,threads=1,free=385982712,total=536870912
18:06:29 [junit4] 2> NOTE: All tests run in this JVM:
[TestIntRangeFieldQueries, TestLucene50TermVectorsFormat,
TestXYPointShapeQueries, TestSortRescorer, TestFloatRangeFieldQueries,
TestFlushByRamOrCountsPolicy, TestSimpleFSLockFactory,
TestFilterDirectoryReader, TestBytesRef, TestCompiledAutomaton,
TestBlockMaxConjunction, TestAddIndexes, TestDirectoryReader, TestLatLonShape,
TestIndependenceSaturated, TestAxiomaticF1LOG, TestLucene84PostingsFormat,
TestLucene50LiveDocsFormat, TestReusableStringReader, TestPolygon2D,
TestPerFieldDocValuesFormat, TestFixedLengthBytesRefArray,
TestSimpleAttributeImpl, TestQueryBuilder, TestFastLZ4,
TestFastCompressionMode, Test2BPagedBytes, TestPayloads, TestMultiTermsEnum,
TestCircle, TestLogMergePolicy, TestPointQueries, TestByteSlices,
TestBufferedChecksum, TestExceedMaxTermLength, TestSegmentInfos,
Test2BSortedDocValuesFixedSorted, TestDocValuesIndexing, TestIndexWriter]
18:06:29 [junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestIndexWriter -Dtests.seed=83A78A6E34594E08 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=teo-KE
-Dtests.timezone=Antarctica/Rothera -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
18:06:29 [junit4] ERROR 0.00s J3 | TestIndexWriter (suite) <<<
18:06:29 [junit4] > Throwable #1: java.lang.AssertionError: The test or
suite printed 9264 bytes to stdout and stderr, even though the limit was set to
8192 bytes. Increase the limit with @Limit, ignore it completely with
@SuppressSysoutChecks or run with -Dtests.verbose=true
18:06:29 [junit4] > at
__randomizedtesting.SeedInfo.seed([83A78A6E34594E08]:0)
18:06:29 [junit4] > at
java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
> Can we merge small segments during refresh, for faster searching?
> -----------------------------------------------------------------
>
> Key: LUCENE-8962
> URL: https://issues.apache.org/jira/browse/LUCENE-8962
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
> Reporter: Michael McCandless
> Priority: Major
> Fix For: 8.6
>
> Attachments: LUCENE-8962_demo.png, failed-tests.patch
>
> Time Spent: 18.5h
> Remaining Estimate: 0h
>
> With near-real-time search we ask {{IndexWriter}} to write all in-memory
> segments to disk and open an {{IndexReader}} to search them, and this is
> typically a quick operation.
> However, when you use many threads for concurrent indexing, {{IndexWriter}}
> will accumulate write many small segments during {{refresh}} and this then
> adds search-time cost as searching must visit all of these tiny segments.
> The merge policy would normally quickly coalesce these small segments if
> given a little time ... so, could we somehow improve {{IndexWriter'}}s
> refresh to optionally kick off merge policy to merge segments below some
> threshold before opening the near-real-time reader? It'd be a bit tricky
> because while we are waiting for merges, indexing may continue, and new
> segments may be flushed, but those new segments shouldn't be included in the
> point-in-time segments returned by refresh ...
> One could almost do this on top of Lucene today, with a custom merge policy,
> and some hackity logic to have the merge policy target small segments just
> written by refresh, but it's tricky to then open a near-real-time reader,
> excluding newly flushed but including newly merged segments since the refresh
> originally finished ...
> I'm not yet sure how best to solve this, so I wanted to open an issue for
> discussion!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
