[
https://issues.apache.org/jira/browse/LUCENE-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094930#comment-17094930
]
Chris M. Hostetter commented on LUCENE-9191:
--------------------------------------------
Similar root causes in other tests in other builds w/diff seeds...
{noformat}
Checking out Revision ecc98e8698a3ce8efa51712686697c0f33afab4d
(refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ecc98e8698a3ce8efa51712686697c0f33afab4d
Commit message: "LUCENE-7788: fail precommit on unparameterised log messages
and examine for wasted work/objects"
> git rev-list --no-walk 03363f413f2134594b012175deb3f10ec9384400 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Cleaning up
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data
Updating http://svn.apache.org/repos/asf/lucene/test-data at revision
'2020-04-24T17:44:24.647 +0000'
At revision 1876938
No emails were triggered.
[checkout] $ /home/jenkins/tools/ant/apache-ant-1.8.4/bin/ant -file build.xml
-Dtests.multiplier=2
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-
BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt
jenkins-nightly-badapples
Buildfile:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/checkout/build.xml
jenkins-nightly-badapples:
-print-java-info:
[java-info] java version "11.0.4"
[java-info] Java(TM) SE Runtime Environment (11.0.4+10-LTS, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (11.0.4+10-LTS, Oracle
Corporation)
[java-info] Test args: [-XX:TieredStopAtLevel=1]
...
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestSameScoresWithThreads -Dtests.method=test
-Dtests.seed=6B89EFC89E940CDC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt
-Dtests.locale=ar-SY -Dtests.timezone=Africa/Dar_es_Salaam
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.02s J1 | TestSameScoresWithThreads.test <<<
[junit4] > Throwable #1: java.nio.charset.MalformedInputException: Input
length = 1
[junit4] > at
__randomizedtesting.SeedInfo.seed([6B89EFC89E940CDC:E3DDD01230686124]:0)
[junit4] > at
java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
[junit4] > at
java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
[junit4] > at
java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
[junit4] > at
java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
[junit4] > at
java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
[junit4] > at
java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
[junit4] > at
java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
[junit4] > at
org.apache.lucene.util.LineFileDocs.open(LineFileDocs.java:175)
[junit4] > at
org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:65)
[junit4] > at
org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:69)
[junit4] > at
org.apache.lucene.search.TestSameScoresWithThreads.test(TestSameScoresWithThreads.java:49)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
...
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestAllFilesHaveCodecHeader -Dtests.method=test
-Dtests.seed=6B89EFC89E940CDC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt
-Dtests.locale=ar-LY -Dtests.timezone=Africa/Windhoek -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestAllFilesHaveCodecHeader.test <<<
[junit4] > Throwable #1: java.nio.charset.MalformedInputException: Input
length = 1
[junit4] > at
__randomizedtesting.SeedInfo.seed([6B89EFC89E940CDC:E3DDD01230686124]:0)
[junit4] > at
java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
[junit4] > at
java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
[junit4] > at
java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
[junit4] > at
java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
[junit4] > at
java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
[junit4] > at
java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
[junit4] > at
java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
[junit4] > at
org.apache.lucene.util.LineFileDocs.open(LineFileDocs.java:175)
[junit4] > at
org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:65)
[junit4] > at
org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:69)
[junit4] > at
org.apache.lucene.index.TestAllFilesHaveCodecHeader.test(TestAllFilesHaveCodecHeader.java:44)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
...
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestAllFilesHaveChecksumFooter -Dtests.method=test
-Dtests.seed=6B89EFC89E940CDC -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-NightlyTests-master/test-data/enwiki.random.lines.txt
-Dtests.locale=vi -Dtests.timezone=America/Mexico_City -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestAllFilesHaveChecksumFooter.test <<<
[junit4] > Throwable #1: java.nio.charset.MalformedInputException: Input
length = 1
[junit4] > at
__randomizedtesting.SeedInfo.seed([6B89EFC89E940CDC:E3DDD01230686124]:0)
[junit4] > at
java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
[junit4] > at
java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
[junit4] > at
java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
[junit4] > at
java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
[junit4] > at
java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
[junit4] > at
java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
[junit4] > at
java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
[junit4] > at
org.apache.lucene.util.LineFileDocs.open(LineFileDocs.java:175)
[junit4] > at
org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:65)
[junit4] > at
org.apache.lucene.util.LineFileDocs.<init>(LineFileDocs.java:69)
[junit4] > at
org.apache.lucene.index.TestAllFilesHaveChecksumFooter.test(TestAllFilesHaveChecksumFooter.java:41)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
> Fix linefiledocs compression or replace in tests
> ------------------------------------------------
>
> Key: LUCENE-9191
> URL: https://issues.apache.org/jira/browse/LUCENE-9191
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Robert Muir
> Assignee: Michael McCandless
> Priority: Major
> Fix For: 8.6
>
> Attachments: LUCENE-9191.patch, LUCENE-9191.patch
>
>
> LineFileDocs(random) is very slow, even to open. It does a very slow "random
> skip" through a gzip compressed file.
> For the analyzers tests, in LUCENE-9186 I simply removed its usage, since
> TestUtil.randomAnalysisString is superior, and fast. But we should address
> other tests using it, since LineFileDocs(random) is slow!
> I think it is also the case that every lucene test has probably tested every
> LineFileDocs line many times now, whereas randomAnalysisString will invent
> new ones.
> Alternatively, we could "fix" LineFileDocs(random), e.g. special compression
> options (in blocks)... deflate supports such stuff. But it would make it even
> hairier than it is now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
