[
https://issues.apache.org/jira/browse/SOLR-13778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949681#comment-16949681
]
Chris M. Hostetter commented on SOLR-13778:
-------------------------------------------
I just realized we're seeing a slightly _different_ SSLException from Uwe's
java13 windows VMs...
{noformat}
[junit4] > Throwable #1:
org.apache.solr.client.solrj.SolrServerException: IOException occurred when
talking to server at: https://127.0.0.1:551
21/solr
[junit4] > at
__randomizedtesting.SeedInfo.seed([E2C1EFE3F69FB5C6:35E9A23BE77FFC28]:0)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:679)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
[junit4] > at
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
[junit4] > at
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
[junit4] > at
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
[junit4] > at
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
[junit4] > at
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:228)
[junit4] > at
org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:549)
[junit4] > at
org.apache.solr.cloud.TestCloudSearcherWarming.tearDown(TestCloudSearcherWarming.java:79)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:567)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:830)
[junit4] > Caused by: javax.net.ssl.SSLException: An established
connection was aborted by the software in your host machine
[junit4] > at
java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
[junit4] > at
java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
[junit4] > at
java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
[junit4] > at
java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
[junit4] > at
java.base/sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1652)
[junit4] > at
java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1038)
[junit4] > at
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] > at
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] > at
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] > at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] > at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] > at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] > at
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] > at
org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] > at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] > at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] > at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] > at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] > at
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] > at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
[junit4] > at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] > at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] > at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:564)
[junit4] > ... 47 more
[junit4] > Suppressed: java.net.SocketException: An established
connection was aborted by the software in your host machine
[junit4] > at
java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:421)
[junit4] > at
java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:441)
[junit4] > at
java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:825)
[junit4] > at
java.base/java.net.Socket$SocketOutputStream.write(Socket.java:989)
[junit4] > at
java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:82)
[junit4] > at
java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:355)
[junit4] > ... 69 more
[junit4] > Caused by: java.net.SocketException: An established connection
was aborted by the software in your host machine
[junit4] > at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:326)
[junit4] > at
java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:351)
[junit4] > at
java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:802)
[junit4] > at
java.base/java.net.Socket$SocketInputStream.read(Socket.java:919)
[junit4] > at
java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:450)
[junit4] > at
java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
[junit4] > at
java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1409)
[junit4] > at
java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1022)
[junit4] > ... 65 more
{noformat}
As before, these only happen on Windows jobs (and Uwe is the only one running
windows jobs) and the root cause SocketException is only reported (in this case
as both 'Caused By' and 'Suppressed') when they are part of these
SSLExceptions...
{noformat}
$ zgrep -c 'javax.net.ssl.SSLException: An established connection was aborted
by the software in your host machine' */*/*/jenkins.log.txt.gz | grep -v ':0$'
thetaphi/Lucene-Solr-8.x-Windows/489/jenkins.log.txt.gz:6
thetaphi/Lucene-Solr-8.x-Windows/491/jenkins.log.txt.gz:6
thetaphi/Lucene-Solr-8.x-Windows/494/jenkins.log.txt.gz:5
thetaphi/Lucene-Solr-8.x-Windows/499/jenkins.log.txt.gz:6
thetaphi/Lucene-Solr-master-Windows/8177/jenkins.log.txt.gz:11
thetaphi/Lucene-Solr-master-Windows/8180/jenkins.log.txt.gz:11
thetaphi/Lucene-Solr-master-Windows/8181/jenkins.log.txt.gz:6
thetaphi/Lucene-Solr-master-Windows/8183/jenkins.log.txt.gz:23
$ zgrep 'javax.net.ssl.SSLException: An established connection was aborted by
the software in your host machine' */*/*/jenkins.log.txt.gz | wc -l
74
$ zgrep 'java.net.SocketException: An established connection was aborted by the
software in your host machine' */*/*/jenkins.log.txt.gz | wc -l
148
$ zgrep 'java.net.SocketException: An established connection was aborted by the
software in your host machine' thetaphi/*Windows/*/jenkins.log.txt.gz | wc -l
148
{noformat}
It's probably worth noting for posterity the java-version info for both types
of exceptions...
{noformat}
$ zgrep -l 'javax.net.ssl.SSLException: Software caused connection abort: recv
failed' thetaphi/*Windows/*/jenkins.log.txt.gz | xargs zgrep -h -F
'[java-info]' | grep -v -E 'java version|Test args|Runtime Environment' | sort
| uniq -c
3 [java-info] OpenJDK 64-Bit Server VM (11.0.4+11, AdoptOpenJDK)
5 [java-info] OpenJDK 64-Bit Server VM (12.0.1+12, AdoptOpenJDK)
$ zgrep -l 'javax.net.ssl.SSLException: An established connection was aborted
by the software in your host machine' thetaphi/*Windows/*/jenkins.log.txt.gz |
xargs zgrep -h -F '[java-info]' | grep -v -E 'java version|Test args|Runtime
Environment' | sort | uniq -c
5 [java-info] OpenJDK 64-Bit Server VM (13+33, Oracle Corporation)
3 [java-info] OpenJDK 64-Bit Server VM (14-ea+14-570, Oracle Corporation)
{noformat}
----
Uwe: any thoughts on how we should deal with these? ... they currently reprsent
the cause of the biggest chunk of recurring failures in Solr test cases.
> Windows JDK SSL Test Failure trend: SSLException: Software caused connection
> abort: recv failed
> -----------------------------------------------------------------------------------------------
>
> Key: SOLR-13778
> URL: https://issues.apache.org/jira/browse/SOLR-13778
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Chris M. Hostetter
> Priority: Major
>
> Now that Uwe's jenkins build has been correctly reporting it's build results
> for my [automated
> reports|http://fucit.org/solr-jenkins-reports/failure-report.html] to pick
> up, I've noticed a pattern of failures that indicate a definite problem with
> using SSL on Windows (even with java 11.0.4
> )
> The symptommatic stack traces all contain...
> {noformat}
> ...
> [junit4] > Caused by: javax.net.ssl.SSLException: Software caused
> connection abort: recv failed
> [junit4] > at
> java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
> ...
> [junit4] > Caused by: java.net.SocketException: Software caused
> connection abort: recv failed
> [junit4] > at
> java.base/java.net.SocketInputStream.socketRead0(Native Method)
> ...
> {noformat}
> I suspect this may be related to
> [https://bugs.openjdk.java.net/browse/JDK-8209333] but i have no concrete
> evidence to back this up.
> I'll post some details of my analysis in comments...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
