Re: SSL Unit Tests Failing

[Igal Sapir]

Mark / Chris,

On 10/2/2018 6:36 AM, Mark Thomas wrote:
On 02/10/18 06:58, Igal Sapir wrote:
When trying to run the unit test cases with `ant clean test` on the current
trunk [1] I am getting two (per connector) failures:
     org.apache.tomcat.util.net.openssl.ciphers.TestCipher FAILED [2]

org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser
FAILED [3]

     Server version: Apache Tomcat/9.0.13-dev
     Server built:   Oct 2 2018 05:24:55 UTC
     Server number:  9.0.13.0
     OS Name:        Linux
     OS Version:     4.18.9-200.fc28.x86_64
     Architecture:   amd64
     JVM Version:    1.8.0_181-b13
     JVM Vendor:     Oracle Corporation

Am I missing something?  Other than the obvious "missing ciphers", that is.
These tests are all particularly sensitive to the versions of OpenSSL,
Java and the implementation of Java used.

Generally, those tests are there to ensure that the code that translates
between JSSE cipher definitions and OpenSSL definitions is correct.

If you see a failure it may indicate that:

- the test has a bug

- you are running with an older version of OpenSSL that behaves
   differently from the latest version (we try and keep pace with the
   latest)

- OpenSSL has changed behaviour and we need to update our translation
   code to align with it (unusual)

- OpenSSL has changed behaviour and we need to update our tests to align
   with it (most frequent).

Thank you both for the detailed explanation.  I suspected that I should 
had added the OpenSSL version to the OP.  On that Fedora machine I have 
OpenSSL 1.1.0i-fips 14 Aug 2018

I tried the same tests on a Windows 10 machine.  Below are some 
discrepancies/peculiarities that I've noticed (I'd be happy to improve 
the test cases if possible):

On the Linux box I have OpenSSL installed and on the PATH.  On Windows I 
used version OpenSSL 1.1.1  11 Sep 2018 and specified it via the 
`test.openssl.path` property.  I checked the value of 
`test.openssl.exists` and it showed the expected `true`.  Both Windows 
and Fedora generated an output file for 
test/org/apache/tomcat/util/net/openssl/TestOpenSSLConf.java [1]. Both, 
however, reported "Found OpenSSL version 0x0" which I find strange?

On Windows, only the output [2] for the file mentioned above is in the 
output/build/logs, while on Fedora I also have output from the 3 Test 
files from test/org/apache/tomcat/util/net/openssl/ciphers/. Does that 
mean that these tests were not run on Windows?

I wanted to check the Gump output to compare with my local results. I 
found this URL, which I'm not sure if it is the right one or not - 
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk/index.html - as 
it says "Project build output found here..." but without any links or 
any other information.

I would like at the very least to add the output of `openssl version` to 
the Ant output, perhaps at the `test.openssl.exists` target.  If there 
are no objections I will add that.

[1] 
https://github.com/apache/tomcat/blob/trunk/test/org/apache/tomcat/util/net/openssl/TestOpenSSLConf.java

[2] Windows output of o.a.t.util.net.openssl.TestOpenSSLConf

Testsuite: org.apache.tomcat.util.net.openssl.TestOpenSSLConf
Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 1.994 sec
------------- Standard Error -----------------
02-Oct-2018 11:23:28.394 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testOpenSSLConfCmdCipher]
02-Oct-2018 11:23:28.618 INFO [main] 
org.apache.tomcat.util.net.openssl.TestOpenSSLConf.testOpenSSLConfCmdCipher 
Found OpenSSL version 0x0
02-Oct-2018 11:23:28.808 INFO [main] org.apache.coyote.AbstractProtocol.destroy 
Destroying ProtocolHandler ["https-jsse-nio-127.0.0.1-auto-1"]
02-Oct-2018 11:23:28.895 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testOpenSSLConfCmdProtocol]
02-Oct-2018 11:23:28.924 INFO [main] 
org.apache.tomcat.util.net.openssl.TestOpenSSLConf.testOpenSSLConfCmdProtocol 
Found OpenSSL version 0x0
02-Oct-2018 11:23:28.926 INFO [main] org.apache.coyote.AbstractProtocol.destroy 
Destroying ProtocolHandler ["https-jsse-nio-127.0.0.1-auto-2"]
------------- ---------------- ---------------

Testcase: testOpenSSLConfCmdCipher took 1.755 sec
        SKIPPED: This test is only for OpenSSL based SSL connectors
Testcase: testOpenSSLConfCmdCipher took 1.759 sec
Testcase: testOpenSSLConfCmdProtocol took 0.037 sec
        SKIPPED: This test is only for OpenSSL based SSL connectors
Testcase: testOpenSSLConfCmdProtocol took 0.037 sec






There is overlap between some of the above cases.

I see different failures when I run locally. Your question has made me
curious to find out why.

Gump is usually very good at catching changes. I normally don;t worry
unless I see a failure in these tests on Gump. Expanding the
combinations of Tomcat and OpenSSL that we test with there is still on
the TODO list.

Mark


Thanks,

Igal

[1] git-svn-id: https://svn.apache.org/repos/asf/tomcat/trunk@1842498
13f79535-47bb-0310-9956-ffa450edef68

[2] Testsuite: org.apache.tomcat.util.net.openssl.ciphers.TestCipher
Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.697 sec

Testcase: testNames took 0.168 sec
Testcase: testAllOpenSSLCiphersMapped took 0.361 sec
     FAILED
No mapping found in IBM's JSSE implementation for
ECDHE-PSK-3DES-EDE-CBC-SHA+TLSv1 when one was expected

junit.framework.AssertionFailedError: No mapping found in IBM's JSSE
implementation for ECDHE-PSK-3DES-EDE-CBC-SHA+TLSv1 when one was expected

     at
org.apache.tomcat.util.net.openssl.ciphers.TestCipher.testAllOpenSSLCiphersMapped(TestCipher.java:66)

Testcase: testOpenSSLCipherAvailability took 0.06 sec
     FAILED
ECDHE-ARIA128-GCM-SHA256+TLSv1.2 DHE-RSA-ARIA128-GCM-SHA256+TLSv1.2
DHE-DSS-ARIA256-GCM-SHA384+TLSv1.2 ECDHE-ECDSA-ARIA128-GCM-SHA256+TLSv1.2
ARIA256-GCM-SHA384+TLSv1.2 ECDHE-ARIA256-GCM-SHA384+TLSv1.2
DHE-RSA-ARIA256-GCM-SHA384+TLSv1.2 RSA-PSK-ARIA256-GCM-SHA384+TLSv1.2
ECDHE-ECDSA-ARIA256-GCM-SHA384+TLSv1.2 ARIA128-GCM-SHA256+TLSv1.2
DHE-PSK-ARIA128-GCM-SHA256+TLSv1.2 RSA-PSK-ARIA128-GCM-SHA256+TLSv1.2
DHE-DSS-ARIA128-GCM-SHA256+TLSv1.2 PSK-ARIA256-GCM-SHA384+TLSv1.2
DHE-PSK-ARIA256-GCM-SHA384+TLSv1.2 PSK-ARIA128-GCM-SHA256+TLSv1.2
expected:<0> but was:<16>
junit.framework.AssertionFailedError: ECDHE-ARIA128-GCM-SHA256+TLSv1.2
DHE-RSA-ARIA128-GCM-SHA256+TLSv1.2 DHE-DSS-ARIA256-GCM-SHA384+TLSv1.2
ECDHE-ECDSA-ARIA128-GCM-SHA256+TLSv1.2 ARIA256-GCM-SHA384+TLSv1.2
ECDHE-ARIA256-GCM-SHA384+TLSv1.2 DHE-RSA-ARIA256-GCM-SHA384+TLSv1.2
RSA-PSK-ARIA256-GCM-SHA384+TLSv1.2 ECDHE-ECDSA-ARIA256-GCM-SHA384+TLSv1.2
ARIA128-GCM-SHA256+TLSv1.2 DHE-PSK-ARIA128-GCM-SHA256+TLSv1.2
RSA-PSK-ARIA128-GCM-SHA256+TLSv1.2 DHE-DSS-ARIA128-GCM-SHA256+TLSv1.2
PSK-ARIA256-GCM-SHA384+TLSv1.2 DHE-PSK-ARIA256-GCM-SHA384+TLSv1.2
PSK-ARIA128-GCM-SHA256+TLSv1.2  expected:<0> but was:<16>
     at
org.apache.tomcat.util.net.openssl.ciphers.TestCipher.testOpenSSLCipherAvailability(TestCipher.java:100)

[3] Testsuite:
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser
Tests run: 86, Failures: 40, Errors: 0, Skipped: 1, Time elapsed: 5.473 sec
------------- Standard Error -----------------
Error in cipher list
140015003477824:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140182557382464:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140372866819904:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
139680405661504:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140699554305856:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
139897177433920:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
139891985295168:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140442752255808:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
139855064180544:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140598129956672:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
139768227612480:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
139839666202432:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140216997062464:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140194450589504:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

Error in cipher list
140636605155136:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no
cipher match:ssl/ssl_lib.c:2193:

<snip/>

Testcase: testARIA128 took 0.535 sec
     FAILED
Expected 8 ciphers but got 0 for the specification 'ARIA128'
expected:<[TLS_DHE_DSS_WITH_ARIA_128_GCM_SHA256,
TLS_DHE_PSK_WITH_ARIA_128_GCM_SHA256, TLS_DHE_RSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256, TLS_PSK_WITH_ARIA_128_GCM_SHA256,
TLS_RSA_PSK_WITH_ARIA_128_GCM_SHA256, TLS_RSA_WITH_ARIA_128_GCM_SHA256]>
but was:<[]>
junit.framework.AssertionFailedError: Expected 8 ciphers but got 0 for the
specification 'ARIA128' expected:<[TLS_DHE_DSS_WITH_ARIA_128_GCM_SHA256,
TLS_DHE_PSK_WITH_ARIA_128_GCM_SHA256, TLS_DHE_RSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256, TLS_PSK_WITH_ARIA_128_GCM_SHA256,
TLS_RSA_PSK_WITH_ARIA_128_GCM_SHA256, TLS_RSA_WITH_ARIA_128_GCM_SHA256]>
but was:<[]>
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testSpecification(TestOpenSSLCipherConfigurationParser.java:588)
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testARIA128(TestOpenSSLCipherConfigurationParser.java:541)

Testcase: testARIA256 took 0.063 sec
     FAILED
Expected 8 ciphers but got 0 for the specification 'ARIA256'
expected:<[TLS_DHE_DSS_WITH_ARIA_256_GCM_SHA384,
TLS_DHE_PSK_WITH_ARIA_256_GCM_SHA384, TLS_DHE_RSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384, TLS_PSK_WITH_ARIA_256_GCM_SHA384,
TLS_RSA_PSK_WITH_ARIA_256_GCM_SHA384, TLS_RSA_WITH_ARIA_256_GCM_SHA384]>
but was:<[]>
junit.framework.AssertionFailedError: Expected 8 ciphers but got 0 for the
specification 'ARIA256' expected:<[TLS_DHE_DSS_WITH_ARIA_256_GCM_SHA384,
TLS_DHE_PSK_WITH_ARIA_256_GCM_SHA384, TLS_DHE_RSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384, TLS_PSK_WITH_ARIA_256_GCM_SHA384,
TLS_RSA_PSK_WITH_ARIA_256_GCM_SHA384, TLS_RSA_WITH_ARIA_256_GCM_SHA384]>
but was:<[]>
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testSpecification(TestOpenSSLCipherConfigurationParser.java:588)
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testARIA256(TestOpenSSLCipherConfigurationParser.java:547)

Testcase: testkECDHE took 0.068 sec
     FAILED
Expected 31 ciphers but got 30 for the specification 'kECDHE'
expected:<[TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_AES_256_CBC_SHA,
TLS_ECDH_anon_WITH_NULL_SHA]> but
was:<[TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_anon_WITH_AES_128_CBC_SHA,
TLS_ECDH_anon_WITH_AES_256_CBC_SHA, TLS_ECDH_anon_WITH_NULL_SHA]>
junit.framework.AssertionFailedError: Expected 31 ciphers but got 30 for
the specification 'kECDHE' expected:<[TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_AES_256_CBC_SHA,
TLS_ECDH_anon_WITH_NULL_SHA]> but
was:<[TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_anon_WITH_AES_128_CBC_SHA,
TLS_ECDH_anon_WITH_AES_256_CBC_SHA, TLS_ECDH_anon_WITH_NULL_SHA]>
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testSpecification(TestOpenSSLCipherConfigurationParser.java:588)
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testkECDHE(TestOpenSSLCipherConfigurationParser.java:202)

Testcase: testkECDHe took 0.072 sec
Testcase: testkECDHr took 0.036 sec
Testcase: testkEECDH took 0.057 sec
     FAILED
Expected 31 ciphers but got 30 for the specification 'kEECDH'
expected:<[TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_AES_256_CBC_SHA,
TLS_ECDH_anon_WITH_NULL_SHA]> but
was:<[TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_anon_WITH_AES_128_CBC_SHA,
TLS_ECDH_anon_WITH_AES_256_CBC_SHA, TLS_ECDH_anon_WITH_NULL_SHA]>
junit.framework.AssertionFailedError: Expected 31 ciphers but got 30 for
the specification 'kEECDH' expected:<[TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_AES_128_CBC_SHA, TLS_ECDH_anon_WITH_AES_256_CBC_SHA,
TLS_ECDH_anon_WITH_NULL_SHA]> but
was:<[TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CCM,
TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_CCM,
TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
TLS_ECDHE_ECDSA_WITH_NULL_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_NULL_SHA,
TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_anon_WITH_AES_128_CBC_SHA,
TLS_ECDH_anon_WITH_AES_256_CBC_SHA, TLS_ECDH_anon_WITH_NULL_SHA]>
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testSpecification(TestOpenSSLCipherConfigurationParser.java:588)
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testkEECDH(TestOpenSSLCipherConfigurationParser.java:190)

Testcase: testGOST89MAC took 0.06 sec
Testcase: testCHACHA20 took 0.061 sec
Testcase: testADH took 0.033 sec
     FAILED
Expected 11 ciphers but got 13 for the specification 'ADH'
expected:<[TLS_DH_anon_WITH_AES_128_CBC_SHA,
TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_DH_anon_WITH_AES_128_GCM_SHA256,
TLS_DH_anon_WITH_AES_256_CBC_SHA, TLS_DH_anon_WITH_AES_256_CBC_SHA256,
TLS_DH_anon_WITH_AES_256_GCM_SHA384, TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA256,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA256, TLS_DH_anon_WITH_SEED_CBC_SHA]>
but was:<[SSL_DH_anon_WITH_3DES_EDE_CBC_SHA,
TLS_DH_anon_WITH_3DES_EDE_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA,
TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_DH_anon_WITH_AES_128_GCM_SHA256,
TLS_DH_anon_WITH_AES_256_CBC_SHA, TLS_DH_anon_WITH_AES_256_CBC_SHA256,
TLS_DH_anon_WITH_AES_256_GCM_SHA384, TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA256,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA256, TLS_DH_anon_WITH_SEED_CBC_SHA]>
junit.framework.AssertionFailedError: Expected 11 ciphers but got 13 for
the specification 'ADH' expected:<[TLS_DH_anon_WITH_AES_128_CBC_SHA,
TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_DH_anon_WITH_AES_128_GCM_SHA256,
TLS_DH_anon_WITH_AES_256_CBC_SHA, TLS_DH_anon_WITH_AES_256_CBC_SHA256,
TLS_DH_anon_WITH_AES_256_GCM_SHA384, TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA256,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA256, TLS_DH_anon_WITH_SEED_CBC_SHA]>
but was:<[SSL_DH_anon_WITH_3DES_EDE_CBC_SHA,
TLS_DH_anon_WITH_3DES_EDE_CBC_SHA, TLS_DH_anon_WITH_AES_128_CBC_SHA,
TLS_DH_anon_WITH_AES_128_CBC_SHA256, TLS_DH_anon_WITH_AES_128_GCM_SHA256,
TLS_DH_anon_WITH_AES_256_CBC_SHA, TLS_DH_anon_WITH_AES_256_CBC_SHA256,
TLS_DH_anon_WITH_AES_256_GCM_SHA384, TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA256,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA,
TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA256, TLS_DH_anon_WITH_SEED_CBC_SHA]>
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testSpecification(TestOpenSSLCipherConfigurationParser.java:588)
     at
org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser.testADH(TestOpenSSLCipherConfigurationParser.java:325)

<snip/>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org