On 02/10/18 20:40, Igal Sapir wrote: > Mark / Chris, > > On 10/2/2018 6:36 AM, Mark Thomas wrote: >> On 02/10/18 06:58, Igal Sapir wrote: >>> When trying to run the unit test cases with `ant clean test` on the >>> current >>> trunk [1] I am getting two (per connector) failures: >>> >>> org.apache.tomcat.util.net.openssl.ciphers.TestCipher FAILED [2] >>> >>> org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser >>> >>> FAILED [3] >>> >>> Server version: Apache Tomcat/9.0.13-dev >>> Server built: Oct 2 2018 05:24:55 UTC >>> Server number: 9.0.13.0 >>> OS Name: Linux >>> OS Version: 4.18.9-200.fc28.x86_64 >>> Architecture: amd64 >>> JVM Version: 1.8.0_181-b13 >>> JVM Vendor: Oracle Corporation >>> >>> Am I missing something? Other than the obvious "missing ciphers", >>> that is. >> These tests are all particularly sensitive to the versions of OpenSSL, >> Java and the implementation of Java used. >> >> Generally, those tests are there to ensure that the code that translates >> between JSSE cipher definitions and OpenSSL definitions is correct. >> >> If you see a failure it may indicate that: >> >> - the test has a bug >> >> - you are running with an older version of OpenSSL that behaves >> differently from the latest version (we try and keep pace with the >> latest) >> >> - OpenSSL has changed behaviour and we need to update our translation >> code to align with it (unusual) >> >> - OpenSSL has changed behaviour and we need to update our tests to align >> with it (most frequent). > > Thank you both for the detailed explanation. I suspected that I should > had added the OpenSSL version to the OP. On that Fedora machine I have > OpenSSL 1.1.0i-fips 14 Aug 2018 > > I tried the same tests on a Windows 10 machine. Below are some > discrepancies/peculiarities that I've noticed (I'd be happy to improve > the test cases if possible):
I noticed some errors on Gump overnight so this morning I have build OpenSSL 1.0.2, 1.1.0, 1.1.1 and master locally and tested them against 8.5.x and 9.0.x. I found a couple of bugs: - The ARIA ciphers were not handled correctly so testing against OpenSSL 1.1.0 was always going to fail. This has been fixed. - 8.5.x was missing some code that ensured the OpenSSL libraries as well as the binary was on the path. This meant 8.5.x tests were either going to fail or use a locally installed OpenSSL version. This has also been fixed. > On the Linux box I have OpenSSL installed and on the PATH. On Windows I > used version OpenSSL 1.1.1 11 Sep 2018 and specified it via the > `test.openssl.path` property. I checked the value of > `test.openssl.exists` and it showed the expected `true`. Both Windows > and Fedora generated an output file for > test/org/apache/tomcat/util/net/openssl/TestOpenSSLConf.java [1]. Both, > however, reported "Found OpenSSL version 0x0" which I find strange? That does seem odd. I suspect either the wrong OpenSSL version or no OpenSSL version was found. > On Windows, only the output [2] for the file mentioned above is in the > output/build/logs, while on Fedora I also have output from the 3 Test > files from test/org/apache/tomcat/util/net/openssl/ciphers/. Does that > mean that these tests were not run on Windows? That seems to be a reasonable conclusion. > I wanted to check the Gump output to compare with my local results. I > found this URL, which I'm not sure if it is the right one or not - > http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk/index.html - as > it says "Project build output found here..." but without any links or > any other information. That is the build. The full output is linked just below that line but you probably want the tests which are run as a separate build for each connector. http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio/index.html http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-apr/index.html If you scroll down for any of those pages, you'll find the individual test files for the latest run. > I would like at the very least to add the output of `openssl version` to > the Ant output, perhaps at the `test.openssl.exists` target. If there > are no objections I will add that. +1 Mark --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
