On Tue, Feb 16, 2021 at 04:36:59AM +1100, Joel Sing wrote:
> On 21-02-15 14:49:46, Jan Klemkow wrote:
> > On Sat, Feb 13, 2021 at 03:53:48PM +0100, Theo Buehler wrote:
> > > On Sat, Feb 13, 2021 at 11:58:04AM +0100, Jan Klemkow wrote:
> > > > A coworker of mine has made tests with LibreSSL [1] and found some
> > > > regressions. I took his test descriptions and created the following
> > > > automated regression test. In the repository he described his findings
> > > > in detail. I kept the numbers of the files and subtests in the target
> > > > names for now. So, its easier to match it with his files.
> > > >
> > > > I don't know how to handle the result of "test-01-ssl". Thats why its
> > > > just a comment. Someone may have an idea to handle this properly.
> > > >
> > > > Any comments, wishes or OK's?
> > > >
> > > > [1]: https://github.com/noxxi/libressl-tests
> > >
> > > First of all thanks for the effort!
> > >
> > > The perl script and probably also the Makefile should have a license.
> > >
> > > Please add a check that tests whether the required perl modules are
> > > installed (p5-IO-Socket-SSL and p5-Net-SSLeay) and otherwise prints
> > > SKIPPED and their names, so I can install them if they're not present.
> > > I never remember their exact capitalization and hyphenation...
> > >
> > > Various comments inline, and a patch for openssl(1) at the end that may
> > > simplify some things.
> >
> > This is an updated version of the test including comments and wishes
> > from tb@ and bluhm@.
> >
> > OK?
>
> This currently drives openssl(1) for tests, which means that it is
> testing openssl(1), libssl and libcrypto, when what you're really
> wanting to test is libcrypto's verifier. While this works, the
> problem is that a change or breakage in libssl or openssl(1) results
> in a regress failure for libcrypto. If this is to land in its
> current form it really should be in regress/usr.bin/openssl -
> alternatively, it could be reworked to explicitly test libcrypto's
> APIs and remain here.
Except for the auto chain thing...
I'm ok with it going into regress/usr.bin/openssl/x509/ in essentially
its present form (with jsings and my nits addressed):
mv regress/lib/libcrypto/validate regress/usr.bin/openssl/x509
then change the SUBDIR line in the regress/usr.bin/openssl/Makefile:
-SUBDIR= options
+SUBDIR= options x509
> Some additional comments inline.
I also have a few more.
>
> > Index: regress/lib/libcrypto/validate/Makefile
> > ===================================================================
> > RCS file: regress/lib/libcrypto/validate/Makefile
> > diff -N regress/lib/libcrypto/validate/Makefile
> > --- /dev/null 1 Jan 1970 00:00:00 -0000
> > +++ regress/lib/libcrypto/validate/Makefile 15 Feb 2021 13:38:22 -0000
> > @@ -0,0 +1,133 @@
> > +# $OpenBSD$
> > +
> > +# Copyright (c) 2021 Jan Klemkow <j.klem...@wemelug.de>
> > +#
> > +# Permission to use, copy, modify, and distribute this software for any
> > +# purpose with or without fee is hereby granted, provided that the above
> > +# copyright notice and this permission notice appear in all copies.
> > +#
> > +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> > +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> > +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> > +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> > +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> > +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> > +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> > +
> > +# This regression test is based on manual test descriptions from:
> > +# https://github.com/noxxi/libressl-tests
> > +
> > +# The following port must be installed for the regression tests:
> > +# p5-IO-Socket-SSL perl interface to SSL sockets
> > +
> > +PERL = perl
> > +OPENSSL ?= openssl
> > +
> > +PERL_REQUIRE != perl -Mstrict -Mwarnings -e ' \
> > + eval { require IO::Socket::SSL } or print $@; \
> > +'
> > +.if ! empty (PERL_REQUIRE)
> > +regress:
> > + @echo "${PERL_REQUIRE}"
> > + @echo install these perl packages for additional tests
> > + @echo SKIPPED
> > +.endif
> > +
> > +REGRESS_TARGETS += test-unusual-wildcard-cert-no-CA-client
> > +REGRESS_TARGETS += test-unusual-wildcard-cert-CA-client
>
> I'd would s/unusual-wildcard/inlabel-wildcard/g
>
> > +REGRESS_TARGETS += test-common-wildcard-cert-no-CA-client
> > +REGRESS_TARGETS += test-common wildcard-cert-CA-client
>
> There's a space between "test-common" and "wildcard-cert-CA-client"
> (presumably meant to be a hyphen) - same in two places below.
>
> > +REGRESS_TARGETS += test-verify-unusual-wildcard-cert
> > +REGRESS_TARGETS += test-openssl-verify-common-wildcard-cert
> > +REGRESS_TARGETS += test-chain-certificates-s_server
> > +REGRESS_TARGETS += test-alternative-chain
> > +REGRESS_CLEANUP = cleanup-ssl
> > +REGRESS_SETUP_ONCE = create-libressl-test-certs
> > +
> > +REGRESS_EXPECTED_FAILURES += test-unusual-wildcard-cert-no-CA-client
> > +REGRESS_EXPECTED_FAILURES += test-common-wildcard-cert-no-CA-client
> > +REGRESS_EXPECTED_FAILURES += test-common wildcard-cert-CA-client
> > +REGRESS_EXPECTED_FAILURES += test-verify-unusual-wildcard-cert
> > +REGRESS_EXPECTED_FAILURES += test-alternative-chain
>
> I suspect that some or all of these are expected behaviour, rather
> than failures. We can review and address this once it lands though.
>
> > +create-libressl-test-certs: create-libressl-test-certs.pl
> > + ${PERL} ${.CURDIR}/$@.pl
>
> We can see how this goes, however we may end up wanting to generate
> the certificates and commit them rather than regenerating on each
> run. The other advantage is that p5-IO-Socket-SSL would only be
> needed to regenerate the certificates and not actually run the
> tests.
The error message is verbose but useless to me:
Can't locate IO/Socket/SSL.pm in @INC (you may need to install the
IO::Socket::SSL module) (@INC contains:
/usr/local/libdata/perl5/site_perl/amd64-openbsd
/usr/local/libdata/perl5/site_perl /usr/libdata/perl5/amd64-openbsd
/usr/libdata/perl5) at -e line 1.
install these perl packages for additional tests
Please mention the p5-IO-Socket-SSL name explicitly so I can copy paste
it and feed it into pkg_add if needed.
> > +cleanup-ssl:
> > + rm *.pem *.key
> > +
> > +test-unusual-wildcard-cert-no-CA-client:
> > + # unusual wildcard cert, no CA given to client
> > + # start client
start server
> > + ${OPENSSL} s_server -cert server-unusual-wildcard.pem \
> > + -key server-unusual-wildcard.pem & \
> > + timeout=$$(($$(date +%s) + 5)); \
> > + while fstat -p $$! | ! grep -q 'tcp .* \*:4433$$'; \
> > + do test $$(date +%s) -lt $$timeout || exit 1; done
> > + # start client
> > + echo "Q" | ${OPENSSL} s_client -verify_return_error \
> > + | grep "Verify return code: 21"
> > +
> > +test-unusual-wildcard-cert-CA-client:
> > + # unusual wildcard cert, CA given to client
> > + # start server
> > + ${OPENSSL} s_server -cert server-unusual-wildcard.pem \
> > + -key server-unusual-wildcard.pem & \
> > + timeout=$$(($$(date +%s) + 5)); \
> > + while fstat -p $$! | ! grep -q 'tcp .* \*:4433$$'; \
> > + do test $$(date +%s) -lt $$timeout || exit 1; done
> > + # start client
> > + echo "Q" | ${OPENSSL} s_client -CAfile caR.pem \
> > + | grep "Verify return code: 0"
> > +
> > +test-common-wildcard-cert-no-CA-client:
> > + # common wildcard cert, no CA given to client
> > + # start server
> > + ${OPENSSL} s_server -cert server-common-wildcard.pem \
> > + -key server-common-wildcard.pem & \
> > + timeout=$$(($$(date +%s) + 5)); \
> > + while fstat -p $$! | ! grep -q 'tcp .* \*:4433$$'; \
> > + do test $$(date +%s) -lt $$timeout || exit 1; done
> > + # start client
> > + echo "Q" | ${OPENSSL} s_client \
> > + | grep "Verify return code: 21"
> > +
> > +test-common wildcard-cert-CA-client:
>
> Space also here.
>
> > + # common wildcard cert, CA given to client
> > + # start server
> > + ${OPENSSL} s_server -cert server-unusual-wildcard.pem \
> > + -key server-unusual-wildcard.pem & \
> > + timeout=$$(($$(date +%s) + 5)); \
> > + while fstat -p $$! | ! grep -q 'tcp .* \*:4433$$'; \
> > + do test $$(date +%s) -lt $$timeout || exit 1; done
> > + # start client
> > + echo "Q" | ${OPENSSL} s_client -CAfile caR.pem \
> > + | grep "Verify return code: 21"
> > +
> > +test-verify-unusual-wildcard-cert:
> > + # openssl verify, unusual wildcard cert
> > + ${OPENSSL} verify -CAfile caR.pem server-unusual-wildcard.pem \
> > + | grep "server-unusual-wildcard.pem: OK"
> > +
> > +test-openssl-verify-common-wildcard-cert:
> > + # openssl verify, common wildcard cert
> > + ${OPENSSL} verify -CAfile caR.pem server-common-wildcard.pem \
> > + | grep "server-common-wildcard.pem: OK"
> > +
> > +test-chain-certificates-s_server:
> > + # Not all chain certificates are sent in s_server
> > + # start server
> > + # ${OPENSSL} s_server -cert server-subca.pem -CAfile subcaR.pem
zap the above line.
> > + ${OPENSSL} s_server -cert server-subca-chainS.pem -CAfile subcaR.pem & \
> > + timeout=$$(($$(date +%s) + 5)); \
> > + while fstat -p $$! | ! grep -q 'tcp .* \*:4433$$'; \
> > + do test $$(date +%s) -lt $$timeout || exit 1; done
> > + # start client
> > + ${OPENSSL} s_client -CAfile caR.pem | grep "Verify return code: 0"
> > +
> > +test-alternative-chain:
> > + # alternative chain not found
> > + ${OPENSSL} verify -verbose -trusted caR.pem -untrusted chainSX.pem \
> > + server-subca.pem | grep "server-subca.pem: Ok"
> > +
> > +.include <bsd.regress.mk>
> > Index: regress/lib/libcrypto/validate/create-libressl-test-certs.pl
> > ===================================================================
> > RCS file: regress/lib/libcrypto/validate/create-libressl-test-certs.pl
> > diff -N regress/lib/libcrypto/validate/create-libressl-test-certs.pl
> > --- /dev/null 1 Jan 1970 00:00:00 -0000
> > +++ regress/lib/libcrypto/validate/create-libressl-test-certs.pl 15 Feb
> > 2021 12:54:58 -0000
> > @@ -0,0 +1,111 @@
> > +#!/usr/bin/perl
> > +
> > +# Copyright (c) 2021 Steffen Ullrich <su...@cpan.org>
> > +# Public Domain
> > +
> > +use strict;
> > +use warnings;
> > +use IO::Socket::SSL::Utils;
> > +
> > +# primitive CA - ROOT
> > +my @ca = cert(
> > + CA => 1,
> > + subject => { CN => 'ROOT' }
> > +);
> > +out('caR.pem', pem(crt => $ca[0]));
> > +out('caR.key', pem(key => $ca[1]));
> > +
> > +# server certificate where SAN contains in-label wildcards which are
> > allowed by
> > +# RFC 6125
>
> It is worth noting that per the RFC, a client MAY allow in-label
> wildcards (this is not a MUST or even a SHOULD). Additionally,
> various software does not allow or support this (for example, libtls
> and hence ftp(1)).
>
> > +my @leafcert = cert(
> > + issuer => \@ca,
> > + purpose => 'server',
> > + subject => { CN => 'server.local' },
> > + subjectAltNames => [
> > + [ DNS => 'bar.server.local' ],
> > + [ DNS => 'www*.server.local'],
> > + [ DNS => '*.www.server.local'],
> > + [ DNS => 'foo.server.local' ],
> > + [ DNS => 'server.local' ],
> > + ]
> > +);
> > +out('server-unusual-wildcard.pem', pem(@leafcert));
> > +
> > +@leafcert = cert(
> > + issuer => \@ca,
> > + purpose => 'server',
> > + subject => { CN => 'server.local' },
> > + subjectAltNames => [
> > + [ DNS => 'bar.server.local' ],
> > + [ DNS => '*.www.server.local'],
> > + [ DNS => 'foo.server.local' ],
> > + [ DNS => 'server.local' ],
> > + ]
> > +);
> > +out('server-common-wildcard.pem', pem(@leafcert));
> > +
> > +# alternative CA - OLD_ROOT
> > +my @caO = cert(
> > + CA => 1,
> > + subject => { CN => 'OLD_ROOT' }
> > +);
> > +out('caO.pem', pem(crt => $caO[0]));
> > +out('caO.key', pem(key => $caO[1]));
> > +
> > +# alternative ROOT CA, signed by OLD_ROOT, same key as other ROOT CA
> > +my @caX = cert(
> > + issuer => \@caO,
> > + CA => 1,
> > + subject => { CN => 'ROOT' },
> > + key => $ca[1],
> > +);
> > +out('caX.pem', pem(crt => $caX[0]));
> > +out('caX.key', pem(key => $caX[1]));
> > +
> > +# subCA below ROOT
> > +my @subcaR = cert(
> > + issuer => \@ca,
> > + CA => 1,
> > + subject => { CN => 'SubCA.of.ROOT' }
> > +);
> > +out('subcaR.pem', pem(crt => $subcaR[0]));
> > +out('subcaR.key', pem(key => $subcaR[1]));
> > +out('chainSX.pem', pem($subcaR[0]), pem($caX[0]));
> > +
> > +@leafcert = cert(
> > + issuer => \@subcaR,
> > + purpose => 'server',
> > + subject => { CN => 'server.subca.local' },
> > + subjectAltNames => [
> > + [ DNS => 'server.subca.local' ],
> > + ]
> > +);
> > +out('server-subca.pem', pem(@leafcert));
> > +out('server-subca-chainSX.pem', pem(@leafcert, $subcaR[0], $caX[0]));
> > +out('server-subca-chainS.pem', pem(@leafcert, $subcaR[0]));
> > +
> > +
> > +sub cert { CERT_create(not_after => 10*365*86400+time(), @_) }
> > +sub pem {
> > + my @default = qw(crt key);
> > + my %m = (key => \&PEM_key2string, crt => \&PEM_cert2string);
> > + my $result = '';
> > + while (my $f = shift(@_)) {
> > + my $v;
> > + if ($f =~m{^(key|crt)$}) {
> > + $v = shift(@_);
> > + } else {
> > + $v = $f;
> > + $f = shift(@default) || 'crt';
> > + }
> > + $f = $m{$f} || die "wrong key $f";
> > + $result .= $f->($v);
> > + }
> > + return $result;
> > +}
> > +
> > +sub out {
> > + my $file = shift;
> > + open(my $fh,'>',"$file") or die "failed to create $file: $!";
> > + print $fh @_
> > +}
>
