tl;dr Setting encryption_options.protocol does not control which TLS protocols are accepted, only restricting cipher_suites by protocol does and I think we should fix encryption_options.protocol to actually restrict, and have a proposal to do so at the end of the email.
I've been investigating restricting the TLS protocols to prevent use of TLSv1 & TLSv1.1 for secure internode messaging and streaming connections and think the current implementation needs improvement before the final 4.0 release. The Apache Cassandra documentation page on security https://cassandra.apache.org/doc/latest/operating/security.html mentions "... the JVM defaults for supported protocols and cipher suites are used when encryption is enabled. These can be overidden using the settings in cassandra.yaml, but this is not recommended unless there are policies in place which dictate certain settings or a need to disable vulnerable ciphers or protocols in cases where the JVM cannot be updated." The implication to me there is that the preferred mechanism is to configure the JSSE subsystem. Trawling through documentation, the operator can disable older TLS protocol at the JVM level by creating new security properties file $ cat conf/cassandra-security.properties jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024, \ EC keySize < 224, 3DES_EDE_CBC, anon, NULL, TLSv1, TLSv1.1 And appending to the current security properties using -Djava.security.properties=conf/cassandra-security.properties. This works fine pre-4.0, however the introduction of Netty tcnative which uses OpenSSL under the hood, does not use the java.security.properties to restrict anything. Neither does it implement the calls for supporting the OpenSSL configuration file. It only seems possible to restrict the protocol & ciphers through the Netty SSLContext API. It is possible to disable OpenSSL by setting the Java system property cassandra.disable_tcactive_openssl=true, but it seems undesirable to lose the performance benefit there. Looking in cassandra.yaml, under 'More advanced defaults' there is a ‘protocol' setting, which an operator might expect restricts which TLS protocols are accepted. # More advanced defaults: # protocol: TLS However, setting that to TLSv1.2 had no effect on the protocols the server accepted. Running "openssl s_client -tlsv1 -connect 127.0.0.1:7000" will connect without issue and negotiate a TLSv1.0 session. I found two previous tickets that addressed TLS protocols, first explicitly hard-coding the accepted TLS protocols to disable SSLv3 (due to POODLE) in CASSANDRA-8265 / b93f48a5db321bf7c9fb55a800ed6ab2d6f6b102, and then rely back on Java8 defaults in CASSANDRA-10508 / e4a0a4bf65a87c3aabae4ee0cc35009879e2d455 once they were fixed. CASSANDRA-10508 mentions the ‘protocol' field as a mechanism for specifying the protocol, however according to Java docs, that only verifies the protocol is to the SSL engine supported, and does not restrict negotiation to using it, as the openssl s_client test demonstrates. >From a quick search of the internet, a couple of blog posts recommend setting the cipher suite to only TLSv1.2 valid ciphers and I can confirm that does work, leading to this being logged (at ERROR). ERROR [Messaging-EventLoop-3-2] 2020-09-19T16:17:48,023 : - Failed to properly handshake with peer /127.0.0.1:33826. Closing the channel. io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1.1 is not enabled or supported in server context Caused by: javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1.1 is not enabled or supported in server context While it does work to restrict the protocol, if we start logging the accepted protocols the log will show that the server will negotiate TLS1/TLS1.1 which may get flagged by anybody validating the operators secure connection configuration. I also discovered that if SSL is misconfigured (ciphers, keystone, truststore etc), the node will start up happily but be unable to accept or make any secure internode connections. The current state of the code and documentation is unsatisfactory to me. We should at least improve the documentation to give clear guidance to operators on how they can secure their systems under 4.0/tcnative, however I think we should go further and make the encryption_option.protocol field behave as intended. Here's my proposal: 1) Interpret the current protocol string as a comma separated list of protocols that are accepted. Replace the default EncryptionOptions.protocol of "TLS" with null. 2) If protocol is non-null, call SslContextBuilder.protocols() with the configured protocols in org.apache.cassandra.security.SSLFactory#createNettySslContext 3) Special case the protocol configuration "TLS" to mean {"TLSv1", "TLSv1.1", "TLSv1.2”} for users that have uncommented the default value. Passing “TLS” is invalid in the protocols() call. 4) Hard-code org.apache.cassandra.security.SSLFactory#createSSLContext to pass "TLS" and then restrict to the protocols specified if non-null. It still looks used by the JavaDriverClient and BulkLoader. 5) Add a function to org.apache.cassandra.config.DatabaseDescriptor#applyAll to verify it is possible to create regular and Netty SSL contexts, and log the accepted protocols and ciphers after configuration. 6) Log the negotiated protocol and cipher after the SSL handshake completes, and make sure it is propagated to the Clients table. With those changes, operators should be able to restrict both the protocols and ciphers as well as audit the configuration and past behavior of their system. I'm bringing this to the mailing list so that those with operational experience using secure internode messaging/streaming can raise any issues I've missed, and to be clear about the change to the behavior of setting encryption_option.protocol actually restricting protocol now we are in the 4.0-beta and it is technically a user-visible change. I'll open a JIRA in the next few days incorporating feedback. Jon P.S. While researching, I think I read that there may also be issues when configuring SSL with differing supported ciphers and cipher names for JSSE / tcnative cipher suites, however the same configuration information is passed to both implementations. I haven’t investigated if it is a problem at all, or that just means you need to keep a separate tools configuration or we would need to add some kind of filtering prefix for jsse vs openssl ciphers. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
