Sorry for the delay in getting to this Joel. This is great work - really well thought out and put together. I'm a +1 on it; had the following observations or questions reviewing the CEP:
Rather than going with a new discretely allowed option to a closed list of allowable options in STARTUP, what if we went the route we did with SUPPORTED and offered a variable [string multimap] so we could add future STARTUP options w/out fully revving the protocol spec in the future? (https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v5.spec#L462-L491) Any opinion on us deprecating the existing cassandra.yaml#authenticator: option to be renamed as "fallback_authenticator", and updating the text to reflect that? Or *default_authenticator*; something to denote through the naming that the role of that is secondary to the newly added *authenticator_negotiation*? I'm 50/50 on this; I don't love deprecation (knowing we're going to keep supporting the old nomenclature into perpetuity), but I do think the topic of auth is important enough that a little config disruption and maintenance here to steer new users to the new, more secure method of auth might warrant the change. > `‘requireAuthentication’ should be set to true as soon as all clients are > using other authenticators.` A hot prop or mutable vtable (... did we ever do mutable / change config through vtables? Hm, looks like not yet: https://issues.apache.org/jira/browse/CASSANDRA-15254) so we could change this live on a cluster w/out bouncing would be nice. Be nicer if that change also coincided w/a change in config via 1 UX through the DB. ;) But that's a *different* problem than what you're looking to address so worth deferring on that piece probably. re: thundering herd risk - this tickled my memory. We did some work on pre-caching things in the Auth cache and I *vaguely recall* getting it into its own ThreadPool; I was upstreaming other peoples' code so it's not as durable in my memory as usual. https://issues.apache.org/jira/browse/CASSANDRA-16958 and https://issues.apache.org/jira/browse/CASSANDRA-16957 come to mind. Ah, there we are: https://issues.apache.org/jira/browse/CASSANDRA-17812. We have a separate executor specifically for auth messages in Dispatcher.java; might be worth keeping an eye on this to see if it proves to be a new bottleneck w/a more heavyweight negotiation on connection. The JIRA link in the CEP just links to the ASF C* JIRA - I couldn't find a ticket for this CEP yet in JIRA. Generally we use that to link to a specific ticket; was a minor speed bump just FYI. And we have some prior art in the following: - CASSANDRA-13048: Support SASL mechanism negotiation in existing Authenticators - CASSANDRA-11471: Add SASL mechanism negotiation to the native protocol Might be good to link to those and once we get a JIRA up for CEP-50, we can flag those 2 as duplicates of it and close them out once it's done. Overall - looks great. Again: +1 from me. On Tue, Jul 8, 2025, at 8:34 PM, Joel Shepherd wrote: > Hi Doug - That's an interesting suggestion for a load test: I'll include > something like that in our plans. > > You're right about the logic in RoleManager: it should be doing the right > thing with MutualTlsWithPasswordFallbackAuthenticator. > > Thanks! -- Joel. > > On 7/8/2025 1:52 PM, Doug Rohrer wrote: >> >> +1 from me (I think committer +1s are "binding" on CEPs given our previous >> "how we do things" conversation, but either way, +1)... >> >> One interesting perf test to think about would be the difference between >> negotiated auth with MutualTlsAuthenticator and PasswordAuthenticator and >> the combined MutualTlsWithPasswordFallbackAuthenticator, as I think it'll >> provide a pretty good indication of the (hopefully negligible) performance >> difference when negotiating. >> >> Also, because I was curious about your last comment... >> MutualTlsWithPasswordFallbackAuthenticator _derives_ from >> PasswordAuthenticator, so *instanceof PasswordAuthenticator* checks return >> *true* for instances of it, and therefore (assuming you're talking about >> supportedOptions/alterableOptions in >> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/auth/CassandraRoleManager.java#L167-L172), >> it should work fine. Not that that helps _you_ solve your problem, but at >> least the existing classes should work. >> >> Thanks for putting the CEP together and working on the implementation! >> >> Doug >> >>> On Jul 3, 2025, at 2:38 PM, Joel Shepherd <sheph...@amazon.com> wrote: >>> >>> Thanks Andy - My hope/expectation is this would significantly reduce the >>> amount of friction involved when either implementing or migrating to a new >>> authenticator. I suspect it will benefit more complex environments too, >>> when a single authenticator isn't ideal for all clients. >>> >>> At this point, the stickiest bits I've run into have involved logic that >>> switches on "the" authenticator class, because of the hardcoded dependency >>> on a specific authn implementation, and working out how to sanely extend >>> the logic once it's possible for a single node to be using different >>> authenticators for different client sessions. Solvable but will take some >>> refactoring and might also generate debate about what the right behaviors >>> are in that scenario. An example is the RoleManager which currently creates >>> additional role attributes if the Password Authenticator is in use ... >>> which, now that you mention it, I wonder if that's already broken for the >>> MutualTlsWithPasswordFallbackAuthenticator. Hmm. >>> >>> Anyway, thanks for the feedback -- Joel. >>> >>> On 7/3/2025 7:52 AM, Tolbert, Andy wrote: >>>> >>>> Hi Joel, >>>> >>>> +1 (nb), I think this is a really good idea and well fleshed our CEP! >>>> >>>> The capability to allow the server to support multiple authenticators >>>> would be very useful. CEP-34 added a >>>> 'MutualTlsWithPasswordFallbackAuthenticator' for simultaneously supporting >>>> both mTLS and Password authentication, primarily as a means to introduce >>>> mTLS auth without breaking password auth and also a possible gradual >>>> migration to mTLS, but this only works for combining these two particular >>>> authenticators, and also creates another authenticator to support. >>>> >>>> The migration to any new auth strategy would likely involve the need to >>>> simultaneously support an existing and new auth provider. I think the >>>> approach you describe is well described and should meet this need assuming >>>> users use a driver that supports it. >>>> >>>> In terms of the protocol, utilizing the capabilities of the existing >>>> OPTIONS, STARTUP and SUPPORTED messages to communicate what authenticators >>>> are supported/should be used is pretty clever as it shouldn't require a >>>> protocol version uprev, and hopefully wouldn't be too complicated for a >>>> driver to implement. >>>> >>>> Thanks, >>>> Andy >>>> >>>> On Mon, Jun 30, 2025 at 11:44 AM Joel Shepherd <sheph...@amazon.com> wrote: >>>>> Erm ... and here's the CEP: >>>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-50%3A+Authentication+Negotiation >>>>> >>>>> (Thanks for the heads up, Abe ...) >>>>> >>>>> -- Joel. >>>>> >>>>> On 6/30/2025 9:37 AM, Joel Shepherd wrote: >>>>> > Hello - We would like to propose CEP-50: Authentication Negotiation >>>>> > for adoption by the community: <link> . >>>>> > >>>>> > This CEP proposes minor changes to the initial handshake protocol >>>>> > (OPTIONS, SUPPORTED and STARTUP messages) to enable a client to inform >>>>> > the node of the authenticators supported by the client, and changes in >>>>> > the node's authentication-related areas to enable it to pick its >>>>> > preferred authenticator for each client client connection. The CEP >>>>> > explains why this approach is proposed, instead of implementing a >>>>> > "negotiating authenticator". >>>>> > >>>>> > Authentication negotiation will make it easier and safer for >>>>> > administrators to migrate clusters to stronger authentication >>>>> > mechanisms (including switching on authentication for a cluster that >>>>> > has been using "allow-all" authentication) without downtime, and to >>>>> > support environments where different clients prefer different >>>>> > authentication mechanisms (e.g., username and password for ad-hoc >>>>> > cqlsh access, mutual TLS for programmatic access, etc.), without >>>>> > having to pick a single "lowest common denominator" authenticator for >>>>> > all. Additionally, the proposed changes are intended to be backwards >>>>> > compatible for both clients and nodes. >>>>> > >>>>> > Thanks in advance for your time and feedback. Please keep the >>>>> > discussion on this mailing list thread. >>>>> > >>>>> > Thanks! -- Joel. >>>>> > >>>>> > >>>>> > >>>>> >
