> > ... I just think it is important to make a decision for each refactor > based on the merits and goals in question. If the justification is improved > testing, then testing should be a core goal of the CEP. If it’s enabling a > feature to be upstreamed later, I personally would prefer to tie the > refactors to those features – which I hope will all find broad support for > inclusion; certainly those I have heard of, I am eager to see arrive in > Cassandra.
+1 If the goal is to support entirely external features, we have to decide > what kind of support we offer to these APIs, and this probably needs to be > discussed on a per-API basis with the justification for pluggability > weighed against any constraints this imposes on development. The most > obvious example here is membership and schema, which I think is a primarily > to support an external dependency but we expect this area of the codebase > to be significantly revised over the coming months. Similar topic was brought up with indexes... we do not have a good way of knowing what internal Java APIs must be supported and can't break, so before adding new ones we need to figure that out. On Mon, Oct 25, 2021 at 7:35 AM bened...@apache.org <bened...@apache.org> wrote: > Thanks Henrik for the additional context. > > I’m not personally a fan of modularity only for modularity’s sake. > Everything in software is a balancing act of competing priorities, and > while pluggability supports certain use cases it can slow down development > or prevent deeper integrations by preventing assumptions about how systems > operate. > > To be clear, I’m fully in favour of helping to enable your use cases, I > just think it is important to make a decision for each refactor based on > the merits and goals in question. If the justification is improved testing, > then testing should be a core goal of the CEP. If it’s enabling a feature > to be upstreamed later, I personally would prefer to tie the refactors to > those features – which I hope will all find broad support for inclusion; > certainly those I have heard of, I am eager to see arrive in Cassandra. > > If the goal is to support entirely external features, we have to decide > what kind of support we offer to these APIs, and this probably needs to be > discussed on a per-API basis with the justification for pluggability > weighed against any constraints this imposes on development. The most > obvious example here is membership and schema, which I think is a primarily > to support an external dependency but we expect this area of the codebase > to be significantly revised over the coming months. > > > From: Henrik Ingo <henrik.i...@datastax.com> > Date: Monday, 25 October 2021 at 14:52 > To: dev@cassandra.apache.org <dev@cassandra.apache.org> > Subject: Re: [DISCUSS] CEP-18: Improving Modularity > Hi Benedict > > This CEP is a bundle of APIs arising out of our recent work to re-architect > Cassandra into a more cloud native architecture. What our product marketing > has chosen to call "Serverless" is a variant of Cassandra where we have > separated compute from storage (coordinator vs data node), used S3-like > storage, and made various improvements to better support multi-tenancy in a > single Cassandra (Serverless) cluster. This whitepaper [1] explains this > work in detail for those of you interested to learn more. (Apologies that > it requires registration and the first page may at times sound a bit > marketingy, but it's really the most detailed report we have published so > far.) > > [1] https://www.datastax.com/resources/whitepaper/astra-serverless > > The above work was implemented in a way where by default a user can > continue to run Cassandra in the familiar "classic" way. The APIs > introduced by CEP-18 on the other hand allow alternate or additional > functionality to be provided, which in our case we have used to create a > "serverless" way of deploying a Cassandra cluster. > > The logic behind proposing this bundle of APIs separately, is roughly for > these reasons: > > The APIs touch existing code and functionality, so to minimize risk to the > next Cassandra release, it would make sense to try to complete merging this > work as early as possible in the development cycle. For the same reason, > keeping the new implementations out of this CEP allows us to focus review - > both of the CEP, and the eventual pull requests - on the APIs themselves, > whereas the related implementations (or plug-ins) would add to the scope > quite significantly. On the other hand non-default plugin functionality can > be added later with much lower risk. > > Second, while it's completely fair to ask for context, why was this > particular refactoring or API done in the first place, the assumption for a > CEP like this one is that better defined interfaces, that are better > documented and come with better test coverage than existing code, should be > enough legs to stand on in itself. Also, in the best case a good API will > also enable other implementations than the one we had in mind when > developing the API, so we wouldn't want to tie the discussion too much into > the implementation that happened to be the first. (As an example of this > working out nicely, your own work in CASSANDRA-16926 was for you motivated > by enabling a new kind of testing, but it also just so happens it is the > same work that enables someone to implement remote file storage, which we > therefore could drop from this CEP-18.) > > Conversely also, it was our expectation when proposing this CEP that > "better modularity" at least on a high level should be a fairly > straightforward conversation, while the actual plugins that make up our > "serverless" new architecture may reasonably ignite much more debate, or at > least questions as to how they work. As we have a backlog of several fairly > substantial CEPs lined up, we are trying to be very mindful of the > bandwidth of the developers on this list. For example, last week Jacek also > proposed CEP-17 for discussion. So we are trying to focus the discussion on > what's in CEP-17 and CEP-18 for now. (In addition I remember at least 2 > CEPs that were discussed but not yet voted on. I don't know if this adds to > cognitive load for anyone else than myself.) > > henrik > > On Mon, Oct 25, 2021 at 12:39 PM bened...@apache.org <bened...@apache.org> > wrote: > > > Hi Jeremiah, > > > > My personal view is that work to modularise the codebase should be tied > to > > specific use cases. If improved testing is the purpose of this work, I > > think it would help to include those improved tests that you plan to > > support as goals for the CEP. > > > > If on the other hand some of this work is primarily intended to enable > > certain features, I personally think it would be preferable to tie them > to > > those features - perhaps with their own CEP? > > > > > > From: Jeremiah Jordan <jeremiah.jor...@gmail.com> > > Date: Friday, 22 October 2021 at 16:24 > > To: Cassandra DEV <dev@cassandra.apache.org> > > Subject: [DISCUSS] CEP-18: Improving Modularity > > Hi All, > > As has been seen with the work already started in CEP-10, increasing the > > modularity of our subsystems can improve their testability, and also the > > ability to try new implementations without breaking things. > > > > Our team has been working on doing this and CEP-18 has been created to > > propose adding more modularity to a few different subsystems. > > > > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-18%3A+Improving+Modularity > > > > CASSANDRA-17044 has already been created for Schema Storage changes > related > > to this work and more JIRAs and PRs are to follow for the other > subsystems > > proposed in the CEP. > > > > Thanks, > > -Jeremiah Jordan > > > > > -- > > Henrik Ingo > > +358 40 569 7354 <358405697354> > > [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on > Twitter.] <https://twitter.com/DataStaxEng> [image: Visit us on YouTube.] > < > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_channel_UCqA6zOSMpQ55vvguq4Y0jAg&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=IFj3MdIKYLLXIUhYdUGB0cTzTlxyCb7_VUmICBaYilU&m=bmIfaie9O3fWJAu6lESvWj3HajV4VFwgwgVuKmxKZmE&s=16sY48_kvIb7sRQORknZrr3V8iLTfemFKbMVNZhdwgw&e= > > > [image: Visit my LinkedIn profile.] <https://www.linkedin.com/in/heingo/ > > >
