I think cutting a 5.1 now makes sense But it should include all features from trunk that we consider to be production ready (that includes TCM in my book)
/Marcus On Thu, Oct 09, 2025 at 12:30:09AM +0200, Mick wrote: > > It is fantastic to read this Josh, to see an initiative to create a space to > bring new collaborators and energy into the project. > > Thank you for bringing it forward, representing those down-stream-fork users. > I'm reading this as true to the open source ethos of "scratch your itch", > and I see the value of letting meritocracy do its thing here. Some (not all) > of the objections and concerns I've read seem to be missing the point of what > people are saying they are already doing– and that if we create space and > process for them they can do that work lighter to the benefit of more… > > FTR, I'm not in favour of softening what goes into 5.0.x (see last paragraph > for more), I do believe (despite a long write-up) it can be trivial to > introduce a cassandra-5.1 branch for the pilot, but it would require some up > front agreements. > > My thoughts to this so far… > > - Distinction from the stable branch (Jeff, Stefan): this seems to be about > back porting features from trunk, something we forbid/frown upon doing as it > jeopardises our stable branch. If folk are going to extra effort to back > port into private forks specific features for the value they provide: despite > the risk (and extra effort in private QA-ing) the private back port against > safety and security; and they want to pilot a way to collaborate together to > reduce that effort in the project then we should be supportive of that. Nor > should we hijack our existing stable branches. So yeah, I do see a clear > distinction here. > > - We would need to be careful about what gets back ported (Isaac, Stefan, > Runtian). IMHO this cannot be about "this feature should…" but rather > sticking to the pilot's specific value-add and litmus test: folk are running > this feature in production in their private fork already. So IMHO this > should also exclude any downstream vendor saying "we have customers asking > for this feature so we want to back port it" as I would suggest that's our > slippery slope (this is different to a vendor saying "we are already running > this downstream"). This debunks the floodgates concern– this is not about > what users or vendors "want". This is about what downstream developers a) > are already running in production, and b) are themselves willing to step > forward into the community and do the work in the pilot. We should also be > explicit about the things we cannot accept back ports of: major > protocol/messaging compatibility changes; as Caleb points out. > > - Lifecycle (Jeremiah, Isaac, Stefan): i would suggest that 5.1 becomes a > sibling release branch to 5.0. So yes, it means post 6.0 GA we have an > additional release branch to maintain (and forward merge commits) in > cassandra-5.1, but that it EOLs the same time as 5.0 does. This is a repeat > of what we did with 3.0 and 3.11. I think the overhead of this is minimal– > it might actually make forward merges easier, breaking up the rebasing to > smaller incremental steps (which ironically is parallel value of reducing and > de-risking downstream fork work). But this is still of course a concern to > consider (Stefan's point about CI runs particularly bears weight, see below), > but still I think the question of how long we have to maintain branches is > the bigger concern. Also note (Jeremy), the next major release after 6.0 is > 7.0. Any 6.1 future branch would be a similar back-port of 7.0 dev features > that users have already started using in a downstream production, given the > pilot proves to be successful (this is to be evaluated). > > - More branches, more CI pain (Stefan): yeah, this struck me as a legit > concern. We do now have pre-ci.cassandra.apache.org > <http://pre-ci.cassandra.apache.org/> and I really hope that alleviates most > of the pain here (not just in its availability, but in its accessibility and > use subsequently improve our CI's pipeline efficiency). Accepting that might > not entirely cover it, my suggestion here would be we evaluate whether CI on > the 5.1 is a pre-commit requirement. If a bugfix patch for 5.0 has a > pre-commit run on 5.0 and trunk: like it would today; I'm ok continuing with > that onwards– if it gets the pilot up and running. Seeing the value of the > pilot, I'm happy to contribute to the work involved in adding the upgrade > paths. > > - 5.1 Releases (Francisco): Is this a requirement of the pilot program ? > Saying up front we won't be cutting releases is a surefire way to enforce the > idea this is reserved for downstream devs that solely are looking to > collaborate together on their back ports. But maybe that's a bit harsh. It > would be fair to state clearly that releases off this branch can be fewer, > given its intent. The other question that comes to mind is what QA label > would 5.1.x releases have ? Labelling them as betas is another, but gentler, > approach to messaging what the pilot is for. (And we want to still be > incentivising users to upgrade early and often to the latest GA major > release, and to be incentivising us to be making the upgrade process easier > for users.) > > - 5.1 needs to be maintained past 6.0 GA's date (Jeremiah, Isaac). > Continuing on from above. Users need a window to upgrade from 5.1 to 6.0, so > there has to be an overlap anyway. But I can imagine, and maybe some > downstream users involved here can answer this, that part of the value of > this pilot is that specific dev features in trunk might be a lot harder to > upgrade to than we realise– that it takes ~years to adjust fleets of > production clusters to certain new features in the next major, while other > features are just about back porting but provide value that's simply > unjustifiable for a business to look past (and is what keeps C* being part of > the tech stack in those companies). A 5.1 branch might also serve purpose in > providing specific features that help make the upgrade to the next major > possible/feasible (like support for a blue-green upgrade solution), but this > is purely speculative. > > - How other projects are doing it and their past discussions on what they > decided (Ekaterina). I agree with this– we should reference what they are > doing and why (but also not presume that's how they would do it again today). > Do we know any folk active in any of those projects ? > > > So FTR, based on the discussion so far, I'm really not keen of softening what > goes into 5.0.x, finding having to work on bugs in our stable branches > sometimes hard work, if not at times quite stressful, I really appreciate all > efforts in maintaining their hardness. I see the risk of production > clusters (and our reputation) if we introduce significant bugs a long way > into a stable branches patch versions. This will definitely make folk shy > from upgrades even more. > > IMO the 5.1 pilot seems such the much safer option, so I'm keen we can > address our concerns with it to give those people their room to collaborate > on it (and ofc appreciate the opposing concerns). And added win from this, I > can also see that it helps downstream forks incrementally rebase towards > trunk more often– a huge win for helping find bugs in trunk earlier and to > adopt trunk faster. I see downstream forks struggle to keep up in an > efficient manner, and that rebases onto year+ long new dev branches can be > time-consumingly painful and hard to keep justifying as a ongoing business > priority. I agree that anything to help alleviate this situation: without > removing the option of, or jeopardising/softening, the users safe stable "GA" > experience; would be most welcome. That these folk are stepping forward to > add their efforts in the community under such a pilot experiment is awesome, > and IMHO a strong sign of a healthy growing community. > > Mick > > > > On 7 Oct 2025, at 03:52, Jeff Jirsa <[email protected]> wrote: > > > > I have to admit I feel slow because I genuinely can’t tell what’s > > functionally different from this vs the existing strategy where we … > > selectively write patches for older versions when they’re low risk / high > > reward for safety and security > > > > Setting aside some unspecified nuances you haven’t haven’t defined, what > > makes this different from the existing practice of apply selective patches > > to old releases so users on old builds have a long term stable release that > > gets correctness and security fixes without the risk of new features ? > > > > > > > >> On Oct 6, 2025, at 9:04 AM, Josh McKenzie <[email protected]> wrote: > >> > >> > >> Many large‑scale Cassandra users have had to maintain private feature > >> back-port forks (e.g., CEP‑37, compaction optimization, etc) for years on > >> older branches. That duplication adds risk and pulls time away from > >> upstream contributions which came up as a pain point in discussion at CoC > >> this year. > >> > >> The proposal we came up with: an official, community‑maintained backport > >> branch (e.g. cassandra‑5.1) built on the current GA release that we pilot > >> for a year and then decide if we want to make it official. The branch > >> would selectively accept non‑disruptive improvements that meet criteria we > >> define together. There’s a lot of OSS prior art here (Lucene, httpd, > >> Hadoop, Kafka, Linux kernel, etc). > >> > >> Benefits include reduced duplicated effort, a safer middle ground between > >> trunk and frozen GA releases, faster delivery of vetted features, and > >> community energy going to this branch instead of duplicated on private > >> forks. > >> > >> If you’re interested in helping curate or maintain this branch - or have > >> thoughts on the idea - please reply and voice your thoughts. > >> > >> ~Josh >
