Re: Cassandra project biweekly status update 2022-06-14
> I think it is good to document further things and keep on doing it in time > when discussions happen. I can see this being a benefit both for users and > Cassandra developers. Strong +1 from me here. Having guidance for people working on the codebase to understand what is and isn't considered a public API will help inform how we shape these things and keep things stable for our userbase. On Sun, Jun 26, 2022, at 12:58 PM, Ekaterina Dimitrova wrote: > “+1 to always, by default, maintaining compatibility.” > +1 > > “We also have the discussion wrt what are breaking changes. Are we intending > to evolve what interfaces and behaviour we provide, and to what degree, > compatibility over via these discussions/votes?” > > While I do agree we cannot really have a fully exhaustive list, I think it is > good to document further things and keep on doing it in time when discussions > happen. I can see this being a benefit both for users and Cassandra > developers. > > > On Thu, 16 Jun 2022 at 18:25, Mick Semb Wever wrote: >> >> >>> We've agreed in the past that we want to maintain compatibility and that >>> all changes will be done with compatibility in mind (see >>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle), >>> but we haven't clarified how we make the call on when to bump to next >>> major. >> >> >> +1 to always, by default, maintaining compatibility. >> >> Note, a major bump isn't only about compatibility breakages per se, but a) >> time to clean up deprecated code, and b) delineating upgrade paths. >> >>> The Release Lifecycle states "Introducing a backward incompatibility change >>> needs dev community approval via voting [voting open for 48 hours]." But >>> this is under a section called "Outstanding questions beyond the scope of >>> this document", maybe it's about time we finalize this and update this >>> document? >> >> >> IIRC, though i can easily be wrong, this was meant for breaking changes >> within a major, e.g. after a beta release. Not that the same formality >> cannot also be applied to trunk dev, as it ensures a desired visibility, >> though I wonder if we will solve it in practice most of the time with the >> preceding [DISCUSS] thread. >> >> We also have the discussion wrt what are breaking changes. Are we intending >> to evolve what interfaces and behaviour we provide, and to what degree, >> compatibility over via these discussions/votes?
Cassandra project biweekly status update 2022-06-28
Slower past couple of weeks; a lot of focus on burning down failing tests. We're at 5 weeks without a build lead (details on the role here: https://cwiki.apache.org/confluence/display/CASSANDRA/Build+Lead). I've taken on the role a few times myself this year already and it's really helpful to get a solid understanding of where some of the weaknesses in our CI infra are, where some test infra and code is brittle or weak, and to help keep visibility in the project to test failures as we burn them down leading up to 4.1's GA. Please don't hesitate to reach out via email or slack if you have the bandwidth to help out (a few hours over the course of the week) and I'll be happy to mentor anyone through their first run as build lead. Checking in on 4.1: https://butler.cassandra.apache.org/#/ At about 10 failures up from 8 last check in 2 weeks ago. It's been pretty consistent and a lot of folks are working on burning those down; 4.1 ci has been a lot more stable recently: https://ci-cassandra.apache.org/job/Cassandra-4.1/. Still have 28 tickets that are beta blockers here: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2454. 4 in need of review down from 6 last time. It looks like almost all the tickets are either trivial or test fixes. [New contributor Getting Started] Check out the 12 unassigned tickets we have marked as beta blockers - many are great candidates for someone new to the project and will help us move towards our goal of releasing 4.1: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2454&quickFilter=2160 Reference the "Getting started" guide on our website for help with setting up your environment: https://cassandra.apache.org/_/development/index.html, and the dev community hangs out on https://the-asf.slack.com in #cassandra-dev. We have 13 @cassandra_mentors in the chat room, and feel free to float any questions you have to the channel at large as we're a 24/7 type of community around the world. [Dev list Digest] https://lists.apache.org/list?dev@cassandra.apache.org:lte=2w: The thread about slow custom unit tests running on Mac OS X continues here: https://lists.apache.org/thread/tt2mmkvp5p0os9k15lwssdbsbqff28s6. This isn't something I've personally seen running on OSX (outside a containerized / docker environment) but if anyone's run into this type of issue, please chime in! The last biweekly status email (https://lists.apache.org/thread/tt2mmkvp5p0os9k15lwssdbsbqff28s6) brought over conversation from JIRA about when we make backwards API breaking changes. That conversation is still ongoing; a couple of days ago Ekaterina advocated for us documenting what we consider "public API's" and investing energy into keeping that up to date and I have to say I agree. All too often you can make a change to a tool or subsystem without realizing there would be external parsers or code coupled with that output or interfaces; the more clarity we can provide both for devs working on C* and consumers of these APIs, the smoother working on this will be for all of us. The conversation on the shape of the syntax for CEP-15's multi-key transactions seems to be winding down (https://lists.apache.org/thread/khr33obyxvfhfpjoyzgwbmh7535mf7hv). A lot of great discussion, probing, and collaboration's gone on on that thread. If you have an opinion, perspective, or question that hasn't been covered already in that thread please pipe up as this is going to be a new foundational primitive for us in the database so the cost of revving that API in the future is high. Last but not least, there's been discussion around the Cassandra Corner / Apache Cassandra Corner podcast, nominal use of the mark, whether it needs the Apache branding on it or not, who has access, etc. (https://lists.apache.org/thread/m4j3r8sofpqsqdgfjkjfr2rjc5w0bqj2). It's fantastic to see things like this in the community, for the community, and coordinated openly with the dev community at large and the PMC. Thanks Aaron for everything you're doing with that podcast and being such a great partner to everyone! [CI Trends] https://butler.cassandra.apache.org/#/ Let's diff last biweekly update vs this 3.0: 13 -> 10 3.11: 23 -> 19 4.0: 2 -> 3 4.1: 8 -> 19 (flaky CI run, probably ~ 10) trunk: 20 -> 16 So all but 4.1 trending in the direction we want. A ton of focus is on 4.1 right now so I don't see that as a long-term problem; let's keep up the pressure! [Release progress] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2175 4.1 alpha: Forbidding writetimes and ttl functions for multicell columns hit in CASSANDRA-17628, applying back to the 3.0 line as well. 4.1 beta: 3 test failure fixes, a change to nodetool clientstats to keep the output backwards compatible in CASSANDRA-17715, and a case where writes with CL=ONE + counters + a network partition could lead to timeouts in CASSANDRA-17411 all
Re: Cassandra project biweekly status update 2022-06-14
I don’t think it has to be all that complicated? If it’s a part of our UX it’s probably something we should maintain backwards compatibility for. If it’s part of our internal codebase, probably not. The only two “public” APIs we have inside the codebase that I’m aware of are triggers and secondary indexes, and these are provided with limited warranty and an expectation of technical sophistication for their users. I think there has always been an expectation that users of these features will bear the cost of migration to any new API versions we might introduce between majors. > On 28 Jun 2022, at 19:39, Josh McKenzie wrote: > > >> >> I think it is good to document further things and keep on doing it in time >> when discussions happen. I can see this being a benefit both for users and >> Cassandra developers. > Strong +1 from me here. Having guidance for people working on the codebase to > understand what is and isn't considered a public API will help inform how we > shape these things and keep things stable for our userbase. > >> On Sun, Jun 26, 2022, at 12:58 PM, Ekaterina Dimitrova wrote: >> “+1 to always, by default, maintaining compatibility.” >> +1 >> >> “We also have the discussion wrt what are breaking changes. Are we intending >> to evolve what interfaces and behaviour we provide, and to what degree, >> compatibility over via these discussions/votes?” >> >> While I do agree we cannot really have a fully exhaustive list, I think it >> is good to document further things and keep on doing it in time when >> discussions happen. I can see this being a benefit both for users and >> Cassandra developers. >> >> >> On Thu, 16 Jun 2022 at 18:25, Mick Semb Wever wrote: >> >> >> We've agreed in the past that we want to maintain compatibility and that all >> changes will be done with compatibility in mind (see >> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle), >> but we haven't clarified how we make the call on when to bump to next major. >> >> >> +1 to always, by default, maintaining compatibility. >> >> Note, a major bump isn't only about compatibility breakages per se, but a) >> time to clean up deprecated code, and b) delineating upgrade paths. >> >> The Release Lifecycle states "Introducing a backward incompatibility change >> needs dev community approval via voting [voting open for 48 hours]." But >> this is under a section called "Outstanding questions beyond the scope of >> this document", maybe it's about time we finalize this and update this >> document? >> >> >> IIRC, though i can easily be wrong, this was meant for breaking changes >> within a major, e.g. after a beta release. Not that the same formality >> cannot also be applied to trunk dev, as it ensures a desired visibility, >> though I wonder if we will solve it in practice most of the time with the >> preceding [DISCUSS] thread. >> >> We also have the discussion wrt what are breaking changes. Are we intending >> to evolve what interfaces and behaviour we provide, and to what degree, >> compatibility over via these discussions/votes? >
