> If we cannot meet at least that quality level (Green CI) we should not merge We should probably make it a formally agreed upon point to not merge things unless we're sure they won't destabilize, and thus block release of, a branch. So green CI for a feature (excepting feature-specific tests if it's still a work in progress), experimental flag if we don't consider it prod ready, should be absolute bare minimum for anything to merge really IMO.
On Wed, Oct 25, 2023, at 4:17 AM, Benjamin Lerer wrote: > The proposal includes 3 things: > 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0 > 2. The next release will be 5.1 and will include only Accord and TCM > 3. Merge TCM and Accord right now in 5.1 (making an initial release) > > I am fine with question 1 and do not have a strong opinion on which way to go. > 2. Means that every new feature will have to wait for post 5.1 even if it is > ready before 5.1 is stabilized and shipped. If we do a 5.1 release why not > take it as an opportunity to release more things. I am not saying that we > will. Just that we should let that door open. > 3. There is a need to merge TCM and Accord as maintaining those separate > branches is costly in terms of time and energy. I fully understand that. On > the other hand merging TCM and Accord will make the TCM review harder and I > do believe that this second round of review is valuable as it already > uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon as > it passes CI and continuing the review after the merge. If we cannot meet at > least that quality level (Green CI) we should not merge just for creating an > 5.1.alpha release for the summit. > > Now, I am totally fine with a preview release without numbering and with big > warnings that will only serve as a preview for the summit. > > Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi <berenguerbl...@gmail.com> a > écrit : >> I also think there's many good new features in 5.0 already they'd make a >> good release even on their own. My 2 cts. >> >> On 24/10/23 23:20, Brandon Williams wrote: >> > The catch here is that we don't publish docker images currently. The >> > C* docker images available are not made by us. >> > >> > Kind Regards, >> > Brandon >> > >> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin <pmcfa...@gmail.com> wrote: >> >> Let me make that really easy. Hell yes >> >> >> >> Not everybody runs CCM, I've tried but I've met resistance. >> >> >> >> Compiling your own version usually involves me saying the words "Yes, ant >> >> realclean exists. I'm not trolling you" >> >> >> >> docker pull <image> works on every OS and curates a single node >> >> experience. >> >> >> >> >> >> >> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie <jmcken...@apache.org> >> >> wrote: >> >>> In order for the project to advertise the release outside the dev@ list >> >>> it needs to be a formal release. >> >>> >> >>> That's my reading as well: >> >>> https://www.apache.org/legal/release-policy.html#release-definition >> >>> >> >>> I wonder if there'd be value in us having a cronned job that'd do >> >>> nightly docker container builds on trunk + feature branches, archived >> >>> for N days, and we make that generally known to the dev@ list here so >> >>> folks that want to poke at the current state of trunk or other branches >> >>> could do so with very low friction. We'd probably see more engagement on >> >>> feature branches if it was turn-key easy for other C* devs to spin the >> >>> up and check them out. >> >>> >> >>> For what you're talking about here Patrick (a docker image for folks >> >>> outside the dev@ audience and more user-facing), we'd want to vote on it >> >>> and go through the formal process. >> >>> >> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote: >> >>> >> >>> In order for the project to advertise the release outside the dev@ list >> >>> it needs to be a formal release. That just means that there was a >> >>> release vote and at least 3 PMC members +1’ed it, and there are more +1 >> >>> than there are -1, and we follow all the normal release rules. The ASF >> >>> release process doesn’t care what branch you cut the artifacts from or >> >>> what version you call it. >> >>> >> >>> So the project can cut artifacts for and release a 5.1-alpha1, >> >>> 5.1-dev-preview1, what ever we want to version this thing, from trunk or >> >>> any other branch name we want. >> >>> >> >>> -Jeremiah >> >>> >> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin <pmcfa...@gmail.com> >> >>> wrote: >> >>> >> >>> I would like to have something for developers to use ASAP to try the >> >>> Accord syntax. Very few people have seen it, and I think there's a >> >>> learning curve we can start earlier. >> >>> >> >>> It's my understanding that ASF policy is that it needs to be a project >> >>> release to create a docker image. >> >>> >> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan >> >>> <jeremiah.jor...@gmail.com> wrote: >> >>> >> >>> If we decide to go the route of not merging TCM to the 5.0 branch. Do >> >>> we actually need to immediately cut a 5.1 branch? Can we work on >> >>> stabilizing things while it is in trunk and cut the 5.1 branch when we >> >>> actually think we are near releasing? I don’t see any reason we can not >> >>> cut “preview” artifacts from trunk? >> >>> >> >>> -Jeremiah >> >>> >> >>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad <rustyrazorbl...@apache.org> >> >>> wrote: >> >>> >> >>> I guess at the end of the day, shipping a release with a bunch of >> >>> awesome features is better than holding it back. If there's 2 big >> >>> releases in 6 months the community isn't any worse off. >> >>> >> >>> We either ship something, or nothing, and something is probably better. >> >>> >> >>> Jon >> >>> >> >>> >> >>> On 2023/10/24 16:27:04 Patrick McFadin wrote: >> >>> >> >>> +1 to what you are saying, Josh. Based on the last survey, yes, everyone >> >>> >> >>> was excited about Accord, but SAI and UCS were pretty high on the list. >> >>> >> >>> >> >>> Benedict and I had a good conversation last night, and now I understand >> >>> >> >>> more essential details for this conversation. TCM is taking far more work >> >>> >> >>> than initially scoped, and Accord depends on a stable TCM. TCM is months >> >>> >> >>> behind and that's a critical fact, and one I personally just learned of. >> >>> I >> >>> >> >>> thought things were wrapping up this month, and we were in the testing >> >>> >> >>> phase. I get why that's a topic we are dancing around. Nobody wants to >> >>> say >> >>> >> >>> ship dates are slipping because that's part of our culture. It's >> >>> >> >>> disappointing and, if new information, an unwelcome surprise, but none of >> >>> >> >>> us should be angry or in a blamey mood because I guarantee every one of >> >>> us >> >>> >> >>> has shipped the code late. My reaction yesterday was based on an >> >>> incorrect >> >>> >> >>> assumption. Now that I have a better picture, my point of view is >> >>> changing. >> >>> >> >>> >> >>> Josh's point about what's best for users is crucial. Users deserve stable >> >>> >> >>> code with a regular cadence of features that make their lives easier. If >> >>> we >> >>> >> >>> put 5.0 on hold for TCM + Accord, users will get neither for a very long >> >>> >> >>> time. And I mentioned a disaster yesterday. A bigger disaster would be >> >>> >> >>> shipping Accord with a major bug that causes data loss, eroding community >> >>> >> >>> trust. Accord has to be the most bulletproof of all bulletproof features. >> >>> >> >>> The pressure to ship is only going to increase and that's fertile ground >> >>> >> >>> for that sort of bug. >> >>> >> >>> >> >>> So, taking a step back and with a clearer picture, I support the 5.0 + >> >>> 5.1 >> >>> >> >>> plan mainly because I don't think 5.1 is (or should be) a fast follow. >> >>> >> >>> >> >>> For the user community, the communication should be straightforward. TCM >> >>> + >> >>> >> >>> Accord are turning out to be much more complicated than was originally >> >>> >> >>> scoped, and for good reasons. Our first principle is to provide a stable >> >>> >> >>> and reliable system, so as a result, we'll be de-coupling TCM + Accord >> >>> from >> >>> >> >>> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while >> >>> >> >>> additional hardening and testing is done. We can communicate this in a >> >>> blog >> >>> >> >>> post., >> >>> >> >>> >> >>> To make this much more palatable to our use community, if we can get a >> >>> >> >>> build and docker image available ASAP with Accord, it will allow >> >>> developers >> >>> >> >>> to start playing with the syntax. Up to this point, that hasn't been >> >>> widely >> >>> >> >>> available unless you compile the code yourself. Developers need to >> >>> >> >>> understand how this will work in an application, and up to this point, >> >>> the >> >>> >> >>> syntax is text they see in my slides. We need to get some hands-on and >> >>> that >> >>> >> >>> will get our user community engaged on Accord this calendar year. The >> >>> >> >>> feedback may even uncover some critical changes we'll need to make. Lack >> >>> of >> >>> >> >>> access to Accord by developers is a critical problem we can fix soon and >> >>> >> >>> there will be plenty of excitement there and start building use cases >> >>> >> >>> before the final code ships. >> >>> >> >>> >> >>> I'm bummed but realistic. It sucks that I won't have a pony for >> >>> Christmas, >> >>> >> >>> but maybe one for my birthday? >> >>> >> >>> >> >>> Patrick >> >>> >> >>> >> >>> >> >>> >> >>> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie <jmcken...@apache.org> >> >>> wrote: >> >>> >> >>> >> >>>> Maybe it won't be a glamorous release but shipping >> >>>> 5.0 mitigates our worst case scenario. >> >>>> I disagree with this characterization of 5.0 personally. UCS, SAI, Trie >> >>>> memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are >> >>>> accurate, all combine to make 5.0 a pretty glamorous release IMO >> >>>> independent of TCM and Accord. Accord is a true paradigm-shift >> >>>> game-changer >> >>>> so it's easy to think of 5.0 as uneventful in comparison, and TCM helps >> >>>> resolve one of the biggest pain-points in our system for over a decade, >> >>>> but >> >>>> I think 5.0 is a very meaty release in its own right today. >> >>>> Anyway - I agree with you Brandon re: timelines. If things take longer >> >>>> than we'd hope (which, if I think back, they do roughly 100% of the >> >>>> time on >> >>>> this project), blocking on these features could both lead to a >> >>>> significant >> >>>> delay in 5.0 going out as well as increasing pressure and risk of >> >>>> burnout >> >>>> on the folks working on it. While I believe we all need some balanced >> >>>> urgency to do our best work, being under the gun for something with a >> >>>> hard >> >>>> deadline or having an entire project drag along blocked on you is not >> >>>> where >> >>>> I want any of us to be. >> >>>> Part of why we talked about going to primarily annual calendar-based >> >>>> releases was to avoid precisely this situation, where something that >> >>>> *feels* right at the cusp of merging leads us to delay a release >> >>>> repeatedly. We discussed this a couple times this year: >> >>>> 1: https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3, >> >>>> where Mick proposed a "soft-freeze" for everything w/out exception and >> >>>> 1st >> >>>> week October "hard-freeze", and there was assumed to be lazy consensus >> >>>> 2: https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw, >> >>>> where we kept along with what we discussed in 1 but added in CEP-30 to >> >>>> be >> >>>> waivered in as well. >> >>>> So. We're at a crossroads here where we need to either follow through >> >>>> with >> >>>> what we all agreed to earlier this year, or acknowledge that our best >> >>>> intention of calendar-based releases can't stand up to our optimism and >> >>>> desire to get these features into the next major. >> >>>> There's no immediate obvious better path to me in terms of what's best >> >>>> for >> >>>> our users. This is a situation of risk tolerance with a lot of unknowns >> >>>> that could go either way. >> >>>> Any light that folks active on TCM and Accord could shed in terms of >> >>>> their >> >>>> best and worst-case scenarios on timelines for those features might >> >>>> help us >> >>>> narrow this down a bit. Otherwise, I'm inclined to defer to our past >> >>>> selves >> >>>> and fall back to "we agreed to yearly calendar releases for good reason. >> >>>> Let's stick to our guns." >> >>>> On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote: >> >>>> The concern I have with this is that is a big slippery 'if' that >> >>>> involves development time estimation, and if it tends to take longer >> >>>> than we estimate (as these things tend to do), then I can see a future >> >>>> where 5.0 is delayed until the middle of 2024, and I really don't want >> >>>> that to happen. Maybe it won't be a glamorous release but shipping >> >>>> 5.0 mitigates our worst case scenario. >> >>>> Kind Regards, >> >>>> Brandon >> >>>> On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi <djo...@apache.org> wrote: >> >>>>> I have a strong preference to move out the 5.0 date to have accord and >> >>>> TCM. I don’t see the point in shipping 5.0 without these features >> >>>> especially if 5.1 is going to follow close behind it. >> >>>>> Dinesh >> >>>>> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever <m...@apache.org> wrote: >> >>>>> >> >>>>> The TCM work (CEP-21) is in its review stage but being well past our >> >>>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would >> >>>> like to propose the following. >> >>>>> We merge TCM and Accord only to trunk. Then branch cassandra-5.1 and >> >>>> cut an immediate 5.1-alpha1 release. >> >>>>> I see this as a win-win scenario for us, considering our current >> >>>> situation. (Though it is unfortunate that Accord is included in this >> >>>> scenario because we agreed it to be based upon TCM.) >> >>>>> This will mean… >> >>>>> - We get to focus on getting 5.0 to beta and GA, which already has a >> >>>> ton of features users want. >> >>>>> - We get an alpha release with TCM and Accord into users hands >> >>>>> quickly >> >>>> for broader testing and feedback. >> >>>>> - We isolate GA efforts on TCM and Accord – giving oss and downstream >> >>>> engineers time and patience reviewing and testing. TCM will be the >> >>>> biggest >> >>>> patch ever to land in C*. >> >>>>> - Give users a choice for a more incremental upgrade approach, given >> >>>> just how many new features we're putting on them in one year. >> >>>>> - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with >> >>>> all 4.x versions, just as if it had landed in 5.0. >> >>>>> The risks/costs this introduces are >> >>>>> - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 >> >>>>> branch, >> >>>> and at some point decide to undo this work, while we can throw away the >> >>>> cassandra-5.1 branch we would need to do a bit of work reverting the >> >>>> changes in trunk. This is a _very_ edge case, as confidence levels on >> >>>> the >> >>>> design and implementation of both are already tested and high. >> >>>>> - We will have to maintain an additional branch. I propose that we >> >>>> treat the 5.1 branch in the same maintenance window as 5.0 (like we have >> >>>> with 3.0 and 3.11). This also adds the merge path overhead. >> >>>>> - Reviewing of TCM and Accord will continue to happen post-merge. >> >>>>> This >> >>>> is not our normal practice, but this work will have already received its >> >>>> two +1s from committers, and such ongoing review effort is akin to GA >> >>>> stabilisation work on release branches. >> >>>>> I see no other ok solution in front of us that gets us at least both >> >>>>> the >> >>>> 5.0 beta and TCM+Accord alpha releases this year. Keeping in mind users >> >>>> demand to start experimenting with these features, and our Cassandra >> >>>> Summit >> >>>> in December. >> >>>>> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3 >> >>> >> >>>
