I would like to have something for developers to use ASAP to try the Accord syntax. Very few people have seen it, and I think there's a learning curve we can start earlier.
It's my understanding that ASF policy is that it needs to be a project release to create a docker image. On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan <jeremiah.jor...@gmail.com> wrote: > If we decide to go the route of not merging TCM to the 5.0 branch. Do we > actually need to immediately cut a 5.1 branch? Can we work on stabilizing > things while it is in trunk and cut the 5.1 branch when we actually think > we are near releasing? I don’t see any reason we can not cut “preview” > artifacts from trunk? > > -Jeremiah > > On Oct 24, 2023 at 11:54:25 AM, Jon Haddad <rustyrazorbl...@apache.org> > wrote: > >> I guess at the end of the day, shipping a release with a bunch of awesome >> features is better than holding it back. If there's 2 big releases in 6 >> months the community isn't any worse off. >> >> We either ship something, or nothing, and something is probably better. >> >> Jon >> >> >> On 2023/10/24 16:27:04 Patrick McFadin wrote: >> >> +1 to what you are saying, Josh. Based on the last survey, yes, everyone >> >> was excited about Accord, but SAI and UCS were pretty high on the list. >> >> >> Benedict and I had a good conversation last night, and now I understand >> >> more essential details for this conversation. TCM is taking far more work >> >> than initially scoped, and Accord depends on a stable TCM. TCM is months >> >> behind and that's a critical fact, and one I personally just learned of. I >> >> thought things were wrapping up this month, and we were in the testing >> >> phase. I get why that's a topic we are dancing around. Nobody wants to say >> >> ship dates are slipping because that's part of our culture. It's >> >> disappointing and, if new information, an unwelcome surprise, but none of >> >> us should be angry or in a blamey mood because I guarantee every one of us >> >> has shipped the code late. My reaction yesterday was based on an incorrect >> >> assumption. Now that I have a better picture, my point of view is >> changing. >> >> >> Josh's point about what's best for users is crucial. Users deserve stable >> >> code with a regular cadence of features that make their lives easier. If >> we >> >> put 5.0 on hold for TCM + Accord, users will get neither for a very long >> >> time. And I mentioned a disaster yesterday. A bigger disaster would be >> >> shipping Accord with a major bug that causes data loss, eroding community >> >> trust. Accord has to be the most bulletproof of all bulletproof features. >> >> The pressure to ship is only going to increase and that's fertile ground >> >> for that sort of bug. >> >> >> So, taking a step back and with a clearer picture, I support the 5.0 + 5.1 >> >> plan mainly because I don't think 5.1 is (or should be) a fast follow. >> >> >> For the user community, the communication should be straightforward. TCM + >> >> Accord are turning out to be much more complicated than was originally >> >> scoped, and for good reasons. Our first principle is to provide a stable >> >> and reliable system, so as a result, we'll be de-coupling TCM + Accord >> from >> >> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while >> >> additional hardening and testing is done. We can communicate this in a >> blog >> >> post., >> >> >> To make this much more palatable to our use community, if we can get a >> >> build and docker image available ASAP with Accord, it will allow >> developers >> >> to start playing with the syntax. Up to this point, that hasn't been >> widely >> >> available unless you compile the code yourself. Developers need to >> >> understand how this will work in an application, and up to this point, the >> >> syntax is text they see in my slides. We need to get some hands-on and >> that >> >> will get our user community engaged on Accord this calendar year. The >> >> feedback may even uncover some critical changes we'll need to make. Lack >> of >> >> access to Accord by developers is a critical problem we can fix soon and >> >> there will be plenty of excitement there and start building use cases >> >> before the final code ships. >> >> >> I'm bummed but realistic. It sucks that I won't have a pony for Christmas, >> >> but maybe one for my birthday? >> >> >> Patrick >> >> >> >> >> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie <jmcken...@apache.org> >> wrote: >> >> >> > Maybe it won't be a glamorous release but shipping >> >> > 5.0 mitigates our worst case scenario. >> >> > >> >> > I disagree with this characterization of 5.0 personally. UCS, SAI, Trie >> >> > memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 are >> >> > accurate, all combine to make 5.0 a pretty glamorous release IMO >> >> > independent of TCM and Accord. Accord is a true paradigm-shift >> game-changer >> >> > so it's easy to think of 5.0 as uneventful in comparison, and TCM helps >> >> > resolve one of the biggest pain-points in our system for over a decade, >> but >> >> > I think 5.0 is a very meaty release in its own right today. >> >> > >> >> > Anyway - I agree with you Brandon re: timelines. If things take longer >> >> > than we'd hope (which, if I think back, they do roughly 100% of the >> time on >> >> > this project), blocking on these features could both lead to a >> significant >> >> > delay in 5.0 going out as well as increasing pressure and risk of >> burnout >> >> > on the folks working on it. While I believe we all need some balanced >> >> > urgency to do our best work, being under the gun for something with a >> hard >> >> > deadline or having an entire project drag along blocked on you is not >> where >> >> > I want any of us to be. >> >> > >> >> > Part of why we talked about going to primarily annual calendar-based >> >> > releases was to avoid precisely this situation, where something that >> >> > *feels* right at the cusp of merging leads us to delay a release >> >> > repeatedly. We discussed this a couple times this year: >> >> > 1: https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3, >> >> > where Mick proposed a "soft-freeze" for everything w/out exception and >> 1st >> >> > week October "hard-freeze", and there was assumed to be lazy consensus >> >> > 2: https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw, >> >> > where we kept along with what we discussed in 1 but added in CEP-30 to >> be >> >> > waivered in as well. >> >> > >> >> > So. We're at a crossroads here where we need to either follow through >> with >> >> > what we all agreed to earlier this year, or acknowledge that our best >> >> > intention of calendar-based releases can't stand up to our optimism and >> >> > desire to get these features into the next major. >> >> > >> >> > There's no immediate obvious better path to me in terms of what's best >> for >> >> > our users. This is a situation of risk tolerance with a lot of unknowns >> >> > that could go either way. >> >> > >> >> > Any light that folks active on TCM and Accord could shed in terms of >> their >> >> > best and worst-case scenarios on timelines for those features might >> help us >> >> > narrow this down a bit. Otherwise, I'm inclined to defer to our past >> selves >> >> > and fall back to "we agreed to yearly calendar releases for good reason. >> >> > Let's stick to our guns." >> >> > >> >> > On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote: >> >> > >> >> > The concern I have with this is that is a big slippery 'if' that >> >> > involves development time estimation, and if it tends to take longer >> >> > than we estimate (as these things tend to do), then I can see a future >> >> > where 5.0 is delayed until the middle of 2024, and I really don't want >> >> > that to happen. Maybe it won't be a glamorous release but shipping >> >> > 5.0 mitigates our worst case scenario. >> >> > >> >> > Kind Regards, >> >> > Brandon >> >> > >> >> > On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi <djo...@apache.org> wrote: >> >> > > >> >> > > I have a strong preference to move out the 5.0 date to have accord and >> >> > TCM. I don’t see the point in shipping 5.0 without these features >> >> > especially if 5.1 is going to follow close behind it. >> >> > > >> >> > > Dinesh >> >> > > >> >> > > On Oct 23, 2023, at 4:52 AM, Mick Semb Wever <m...@apache.org> wrote: >> >> > > >> >> > > >> >> > > >> >> > > The TCM work (CEP-21) is in its review stage but being well past our >> >> > cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I would >> >> > like to propose the following. >> >> > > >> >> > > We merge TCM and Accord only to trunk. Then branch cassandra-5.1 and >> >> > cut an immediate 5.1-alpha1 release. >> >> > > >> >> > > I see this as a win-win scenario for us, considering our current >> >> > situation. (Though it is unfortunate that Accord is included in this >> >> > scenario because we agreed it to be based upon TCM.) >> >> > > >> >> > > This will mean… >> >> > > - We get to focus on getting 5.0 to beta and GA, which already has a >> >> > ton of features users want. >> >> > > - We get an alpha release with TCM and Accord into users hands >> quickly >> >> > for broader testing and feedback. >> >> > > - We isolate GA efforts on TCM and Accord – giving oss and downstream >> >> > engineers time and patience reviewing and testing. TCM will be the >> biggest >> >> > patch ever to land in C*. >> >> > > - Give users a choice for a more incremental upgrade approach, given >> >> > just how many new features we're putting on them in one year. >> >> > > - 5.1 w/ TCM and Accord will maintain its upgrade compatibility with >> >> > all 4.x versions, just as if it had landed in 5.0. >> >> > > >> >> > > >> >> > > The risks/costs this introduces are >> >> > > - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 >> branch, >> >> > and at some point decide to undo this work, while we can throw away the >> >> > cassandra-5.1 branch we would need to do a bit of work reverting the >> >> > changes in trunk. This is a _very_ edge case, as confidence levels on >> the >> >> > design and implementation of both are already tested and high. >> >> > > - We will have to maintain an additional branch. I propose that we >> >> > treat the 5.1 branch in the same maintenance window as 5.0 (like we have >> >> > with 3.0 and 3.11). This also adds the merge path overhead. >> >> > > - Reviewing of TCM and Accord will continue to happen post-merge. >> This >> >> > is not our normal practice, but this work will have already received its >> >> > two +1s from committers, and such ongoing review effort is akin to GA >> >> > stabilisation work on release branches. >> >> > > >> >> > > >> >> > > I see no other ok solution in front of us that gets us at least both >> the >> >> > 5.0 beta and TCM+Accord alpha releases this year. Keeping in mind users >> >> > demand to start experimenting with these features, and our Cassandra >> Summit >> >> > in December. >> >> > > >> >> > > >> >> > > 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3 >> >> > > >> >> > > >> >> > >> >> > >> >> > >> >> >>
