> in practice we wait and receive bug reports from downstream testing efforts. > Such testing isn't necessarily possible pre-commit, e.g. third-party and not > feasible to continuously run, nor appropriate to upstream/open-source. > > We want GA releases to be production ready for any cluster at any scale. So I > guess in practice for us Stable Trunk != GA, but that's ok I agree with this sentiment, and I also am uncomfortable with how vague this presently is. I'd be happier with something concrete like the following expected release flow:
1) We freeze a branch 2) To hit RC, we need green circle + no regression on ASF (or green ASF in the future when stable) 3) We need N weeks in this frozen state for people to test it out 4) Once we have both 2 and 3, we RC and GA I definitely think there should be a time-boxed element of it but as far as I know we haven't really settled on that and it's pretty hand-wavy. Probably moot as in the past getting tests to green gave us enough calendar time that folks had plenty of time to test out the betas and RC's, but in a world where tests are stable, that shifts us to needing to set a minimum time on calendar to give folks time to test. Is it too prescriptive to say "we'll be frozen on a branch for at least 8 weeks so folks can test out the betas"? (I ask because I know I can get a little "structure-happy" at times). On Fri, Mar 31, 2023, at 9:17 AM, Mick Semb Wever wrote: > >> We have a lot of significant features that have or will land soon and our >> experience suggests that those merges usually bring their set of >> instabilities. The goal of the proposal was to make sure that we get rid of >> them before TCM and Accord land to allow us to more easily identify the root >> causes of problems. >>> >> >> > Agree with Benjamin that testing in phases, i.e. separate periods of time, > has positives that we can take advantage of. > > > >> a) do we have test failures on circle on trunk right now, and >> b) do we have regressions on trunk on ASF CI compared to 4.1 >> >> Whether or not new features land near the cutoff date or not shouldn't >> impact the above right? > > > I don't think it can be limited to the above. They are our minimum > requirements to getting to beta, to rc, and to GA. But in practice we wait > and receive bug reports from downstream testing efforts. Such testing isn't > necessarily possible pre-commit, e.g. third-party and not feasible to > continuously run, nor appropriate to upstream/open-source. > > We want GA releases to be production ready for any cluster at any scale. So I > guess in practice for us Stable Trunk != GA, but that's ok – just being > honest to the ideal we are moving towards. >
