I share Jacek’s and Stefan’s sentiment about the low value of requiring precommit j11+j17 tests for all changes.
Perhaps this was needed during j17 stabilization but is no longer required? Please correct if I’m missing some context. To have a practical proposal to address this, how about: 1) Define “standard” java version for branch (11 or 17). 2) Define “standard” cassandra.yaml variant for branch (legacy cassandra.yaml or shiny cassandra_latest.yaml). 3) Require green CI on precommit on standard java version + standard cassandra.yaml variant. 4) Any known java-related changes require precommit j11 + j17. 5) Any known configuration changes require precommit tests on all cassandra.yaml variants. 6) All supported java versions + cassandra.yaml variants need to be checked before a release is proposed, to catch any issue missed during 4) or 5). For example: - If j17 is set as “default” java version of the branch cassandra-5.0, then j11 tests are no longer required for patches that don’t touch java-related stuff - if cassandra_latest.yaml becomes the new default configuration for 6.0, then precommit only needs to be run against thatversion - prerelease needs to be run against all cassandra.yaml variants. Wdyt? On Wed, 14 Feb 2024 at 18:25 Štefan Miklošovič <stefan.mikloso...@gmail.com> wrote: > Jon, > > I was mostly referring to Circle CI where we have two pre-commit > workflows. (just click on anything here > https://app.circleci.com/pipelines/github/instaclustr/cassandra) > > java17_pre-commit_tests > > This workflow is compiling & testing everything with Java 17 > > java11_pre-commit_tests > > This workflow is compiling with Java 11 and it contains jobs which are > also run with Java 11 and another set of jobs which run with Java 17. > > The workflow I have so far is that when I want to merge something, it is > required to formally provide builds for both workflows. Maybe I am doing > more work than necessary here but my understanding is that this has to be > done and it is required. > > I think that Jacek was talking also about this and that it is questionable > what value it brings. > > > > On Thu, Feb 15, 2024 at 12:13 AM Jon Haddad <j...@jonhaddad.com> wrote: > >> Stefan, can you elaborate on what you are proposing? It's not clear (at >> least to me) what level of testing you're advocating for. Dropping testing >> both on dev branches, every commit, just on release? In addition, can you >> elaborate on what is a hassle about it? It's been a long time since I >> committed anything but I don't remember 2 JVMs (8 & 11) being a problem. >> >> Jon >> >> >> >> On Wed, Feb 14, 2024 at 2:35 PM Štefan Miklošovič < >> stefan.mikloso...@gmail.com> wrote: >> >>> I agree with Jacek, I don't quite understand why we are running the >>> pipeline for j17 and j11 every time. I think this should be opt-in. >>> Majority of the time, we are just refactoring and coding stuff for >>> Cassandra where testing it for both jvms is just pointless and we _know_ >>> that it will be fine in 11 and 17 too because we do not do anything >>> special. If we find some subsystems where testing that on both jvms is >>> crucial, we might do that, I just do not remember when it was last time >>> that testing it in both j17 and j11 suddenly uncovered some bug. Seems more >>> like a hassle. >>> >>> We might then test the whole pipeline with a different config basically >>> for same time as we currently do. >>> >>> On Wed, Feb 14, 2024 at 9:32 PM Jacek Lewandowski < >>> lewandowski.ja...@gmail.com> wrote: >>> >>>> śr., 14 lut 2024 o 17:30 Josh McKenzie <jmcken...@apache.org> >>>> napisał(a): >>>> >>>>> When we have failing tests people do not spend the time to figure out >>>>> if their logic caused a regression and merge, making things more unstable… >>>>> so when we merge failing tests that leads to people merging even more >>>>> failing tests... >>>>> >>>>> What's the counter position to this Jacek / Berenguer? >>>>> >>>> >>>> For how long are we going to deceive ourselves? Are we shipping those >>>> features or not? Perhaps it is also a good opportunity to distinguish >>>> subsets of tests which make sense to run with a configuration matrix. >>>> >>>> If we don't add those tests to the pre-commit pipeline, "people do not >>>> spend the time to figure out if their logic caused a regression and merge, >>>> making things more unstable…" >>>> I think it is much more valuable to test those various configurations >>>> rather than test against j11 and j17 separately. I can see a really little >>>> value in doing that. >>>> >>>> >>>>
