> Mick and Ekaterina (and everyone really) - any thoughts on what test > coverage, if any, we should commit to for this new configuration? > Acknowledging that we already have *a lot* of CI that we run. >
Branimir in this patch has already done some basic cleanup of test variations, so this is not a duplication of the pipeline. It's a significant improvement. I'm ok with cassandra_latest being committed and added to the pipeline, *if* the authors genuinely believe there's significant time and effort saved in doing so. How many broken tests are we talking about ? Are they consistently broken or flaky ? Are they ticketed up and 5.0-rc blockers ? Having to deal with flakies and broken tests is an unfortunate reality to having a pipeline of 170k tests. Despite real frustrations I don't believe the broken windows analogy is appropriate here – it's more of a leave the campground cleaner… That being said, knowingly introducing a few broken tests is not that either, but still having to deal with a handful of consistently breaking tests for a short period of time is not the same cognitive burden as flakies. There are currently other broken tests in 5.0: VectorUpdateDeleteTest, upgrade_through_versions_test; are these compounding to the frustrations ? It's also been questioned about why we don't just enable settings we recommend. These are settings we recommend for new clusters. Our existing cassandra.yaml needs to be tailored for existing clusters being upgraded, where we are very conservative about changing defaults.
