> The testing module (Property, and stateful harness) has a wide and growing > set of potential consumers: cassandra-ecosystem projects, tooling projects, > automation infrastructure, and contributors who want access to the > property-based and stateful testing work without forking it. Multiple people > and projects have asked to use this code. Changes to it tend to be additive. > It is also exactly the kind of code that can carry strong property-based test > coverage of its own, since that is what it does.
Testing module (namely, Harry / stateful checker, generators, and more [1]), is already a standalone module. Same with simulator. I’m working through a proposal to make Simulator and Harry, alongside with stateful testing tooling, submitted for ASF incubator. It would be great if we can coordinate this work (i.e., migrating things to a separate repository). [1] https://github.com/apache/cassandra-harry/blob/harry-3.0/src/main/java/org/apache/cassandra/harry/checker/ModelChecker.java On Fri, Aug 21, 2026, at 10:11 AM, David Capwell wrote: >> 1. We could split the difference and just do unit tests for each project on >> utils. That'd exercise the code w/out subjecting people working on utils to >> the flakiness of multi-node non-determinism. > The CI coupling debate in this thread is treating all three proposed modules > as equivalent. I don't think they are, and I think separating them changes > the conversation. > > The testing module (Property, and stateful harness) has a wide and growing > set of potential consumers: cassandra-ecosystem projects, tooling projects, > automation infrastructure, and contributors who want access to the > property-based and stateful testing work without forking it. Multiple people > and projects have asked to use this code. Changes to it tend to be additive. > It is also exactly the kind of code that can carry strong property-based test > coverage of its own, since that is what it does. > > The collections and concurrency modules are different. Today their primary > driver is resolving a single fork situation. A project that cannot depend on > Cassandra directly copied utility code to make progress and wants to collapse > that fork onto a shared artifact. That is a legitimate goal. But the cost > model is not the same as the testing module. > > If we accept Benedict's position that we must run CI for all dependent > projects on every utils change, the CI burden on a collections or concurrency > change is much higher than on a testing utility change. Collections and > concurrency primitives are threaded deeply through multiple project builds. A > rename, a generics change, a behavioral refinement that fixes one consumer's > edge case may require coordinated updates across several projects. That is > real work, and it falls on whoever makes the change. If that person is > primarily motivated by one project's needs, the incentive to do that > coordination work is uneven. > > The testing module does not carry the same risk. If I add a new string > generator or add shrinking, the impact on a dependent project's build is > minimal. The argument for running full downstream CI on that change is weak. > > I would like to understand from the thread whether we are actually debating a > single policy that applies to all three modules, or whether we can treat the > modules differently. If the collections and concurrency modules require a > heavier CI gate because of their blast radius, that gate should be evaluated > against who benefits and who pays. If the testing module can operate under a > lighter gate, we should say so explicitly. > > Starting with the testing module and deferring the collections and > concurrency modules until we have the repo structure, the CI model, and the > API lifecycle nailed down is a reasonable path. The testing module has clear > multi-project demand now. The other modules can follow once we have a working > model. > > >> On Aug 21, 2026, at 5:54 AM, Josh McKenzie <[email protected]> wrote: >> >>> 1.) We have annotations to *communicate intent* and allow for >>> experimentation in the shared space. >>> 2.) We require shared library changes to simply *not break the builds* of >>> ecosystem projects. (It seems like the argument here is over whether we'll >>> have a versioning apparatus that *does* allow this?) >>> 3.) Particular ecosystem projects manage the safety of their own >>> consumption of the shared library. >> So been letting this simmer for a couple days. My thinking: >> 1. full CI pass on everything puts a dependency on us having consistently >> green CI or utils becomes the single worst place to try and get something >> merged in the ecosystem. We haven't had consistent green CI in the 12 years >> I've been around; I don't see how we're going to start now. This would >> probably make utils DOA. >> 2. Making sure everything still builds is pretty straightforward and a >> simple enough check. Subsequent CI on other projects would surface >> functional breakage at that time but to Mick's point and subsequent >> discussion, that'd introduce some nasty inefficiency and potential >> ping-ponging between projects. >> 3. We could split the difference and just do unit tests for each project on >> utils. That'd exercise the code w/out subjecting people working on utils to >> the flakiness of multi-node non-determinism. >> It's a tension between when we find a breakage and how much of a lift it is >> to get things merged. I kind of like option 3 above now that I've written it >> out. >> >> I hate doing merge commits on primary C* and generally just find >> multi-branch maintenance tiresome and uninteresting, so at least for me >> personally I like the idea of versioning per-API and just having 1 utils >> branch. @Benedict - I absolutely get where you're coming from w/not wanting >> to deal with a bunch of stupid process when all we want to do is share code, >> but my intuition is the nuts and bolts of the work would actually be pretty >> simple. A concrete example: >> • There's a method in the utils class someone wants to change. It's >> @STABLE. They want to add a new argument. >> • So they copy the @STABLE method to a new one, mark it @BETA, add their >> arg, and continue about their day modifying and iterating on it across a few >> patches. >> • Eventually it seems pretty solid and a strict superset of the old >> @STABLE, so they hit the dev list with a [DISCUSS] proposing they @DEPRECATE >> the old api and flag the current @BETA as @STABLE. Or just hit the list with >> [DISCUSS] saying they're planning on flagging it @STABLE; no reason we can't >> have multiple @STABLE annotations. >> • Lazy consensus, flagged, done. >> There's effectively no operational block to adding or iterating on things in >> utils with the same 0-friction approach as without versioning at all with >> the one exception of needing to type @BETA on a class/method. There *is* a >> moment of friction on the discussion around making the @BETA the new @STABLE >> if deprecating an old API, but even then if we're not removing them my guess >> is it's all going to be pretty smooth. >> >> On Wed, Aug 19, 2026, at 7:43 PM, Caleb Rackliffe wrote: >>> The obvious problem around #3, as others have stated, is when we have an >>> unhappy logical conflict in behavior that breaks one client while it fixes >>> another. That will probably require collaboration across projects no matter >>> what we do or who validates the change, but finding it a little later is >>> something I'd trade away to encourage sharing in the first place. >>> >>> On Wed, Aug 19, 2026 at 6:39 PM Caleb Rackliffe <[email protected]> >>> wrote: >>>> > won't that just discourage people from moving code into utils or >>>> > modifying existing code since then you're subject to the flakiness of >>>> > all the projects CI combined instead of just one project's? >>>> >>>> My honest assessment of my own behavior and motivations leads me to >>>> say...yes. >>>> >>>> It seems reasonable to arrive in a situation where: >>>> >>>> 1.) We have annotations to *communicate intent* and allow for >>>> experimentation in the shared space. >>>> 2.) We require shared library changes to simply *not break the builds* of >>>> ecosystem projects. (It seems like the argument here is over whether we'll >>>> have a versioning apparatus that *does* allow this?) >>>> 3.) Particular ecosystem projects manage the safety of their own >>>> consumption of the shared library. >>>> >>>> On Wed, Aug 19, 2026 at 4:03 PM Josh McKenzie <[email protected]> wrote: >>>>> __ >>>>> (Trying to find the path to progress here, not advocate for any certain >>>>> outcome btw) >>>>> >>>>>> This testing matrix has to be done, and the earlier it is done (and done >>>>>> by the author) the better. >>>>> I like the author owning it. I'm also concerned the author might not have >>>>> context and expertise in a domain that breaks and needs modification >>>>> (i.e. core C* dev changes utils, that change breaks something in >>>>> analytics, ???). It does keep the responsibility for making sure >>>>> breakages are resolved closer to the changer which is also positive - the >>>>> earlier the better indeed. >>>>> >>>>> How do we run CI for everything? That's a huge lift. If we *do* manage to >>>>> get that setup, won't that just discourage people from moving code into >>>>> utils or modifying existing code since then you're subject to the >>>>> flakiness of all the projects CI combined instead of just one project's? >>>>> >>>>> On Wed, Aug 19, 2026, at 3:23 PM, Mick wrote: >>>>>> >>>>>> >>>>>> > Changes to libraries that fix one consumer can >>>>>> > break another: we will need to be able to build and run tests for all >>>>>> > dependent projects when we make changes >>>>>> >>>>>> >>>>>> I agree with Benedict here. >>>>>> This testing matrix has to be done, and the earlier it is done (and done >>>>>> by the author) the better. >>>>>> If we're honest about that, then the API labelling and versioning isn't >>>>>> needed. >>>>>> >>>>>> Pushing the complication onto the consumer in following such api >>>>>> labelling and versions, and then finding out late things don't work for >>>>>> them and another change in the utils repo is needed, doesn't sound fun. >>>>>> Let's just keep it simple and encourage everyone to say close, via >>>>>> authors taking full responsibility for a change they make at such >>>>>> low-level and shared code. >>>>>> >>>>>> >>>>> >>
