On Wed, 12 Jul 2023 at 15:05, Jacek Lewandowski <lewandowski.ja...@gmail.com> wrote:
> I believe some tools can determine which tests make sense to multiplex, > given that some exact lines of code were changed using code coverage > analysis. After the initial run, we should have data from the coverage > analysis, which would tell us which test classes are tainted - that is, > they cover the modified code fragments. > > Using a similar approach, we could detect the coverage differences when > running, say w/wo compression, and discover the tests which cover those > parts of the code. > > That way, we can be smart and save time by precisely pointing to it makes > sense to test more accurately. > This. We already have the beginnings of it in circleci for what to repeat run. If we have a baseline pre-commit pipeline that's quite minimal (the default jdk and config only), then such a pre-detection script can detect what additional test matrix as well as repeated tests are required. This can also be used to determine what test suites don't need to be run. Adding in a results validation script, it would re-use this to determine if the aggregated unit xml results file from a pre-commit CI run contained all the expected test runs for the patch in question. Doing this: going from the script generating "this is the tests you need to run for this patch" to "run that list as one pipeline" using the in-tree scripts becomes very easy. Not related but in general, I keep going back to this for inspiration: https://yetus.apache.org/documentation/in-progress/precommit/
