+1 to Josh which is exactly my line of thought as well. But that is only
valid if we have a solid Jenkins that will eventually run all test
configs. So I think I lost track a bit here. Are you proposing:
1- CircleCI: Run pre-commit a single (the most common/meaningful, TBD)
config of tests
2- Jenkins: Runs post-commit _all_ test configs and emails/notifies you
in case of problems?
Or sthg different like having 1 also in Jenkins?
On 7/7/23 17:55, Andrés de la Peña wrote:
I think 500 runs combining all configs could be reasonable, since it's
unlikely to have config-specific flaky tests. As in five configs with
100 repetitions each.
On Fri, 7 Jul 2023 at 16:14, Josh McKenzie <jmcken...@apache.org> wrote:
Maybe. Kind of depends on how long we write our tests to run
doesn't it? :)
But point taken. Any non-trivial test would start to be something
of a beast under this approach.
On Fri, Jul 7, 2023, at 11:12 AM, Brandon Williams wrote:
On Fri, Jul 7, 2023 at 10:09 AM Josh McKenzie
<jmcken...@apache.org> wrote:
> 3. Multiplexed tests (changed, added) run against all JDK's and
a broader range of configs (no-vnode, vnode default, compression,
etc)
I think this is going to be too heavy...we're taking 500 iterations
and multiplying that by like 4 or 5?
