Add a 'devBranch' jenkins job to that imo: The possibility to run the
full suite + multiplex new tests before commit when you're about to
release a Kraken into the codebase: Accord, TCM, TTL, SAI, Vector,
JDK... So:
1: Pre-commit subset of tests (suites + matrices + env) runs. On green,
merge.
2: Pre-commit 'devBranch' full suite for high risk/disruptive merges: at
reviewer's discretion
3: Post-commit tests (all suites, matrices, env) runs. If failure, link
back to the JIRA where the commit took place
My 2cts
On 10/7/23 17:36, Josh McKenzie wrote:
I'm personally not thinking about CircleCI at all; I'm envisioning a
world where all of us have 1 CI /software/ system (i.e. reproducible
on any env) that we use for pre-commit validation, and then
post-commit happens on reference ASF hardware.
So:
1: Pre-commit subset of tests (suites + matrices + env) runs. On
green, merge.
2: Post-commit tests (all suites, matrices, env) runs. If failure,
link back to the JIRA where the commit took place
Circle would need to remain in lockstep with the requirements for
point 1 here.
On Mon, Jul 10, 2023, at 1:04 AM, Berenguer Blasi wrote:
+1 to Josh which is exactly my line of thought as well. But that is
only valid if we have a solid Jenkins that will eventually run all
test configs. So I think I lost track a bit here. Are you proposing:
1- CircleCI: Run pre-commit a single (the most common/meaningful,
TBD) config of tests
2- Jenkins: Runs post-commit _all_ test configs and emails/notifies
you in case of problems?
Or sthg different like having 1 also in Jenkins?
On 7/7/23 17:55, Andrés de la Peña wrote:
I think 500 runs combining all configs could be reasonable, since
it's unlikely to have config-specific flaky tests. As in five
configs with 100 repetitions each.
On Fri, 7 Jul 2023 at 16:14, Josh McKenzie <jmcken...@apache.org
<mailto:jmcken...@apache.org>> wrote:
Maybe. Kind of depends on how long we write our tests to run
doesn't it? :)
But point taken. Any non-trivial test would start to be
something of a beast under this approach.
On Fri, Jul 7, 2023, at 11:12 AM, Brandon Williams wrote:
On Fri, Jul 7, 2023 at 10:09 AM Josh McKenzie
<jmcken...@apache.org <mailto:jmcken...@apache.org>> wrote:
> 3. Multiplexed tests (changed, added) run against all JDK's
and a broader range of configs (no-vnode, vnode default,
compression, etc)
I think this is going to be too heavy...we're taking 500 iterations
and multiplying that by like 4 or 5?
