[
https://issues.apache.org/jira/browse/MBUILDCACHE-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877461#comment-17877461
]
ASF GitHub Bot commented on MBUILDCACHE-104:
--------------------------------------------
reda-alaoui commented on PR #175:
URL:
https://github.com/apache/maven-build-cache-extension/pull/175#issuecomment-2315794689
Let me try to be more concrete by exposing a simplified example.
I have a maven project with a single module. In this module, I have many
prod classes and many slow tests in this module.
My naive CI (Jenkins) pipeline is a single step: `mvn clean verify`. Running
this takes 5 hours (30 minutes of compilation + 4h30m of test execution).
My CI can execute jobs among multiple machines. So I decide to distribute
the pipelines of my tests among my CI machines.
I split the tests in 3 groups. To assign each test a group, I annotate each
test class with a JUnit 5 `@Tag` annotation with a value of `group1`, `group2`
or `group3`.
My new pipeline is:
1. `mvn clean verify -DskipTests` (~30 minutes)
2. In parallel:
1. `mvn clean verify -Dgroups=group1` (4h30m/3)
2. `mvn clean verify -Dgroups=group2` (4h30m/3)
3. `mvn clean verify -Dgroups=group3` (4h30m/3)
I want:
* steps 2.x to reuse the cache produced by step 1 (I needed
https://github.com/apache/maven-build-cache-extension/pull/177#pullrequestreview-2262014561
to make this one totally work)
* that running again any step without any code change should skip all maven
plugin executions
> Does statement 2 imply that the rerun is a different reactor compile ->
test group 1 and the expectation is that the results of the initial run are
reused?
Yes
> Does parallel[test group 1, test group 2, test group 3] refer to three
different reactor modules running tests?
Yes
> Are compile and any of test group 1, test group 2, test group 3 parts of
the same module?
Not necessarly. You can have a test group spanning across multiple modules.
> You can move the parameter to the effective pom. One way to do this is by
using a profile that defines a literal property value and referencing that
value in a plugin. By doing so, the value will be interpolated to the plugin
parameters and become part of the effective pom.
Yes I know, and as I said I don't want a different checksum between all
those steps.
> Allow multiple cache entries per checksum
> -----------------------------------------
>
> Key: MBUILDCACHE-104
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-104
> Project: Maven Build Cache Extension
> Issue Type: New Feature
> Reporter: Réda Housni Alaoui
> Priority: Major
> Labels: pull-request-available
>
> I have the following use case: a CI pipeline with a compile step *then*
> multiple parallelized test steps. Tests are split in 3 groups. Each test
> group execution specify the group via maven-surefire-plugin:test property
> 'groups'.
> Maven phase for compilation: process-test-classes
> Maven phase for tests: verify
> The compilation should use remote cache as much as possible and end up
> populating it.
> Each test execution should either use the populated cache ending with
> process-test-classes phase or a cache representing the result of the
> execution of the current test group (value for 'groups').
> The issue I have is that this extension can only store a single cache entry
> per input checksum. But I need to persist the build result of each test group.
> The easiest solution I can think of is having 4 cache zones. The first for
> the compile phase (the default zone), the 3 remaining for each test group.
> I need the compile phase to read/write from/to the default cache zone.
> I need each test group to read from its own cache zone then the default cache
> zone, *in this order*. At the end, it should write to its own cache zone.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)