[ 
https://issues.apache.org/jira/browse/MBUILDCACHE-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877461#comment-17877461
 ] 

ASF GitHub Bot commented on MBUILDCACHE-104:
--------------------------------------------

reda-alaoui commented on PR #175:
URL: 
https://github.com/apache/maven-build-cache-extension/pull/175#issuecomment-2315794689

   Let me try to be more concrete by exposing a simplified example.
   
   I have a maven project with a single module. In this module, I have many 
prod classes and many slow tests in this module. 
   
   My naive CI (Jenkins) pipeline is a single step: `mvn clean verify`. Running 
this takes 5 hours (30 minutes of compilation + 4h30m of test execution).
   
   My CI can execute jobs among multiple machines. So I decide to distribute 
the pipelines of my tests among my CI machines.
   I split the tests in 3 groups. To assign each test a group, I annotate each 
test class with a JUnit 5 `@Tag` annotation with a value of `group1`, `group2` 
or `group3`.
   My new pipeline is:
   1. `mvn clean verify -DskipTests` (~30 minutes)
   2. In parallel:
       1. `mvn clean verify -Dgroups=group1` (4h30m/3)
       2. `mvn clean verify -Dgroups=group2` (4h30m/3)
       3. `mvn clean verify -Dgroups=group3` (4h30m/3)
   
   I want:
   * steps 2.x to reuse the cache produced by step 1 (I needed 
https://github.com/apache/maven-build-cache-extension/pull/177#pullrequestreview-2262014561
 to make this one totally work)
   * that running again any step without any code change should skip all maven 
plugin executions
   
   > Does statement 2 imply that the rerun is a different reactor compile -> 
test group 1 and the expectation is that the results of the initial run are 
reused?
   
   Yes
   
   > Does parallel[test group 1, test group 2, test group 3] refer to three 
different reactor modules running tests?
   
   Yes
   
   > Are compile and any of test group 1, test group 2, test group 3 parts of 
the same module?
   
   Not necessarly. You can have a test group spanning across multiple modules.
   
   > You can move the parameter to the effective pom. One way to do this is by 
using a profile that defines a literal property value and referencing that 
value in a plugin. By doing so, the value will be interpolated to the plugin 
parameters and become part of the effective pom.
   
   Yes I know, and as I said I don't want a different checksum between all 
those steps.




> Allow multiple cache entries per checksum
> -----------------------------------------
>
>                 Key: MBUILDCACHE-104
>                 URL: https://issues.apache.org/jira/browse/MBUILDCACHE-104
>             Project: Maven Build Cache Extension
>          Issue Type: New Feature
>            Reporter: Réda Housni Alaoui
>            Priority: Major
>              Labels: pull-request-available
>
> I have the following use case: a CI pipeline with a compile step *then* 
> multiple parallelized test steps. Tests are split in 3 groups. Each test 
> group execution specify the group via maven-surefire-plugin:test property 
> 'groups'.
> Maven phase for compilation: process-test-classes
> Maven phase for tests: verify
> The compilation should use remote cache as much as possible and end up 
> populating it.
> Each test execution should either use the populated cache ending with 
> process-test-classes phase or a cache representing the result of the 
> execution of the current test group (value for 'groups').
> The issue I have is that this extension can only store a single cache entry 
> per input checksum. But I need to persist the build result of each test group.
> The easiest solution I can think of is having 4 cache zones. The first for 
> the compile phase (the default zone), the 3 remaining for each test group. 
> I need the compile phase to read/write from/to the default cache zone. 
> I need each test group to read from its own cache zone then the default cache 
> zone, *in this order*. At the end, it should write to its own cache zone.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to