Igor Dianov created MBUILDCACHE-80:
--------------------------------------
Summary: Incremental builds with a higher goal than the highest
cached goal is rebuilding the full project from scratch
Key: MBUILDCACHE-80
URL: https://issues.apache.org/jira/browse/MBUILDCACHE-80
Project: Maven Build Cache Extension
Issue Type: Bug
Affects Versions: 1.1.0, 1.0.1, 1.2.0
Environment: Apache Maven 3.9.6
(bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: C:\Users\sdkman\candidates\maven\current
Java version: 21.0.1, vendor: Eclipse Adoptium, runtime:
C:\Users\sdkman\candidates\java\current
Default locale: en_US, platform encoding: UTF8
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"
Reporter: Igor Dianov
We are trying to use the Maven build cache extension on a large multi-module
project with ~180 modules by caching builds in the CI workflows to be able to
reuse artifacts in the build cache between main and feature branches. The final
build artifacts (i.e. jar, sources, apidocs) are used to build application
Docker images and publish Maven modules into Nexus repository during deploy
phase for releases.
*Expected Behavior*
When executing a build for a higher goal then the currently highest cached
goal, the build extension should skip cached mojo executions, restore the
cached artifacts (i.e. jar, javadocs, sources) into the project build directory
and run remaining mojo executions for the increment, i.e. javadocs, sources,
Docker images between verify -> deploy incremental build. After successful
completion, the build cache info should be updated to record the new highest
cached goal with incremental mojo executions and artifacts.
*Current Behavior*
When executing a build for a higher goal (i.e. deploy) then the currently
highest cached goal (i.e. verify), the extension skips cached executions and
runs mojos between cached and current goals while missing to restore cached
final artifacts into the project build directory. After that, it runs the full
build again from the begining to rebuild the artifacts and save build cache.
Instead of reducing the build time by reusing already packaged artifacts and
executions, it almost doubles the time to re-run the deploy for release from
scratch. It also causes the Maven source plugin (3.3.0) to fail due to a
duplicate sources artifact error, causing the deploy build to fail.
*Possible Solution*
It should be possible to restore cached artifacts into project build directory
while avoiding to re-run full build again after restoring already cached mojo
executions.
*Steps to Reproduce*
1. Run *mvn clean package -DprojectVersion=1.1.0* from the
*tests/java/projects/build-extensions* directory.
{code:java}
[INFO] Scanning for projects...
[INFO] Loading cache configuration from
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\.mvn\maven-build-cache-config.xml
[INFO] Using XX hash algorithm for cache
[INFO]
[INFO] -----------{-}< org.apache.maven.caching.test.simple:simple
>{-}------------
[INFO] Building simple 0.0.1-SNAPSHOT
[INFO] from pom.xml
[INFO] -------------------------------{-}[ jar
]{-}--------------------------------
[INFO]
[INFO] — clean:3.2.0:clean (default-clean) @ simple —
[INFO] Going to calculate checksum for project
[groupId=org.apache.maven.caching.test.simple, artifactId=simple]
[INFO] Scanning plugins configurations to find input files. Probing is enabled,
values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 12, plugins: 3 millis
[INFO] Project inputs calculated in 30 ms. XX checksum [8e6f2406cb760579]
calculated in 15 ms.
[INFO] Attempting to restore project
org.apache.maven.caching.test.simple:simple from build cache
[INFO] Local build was not found by checksum 8e6f2406cb760579 for
org.apache.maven.caching.test.simple:simple
[INFO]
[INFO] — resources:3.3.1:resources (default-resources) @ simple —
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\src\main\resources
[INFO]
[INFO] — compiler:3.11.0:compile (default-compile) @ simple —
[INFO] Changes detected - recompiling the module! :source
[WARNING] File encoding has not been set, using platform encoding UTF8, i.e.
build is platform dependent!
[INFO] Compiling 1 source file with javac [debug target 1.8] to target\classes
[WARNING] bootstrap class path not set in conjunction with -source 8
[WARNING] source value 8 is obsolete and will be removed in a future release
[INFO] — compiler:3.11.0:testCompile (default-testCompile) @ simple —
[INFO] No sources to compile
[INFO]
[INFO] — surefire:3.2.2:test (default-test) @ simple —
[INFO] No tests to run.
[INFO]
[INFO] — jar:3.3.0:jar (default-jar) @ simple —
[INFO] Building jar:
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target\simple-0.0.1-SNAPSHOT.jar
[INFO] Saved Build to local file:
C:\Users\.m2\build-cache\v1\org.apache.maven.caching.test.simple\simple\8e6f2406cb760579\local\buildinfo.xml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.679 s
[INFO] Finished at: 2024-01-28T19:12:36-08:00
[INFO] ------------------------------------------------------------------------
{code}
2. Run *mvn clean install -DprojectVersion=1.1.0*
{code:java}
[INFO] Scanning for projects...
[INFO] Loading cache configuration from
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\.mvn\maven-build-cache-config.xml
[INFO] Using XX hash algorithm for cache
[INFO]
[INFO] -----------{-}< org.apache.maven.caching.test.simple:simple
>{-}------------
[INFO] Building simple 0.0.1-SNAPSHOT
[INFO] from pom.xml
[INFO] -------------------------------{-}[ jar
]{-}--------------------------------
[INFO]
[INFO] — clean:3.2.0:clean (default-clean) @ simple —
[INFO] Deleting
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target
[INFO] Going to calculate checksum for project
[groupId=org.apache.maven.caching.test.simple, artifactId=simple]
[INFO] Scanning plugins configurations to find input files. Probing is enabled,
values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 6, plugins: 3 millis
[INFO] Project inputs calculated in 24 ms. XX checksum [8e6f2406cb760579]
calculated in 15 ms.
[INFO] Attempting to restore project
org.apache.maven.caching.test.simple:simple from build cache
[INFO] Local build found by checksum 8e6f2406cb760579
[INFO] Found cached build, restoring
org.apache.maven.caching.test.simple:simple from cache by checksum
8e6f2406cb760579
[INFO] Project org.apache.maven.caching.test.simple:simple restored partially.
Highest cached goal: package, requested: install
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] Skipping plugin execution (cached): jar:jar
[INFO]
[INFO] — install:3.1.1:install (default-install) @ simple —
[INFO] Installing
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\pom.xml
to
C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.pom
[INFO] Installing
C:\Users\.m2\build-cache\v1\org.apache.maven.caching.test.simple\simple\8e6f2406cb760579\local\simple.jar
to
C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] — resources:3.3.1:resources (default-resources) @ simple —
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\src\main\resources
[INFO]
[INFO] — compiler:3.11.0:compile (default-compile) @ simple —
[INFO] Changes detected - recompiling the module! :source
[WARNING] File encoding has not been set, using platform encoding UTF8, i.e.
build is platform dependent!
[INFO] Compiling 1 source file with javac [debug target 1.8] to target\classes
[WARNING] bootstrap class path not set in conjunction with -source 8
[WARNING] source value 8 is obsolete and will be removed in a future release
[WARNING] target value 8 is obsolete and will be removed in a future release
[WARNING] To suppress warnings about obsolete options, use -Xlint:-options.
[INFO]
[INFO] — resources:3.3.1:testResources (default-testResources) @ simple —
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources,
i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\src\test\resources
[INFO]
[INFO] — compiler:3.11.0:testCompile (default-testCompile) @ simple —
[INFO] No sources to compile
[INFO]
[INFO] — surefire:3.2.2:test (default-test) @ simple —
[INFO] No tests to run.
[INFO]
[INFO] — jar:3.3.0:jar (default-jar) @ simple —
[INFO] Building jar:
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target\simple-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] — install:3.1.1:install (default-install) @ simple —
[INFO] Installing
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\pom.xml
to
C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.pom
[INFO] Installing
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target\simple-0.0.1-SNAPSHOT.jar
to
C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.jar
[INFO] Saved Build to local file:
C:\Users\.m2\build-cache\v1\org.apache.maven.caching.test.simple\simple\8e6f2406cb760579\local\buildinfo.xml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.956 s
[INFO] Finished at: 2024-01-28T19:13:10-08:00
[INFO] ------------------------------------------------------------------------
{code}
3. Run *mvn clean install -DprojectVersion=1.1.0*
{code:java}
[INFO] Scanning for projects...
[INFO] Loading cache configuration from
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\.mvn\maven-build-cache-config.xml
[INFO] Using XX hash algorithm for cache
[INFO]
[INFO] -----------{-}< org.apache.maven.caching.test.simple:simple
>{-}------------
[INFO] Building simple 0.0.1-SNAPSHOT
[INFO] from pom.xml
[INFO] -------------------------------{-}[ jar
]{-}--------------------------------
[INFO]
[INFO] — clean:3.2.0:clean (default-clean) @ simple —
[INFO] Deleting
C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target
[INFO] Going to calculate checksum for project
[groupId=org.apache.maven.caching.test.simple, artifactId=simple]
[INFO] Scanning plugins configurations to find input files. Probing is enabled,
values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 6, plugins: 3 millis
[INFO] Project inputs calculated in 24 ms. XX checksum [8e6f2406cb760579]
calculated in 13 ms.
[INFO] Attempting to restore project
org.apache.maven.caching.test.simple:simple from build cache
[INFO] Local build found by checksum 8e6f2406cb760579
[INFO] Found cached build, restoring
org.apache.maven.caching.test.simple:simple from cache by checksum
8e6f2406cb760579
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] Skipping plugin execution (cached): jar:jar
[INFO] Skipping plugin execution (cached): install:install
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.667 s
[INFO] Finished at: 2024-01-28T19:18:51-08:00
[INFO] ------------------------------------------------------------------------
{code}
4. Check target directory contexts: *ls target/ -la*
{code:java}
drwxr-xr-x 1 igdianov 197121 0 Jan 28 19:18 .
drwxr-xr-x 1 igdianov 197121 0 Jan 28 19:18 ..
drwxr-xr-x 1 igdianov 197121 0 Jan 28 19:18 maven-incremental
{code}
*Context (Environment)*
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: C:\Users\sdkman\candidates\maven\current
Java version: 21.0.1, vendor: Eclipse Adoptium, runtime:
C:\Users\sdkman\candidates\java\current
Default locale: en_US, platform encoding: UTF8
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
