[
https://issues.apache.org/jira/browse/MNG-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090859#comment-17090859
]
Filip Kaliński commented on MNG-6843:
-------------------------------------
As discussed on the Github, I have tried another workaround by using
ThreadLocal for MavenProject dependency artifact list related fields and it
works for me, at least. It's
[https://github.com/fkalinski/maven/commit/b38e3c2bfb6183c0f45214012112a1c1ca4048bd].
I have also created a Maven extension
[https://github.com/fkalinski/parallel-workaround] to make it possible to use
the workaround without modified Maven.
The problem looks to be triggered by aggregator mojos. In my case by
maven-help-plugin:effective-pom, which was executed for every project. Every
time it was called, the loop over all projects called
MavenProject.setArtifactFilter in MojoExecutor.ensureDependenciesAreResolved(),
leading to a very high chance of project dependency artifacts being
overwritten. I have just moved the call to effective-pom from the parent
project to the top-level project, to have it called once, and it was sufficient.
Of course Maven needs to properly handle also such cases, for sure there are
more legitimate uses of aggregator mojos. Maybe cloning of the projects only
for the execution of aggregators would be a good approach?
> Parallel build fails due to missing JAR artifacts in compilePath
> ----------------------------------------------------------------
>
> Key: MNG-6843
> URL: https://issues.apache.org/jira/browse/MNG-6843
> Project: Maven
> Issue Type: Bug
> Components: core
> Affects Versions: 3.6.3
> Environment: - Linux (tested Docker using maven:3-jdk-8 tag): happens
> most times.
> - Windows 10: happens sometimes.
> Reporter: Stepan Hrbacek
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Build of our multi module (57) Java maven project is failing phase when
> running it as parallel in 4 threads (mvn -T 4 clean install). The failure
> happens during compilation because packages/classes from compile dependencies
> cannot be found:
> {noformat}
> [main] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile
> (default-compile) on project common: Compilation failure: Compilation
> failure:
> [main] [ERROR]
> /home/common/src/main/java/com/foo/ZonedDateTimeParser.java:[6,32] package
> org.apache.commons.lang3 does not exist{noformat}
> After enabling debug logging (with thread names) I have found out that a
> compile path of the failing module is empty (besides target/classes):
> When running in 4 threads (-T 4):
> {noformat}
> [BuilderThread 2] [DEBUG] (f) compilePath = [/home/common/target/classes]
> ...
> [BuilderThread 2] [DEBUG] Command line options:
> [BuilderThread 2] [DEBUG] -d /home/common/target/classes -classpath
> /home/common/target/classes: -sourcepath
> /home/common/src/main/java:/home/common/target/generated-sources/annotations:
> -s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8
> -source 1.8 -encoding UTF-8{noformat}
> When running in a single thread (-T 1):
> {noformat}
> [BuilderThread 0] [DEBUG] (f) compilePath = [/home/common/target/classes,
> /root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar,
>
> /root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar,
>
> /root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar,
>
> /root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar,
> /root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar,
> /root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar,
> /root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar,
>
> /root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar,
>
> /root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar]
> ...
> [BuilderThread 0] [DEBUG] Command line options:
> [BuilderThread 0] [DEBUG] -d /home/common/target/classes -classpath
> /home/common/target/classes:/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar:/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar:/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar:
> -sourcepath
> /home/common/src/main/java:/home/common/target/generated-sources/annotations:
> -s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8
> -source 1.8 -encoding UTF-8{noformat}
> After adding custom log messages I have found out that the root cause is that
> org.apache.maven.project.MavenProject.setArtifactFilter() is called with null
> artifactFilter parameter. The call happens for the failing module from a
> thread that is building another module. The call stack is:
> {code:java}
> "BuilderThread 0@2513" prio=5 tid=0xe nid=NA runnable
> java.lang.Thread.State: RUNNABLE
> at
> org.apache.maven.project.MavenProject.setArtifactFilter(MavenProject.java:1437)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:279)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:202)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> The MavenProject objects are shared among all build threads via MavenSession
> - member {{org.apache.maven.execution.MavenSession#projects}}.
> As all the MavenSession objects are created by cloning an initial
> MavenSession object, I have added cloning of projects into the MavenSession
> clone() method. With this change the build succeeded:
> {code:java}
> @Override
> public MavenSession clone()
> {
> try
> {
> MavenSession thisClone = (MavenSession) super.clone();
> thisClone.setProjects(getProjectsClone());
> return thisClone;
> }
> catch ( CloneNotSupportedException e )
> {
> throw new RuntimeException( "Bug", e );
> }
> }
> private synchronized List<MavenProject> getProjectsClone()
> {
> if (projects == null) {
> return null;
> } else {
> if (projects.isEmpty()) {
> return Collections.emptyList();
> } else {
> List<MavenProject> clonedProjects = new
> ArrayList<>(projects.size());
> for (MavenProject project : projects) {
> clonedProjects.add(project.clone());
> } return clonedProjects;
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
