[
https://issues.apache.org/jira/browse/MNG-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091327#comment-17091327
]
Filip Kaliński commented on MNG-6843:
-------------------------------------
I have added a reproducible test case at
[https://github.com/fkalinski/parallel-testbed]
There is a script for generating a number of child modules. For 20 child
modules and 8 threads the build always fail for me. After applying the
workaround it always succeed.
Of course there can be problems due to the codebase not really prepared for
concurency, that can have different ways of fixing...
> Parallel build fails due to missing JAR artifacts in compilePath
> ----------------------------------------------------------------
>
> Key: MNG-6843
> URL: https://issues.apache.org/jira/browse/MNG-6843
> Project: Maven
> Issue Type: Bug
> Components: core
> Affects Versions: 3.6.3
> Environment: - Linux (tested Docker using maven:3-jdk-8 tag): happens
> most times.
> - Windows 10: happens sometimes.
> Reporter: Stepan Hrbacek
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Build of our multi module (57) Java maven project is failing phase when
> running it as parallel in 4 threads (mvn -T 4 clean install). The failure
> happens during compilation because packages/classes from compile dependencies
> cannot be found:
> {noformat}
> [main] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile
> (default-compile) on project common: Compilation failure: Compilation
> failure:
> [main] [ERROR]
> /home/common/src/main/java/com/foo/ZonedDateTimeParser.java:[6,32] package
> org.apache.commons.lang3 does not exist{noformat}
> After enabling debug logging (with thread names) I have found out that a
> compile path of the failing module is empty (besides target/classes):
> When running in 4 threads (-T 4):
> {noformat}
> [BuilderThread 2] [DEBUG] (f) compilePath = [/home/common/target/classes]
> ...
> [BuilderThread 2] [DEBUG] Command line options:
> [BuilderThread 2] [DEBUG] -d /home/common/target/classes -classpath
> /home/common/target/classes: -sourcepath
> /home/common/src/main/java:/home/common/target/generated-sources/annotations:
> -s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8
> -source 1.8 -encoding UTF-8{noformat}
> When running in a single thread (-T 1):
> {noformat}
> [BuilderThread 0] [DEBUG] (f) compilePath = [/home/common/target/classes,
> /root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar,
>
> /root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar,
>
> /root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar,
>
> /root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar,
> /root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar,
> /root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar,
> /root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar,
>
> /root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar,
>
> /root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar]
> ...
> [BuilderThread 0] [DEBUG] Command line options:
> [BuilderThread 0] [DEBUG] -d /home/common/target/classes -classpath
> /home/common/target/classes:/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar:/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar:/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar:
> -sourcepath
> /home/common/src/main/java:/home/common/target/generated-sources/annotations:
> -s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8
> -source 1.8 -encoding UTF-8{noformat}
> After adding custom log messages I have found out that the root cause is that
> org.apache.maven.project.MavenProject.setArtifactFilter() is called with null
> artifactFilter parameter. The call happens for the failing module from a
> thread that is building another module. The call stack is:
> {code:java}
> "BuilderThread 0@2513" prio=5 tid=0xe nid=NA runnable
> java.lang.Thread.State: RUNNABLE
> at
> org.apache.maven.project.MavenProject.setArtifactFilter(MavenProject.java:1437)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:279)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:202)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> The MavenProject objects are shared among all build threads via MavenSession
> - member {{org.apache.maven.execution.MavenSession#projects}}.
> As all the MavenSession objects are created by cloning an initial
> MavenSession object, I have added cloning of projects into the MavenSession
> clone() method. With this change the build succeeded:
> {code:java}
> @Override
> public MavenSession clone()
> {
> try
> {
> MavenSession thisClone = (MavenSession) super.clone();
> thisClone.setProjects(getProjectsClone());
> return thisClone;
> }
> catch ( CloneNotSupportedException e )
> {
> throw new RuntimeException( "Bug", e );
> }
> }
> private synchronized List<MavenProject> getProjectsClone()
> {
> if (projects == null) {
> return null;
> } else {
> if (projects.isEmpty()) {
> return Collections.emptyList();
> } else {
> List<MavenProject> clonedProjects = new
> ArrayList<>(projects.size());
> for (MavenProject project : projects) {
> clonedProjects.add(project.clone());
> } return clonedProjects;
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
