[ 
https://issues.apache.org/jira/browse/MNG-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Falko Modler updated MNG-7335:
------------------------------
    Description: 
*The following was cloned from MNG-6843 which was fixed in 3.8.2 and 3.8.3, but 
came back in 3.8.4 due to MNG-7312:*

Build of our multi module (57) Java maven project is failing phase when running 
it as parallel in 4 threads (mvn -T 4 clean install). The failure happens 
during compilation because packages/classes from compile dependencies cannot be 
found:
{noformat}
[main] [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) 
on project common: Compilation failure: Compilation failure: 
[main] [ERROR] 
/home/common/src/main/java/com/foo/ZonedDateTimeParser.java:[6,32] package 
org.apache.commons.lang3 does not exist{noformat}
After enabling debug logging (with thread names) I have found out that a 
compile path of the failing module is empty (besides target/classes):

When running in 4 threads (-T 4):
{noformat}
[BuilderThread 2] [DEBUG] (f) compilePath = [/home/common/target/classes]
...
[BuilderThread 2] [DEBUG] Command line options:
[BuilderThread 2] [DEBUG] -d /home/common/target/classes -classpath 
/home/common/target/classes: -sourcepath 
/home/common/src/main/java:/home/common/target/generated-sources/annotations: 
-s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8 
-source 1.8 -encoding UTF-8{noformat}
When running in a single thread (-T 1):
{noformat}
[BuilderThread 0] [DEBUG] (f) compilePath = [/home/common/target/classes, 
/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar,
 
/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar,
 
/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar,
 
/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar,
 /root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar, 
/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar, 
/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar,
 
/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar,
 
/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar]
...
[BuilderThread 0] [DEBUG] Command line options:
[BuilderThread 0] [DEBUG] -d /home/common/target/classes -classpath 
/home/common/target/classes:/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar:/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar:/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar:
 -sourcepath 
/home/common/src/main/java:/home/common/target/generated-sources/annotations: 
-s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8 
-source 1.8 -encoding UTF-8{noformat}
After adding custom log messages I have found out that the root cause is that 
org.apache.maven.project.MavenProject.setArtifactFilter() is called with null 
artifactFilter parameter. The call happens for the failing module from a thread 
that is building another module. The call stack is:
{code:java}
"BuilderThread 0@2513" prio=5 tid=0xe nid=NA runnable
 java.lang.Thread.State: RUNNABLE
 at 
org.apache.maven.project.MavenProject.setArtifactFilter(MavenProject.java:1437)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:279)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:202)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
 at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
 at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
 at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748){code}
The MavenProject objects are shared among all build threads via MavenSession - 
member {{org.apache.maven.execution.MavenSession#projects}}.
 As all the MavenSession objects are created by cloning an initial MavenSession 
object, I have added cloning of projects into the MavenSession clone() method. 
With this change the build succeeded:
{code:java}
@Override
public MavenSession clone()
{
    try
    {
        MavenSession thisClone = (MavenSession) super.clone();
        thisClone.setProjects(getProjectsClone());
        return thisClone;
    }
    catch ( CloneNotSupportedException e )
    {
        throw new RuntimeException( "Bug", e );
    }
}

private synchronized List<MavenProject> getProjectsClone()
{
    if (projects == null) {
        return null;
    } else {
        if (projects.isEmpty()) {
            return Collections.emptyList();
        } else {
            List<MavenProject> clonedProjects = new 
ArrayList<>(projects.size());
            for (MavenProject project : projects) {
                clonedProjects.add(project.clone());
            }            return clonedProjects;
        }
    }
}
{code}

  was:
*The following was cloned from MNG-6843 which was fixed in 3.8.2 and 3.8.3, but 
cam back in 3.8.4 due to MNG-7312:*

Build of our multi module (57) Java maven project is failing phase when running 
it as parallel in 4 threads (mvn -T 4 clean install). The failure happens 
during compilation because packages/classes from compile dependencies cannot be 
found:
{noformat}
[main] [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) 
on project common: Compilation failure: Compilation failure: 
[main] [ERROR] 
/home/common/src/main/java/com/foo/ZonedDateTimeParser.java:[6,32] package 
org.apache.commons.lang3 does not exist{noformat}
After enabling debug logging (with thread names) I have found out that a 
compile path of the failing module is empty (besides target/classes):

When running in 4 threads (-T 4):
{noformat}
[BuilderThread 2] [DEBUG] (f) compilePath = [/home/common/target/classes]
...
[BuilderThread 2] [DEBUG] Command line options:
[BuilderThread 2] [DEBUG] -d /home/common/target/classes -classpath 
/home/common/target/classes: -sourcepath 
/home/common/src/main/java:/home/common/target/generated-sources/annotations: 
-s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8 
-source 1.8 -encoding UTF-8{noformat}
When running in a single thread (-T 1):
{noformat}
[BuilderThread 0] [DEBUG] (f) compilePath = [/home/common/target/classes, 
/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar,
 
/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar,
 
/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar,
 
/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar,
 /root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar, 
/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar, 
/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar,
 
/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar,
 
/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar]
...
[BuilderThread 0] [DEBUG] Command line options:
[BuilderThread 0] [DEBUG] -d /home/common/target/classes -classpath 
/home/common/target/classes:/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar:/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar:/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar:
 -sourcepath 
/home/common/src/main/java:/home/common/target/generated-sources/annotations: 
-s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8 
-source 1.8 -encoding UTF-8{noformat}
After adding custom log messages I have found out that the root cause is that 
org.apache.maven.project.MavenProject.setArtifactFilter() is called with null 
artifactFilter parameter. The call happens for the failing module from a thread 
that is building another module. The call stack is:
{code:java}
"BuilderThread 0@2513" prio=5 tid=0xe nid=NA runnable
 java.lang.Thread.State: RUNNABLE
 at 
org.apache.maven.project.MavenProject.setArtifactFilter(MavenProject.java:1437)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:279)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:202)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
 at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
 at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
 at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
 at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748){code}
The MavenProject objects are shared among all build threads via MavenSession - 
member {{org.apache.maven.execution.MavenSession#projects}}.
 As all the MavenSession objects are created by cloning an initial MavenSession 
object, I have added cloning of projects into the MavenSession clone() method. 
With this change the build succeeded:
{code:java}
@Override
public MavenSession clone()
{
    try
    {
        MavenSession thisClone = (MavenSession) super.clone();
        thisClone.setProjects(getProjectsClone());
        return thisClone;
    }
    catch ( CloneNotSupportedException e )
    {
        throw new RuntimeException( "Bug", e );
    }
}

private synchronized List<MavenProject> getProjectsClone()
{
    if (projects == null) {
        return null;
    } else {
        if (projects.isEmpty()) {
            return Collections.emptyList();
        } else {
            List<MavenProject> clonedProjects = new 
ArrayList<>(projects.size());
            for (MavenProject project : projects) {
                clonedProjects.add(project.clone());
            }            return clonedProjects;
        }
    }
}
{code}


> [Regression] Parallel build fails due to missing JAR artifacts in compilePath
> -----------------------------------------------------------------------------
>
>                 Key: MNG-7335
>                 URL: https://issues.apache.org/jira/browse/MNG-7335
>             Project: Maven
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 3.6.3
>         Environment: - Linux (tested Docker using maven:3-jdk-8 tag): happens 
> most times.
> - Windows 10: happens sometimes.
>            Reporter: Falko Modler
>            Assignee: Robert Scholte
>            Priority: Major
>             Fix For: 3.8.2, 4.0.0, 4.0.0-alpha-1
>
>
> *The following was cloned from MNG-6843 which was fixed in 3.8.2 and 3.8.3, 
> but came back in 3.8.4 due to MNG-7312:*
> Build of our multi module (57) Java maven project is failing phase when 
> running it as parallel in 4 threads (mvn -T 4 clean install). The failure 
> happens during compilation because packages/classes from compile dependencies 
> cannot be found:
> {noformat}
> [main] [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile 
> (default-compile) on project common: Compilation failure: Compilation 
> failure: 
> [main] [ERROR] 
> /home/common/src/main/java/com/foo/ZonedDateTimeParser.java:[6,32] package 
> org.apache.commons.lang3 does not exist{noformat}
> After enabling debug logging (with thread names) I have found out that a 
> compile path of the failing module is empty (besides target/classes):
> When running in 4 threads (-T 4):
> {noformat}
> [BuilderThread 2] [DEBUG] (f) compilePath = [/home/common/target/classes]
> ...
> [BuilderThread 2] [DEBUG] Command line options:
> [BuilderThread 2] [DEBUG] -d /home/common/target/classes -classpath 
> /home/common/target/classes: -sourcepath 
> /home/common/src/main/java:/home/common/target/generated-sources/annotations: 
> -s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8 
> -source 1.8 -encoding UTF-8{noformat}
> When running in a single thread (-T 1):
> {noformat}
> [BuilderThread 0] [DEBUG] (f) compilePath = [/home/common/target/classes, 
> /root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar,
>  
> /root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar,
>  
> /root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar,
>  
> /root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar,
>  /root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar, 
> /root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar, 
> /root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar,
>  
> /root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar,
>  
> /root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar]
> ...
> [BuilderThread 0] [DEBUG] Command line options:
> [BuilderThread 0] [DEBUG] -d /home/common/target/classes -classpath 
> /home/common/target/classes:/root/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/root/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/root/.m2/repository/org/apache/commons/commons-collections4/4.3/commons-collections4-4.3.jar:/root/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/root/.m2/repository/org/jooq/jool-java-8/0.9.14/jool-java-8-0.9.14.jar:/root/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar:/root/.m2/repository/org/springframework/spring-beans/5.1.8.RELEASE/spring-beans-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-core/5.1.8.RELEASE/spring-core-5.1.8.RELEASE.jar:/root/.m2/repository/org/springframework/spring-context/5.1.8.RELEASE/spring-context-5.1.8.RELEASE.jar:
>  -sourcepath 
> /home/common/src/main/java:/home/common/target/generated-sources/annotations: 
> -s /home/common/target/generated-sources/annotations -g -nowarn -target 1.8 
> -source 1.8 -encoding UTF-8{noformat}
> After adding custom log messages I have found out that the root cause is that 
> org.apache.maven.project.MavenProject.setArtifactFilter() is called with null 
> artifactFilter parameter. The call happens for the failing module from a 
> thread that is building another module. The call stack is:
> {code:java}
> "BuilderThread 0@2513" prio=5 tid=0xe nid=NA runnable
>  java.lang.Thread.State: RUNNABLE
>  at 
> org.apache.maven.project.MavenProject.setArtifactFilter(MavenProject.java:1437)
>  at 
> org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:279)
>  at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:202)
>  at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
>  at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
>  at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
>  at 
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
>  at 
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748){code}
> The MavenProject objects are shared among all build threads via MavenSession 
> - member {{org.apache.maven.execution.MavenSession#projects}}.
>  As all the MavenSession objects are created by cloning an initial 
> MavenSession object, I have added cloning of projects into the MavenSession 
> clone() method. With this change the build succeeded:
> {code:java}
> @Override
> public MavenSession clone()
> {
>     try
>     {
>         MavenSession thisClone = (MavenSession) super.clone();
>         thisClone.setProjects(getProjectsClone());
>         return thisClone;
>     }
>     catch ( CloneNotSupportedException e )
>     {
>         throw new RuntimeException( "Bug", e );
>     }
> }
> private synchronized List<MavenProject> getProjectsClone()
> {
>     if (projects == null) {
>         return null;
>     } else {
>         if (projects.isEmpty()) {
>             return Collections.emptyList();
>         } else {
>             List<MavenProject> clonedProjects = new 
> ArrayList<>(projects.size());
>             for (MavenProject project : projects) {
>                 clonedProjects.add(project.clone());
>             }            return clonedProjects;
>         }
>     }
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Reply via email to