[ https://issues.apache.org/jira/browse/MCLEAN-117 ]
Peter De Maeyer deleted comment on MCLEAN-117:
----------------------------------------
was (Author: peterdm):
I think there is another limitation. Looking at the code {{Cleaner.delete}},
{{fast}} mode together with {{includes/excludes}} has a performance impact. As
soon as a file is excluded from a directory, it will cause the
{{parentDir.delete}} to fail because the dir is obviously not empty, which will
result in a {{System.gc()}} on Windows + 3 retries with back-off delays of 50 +
250 + 750 milliseconds, only to fail 3 times over again for the same reason.
Depending on {{failOnError}} this will actually fail the build or not, but in
any case it will cause an unnecessary delay. This could be fixed, by rewriting
the {{Cleaner}} to leverage {{Files.walkFileTree(Path, FileVisitor)}} + being
smarter about when to (not) delete a directory. MCLEAN-126 won't fix it, but is
a prerequisite first step towards improving this.
> Threading issue with fast=true on multi-module project.
> -------------------------------------------------------
>
> Key: MCLEAN-117
> URL: https://issues.apache.org/jira/browse/MCLEAN-117
> Project: Maven Clean Plugin
> Issue Type: Bug
> Affects Versions: 3.3.2
> Reporter: Dan Mclaughlin
> Priority: Major
>
> We recently moved to new Macbooks with the M3 Max and have started seeing
> random failures where builds will fail trying to access files in target. Our
> project is a multi-module project with multiple sub-modules, with 50+ modules
> in total. For example...
> pom.xml (parent)
> module1 (parent is root)
> pom.xml
> sub-module1 (parent is module1
> pom.xml
> sub-sub-module (parent is sub-module1)
> sub-module2 (parent is module1)
> pom.xml
> module2 (parent is root)
> pom.xml
> sub-module1 (parent is module1
> pom.xml
> sub-sub-module (parent is sub-module2)
> sub-module2 (parent is module2)
> pom.xml
> module3, 4, 5, etc...
> We build using -TC1, so on the M3 Max there are 16 threads used during the
> build.
> We didn't see this problem often running on our older laptops; we rarely saw
> it on our Windows laptops because they only had 8 cores, but we would
> occasionally see it on our Macbook Pro 2019 models with i9 8 core (16 with
> hyperthreading). Now that we are using the new M3 Max we can't get a build
> to finish without failures.
> We see that things fail randomly when trying to access files in the target.
> For example...
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:3.6.1:analyze-only (analyze)
> on project authservice-webapp: Cannot analyze dependencies:
> /Users/username/workspace/trunk/common-resources/target/common-resources-28.0.0-SNAPSHOT.jar
> -> [Help 1]
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:3.6.1:copy-dependencies
> (copy-dependencies) on project spatialdata-loader: Error copying artifact
> from
> /Users/username/workspace/trunk/spatialdata/ant-spring-runner/target/ant-spring-runner-28.0.0-SNAPSHOT.jar
> to
> /Users/username/workspace//trunk/spatialdata/spatialdata-loader/target/lib/ant-spring-runner-28.0.0-SNAPSHOT.jar:
> File
> /Users/username/workspace/trunk/spatialdata/ant-spring-runner/target/ant-spring-runner-28.0.0-SNAPSHOT.jar
> does not exist ->
> [Help 1]
> To debug it down to possibly a threading issue related to using fast with
> maven-clean-plugin a tested the following...
> If I build using "mvn clean install" the build always fails and in random
> places.
> If I build with -T1 then the build never fails. It also doesn't fail with up
> to -T7, but failures start at -T8 or -TC1.
> If I run mvn clean first, then mvn install, then the build never fails.
> If I build using "mvn clean install", but I disable fast, then the build
> never fails.
> This is our plugin configuration. All I have to do is set fast to false, and
> we won't see any of the random failures as I described above. So it seems
> like there is some threading issue with fast where files are being deleted.
> {code:java}
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-clean-plugin</artifactId>
> <version>${maven-clean-plugin.version}</version>
> <configuration>
> <filesets>
> <fileset>
> <directory>${project.basedir}</directory>
> <followSymlinks>true</followSymlinks>
> <useDefaultExcludes>true</useDefaultExcludes>
> <includes>
> <include>*.tgz</include>
> <include>src/**/generated-sources/**</include>
> <include>dist/**</include>
> <include>app/scripts/beans/**</include>
> <include>src/scripts/beans/**</include>
> <!-- <include>.tmp/</include>-->
> </includes>
> </fileset>
> </filesets>
> <verbose>false</verbose>
> <failOnError>true</failOnError>
> <retryOnError>true</retryOnError>
> <fast>true</fast>
> <fastDir>${project.build.directory}/.clean</fastDir>
> <fastMode>background</fastMode>
> </configuration>
> </plugin>{code}
> One place it will randomly fail is a sub-module like this...
> root
> pom.xml
> module A
> pom.xml
> sub-module 1
> pom.xml
> sub-module 2
> pom.xml (sub-module 2 has a runtime dependency on module
> A/sub-module 1)
> If I build module A by itself, then it never fails; it will build module A
> and all of its sub-modules without failures. But if I build the project's
> root, sub-module2 will almost always fails (90% of the time) when it tries to
> copy its runtime dependency from sub-module1 because the jar is missing from
> the target in sub-module1.
> This is the related failure...
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:3.6.1:copy-dependencies
> (copy-dependencies) on project sub-module2: Error copying artifact from
> /Users/username/workspace/trunk/moduleA/sub-module1/target/sub-module1-28.0.0-SNAPSHOT.jar
> to
> /Users/username/workspace/trunk/moduleA/sub-module2/target/lib/sub-module1-28.0.0-SNAPSHOT.jar:
> File
> /Users/username/workspace/trunk/moduleA/sub-module1/target/sub-module1-28.0.0-SNAPSHOT.jar
> does not exist
> It could very likely be a bug in the reactor, but given that simply changing
> fast to false makes the problem go away, it seems to point the finger at the
> maven-clean-plugin.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
