[
https://issues.apache.org/jira/browse/MRESOLVER-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17693545#comment-17693545
]
Michael Osipov commented on MRESOLVER-325:
------------------------------------------
OK, I cracked it. This is purely a Windows problem, any system which adheres to
POSIX is supposed to implement {{rename(2)}} atomic:
https://pubs.opengroup.org/onlinepubs/000095399/functions/rename.html.
Now, the code for Windows does this:
https://github.com/openjdk/jdk17u-dev/blob/aa11a935373686bf5d67afb89368931c995824f6/src/java.base/windows/classes/sun/nio/fs/WindowsFileCopy.java#L307-L320
{{MoveFileEx}} does not mention anything about atomic. Interesting read:
https://github.com/python/cpython/issues/53074. My testing even with single
thread leads to the assumption that {{MoveFileEx}} is executed delayed and
fails under high/fast load which means that the target file is blocked while
the next operation is already trying to overwrite it again and it fails. So
{{MoveFileEx}} returns although the move isn't complete.
To verify this I have applied the following patch:
{noformat}
diff --git
a/maven-resolver-util/src/main/java/org/eclipse/aether/util/FileUtils.java
b/maven-resolver-util/src/main/java/org/eclipse/aether/util/FileUtils.java
index f12d5b82..8b4a1960 100644
--- a/maven-resolver-util/src/main/java/org/eclipse/aether/util/FileUtils.java
+++ b/maven-resolver-util/src/main/java/org/eclipse/aether/util/FileUtils.java
@@ -26,6 +26,10 @@ import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.concurrent.ThreadLocalRandom;
+import com.sun.jna.platform.win32.Kernel32;
+import com.sun.jna.platform.win32.WinBase;
+import com.sun.jna.platform.win32.WinDef.DWORD;
+
import static java.util.Objects.requireNonNull;
/**
@@ -118,7 +122,8 @@ public final class FileUtils
@Override
public void move() throws IOException
{
- Files.move( tempFile, file, StandardCopyOption.ATOMIC_MOVE );
+ Kernel32.INSTANCE.MoveFileEx(
tempFile.toAbsolutePath().toString(), file.toAbsolutePath().toString(),
+ new DWORD( WinBase.MOVEFILE_REPLACE_EXISTING |
WinBase.MOVEFILE_WRITE_THROUGH ) );
}
@Override
{noformat}
{{MOVEFILE_WRITE_THROUGH}} guarantees that the method does *not* exit before
the operation is complete. Which is actually what we need. With that patch
single thread and multi thread executions ran multiple times without issues.
So, [~laeubi], nothing is wrong with Tycho. the Java implementation is just
CRAP.
What now? Well, I replaced ATOMIC with REPLACE_EXISTING, now if you look at
that code in JDK it will delete the target which might be subject to move from
a previous run and then use again {{MoveFileEx}} without any flags which is
again non-blocking and we are back to the previous problem causing a chicken
and egg problem. Again, this is under high volume like here. This fails with
exceptions as well.
Why actually gson? Well, gson JAR AND gson sources JAR are downloaded with a
diff of a few hundred milliseconds and Windows cannot handle this. WHAT A JOKE.
While the additional flag won't make fully atomic, there is another function
for this, it reduces the possible failure to a minimum.
Thoughts?
PS: This needs to be reported with the OpenJDK team. Many others have suffered
from this because of the missing flags.
> [REGRESSION] Suddenly seeing I/O errors under windows aborting the build
> ------------------------------------------------------------------------
>
> Key: MRESOLVER-325
> URL: https://issues.apache.org/jira/browse/MRESOLVER-325
> Project: Maven Resolver
> Issue Type: Bug
> Components: Resolver
> Affects Versions: 1.9.4
> Reporter: Christoph Läubrich
> Priority: Major
> Fix For: 1.9.6
>
>
> If one runs a build that otherwise works fine on 3.8.x with 3.9 we now get
> the following exception (full output can be found here
> https://github.com/eclipse-platform/eclipse.platform/actions/runs/4211467991/jobs/7309831666):
> {code:java}
> Error: 5.889 [ERROR] Internal error: java.io.UncheckedIOException:
> java.nio.file.AccessDeniedException:
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories.15650462061630955031.tmp
> ->
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories
> -> [Help 1]
> org.apache.maven.InternalErrorException: Internal error:
> java.io.UncheckedIOException: java.nio.file.AccessDeniedException:
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories.15650462061630955031.tmp
> ->
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:108)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:77)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:568)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> Caused by: java.io.UncheckedIOException: java.nio.file.AccessDeniedException:
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories.15650462061630955031.tmp
> ->
> C:\Users\runneradmin\.m2\repository\com\google\code\gson\gson\2.10.1\_remote.repositories
> at org.eclipse.aether.internal.impl.DefaultTrackingFileManager.update
> (DefaultTrackingFileManager.java:121)
> at
> org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.addRepo
> (EnhancedLocalRepositoryManager.java:274)
> at
> org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.addArtifact
> (EnhancedLocalRepositoryManager.java:252)
> at org.eclipse.aether.internal.impl.EnhancedLocalRepositoryManager.add
> (EnhancedLocalRepositoryManager.java:225)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.evaluateDownloads
> (DefaultArtifactResolver.java:680)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads
> (DefaultArtifactResolver.java:592)
> at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:478)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts
> (DefaultArtifactResolver.java:278)
> at
> org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact
> (DefaultArtifactResolver.java:255)
> at
> org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact
> (DefaultRepositorySystem.java:296)
> at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:197)
> at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve
> (DefaultArtifactResolver.java:413)
> at org.apache.maven.repository.legacy.LegacyRepositorySystem.resolve
> (LegacyRepositorySystem.java:332)
> at
> org.eclipse.tycho.osgi.configuration.MavenDependenciesResolverConfigurer.resolve
> (MavenDependenciesResolverConfigurer.java:104)
> at org.eclipse.tycho.core.shared.MavenDependenciesResolver.resolve
> (MavenDependenciesResolver.java:60)
> at org.eclipse.tycho.core.resolver.MavenTargetDefinitionContent.<init>
> (MavenTargetDefinitionContent.java:262)
> at
> org.eclipse.tycho.p2resolver.TargetDefinitionResolver.resolveContentWithExceptions
> (TargetDefinitionResolver.java:179)
> at org.eclipse.tycho.p2resolver.TargetDefinitionResolver.resolveContent
> (TargetDefinitionResolver.java:110)
> at
> org.eclipse.tycho.p2resolver.TargetDefinitionResolverService.resolveFromArguments
> (TargetDefinitionResolverService.java:90)
> at java.util.concurrent.ConcurrentHashMap.computeIfAbsent
> (ConcurrentHashMap.java:1708)
> at
> org.eclipse.tycho.p2resolver.TargetDefinitionResolverService.getTargetDefinitionContent
> (TargetDefinitionResolverService.java:65)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.resolveTargetDefinitions
> (TargetPlatformFactoryImpl.java:214)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.createTargetPlatform
> (TargetPlatformFactoryImpl.java:164)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.createTargetPlatform
> (TargetPlatformFactoryImpl.java:137)
> at
> org.eclipse.tycho.p2resolver.TargetPlatformFactoryImpl.createTargetPlatform
> (TargetPlatformFactoryImpl.java:88)
> at
> org.eclipse.tycho.p2resolver.ReactorRepositoryManagerImpl.computePreliminaryTargetPlatform
> (ReactorRepositoryManagerImpl.java:61)
> at
> org.eclipse.tycho.p2resolver.P2DependencyResolver.computePreliminaryTargetPlatform
> (P2DependencyResolver.java:202)
> at org.eclipse.tycho.core.resolver.DefaultTychoResolver.resolveProject
> (DefaultTychoResolver.java:130)
> at
> org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.lambda$resolveProjects$2
> (TychoMavenLifecycleParticipant.java:256)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept
> (ForEachOps.java:183)
> at java.util.stream.SpinedBuffer$1Splitr.forEachRemaining
> (SpinedBuffer.java:364)
> at java.util.stream.AbstractPipeline.copyInto (AbstractPipeline.java:509)
> at java.util.stream.ForEachOps$ForEachTask.compute (ForEachOps.java:290)
> at java.util.concurrent.CountedCompleter.exec (CountedCompleter.java:754)
> at java.util.concurrent.ForkJoinTask.doExec (ForkJoinTask.java:373)
> at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec
> (ForkJoinPool.java:1182)
> at java.util.concurrent.ForkJoinPool.scan (ForkJoinPool.java:1655)
> at java.util.concurrent.ForkJoinPool.runWorker (ForkJoinPool.java:1622)
> at java.util.concurrent.ForkJoinWorkerThread.run
> (ForkJoinWorkerThread.java:165)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
