[
https://issues.apache.org/jira/browse/HADOOP-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated HADOOP-10606:
------------------------------------------
Assignee: (was: Brahma Reddy Battula)
> NodeManager cannot launch container when using RawLocalFileSystem for
> fs.file.impl
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-10606
> URL: https://issues.apache.org/jira/browse/HADOOP-10606
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, io, util
> Affects Versions: 2.4.0
> Environment: The environment does not matter with this issue. But I
> use Windows 8 64bits, Open JDK 7.0.
> Reporter: BoYang
> Priority: Critical
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Node manager failed to launch container when I set fs.file.impl to
> org.apache.hadoop.fs.RawLocalFileSystem in core-site.xml.
> The log is:
> WARN ContainersLauncher #11
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch
> - Failed to launch container.
> java.lang.ClassCastException: org.apache.hadoop.fs.RawLocalFileSystem cannot
> be cast to org.apache.hadoop.fs.LocalFileSystem
> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:339)
> at
> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:270)
> at
> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:344)
> at
> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
> at
> org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLogPathForWrite(LocalDirsHandlerService.java:307)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:185)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> The issue is in
> hadoop-common-project\hadoop-common\src\main\java\org\apache\hadoop\fs\LocalDirAllocator.java.
> It invokes FileSystem.getLocal(), which tries to convert the FileSystem to
> LocalFileSystem and will fail when FileSystem object is RawLocalFileSystem
> (RawLocalFileSystem is not a sub class of LocalFileSystem).
> public static LocalFileSystem getLocal(Configuration conf)
> throws IOException {
> return (LocalFileSystem)get(LocalFileSystem.NAME, conf);
> }
> The fix for LocalDirAllocator.java seems to be invoking LocalFileSystem.get()
> instead of LocalFileSystem.getLocal()?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
