[
https://issues.apache.org/jira/browse/HADOOP-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15055248#comment-15055248
]
Inigo Goiri commented on HADOOP-12636:
--------------------------------------
One example is when S3AFileSystem is in the path and we don't have the AWS SDK
in the path. Even when we are trying to load HDFS, we get this fatal error:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.fs.s3a.S3AFileSystem could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:224)
at java.util.ServiceLoader.access$100(ServiceLoader.java:181)
at
java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:377)
at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
at
org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2617)
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2633)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2650)
at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2689)
at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2671)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:342)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2284)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2182)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2229)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2409)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2433)
Caused by: java.lang.NoClassDefFoundError:
com/amazonaws/auth/AWSCredentialsProvider
at java.lang.Class.getDeclaredConstructors0(Native Method)
at
java.lang.Class.privateGetDeclaredConstructors(Class.java:2532)
at java.lang.Class.getConstructor0(Class.java:2842)
at java.lang.Class.newInstance(Class.java:345)
at
java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:373)
... 14 more
Caused by: java.lang.ClassNotFoundException:
com.amazonaws.auth.AWSCredentialsProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 19 more
I propose to capture these exception in loadFilesystems and when the user
actually tries to use the failed Filesystem will get the proper error.
> Catch exception in loadFileSystems()
> -------------------------------------
>
> Key: HADOOP-12636
> URL: https://issues.apache.org/jira/browse/HADOOP-12636
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.6.2
> Reporter: Inigo Goiri
> Assignee: Inigo Goiri
> Fix For: 2.6.3
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> loadFileSystems() loads all the Filesystems in the path. However, some
> Filesystems cannot be initialized. There is no point on failing the startup
> because a Filesystem that won't be used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
