Sangjin Lee created HADOOP-11263:
------------------------------------
Summary: NativeS3FileSystem doesn't work with hadoop-client
Key: HADOOP-11263
URL: https://issues.apache.org/jira/browse/HADOOP-11263
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.4.0
Reporter: Sangjin Lee
When you start using the NativeS3FileSystem (which is in hadoop-common) based
on the hadoop-client set of jars, it fails with a ClassNotFoundException:
{noformat}
Caused by: java.lang.NoClassDefFoundError: org/jets3t/service/ServiceException
at
org.apache.hadoop.fs.s3native.NativeS3FileSystem.createDefaultStore(NativeS3FileSystem.java:280)
at
org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:270)
at
com.twitter.twadoop.util.hadoop.NativeS3FileSystemWrapper.initialize(NativeS3FileSystemWrapper.java:34)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2438)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2472)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2454)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:384)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
{noformat}
NativeS3FileSystem depends on a library called jets3t, which is not found in
the hadoop-client build. It turns out that this library was specifically
excluded in the hadoop-client pom.xml:
{noformat}
<exclusion>
<groupId>net.java.dev.jets3t</groupId>
<artifactId>jets3t</artifactId>
</exclusion>
{noformat}
This strikes me as an issue, as a component that's part of hadoop-common cannot
run with a hadoop-client build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
