[
https://issues.apache.org/jira/browse/HADOOP-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883816#comment-17883816
]
Steve Loughran commented on HADOOP-19282:
-----------------------------------------
yes it is huge, but if you have ever tried to debug the problem related to
cross version AWS artefacts or incompatibilities between the libraries the SDK
depends upon and those which Hadoop and downstream Applications depend on -you
will soon why unified SDK with a shaded set of libraries is the only way to
avoid a lot of problems.
You are free to use the sdk-s3 jar and it's dependencies in your own
deployment. We try to use reflection where needed to avoid being completely
dependent on the sdk-bundle. You will, sadly. be left to resolve library
incompatibilities.
One thing which has been discussed is to ship some tarballs of the hadoop
binaries without the SDK. If you want to get involved in that I can show you
where to begin. We would need the the ant build.xml in
https://github.com/apache/hadoop-release-support to take the existing x86 and
aarch64 .tar.gz files, expand them, delete the AWS binary, recompress them to
new tar.gz files with different names and then sign these again.
Contributions welcome.
> hadoop-aws exclude aws-java-sdk-bundle dependencies. Use aws-java-sdk-s3
> instead
> --------------------------------------------------------------------------------
>
> Key: HADOOP-19282
> URL: https://issues.apache.org/jira/browse/HADOOP-19282
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: melin
> Priority: Major
>
> hadoop-aws directly depends on aws-java-sdk-bundle, and the size of
> aws-java-sdk-bundle is too large.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
