[
https://issues.apache.org/jira/browse/HADOOP-17197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17175040#comment-17175040
]
Mingliang Liu commented on HADOOP-17197:
----------------------------------------
I'm not fond of moving back to individual jar files from the current sdk bundle
jar. The benefit of smaller dependency size can not justify all the headaches
we've had (we mean both S3A developers and users).
> Decrease size of s3a dependencies
> ---------------------------------
>
> Key: HADOOP-17197
> URL: https://issues.apache.org/jira/browse/HADOOP-17197
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Sahil Takiar
> Priority: Major
>
> S3A currently has a dependency on the aws-java-sdk-bundle, which includes the
> SDKs for all AWS services. The jar file for the current version is about 120
> MB, but continues to grow (the latest is about 170 MB). Organic growth is
> expected as more and more AWS services are created.
> The aws-java-sdk-bundle jar file is shaded as well, so it includes all
> transitive dependencies.
> It would be nice if S3A could depend on smaller jar files in order to
> decrease the size of jar files pulled in transitively by clients. Decreasing
> the size of dependencies is particularly important for Docker files, where
> image pull times can be affected by image size.
> One solution here would be for S3A to publish its own shaded jar which
> includes the SDKs for all needed AWS Services (e.g. S3, DynamoDB, etc.) along
> with the transitive dependencies for the individual SDKs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
