[
https://issues.apache.org/jira/browse/HADOOP-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378749#comment-16378749
]
Steve Loughran commented on HADOOP-15264:
-----------------------------------------
Think we need some explicit policy w.r.t JAR updates in our s3a testing docs
Something like
* JAR updates potentially brittle, esp the 1.XX to 1.YY. Even the minor ones
have surprises (+point to links related to abort(), NPEs)
* regression test everything with/without s3guard, scale including assumed roles
* + against > 1 endpoint, especially a v4
* Maybe even try v2 signing
* Seek tests from anyone with internal endpoints. e.g [~ehiggs]
* Don't panic about new deprecation warnings, be selective about whether to
upgrade, as its not always clear the new versions are better in terms of
maintenance
* do a mvn dep of the hadoop-aws and hadoop-cloudstore modules, review, reject
any new additions
* any downstream tests you can pull off too, including my little [cloudstore
diags|https://github.com/steveloughran/cloudstore]
* *review the logs*. Those extra warnings and stack traces didn't cause
problems in the tests, but showed up when we looked at things
> AWS "shaded" SDK 1.271 is pulling in netty 4.1.17
> -------------------------------------------------
>
> Key: HADOOP-15264
> URL: https://issues.apache.org/jira/browse/HADOOP-15264
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Attachments: HADOOP-15264-001.patch
>
>
> The latest versions of the AWS Shaded SDK are declaring a dependency on netty
> 4.1.17
> {code}
> [INFO] +- org.apache.hadoop:hadoop-aws:jar:3.2.0-SNAPSHOT:compile
> [INFO] | \- com.amazonaws:aws-java-sdk-bundle:jar:1.11.271:compile
> [INFO] | +- io.netty:netty-codec-http:jar:4.1.17.Final:compile
> [INFO] | +- io.netty:netty-codec:jar:4.1.17.Final:compile
> [INFO] | +- io.netty:netty-handler:jar:4.1.17.Final:compile
> [INFO] | +- io.netty:netty-buffer:jar:4.1.17.Final:compile
> [INFO] | +- io.netty:netty-common:jar:4.1.17.Final:compile
> [INFO] | +- io.netty:netty-transport:jar:4.1.17.Final:compile
> [INFO] | \- io.netty:netty-resolver:jar:4.1.17.Final:compile
> {code}
> We either exclude these or roll back HADOOP-15040.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
