[
https://issues.apache.org/jira/browse/HADOOP-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281647#comment-15281647
]
Steve Loughran commented on HADOOP-13050:
-----------------------------------------
I'll make this patch something where you can set the aws version on the command
line; that way people can build hadoop with their own choice of version without
having to edit the poms.
Dependencies of the 1.10.77 full SDK below
* there's a new jackson dependency, {{jackson-dataformat-cbor}} not used
anywhere in hadoop —so not under hadoop's version management. It' needs to be
explicitly declared in hadoop-project and then depended on in hadoop-aws.
* yes, a new joda-time
* the httpcomponents are as 10.6 as before. Meaning HADOOP-12767 exists in
Hadoop 2.8 today
{code}
[INFO] +- com.amazonaws:aws-java-sdk:jar:1.10.77:compile
[INFO] | +- com.amazonaws:aws-java-sdk-support:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-simpledb:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-storagegateway:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-route53:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile
[INFO] | | +- (com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-importexport:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-sts:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-rds:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-redshift:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-elasticbeanstalk:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-glacier:jar:1.10.77:compile
[INFO] | | +- (com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | | +- (com.amazonaws:aws-java-sdk-sns:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | | +- (com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-iam:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-datapipeline:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-elasticloadbalancing:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-emr:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-elasticache:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-elastictranscoder:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-ec2:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.77:compile
[INFO] | | +- (com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-sns:jar:1.10.77:compile
[INFO] | | +- (com.amazonaws:aws-java-sdk-sqs:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudtrail:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-logs:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-events:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cognitoidentity:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cognitosync:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-directconnect:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudformation:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudfront:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-kinesis:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-opsworks:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-ses:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-autoscaling:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudsearch:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudwatchmetrics:jar:1.10.77:compile
[INFO] | | +- (com.amazonaws:aws-java-sdk-cloudwatch:jar:1.10.77:compile -
omitted for duplicate)
[INFO] | | +- (com.amazonaws:aws-java-sdk-dynamodb:jar:1.10.77:compile -
omitted for duplicate)
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-swf-libraries:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-simpleworkflow:jar:1.10.77:compile
- omitted for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-codedeploy:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-codepipeline:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-config:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-lambda:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-ecs:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-ecr:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cloudhsm:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-ssm:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-workspaces:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-machinelearning:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-directory:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-efs:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-codecommit:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-devicefarm:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-elasticsearch:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-waf:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +-
com.amazonaws:aws-java-sdk-marketplacecommerceanalytics:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-inspector:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-iot:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-api-gateway:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-acm:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-gamelift:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-dms:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +-
com.amazonaws:aws-java-sdk-marketplacemeteringservice:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | +- com.amazonaws:aws-java-sdk-cognitoidp:jar:1.10.77:compile
[INFO] | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile - omitted
for duplicate)
[INFO] | \- com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile
[INFO] | +- (commons-logging:commons-logging:jar:1.1.3:compile - version
managed from 1.1.1; omitted for duplicate)
[INFO] | +- (org.apache.httpcomponents:httpclient:jar:4.2.5:compile -
version managed from 4.3.6; omitted for duplicate)
[INFO] | +- (com.fasterxml.jackson.core:jackson-databind:jar:2.2.3:compile
- version managed from 2.5.3; omitted for duplicate)
[INFO] | +-
com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.5.3:compile
[INFO] | | \- (com.fasterxml.jackson.core:jackson-core:jar:2.2.3:compile -
version managed from 2.5.3; omitted for duplicate)
[INFO] | \- joda-time:joda-time:jar:2.8.1:compile
{code}
> Upgrade to AWS SDK 10.10+ for Java 8u60+
> ----------------------------------------
>
> Key: HADOOP-13050
> URL: https://issues.apache.org/jira/browse/HADOOP-13050
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, fs/s3
> Affects Versions: 2.7.2
> Reporter: Steve Loughran
>
> HADOOP-13044 highlights that AWS SDK 10.6 —shipping in Hadoop 2.7+, doesn't
> work on open jdk >= 8u60, because a change in the JDK broke the version of
> Joda time that AWS uses.
> Fix, update the JDK. Though, that implies updating http components:
> HADOOP-12767.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
