[
https://issues.apache.org/jira/browse/HADOOP-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17621413#comment-17621413
]
Greg Senia commented on HADOOP-17705:
-------------------------------------
Also fixes it on Hadoop 2.x versions if the SDK is updated to 1.11.45 with the
aws-sdk patched... We have a case open with AWS to force them to address this.
https://github.com/aws/aws-sdk-java/pull/2537
> S3A to add option fs.s3a.endpoint.region to set AWS region
> ----------------------------------------------------------
>
> Key: HADOOP-17705
> URL: https://issues.apache.org/jira/browse/HADOOP-17705
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Mehakmeet Singh
> Assignee: Mehakmeet Singh
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.2
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Currently, AWS region is either constructed via the endpoint URL, by making
> an assumption that the 2nd component after delimiter "." is the region in
> endpoint URL, which doesn't work for private links and sets the default to
> us-east-1 thus causing authorization issue w.r.t the private link.
> The option fs.s3a.endpoint.region allows this to be explicitly set
> h2. how to set the s3 region on older hadoop releases
> For anyone who needs to set the signing region on older versions of the s3a
> client *you do not need this festure*. instead just provide a custom endpoint
> to region mapping json file
> # Download the default region mapping file
> [awssdk_config_default.json|https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/resources/com/amazonaws/internal/config/awssdk_config_default.json]
> # Add a new regular expression to map the endpoint/hostname to the target
> region
> # Save the file as {{/etc/hadoop/conf/awssdk_config_override.json}}
> # verify basic hadop fs -ls commands work
> # copy to the rest of the cluster.
> # There should be no need to restart any services
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
