Steve Loughran created HADOOP-19490:
---------------------------------------
Summary: S3A: AWS SDK 2.30+ incompatible with third party stores
Key: HADOOP-19490
URL: https://issues.apache.org/jira/browse/HADOOP-19490
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Steve Loughran
Changes in the SDK related to a new AWS SDK Feature, Default integritY
Protection
{{https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html}}
break all intereraction with third party S3 Stores.
See {{https://github.com/aws/aws-sdk-java-v2/issues/5801}}
There are documented mechanisms to turn this off
* an environment variable
* a change in ~/..aws/config
* a system property
* looks like a new builder option, though the docs don't cover it.
That checksum builder option looks like the only viable strategy, but it will
need testing. maybe even make it a property which can be enabled/disabled, with
tests and docs.
Whoever wants a 2.30.x feature either gets to do the fixing and testing. For
3.4.2 it'll be an older release.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
