[
https://issues.apache.org/jira/browse/HADOOP-19492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-19492:
------------------------------------
Description:
Code changes related to HADOOP-19485.
AwsSdkWorkarounds no longer needs to cut back on transfer manager logging
(HADOOP-19272) :
- Remove log downgrade and change assertion to expect nothing to be logged.
- remove false positives from log.
ITestS3AEndpointRegion failure:
- Change in state of AwsExecutionAttribute.ENDPOINT_OVERRIDDEN
attribute requires test tuning to match.
Some tests against third-party stores fail
- Includes fix for the assumeStoreAwsHosted() logic.
- Documents how to turn off multipart uploads with third-party stores.
was:
ITestS3AEndpointRegion.testCentralEndpointAndNullRegionWithCRUD with non aws
bucket/credentials fails.
Fix: don't do that.
{code}
[ERROR]
testCentralEndpointAndNullRegionWithCRUD(org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion)
Time elapsed: 1.88 s <<< ERROR!
java.nio.file.AccessDeniedException:
s3a://dellecs719/test/testCentralEndpointAndNullRegionWithCRUD/srcdir:
getFileStatus on
s3a://dellecs719/test/testCentralEndpointAndNullRegionWithCRUD/srcdir:
software.amazon.awssdk.services.s3.model.S3Exception: The AWS Access Key Id you
provided does not exist in our records. (Service: S3, Status Code: 403, Request
ID: 2CZF3PJDSKRYABWS, Extended Request ID:
h8T/uz2tPH4zwA6W/dTqqSZfm5lAT9JetkO8moPu9R9L5QkqTP0VwfWrAZWo3J/WMy/nollnxT8=):InvalidAccessKeyId
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:271)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:158)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4088)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3954)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$MkdirOperationCallbacksImpl.probePathStatus(S3AFileSystem.java:3811)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.probePathStatusOrNull(MkdirOperation.java:218)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.getPathStatusExpectingDir(MkdirOperation.java:239)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:134)
at
org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:59)
at
org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation.apply(ExecutingStoreOperation.java:76)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2844)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2863)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:3782)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2496)
at
org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion.assertOpsUsingNewFs(ITestS3AEndpointRegion.java:541)
at
org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion.testCentralEndpointAndNullRegionWithCRUD(ITestS3AEndpointRegion.java:495)
{code}
> S3A: some tests failing on third-party stores
> ---------------------------------------------
>
> Key: HADOOP-19492
> URL: https://issues.apache.org/jira/browse/HADOOP-19492
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Fix For: 3.5.0, 3.4.2
>
>
> Code changes related to HADOOP-19485.
> AwsSdkWorkarounds no longer needs to cut back on transfer manager logging
> (HADOOP-19272) :
> - Remove log downgrade and change assertion to expect nothing to be logged.
> - remove false positives from log.
> ITestS3AEndpointRegion failure:
> - Change in state of AwsExecutionAttribute.ENDPOINT_OVERRIDDEN
> attribute requires test tuning to match.
> Some tests against third-party stores fail
> - Includes fix for the assumeStoreAwsHosted() logic.
> - Documents how to turn off multipart uploads with third-party stores.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
