[
https://issues.apache.org/jira/browse/HADOOP-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17942270#comment-17942270
]
ASF GitHub Bot commented on HADOOP-19527:
-----------------------------------------
steveloughran commented on code in PR #7592:
URL: https://github.com/apache/hadoop/pull/7592#discussion_r2035938447
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java:
##########
@@ -1709,6 +1710,24 @@ public static void skipIfEncryptionNotSet(Configuration
configuration,
}
}
+ /**
+ * Skip a test if encryption algorithm is not empty, or if it is set to
anything other than AES256.
+ *
+ * @param configuration configuration
+ * @throws IOException if the secret lookup fails.
+ */
+ public static void skipIfEncryptionSet(Configuration configuration) throws
IOException {
Review Comment:
change title to skipForAnyEncryptionExceptSSES3()
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractAnalyticsStreamVectoredRead.java:
##########
@@ -44,6 +48,19 @@ public ITestS3AContractAnalyticsStreamVectoredRead(String
bufferType) {
protected Configuration createConfiguration() {
Configuration conf = super.createConfiguration();
enableAnalyticsAccelerator(conf);
+
+ // If encryption is set, some AAL tests will fail. This is because AAL
caches the head request response, and uses
+ // the eTag when making a GET request. When using encryption, the eTag is
no longer a hash of the object content,
+ // and is not always the same when the same object is created multiple
times. This test creates the file
+ // vectored_file.txt before running each test, which will have a different
eTag when using encryption, leading to
+ // preconditioned failures. This issue is tracked in:
+ // https://github.com/awslabs/analytics-accelerator-s3/issues/218
+ try {
+ skipIfEncryptionSet(conf);
+ } catch (IOException e) {
Review Comment:
do this in the method itself
> S3A: testVectoredReadAfterNormalRead() failing with 412 response from S3
> ------------------------------------------------------------------------
>
> Key: HADOOP-19527
> URL: https://issues.apache.org/jira/browse/HADOOP-19527
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.5.0, 3.4.2
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Blocker
> Labels: pull-request-available
>
> This is surfacing on a bucket using versionid for change detection: block
> reads are failing in the test
> {{ITestS3AContractAnalyticsStreamVectoredRead.testVectoredReadAfterNormalRead()}}
> {code}
> Caused by: software.amazon.awssdk.services.s3.model.S3Exception: At least one
> of the pre-conditions you specified did not hold (Service: S3, Status Code:
> 412, Request ID: 0AN2EB8QXC75HH0T, Extended Request ID:
> U5l/UnIF4n3NO1mrZVzS2vv72F3LgUoVJxR4XodUSaTWCerfjmmpH45CbFGKkTkfgfnykwzseGo=)
> at
> software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:104)
> {code}
> * this is the normal readFully() call, before the vectored one
> * it worked last week
> * also found on branch-3.4 before the SDK update, so not an issue caused by
> the SDK unless my maven repo is badly contaminated
> * seems unrelated to versioning -still there when disabled.
> * applies on unversioned s3 express store too.
> About the main way I could see this surface is if the test file is less than
> the actual length of file created, so the GET is rejected for reading off the
> end (the openfile passes in the length to save the HEAD)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
