[
https://issues.apache.org/jira/browse/HADOOP-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17941947#comment-17941947
]
Ahmar Suhail edited comment on HADOOP-19527 at 4/8/25 4:43 PM:
---------------------------------------------------------------
ok took me a few hours but I get it now.
This happens when encryption is enabled (eg: SSE-KMS), because in that case the
etag is not the md5 of the object content. When making a GET request, AAL does
GetRequest.builder()
.s3Uri(this.objectKey.getS3URI())
.range(this.range)
.etag(this.objectKey.getEtag())
the etag comes from cached HeadObjectResponse. When you run the whole test
suite, head object gets cached from a previous HEAD. Then in this test, the
cached value gets used, but since with SSE-KMS the value will change every time
the vectored object vectored_file.txt gets created, this will fail.
When not using SSE-KMS, the etag is always the same, as it is just the md5 of
the object content which never changes.
was (Author: JIRAUSER283484):
ok took me a few hours but I get it now.
This happens when encryption is enabled (eg: SSE-KMS), because in that case the
etag is not the md5 of the object content. When making a GET request, AAL does
GetRequest.builder()
.s3Uri(this.objectKey.getS3URI())
.range(this.range)
.etag(this.objectKey.getEtag())
the etag comes from cached HeadObjectResponse. When you run the whole test
suite, head object gets cached from a previous HEAD. Then in this test, the
cached value gets used, but since with SSE-KMS the value will change every time
the vectored object vectored_file.txt gets created, this will fail.
When not using SSE-KMS, the etag is always the same, as it is just the md5 of
the object content which never changes.
> S3A: testVectoredReadAfterNormalRead() failing with 412 response from S3
> ------------------------------------------------------------------------
>
> Key: HADOOP-19527
> URL: https://issues.apache.org/jira/browse/HADOOP-19527
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.5.0, 3.4.2
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Blocker
>
> This is surfacing on a bucket using versionid for change detection: block
> reads are failing in the test
> {{ITestS3AContractAnalyticsStreamVectoredRead.testVectoredReadAfterNormalRead()}}
> {code}
> Caused by: software.amazon.awssdk.services.s3.model.S3Exception: At least one
> of the pre-conditions you specified did not hold (Service: S3, Status Code:
> 412, Request ID: 0AN2EB8QXC75HH0T, Extended Request ID:
> U5l/UnIF4n3NO1mrZVzS2vv72F3LgUoVJxR4XodUSaTWCerfjmmpH45CbFGKkTkfgfnykwzseGo=)
> at
> software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:104)
> {code}
> * this is the normal readFully() call, before the vectored one
> * it worked last week
> * also found on branch-3.4 before the SDK update, so not an issue caused by
> the SDK unless my maven repo is badly contaminated
> * seems unrelated to versioning -still there when disabled.
> * applies on unversioned s3 express store too.
> About the main way I could see this surface is if the test file is less than
> the actual length of file created, so the GET is rejected for reading off the
> end (the openfile passes in the length to save the HEAD)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
