[
https://issues.apache.org/jira/browse/HADOOP-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023626#comment-17023626
]
Hudson commented on HADOOP-16746:
---------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17901 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17901/])
HADOOP-16746. mkdirs and s3guard Authoritative mode. (stevel: rev
7f40e6688a5716fca53e1090d8347a43064d6d43)
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestDynamoDBMetadataStoreScale.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardWriteBack.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/TestS3Guard.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/MetadataStoreTestBase.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DynamoDBMetadataStore.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/MetadataStore.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestRestrictedReadAccess.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/BulkOperationState.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardFsck.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestPartialDeleteFailures.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestDynamoDBMetadataStore.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java
* (edit)
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestDynamoDBMetadataStoreAuthoritativeMode.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/LocalMetadataStore.java
* (edit)
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/NullMetadataStore.java
> S3A empty dir markers are not created in s3guard as authoritative
> -----------------------------------------------------------------
>
> Key: HADOOP-16746
> URL: https://issues.apache.org/jira/browse/HADOOP-16746
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0, 3.2.1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Critical
> Fix For: 3.3.0
>
>
> Newly created empty dirs, or markers created after delete operations, are not
> marked in S3Guard as auth. This has adverse consequences in that following
> changes (i.e. new files) don't get marked as auth either...it needs a
> listFiles call to scan the source and mark as auth.
> I could stick a quick fix in to HADOOP-16697, but don't want to as I don't
> like what that would mean. Essentially, finishedWrite() need to recognise
> when an empty directory markers being created (it does this already) and then
> *always* declare it as auth.
> I'd prefer for the mkdirs operation to pass a flag all the way through to
> finishedWrite so that it doesn't need to infer this. The WriteOpContext of
> HADOOP-16134 would be the way to do this. Yes it's a big change but it would
> be extensible -and I already have some plans there.
> Instead it will be a follow-up.
> The tests for this problem are part of HADOOP-16697, just disabled for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
