[
https://issues.apache.org/jira/browse/HADOOP-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17016022#comment-17016022
]
Steve Loughran commented on HADOOP-16746:
-----------------------------------------
the "simple" feature is already ~in, just handled wrongly in DDB metastore
* fix
* tests for auth break where they assumed that creating a dir resulted in it
being non-auth. Without opening a test-only option, need to rely on other
quirks here
At same time
* reduce #of GET/PUT request on DDB when creating entries
* PUT of a file doesn't change auth status of parent dir
* PutOperations enum includes a "mkdir", so trace of operations shows which
were related to a mkdir
> s3a empty dir markers are not created in s3guard as authoritative
> -----------------------------------------------------------------
>
> Key: HADOOP-16746
> URL: https://issues.apache.org/jira/browse/HADOOP-16746
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> Newly created empty dirs, or markers created after delete operations, are not
> marked in S3Guard as auth. This has adverse consequences in that following
> changes (i.e. new files) don't get marked as auth either...it needs a
> listFiles call to scan the source and mark as auth.
> I could stick a quick fix in to HADOOP-16697, but don't want to as I don't
> like what that would mean. Essentially, finishedWrite() need to recognise
> when an empty directory markers being created (it does this already) and then
> *always* declare it as auth.
> I'd prefer for the mkdirs operation to pass a flag all the way through to
> finishedWrite so that it doesn't need to infer this. The WriteOpContext of
> HADOOP-16134 would be the way to do this. Yes it's a big change but it would
> be extensible -and I already have some plans there.
> Instead it will be a follow-up.
> The tests for this problem are part of HADOOP-16697, just disabled for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
