[
https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Mackrory updated HADOOP-14488:
-----------------------------------
Attachment: HADOOP-14488-HADOOP-13345-004.patch
So after digging into making sure this dealt correctly with nested directories,
I found a few other short-comings in the InconsistentAmazonS3Client as
currently used.
* There are separate entry points for individual and bulk deletes - when I
added tests for delete tracking I only started intercepting individual deletes.
* I made the isChild() function usable for both immediate children and all
other descendants, and corrected the logic for adding prefixes too.
* Added a test that the client returns the same number of object summaries and
common prefixes before and after (but still during the delay) a delete.
The test is really a meta-test and doesn't belong in this class. It's just
testing correct delete behavior in the Inconsistent client, so if anywhere I'd
add it to ITestS3GuardDeleteTracking, which is added in my patches for
HADOOP-14457. I suspect this will be otherwise ready to commit before that one,
so once we're at that point I'll plan on switching the order I have the patches
locally and moving testInconsistentS3ClientDeletes to
ITestS3GuardDeleteTracking.
> s3guard listStatus fails after renaming file into directory
> -----------------------------------------------------------
>
> Key: HADOOP-14488
> URL: https://issues.apache.org/jira/browse/HADOOP-14488
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
> Priority: Blocker
> Attachments: HADOOP-14488-HADOOP-13345-001.patch,
> HADOOP-14488-HADOOP-13345-002.patch, HADOOP-14488-HADOOP-13345-003.patch,
> HADOOP-14488-HADOOP-13345-004.patch, output.txt
>
>
> Running scala integration test with inconsistent s3 client & local DDB enabled
> {code}
> fs.rename("work/task-00/part-00", work)
> fs.listStatus(work)
> {code}
> The list status work fails with a message about the childStatus not being a
> child of the parent.
> Hypothesis: rename isn't updating the child path entry
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
