[
https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716696#comment-15716696
]
Aaron Fabbri commented on HADOOP-13449:
---------------------------------------
I think this is the list of outstanding items to get integration tests passing.
1. Dealing with anonymous / reduced privilege bucket credentials ([~steve_l]'s
previous comment). We should discuss separately... maybe separate JIRA? I
have some other related requirements around table <-> bucket mappings.
2. Updating {{S3AFileStatus#isEmptyDirectory()}}. move(), put(), delete(), and
deleteSubtree() will need to maintain the parent dir's empty bit and/or
invalidate it's state. I think the basic logic used in LocalMetadataStore
should work fine for now.
3. deleteSubtree(path) assumes that any deleted subtree is fully recorded in
the MetadataStore. Best solution, IMO, is to query for all entries that have
'path' as an ancestor. Hoping we can use a prefix scan to keep that efficient.
[~liuml07] would love to hear your DynamoDB expertise on that idea?
I'm working on #2 at the moment. I wrote a new integration test
{{ITestS3AEmptyDirectory}} that exercises a directory going from
empty->non-empty and vice-versa. Much easier to debug that case in isolation!
It passes for LocalMetadataStore but, as expected, fails for DDB still.
> S3Guard: Implement DynamoDBMetadataStore.
> -----------------------------------------
>
> Key: HADOOP-13449
> URL: https://issues.apache.org/jira/browse/HADOOP-13449
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Mingliang Liu
> Attachments: HADOOP-13449-HADOOP-13345.000.patch,
> HADOOP-13449-HADOOP-13345.001.patch, HADOOP-13449-HADOOP-13345.002.patch,
> HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch,
> HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch,
> HADOOP-13449-HADOOP-13345.007.patch, HADOOP-13449-HADOOP-13345.008.patch,
> HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch
>
>
> Provide an implementation of the metadata store backed by DynamoDB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
