[
https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717021#comment-15717021
]
Aaron Fabbri commented on HADOOP-13449:
---------------------------------------
I did a little research on #3. It looks like you cannot do a prefix scan on a
partition key for DynamoDB. This seems to imply that, considering an operation
{{deleteSubtree(delete_path)}}, a simple search by prefix to find all entries
with paths that begin with {{delete_path}} would actually be a full table scan.
If I'm right, that is unfortunate.
The problem with the existing deleteSubtree(delete_path) implementation is that
all the children under delete_path might not be reachable from delete_path by
doing a simple tree walk over the state in the MetadataStore. The algorithm
would work, however, if, when we created a file, we also created all its
ancestor directories up to the root. This would establish an invariant that
{quote}
For any path p in DDB MetadataStore
For each ancestor a_i from p to the root
a_i is in DDB MetadataStore
{quote}
This actually sounds reasonable. Can we do it without changing the
{{MetadataStore}} interface? I think we can: when we create(path), we always
have the full absolute 'path', so we know the names of the ancestors all the
way to the root.
Thoughts?
> S3Guard: Implement DynamoDBMetadataStore.
> -----------------------------------------
>
> Key: HADOOP-13449
> URL: https://issues.apache.org/jira/browse/HADOOP-13449
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Mingliang Liu
> Attachments: HADOOP-13449-HADOOP-13345.000.patch,
> HADOOP-13449-HADOOP-13345.001.patch, HADOOP-13449-HADOOP-13345.002.patch,
> HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch,
> HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch,
> HADOOP-13449-HADOOP-13345.007.patch, HADOOP-13449-HADOOP-13345.008.patch,
> HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch
>
>
> Provide an implementation of the metadata store backed by DynamoDB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
