[
https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904022#comment-15904022
]
Aaron Fabbri commented on HADOOP-14154:
---------------------------------------
{quote}
Would that isAuthoritative flag have to be setup by higher level applications
like Pig/Hive/MR?
{quote}
No, it is internal to S3A. S3A can tell when it has the full listing for a
directory, and simply conveys that to the MetadataStore by setting the
isAuthoritative bit.
e.g. in {{S3Guard#dirListingUnion(..)}}, it always sets the flag when it puts
the listing into the MetadataStore, since this function always has the full
listing for the directory:
{code}
dirMeta.setAuthoritative(true); // This is the full directory contents
{code}
> Set isAuthoritative flag when creating DirListingMetadata in DynamoDBMetaStore
> ------------------------------------------------------------------------------
>
> Key: HADOOP-14154
> URL: https://issues.apache.org/jira/browse/HADOOP-14154
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Rajesh Balamohan
> Priority: Minor
> Attachments: HADOOP-14154-HADOOP-13345.001.patch,
> HADOOP-14154-HADOOP-13345.002.patch
>
>
> Currently {{DynamoDBMetaStore::listChildren}} does not populate
> {{isAuthoritative}} flag when creating {{DirListingMetadata}}.
> This causes additional S3 lookups even when users have enabled
> {{fs.s3a.metadatastore.authoritative}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
