[
https://issues.apache.org/jira/browse/HADOOP-14576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075123#comment-16075123
]
Aaron Fabbri commented on HADOOP-14576:
---------------------------------------
Thanks for filing the JIRA, [~mackrorysd].
{quote}
My concern with failing over to S3 for non-auth read is what happens when
you're listing stuff that isn't consistent on S3 yet. IMO non-auth mode is
really just to enable lazily loading data that already existed or that is added
outside of S3Guard. I don't think it should weaken guarantees in the presence
of partitioning events in DynamoDB.
{quote}
Yeah I think both arguments have merit.
We could argue that failing back to basic S3 without consistency is better than
failing a job. We could also have a configuration flag that lets users choose
either behavior. Since the chance of inconsistency is pretty low, there is
a good probability that running in degraded mode (no S3Guard) until the table
comes back would be successful.
I'm not sure authoritative mode matters? The client being in S3guard
authoritative mode just means the the FS client *may* skip round trips to S3
*if* the metadatastore reports it has a full listing. Since MetadatStore
throws an error, the "if metadatastore reports full listing" condition is not
met.
> DynamoDB tables may leave ACTIVE state after initial connection
> ---------------------------------------------------------------
>
> Key: HADOOP-14576
> URL: https://issues.apache.org/jira/browse/HADOOP-14576
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Sean Mackrory
>
> We currently only anticipate tables not being in the ACTIVE state when first
> connecting. It is possible for a table to be in the ACTIVE state and move to
> an UPDATING state during partitioning events. Attempts to read or write
> during that time will result in an AmazonServerException getting thrown. We
> should try to handle that better...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
