Rajesh Balamohan created HADOOP-13936:
-----------------------------------------
Summary: S3Guard: DynamoDB can go out of sync with
S3AFileSystem::delete operation
Key: HADOOP-13936
URL: https://issues.apache.org/jira/browse/HADOOP-13936
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Rajesh Balamohan
Priority: Minor
As a part of {{S3AFileSystem.delete}} operation {{innerDelete}} is invoked,
which deletes keys from S3 in batches (default is 1000). But DynamoDB is
updated only at the end of this operation. This can cause issues when deleting
large number of keys.
E.g, it is possible to get exception after deleting 1000 keys and in such cases
dynamoDB would not be updated. This can cause DynamoDB to go out of sync.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
