ajfabbri commented on a change in pull request #951: HADOOP-15183. S3Guard
store becomes inconsistent after partial failure of rename
URL: https://github.com/apache/hadoop/pull/951#discussion_r295519756
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DynamoDBMetadataStore.java
##########
@@ -899,6 +915,9 @@ public void addAncestors(
// a directory entry will go in.
PathMetadata directory = get(parent);
if (directory == null || directory.isDeleted()) {
+ if (entryFound) {
+ LOG.warn("Inconsistent S3Guard table: adding directory {}", parent);
+ }
Review comment:
also, we might as well do the `depth(path)` get operations in parallel if
they always happen, and the `break` behavior you remove is not configurable. In
terms of write latency it would remove `depth(path)-1` round trips (approx.).
Proposing this as a followup JIRA, not doing it here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
