bgaborg commented on a change in pull request #951: HADOOP-15183. S3Guard store
becomes inconsistent after partial failure of rename
URL: https://github.com/apache/hadoop/pull/951#discussion_r293001205
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java
##########
@@ -149,14 +148,63 @@ public static MetadataStore getMetadataStore(FileSystem
fs)
public static S3AFileStatus putAndReturn(MetadataStore ms,
S3AFileStatus status,
S3AInstrumentation instrumentation) throws IOException {
+ return putAndReturn(ms, status, instrumentation, null);
+ }
+
+ /**
+ * Helper function which puts a given S3AFileStatus into the MetadataStore
and
+ * returns the same S3AFileStatus. Instrumentation monitors the put
operation.
+ * @param ms MetadataStore to {@code put()} into.
+ * @param status status to store
+ * @param instrumentation instrumentation of the s3a file system
+ * @param operationState possibly-null metastore state tracker.
+ * @return The same status as passed in
+ * @throws IOException if metadata store update failed
+ */
+ @RetryTranslated
+ public static S3AFileStatus putAndReturn(
+ final MetadataStore ms,
+ final S3AFileStatus status,
+ final S3AInstrumentation instrumentation,
+ @Nullable final BulkOperationState operationState) throws IOException {
long startTimeNano = System.nanoTime();
- ms.put(new PathMetadata(status));
- instrumentation.addValueToQuantiles(S3GUARD_METADATASTORE_PUT_PATH_LATENCY,
- (System.nanoTime() - startTimeNano));
- instrumentation.incrementCounter(S3GUARD_METADATASTORE_PUT_PATH_REQUEST,
1);
+ try {
+ ms.put(new PathMetadata(status), operationState);
+ } finally {
Review comment:
maybe add a log message on exception and rethrow
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
