[
https://issues.apache.org/jira/browse/HADOOP-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18016984#comment-18016984
]
ASF GitHub Bot commented on HADOOP-19658:
-----------------------------------------
anujmodi2021 commented on code in PR #7914:
URL: https://github.com/apache/hadoop/pull/7914#discussion_r2309688341
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCreate.java:
##########
@@ -2236,6 +2238,98 @@ public void
testFailureInGetPathStatusDuringCreateRecovery() throws Exception {
}
}
+ /**
+ * Test to simulate a successful create operation followed by a connection
reset
+ * on the response, triggering a retry.
+ *
+ * This test verifies that the create operation is retried in the event of a
+ * connection reset during the response phase. The test creates a mock
+ * AzureBlobFileSystem and its associated components to simulate the create
+ * operation and the connection reset. It then verifies that the create
+ * operation is retried once before succeeding.
+ *
+ * @throws Exception if an error occurs during the test execution.
+ */
+ @Test
+ public void testCreateIdempotencyForNonHnsBlob() throws Exception {
+ assumeThat(isAppendBlobEnabled()).as("Not valid for APPEND
BLOB").isFalse();
+ // Create a spy of AzureBlobFileSystem
+ try (AzureBlobFileSystem fs = Mockito.spy(
+ (AzureBlobFileSystem) FileSystem.newInstance(getRawConfiguration()))) {
+ assumeHnsDisabled();
+ // Create a spy of AzureBlobFileSystemStore
+ AzureBlobFileSystemStore store = Mockito.spy(fs.getAbfsStore());
+ assumeBlobServiceType();
Review Comment:
We can move all assume before any other statement
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java:
##########
@@ -1702,6 +1705,85 @@ public void testRenamePathRetryIdempotency() throws
Exception {
}
}
+ /**
+ * Test to simulate a successful copy blob operation followed by a
connection reset
+ * on the response, triggering a retry.
+ *
+ * This test verifies that the copy blob operation is retried in the event
of a
+ * connection reset during the response phase. The test creates a mock
+ * AzureBlobFileSystem and its associated components to simulate the copy
blob
+ * operation and the connection reset. It then verifies that the create
+ * operation is retried once before succeeding.
+ *
+ * @throws Exception if an error occurs during the test execution.
+ */
+ @Test
+ public void testRenameIdempotencyForNonHnsBlob() throws Exception {
+ assumeThat(isAppendBlobEnabled()).as("Not valid for APPEND
BLOB").isFalse();
+ // Create a spy of AzureBlobFileSystem
+ try (AzureBlobFileSystem fs = Mockito.spy(
+ (AzureBlobFileSystem) FileSystem.newInstance(getRawConfiguration()))) {
+ assumeHnsDisabled();
Review Comment:
Same here, move all assume to first few lines
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##########
@@ -509,9 +509,30 @@ public AbfsRestOperation createPath(final String path,
final TracingContext tracingContext) throws AzureBlobFileSystemException
{
AbfsRestOperation op;
if (isFileCreation) {
- // Create a file with the specified parameters
- op = createFile(path, overwrite, permissions, isAppendBlob, eTag,
- contextEncryptionAdapter, tracingContext);
+ AbfsRestOperation statusOp = null;
+ try {
+ // Check if the file already exists by calling GetPathStatus
+ statusOp = getPathStatus(path, false, tracingContext, null);
Review Comment:
In case of override true, flow might come here with already a Head call done
on path.
Can we avoid this head call in that case?
> ABFS: [FNS Over Blob] Support create and rename idempotency on FNS Blob from
> client side
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-19658
> URL: https://issues.apache.org/jira/browse/HADOOP-19658
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: hadoop-azure
> Affects Versions: 3.4.1
> Reporter: Anmol Asrani
> Assignee: Anmol Asrani
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.1
>
>
> Support create and rename idempotency on FNS Blob from client side
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
