[
https://issues.apache.org/jira/browse/HADOOP-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Marqardt reopened HADOOP-17015:
--------------------------------------
We should revisit PR 2021 and try to find a better solution for rename. Users
expect Rename to be atomic. The service implementation is atomic, but we have
this client-side idempotency issue. This fix relies on time and assumes that
if the destination was recently updated while we are executing a retry policy,
that we succeeded. This may not be the case. For example, users may rely on
rename (with overwrite = false) of a file to synchronize or act like a
distributed lock, so who ever renames successfully acquires the lock. With the
fix in PR 2021, more than one caller could acquire this lock at the same time.
Instead, I think we could allow the client to provide a UUID for the rename
operation and persist this UUID in the metadata of the destination blob upon
successful completion of a rename, then if we get into this idempotency issue
and the client gets a 404 source does not exist, we can check the destination
blob's metadata to see if the UUID is a match.
> ABFS: Make PUT and POST operations idempotent
> ---------------------------------------------
>
> Key: HADOOP-17015
> URL: https://issues.apache.org/jira/browse/HADOOP-17015
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2.1
> Reporter: Sneha Vijayarajan
> Assignee: Sneha Vijayarajan
> Priority: Major
> Fix For: 3.4.0
>
>
> Currently when a PUT or POST operation timeouts and the server has already
> successfully executed the operation, there is no check in driver to see if
> the operation did succeed or not and just retries the same operation again.
> This can cause driver to through invalid user errors.
>
> Sample scenario:
> # Rename request times out. Though server has successfully executed the
> operation.
> # Driver retries rename and get source not found error.
> In the scenario, driver needs to check if rename is being retried and success
> if source if not found, but destination is present.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
