[
https://issues.apache.org/jira/browse/HADOOP-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961361#comment-14961361
]
Chris Nauroth commented on HADOOP-12334:
----------------------------------------
Hello [~gouravk]. I would not suggest trying to write a test for this against
live Azure Storage. The patch relates to client-side error handling of
server-side errors. I don't think there is a reliable way to force the service
to return those errors.
Instead, I was thinking we might be able to use mocks to simulate the errors
and confirm that it can fall back to a client-side copy.
{{TestAzureFileSystemErrorConditions}} is an example of a test suite that
already does something similar, so you might try looking at that one as a
sample.
Please take a look, and let me know if you think it can be done or if you think
it's not feasible. Thanks!
> Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage
> Throttling after retries
> -----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-12334
> URL: https://issues.apache.org/jira/browse/HADOOP-12334
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools
> Reporter: Gaurav Kanade
> Assignee: Gaurav Kanade
> Attachments: HADOOP-12334.01.patch, HADOOP-12334.02.patch,
> HADOOP-12334.03.patch, HADOOP-12334.04.patch, HADOOP-12334.05.patch,
> HADOOP-12334.06.patch
>
>
> HADOOP-11693 mitigated the problem of HMaster aborting regionserver due to
> Azure Storage Throttling event during HBase WAL archival. The way this was
> achieved was by applying an intensive exponential retry when throttling
> occurred.
> As a second level of mitigation we will change the mode of copy operation if
> the operation fails even after all retries -i.e. we will do a client side
> copy of the blob and then copy it back to destination. This operation will
> not be subject to throttling and hence should provide a stronger mitigation.
> However it is more expensive, hence we do it only in the case we fail after
> all retries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
