[
https://issues.apache.org/jira/browse/HADOOP-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900959#comment-14900959
]
Chris Nauroth commented on HADOOP-12334:
----------------------------------------
[~gouravk], thank you for the updated patch and the responses.
In patch v06, I still see a possibility that streams won't get closed after
exceptions from within the {{finally}} block.
{code}
} finally {
if(ipStream != null) {
ipStream.close();
}
if(opStream != null) {
opStream.close();
}
}
{code}
You could address this by using {{org.apache.hadoop.io.IOUtils#closeStream}},
which would handle the null check, the close, and the exception handling.
It's great to hear that there has been manual testing done. Could you also
please take another look at the possibility of adding a unit test? For
example, {{TestAzureFileSystemErrorConditions}} shows use of some special
testing hooks to simulate server-side failures. Is it possible to do something
similar here? Sorry to push on this, but this has become a somewhat complex
code path, and it would be great if we could get automated test coverage of it
to avoid regressions.
> Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage
> Throttling after retries
> -----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-12334
> URL: https://issues.apache.org/jira/browse/HADOOP-12334
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools
> Reporter: Gaurav Kanade
> Assignee: Gaurav Kanade
> Attachments: HADOOP-12334.01.patch, HADOOP-12334.02.patch,
> HADOOP-12334.03.patch, HADOOP-12334.04.patch, HADOOP-12334.05.patch,
> HADOOP-12334.06.patch
>
>
> HADOOP-11693 mitigated the problem of HMaster aborting regionserver due to
> Azure Storage Throttling event during HBase WAL archival. The way this was
> achieved was by applying an intensive exponential retry when throttling
> occurred.
> As a second level of mitigation we will change the mode of copy operation if
> the operation fails even after all retries -i.e. we will do a client side
> copy of the blob and then copy it back to destination. This operation will
> not be subject to throttling and hence should provide a stronger mitigation.
> However it is more expensive, hence we do it only in the case we fail after
> all retries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
