[
https://issues.apache.org/jira/browse/HADOOP-13826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743421#comment-15743421
]
Thomas Demoor commented on HADOOP-13826:
----------------------------------------
[~mackrorysd] your last patch seems simple but I think it might do the trick. I
like simple solutions.
{{putObject()}} uses the (unbounded) TransferManager, {{putObjectDirect()}}
and {{uploadPart()}} use the bounded threadpool so I think the (potentially)
memory intensive parts are nicely isolated and under control.
My only slight concern is that now both pools can have {{MAX_THREADS}} active.
From my reading of the code, both threadpools cannot be doing large object PUTs
at the same time (an instance of s3a uses the block-based uploads or the
regular S3AOutputstream, never both at the same time). What is possible, is
that during a large block-based upload, which is saturating the bounded
executor, another client thread might {{rename}} a directory, invoking a lot of
parallel copies, hence saturating the TransferManager. But copies are are not
data-intensive (see below) so I assume this is manageable.
I like [[email protected]]'s ideas for further separating out the different
types of operations, but have one remark: for me COPY is not similar to PUT.
COPY is completely server-side and is thus generally much less
resource-intensive and much quicker than PUT (the smaller your bandwidth to S3,
the bigger the difference becomes).
> S3A Deadlock in multipart copy due to thread pool limits.
> ---------------------------------------------------------
>
> Key: HADOOP-13826
> URL: https://issues.apache.org/jira/browse/HADOOP-13826
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 2.7.3
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Priority: Critical
> Attachments: HADOOP-13826.001.patch, HADOOP-13826.002.patch,
> HADOOP-13826.003.patch
>
>
> In testing HIVE-15093 we have encountered deadlocks in the s3a connector. The
> TransferManager javadocs
> (http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html)
> explain how this is possible:
> {quote}It is not recommended to use a single threaded executor or a thread
> pool with a bounded work queue as control tasks may submit subtasks that
> can't complete until all sub tasks complete. Using an incorrectly configured
> thread pool may cause a deadlock (I.E. the work queue is filled with control
> tasks that can't finish until subtasks complete but subtasks can't execute
> because the queue is filled).{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
