[
https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278034#comment-15278034
]
Steve Loughran commented on HADOOP-13113:
-----------------------------------------
I think our tests generate race conditions more than most other bits of code
do, as they are repeatedly working on the same bits of the objects store and
trying to do lots of small operations.
Looks like you are hitting the delayed-listing-consistency problem which S3 has
—here on a delete, rather than an addition.
why not have that test not fail immediately if the root dir isn't empty, but
sleep and spin for a bit until the dir empties up.
> Enable parallel test execution for hadoop-aws.
> ----------------------------------------------
>
> Key: HADOOP-13113
> URL: https://issues.apache.org/jira/browse/HADOOP-13113
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch
>
>
> The full hadoop-aws test suite takes ~30 minutes to execute. The tests spend
> most of their time blocked on network I/O with the S3 back-end, but they
> don't saturate the bandwidth of the NIC. We can improve overall execution
> time by enabling parallel test execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
