[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13614:
------------------------------------
Status: Patch Available (was: Open)
latest patch a github PR. This does more as I decided I could make this the
patch to fix up some of the test niggles that had been irritating us all,
namely: different parent classes for tests, still too many not running in
parallel.
* where possible give all the s3a tests the superclass of
{{AbstractS3ATestBase}}, clean up children.
* ITestS3AFileSystemContract has path logic reworked *so test can run in
parallel*
* fixed up yarn test to work in parallel too, along with BlockingThreadPool and
FastOutputStream
* on the files changed, moved the copyright comment to being a simple comment
rather than a javadoc one.
This conflicts a bit with HADOOP-13560; this patch should go in first and then
I'll address the diffs
Tested: s3 ireland; if I skipthe s3n and s3 tests, it takes ~4:20 to do a test
run now
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---------------------------------------------------------------------------
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
