[
https://issues.apache.org/jira/browse/HADOOP-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13614:
------------------------------------
Attachment: HADOOP-13614-branch-2-007.patch
patch 007
* apply to branch-2
* verify that all appears well.
* Tune more tests, including ITestS3AMiniYarnCluster.
A big change here is fixing up the scale tests to work as subclasses of the
S3AScaleTestBase indirect subclasses of AbstractFSContractTestBase, because
that sets up the test timeout rule. Rather than have a field of the same name
and hope that its timeout gets picked up, I've tuned how timeouts get set up,
so the subclasses do it. All well and good, except those subclasses are being
called during the initialization of the base class, that is: before the
subclasses are full inited. I don't ever like doing that, though it is working
here.
Tested S3a ireland, two (unrelated) failures
{code}
ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:97->Assert.fail:88
Deletion of child entries failed, still have1
s3a://hwdev-steve-ireland-new/fork-5
ITestS3AFileContextStatistics>FCStatisticsBaseTest.testStatistics:102->verifyWrittenBytes:54
expected:<512> but was:<0>
{code}
Time to run without scale tests, ~4 minutes, going to 6:30 with the scale tests
@ default size in.
> Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
> ---------------------------------------------------------------------------
>
> Key: HADOOP-13614
> URL: https://issues.apache.org/jira/browse/HADOOP-13614
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-13614-branch-2-001.patch,
> HADOOP-13614-branch-2-002.patch, HADOOP-13614-branch-2-002.patch,
> HADOOP-13614-branch-2-004.patch, HADOOP-13614-branch-2-005.patch,
> HADOOP-13614-branch-2-006.patch, HADOOP-13614-branch-2-007.patch, testrun.txt
>
>
> Some of the slow test cases contain tests that are now obsoleted by newer
> ones. For example, {{ITestS3ADeleteManyFiles}} has the test case
> {{testOpenCreate()}} which writes then reads files up 25 MB.
> Have a look at which of the s3a tests are taking time, review them to see if
> newer tests have superceded the slow ones; and cut them where appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
