[
https://issues.apache.org/jira/browse/HADOOP-11984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546761#comment-14546761
]
Chris Nauroth commented on HADOOP-11984:
----------------------------------------
That was pretty good overall. I see recent hadoop-hdfs test runs have been
taking ~2.5 hours. This one was ~45 minutes. There are a couple of things to
follow up on before we can commit this:
# {{TestRenameWithSnapshots}} failed due to {{OutOfMemoryError}}. I don't know
yet if this is a problem in the test code or if we really over-taxed the
Jenkins host by running too many JVM processes.
# Numerous tests were reported as timed out, but they actually passed. The
test-patch.sh logic that checks for timed out tests works by looking for a
"Running" line that is not followed by a "Tests run" line. This logic won't
work with parallel execution, because the output from multiple JUnit processes
gets interleaved non-deterministically. We'll need to figure out a different
way to do the timed out check.
# The parallel-tests profile is only implemented for hadoop-common and
hadoop-hdfs, so other sub-modules won't get the benefit yet. MapReduce is
tracked in MAPREDUCE-4980. I don't think we have a YARN jira. We don't have
to wait for these to get done. Those modules just won't get the benefit until
they're done.
Next, I'm going to try another patch with a forced hadoop-common change to see
how that does.
> Enable parallel JUnit tests in pre-commit.
> ------------------------------------------
>
> Key: HADOOP-11984
> URL: https://issues.apache.org/jira/browse/HADOOP-11984
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-11984.001.patch
>
>
> HADOOP-9287 and related issues implemented the parallel-tests Maven profile
> for running JUnit tests in multiple concurrent processes. This issue
> proposes to activate that profile during pre-commit to speed up execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
