[
https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146129#comment-16146129
]
Steve Loughran commented on HADOOP-14553:
-----------------------------------------
bq. {{mvn test}} must run all the tests. This behavior should not change. I
think this should be the standard command to run tests for a project. Also,
people will not be aware of the change we are making, so please add new
arguments to mvn but do not change existing behavior.
Maven has two test phases in its lifecycle: the unit tests & the integration
tests.This patch keeps all the unit tests in the test phase, editing the POM so
that {{mvn test}} and hence jenkins & yetus will run them *always*; they're
now really "unit" tests. In contrast: The integration tests are meant to be
that: the system integration tests. It kicks off after the packaging is done
and the unit tests are passed, and lets you do more stuff [related to testing
with/against real
systems.|http://maven.apache.org/surefire/maven-failsafe-plugin/]. I think
we'll probably be reviewing all the object stores' tests and split them off the
same way, as and when someone (else) sits down to make test runs finish in
single digits of minutes.
The good news:
# run {{mvn verify}} and everything is run, unit and integration. That happens
today on any module, even those without any integration
# make the test review process "declare that you executed mvn verify" and the
process ensures all tests get run
# yetus and jenkins will run the "mvn test" suite on every patch which touches
hadoop-common. This reduces the risk of other changes breaking things.
# You can still run individual tests in your IDE, which is generally the
fastest way to get a feature/test working.
# Even though the IT test phase is halted if the unit tests fail, you can
bypass all the unit tests just by making up an nonexistent test {{mvn verify
-Dtest=moo -Dit.test=ITestSomething}}
This is the evolving best practise in the project related to unit & system
testing; expect more of, especially in modules I'm involved in.
bq. 2) The Results summary is no longer consolidated. For example, when you run
mvn -T 1C -Dparallel-tests clean verify, at the end it looks like only a few
tests were run, but if you scroll up in the console output you see that there
were several runs and reports. Lets summarize the results in a single line at
the end.
This is because there are flour test runs: the sequential and parallel unit
test runs, the parallel integration tests and the final set of IT tests which
don't parallelise. Each test run produces its own list of results in the
console output, and its own set of XML docs. target/surefire-reports collect
the surefire ones (unit) and target/failsafe-reports for the failsafe ones.
It's all the same XML format developed for <junit> back in 2000 and this JUnit
stuff was new.
The way to coalesce the results is to merge the XML docs, usually with XSLT, to
produce HTML site doc, or whatever jenkins does. Now if you want console
output, that's probably doable too...I think there's some in Stack overflow for
an <xslt> task to do the conversion, albeit after everything.
{code}
# the unit tests
mvn surefire-report:report-only
# integration tests
mvn surefire-report:failsafe-report-only
{code}
These add some new html files in target/site with report summaries
If you want both reports, the azure markdown and the images needed to make the
test reports look good:
{code}
mvn site:site surefire-report:failsafe-report-only surefire-report:report
{code}
bq. 3) The tests currently marked scale are not all scale tests. Several are
functional tests, like those in ITestBlockBlobInputStream.java. It is important
for these tests to be run prior to each check-in. Unless you have added new
scale tests (I did not look at the history of every scale test), all of the
tests need to be run before check-in when we run mvn test.
Maybe the way to view them is "scaleable" tests: tests which can run at a small
scale & configured to run at larger scales if the caller wants it, but will
always run. I'll see what I can do there.
> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>
> Key: HADOOP-14553
> URL: https://issues.apache.org/jira/browse/HADOOP-14553
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch,
> HADOOP-14553-003.patch, HADOOP-14553-004.patch, HADOOP-14553-005.patch,
> HADOOP-14553-006.patch, HADOOP-14553-007.patch, HADOOP-14553-008.patch,
> HADOOP-14553-009.patch, HADOOP-14553-010.patch, HADOOP-14553-011.patch,
> HADOOP-14553-012.patch, HADOOP-14553-014.patch, HADOOP-14553-015.patch
>
>
> The Azure tests are slow to run as they are serialized, as they are all
> called Test* there's no clear differentiation from unit tests which Jenkins
> can run, and integration tests which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize
> (which includes having separate paths for every test suite). The code in
> hadoop-aws's POM &c show what to do.
> *UPDATE August 4, 2017*: Adding a list of requirements to clarify the
> acceptance criteria for this JIRA:
> # Parallelize test execution
> # Define test groups: i) UnitTests - self-contained, executed by Jenkins, ii)
> IntegrationTests - requires Azure Storage account, executed by engineers
> prior to check-in, and if needed, iii) ScaleTests – long running performance
> and scalability tests.
> # Define configuration profiles to run tests with different settings. Allows
> an engineer to run “IntegrationTests” with fs.azure.secure.mode = true and
> false. Need to review settings to see what else would benefit.
> # Maven commands to run b) and c). Turns out it is not easy to do with
> Maven, so we might have to run it multiple times to run with different
> configuration settings.
> # Document how to add and run tests and the process for contributing to
> Apache Hadoop. Steve shared an example at
> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
>
> # UnitTests should run in under 2 minutes and IntegrationTests should run in
> under 15 minutes, even on slower network connections. (These are rough goals)
> # Ensure test data (containers/blobs/etc) is deleted. Exceptions for large
> persistent content used repeatedly to expedite test execution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
