Author: dfabulich
Date: Wed Jan 16 13:58:19 2008
New Revision: 612582
URL: http://svn.apache.org/viewvc?rev=612582&view=rev
Log:
Updated plugin testing documentation
Modified:
maven/site/trunk/src/site/apt/plugin-developers/plugin-testing.apt
Modified: maven/site/trunk/src/site/apt/plugin-developers/plugin-testing.apt
URL:
http://svn.apache.org/viewvc/maven/site/trunk/src/site/apt/plugin-developers/plugin-testing.apt?rev=612582&r1=612581&r2=612582&view=diff
==============================================================================
--- maven/site/trunk/src/site/apt/plugin-developers/plugin-testing.apt
(original)
+++ maven/site/trunk/src/site/apt/plugin-developers/plugin-testing.apt Wed Jan
16 13:58:19 2008
@@ -31,24 +31,38 @@
Currently, Maven only supports unit testing out of the box. This document is
intended to help Maven Developers to
test Plugins with Unit Tests, Integration Tests or Functional tests.
- <<Note>>: It is a <<DRAFT>> document, only the unit testing part is more
official.
+ <<Note: There are a lot of different ways to test a Maven plugin.>> For a
review of different strategies and tools, please refer to
{{{http://docs.codehaus.org/display/MAVENUSER/Review+of+Plugin+Testing+Strategies}Review
of Plugin Testing Strategies}}
-Unit Tests
+Testing Styles: Unit Testing vs. Functional/Integration Testing
- Typically, these tests will be executed with the <<<test>>> phase.
+ A unit test attempts to verify a mojo as an isolated unit, by mocking out the
rest of the Maven environment. A mojo unit test does not attempt to run your
plugin in the context of a real Maven build. Unit tests are designed to be
fast.
+
+ A functional/integration test attempts to use a mojo in a real Maven build,
by launching a real instance of Maven in a real project. Normally this
requires you to construct special dummy Maven projects with real POM files.
Often this requires you to have already installed your plugin into your local
repository so it can be used in a real Maven build. Functional tests run much
more slowly than unit tests, but they can catch bugs that you may not catch
with unit tests.
+
+ The general wisdom is that your code should be mostly tested with unit tests,
but should also have some functional tests.
-* Junit
+Unit Tests
- Globally, you need to create a *Test (by convention) class which <<<extends
TestCase>>>.
+* Using JUnit alone
- If you need Plexus container services, you need to <<<extends
PlexusTestCase>>> instead of TestCase.
+ In principle, you can write a unit test of a plugin Mojo the same way you'd
write any other JUnit test case, by writing a class that <<<extends TestCase>>>.
+
+ However, most mojos need more information to work properly. For example,
you'll probably need to inject a reference to a MavenProject, so your mojo can
query project variables.
+
+* Using PlexusTestCase
+
+ Mojo variables are injected using Plexus, and many Mojos are written to take
specific advantage of the Plexus container (by executing a lifecycle or having
various injected dependencies).
+
+ If you all you need is Plexus container services, you can write your class
with <<<extends PlexusTestCase>>> instead of TestCase.
+
+ With that said, if you need to inject Maven objects into your mojo, you'll
probably prefer to use the maven-plugin-testing-harness.
* maven-plugin-testing-harness
The
{{{https://svn.apache.org/repos/asf/maven/shared/trunk/maven-plugin-testing-harness/}maven-plugin-testing-harness}}
- is attended to test the
<<<org.apache.maven.reporting.AbstractMavenReport#execute()>>> implementation.
+ is explicitly intended to test the
<<<org.apache.maven.reporting.AbstractMavenReport#execute()>>> implementation.
- Globally, you need to include <<<maven-plugin-testing-harness>>> as
dependency, and create a *MojoTest (by convention)
+ In general, you need to include <<<maven-plugin-testing-harness>>> as
dependency, and create a *MojoTest (by convention)
class which <<<extends AbstractMojoTestCase>>>.
+-----+
@@ -95,15 +109,133 @@
For more information, please refer to
{{{http://docs.codehaus.org/display/MAVEN/Maven+Plugin+Harness}Maven Plugin
Harness Wiki}}
-Integration testing
+Integration/Functional testing
+
+* maven-verifier
+
+ maven-verifier tests are run using JUnit or TestNG, and provide a simple
class allowing you to launch Maven and assert on its log file and built
artifacts. It also provides a ResourceExtractor, which extracts a Maven
project from your src/test/resources directory into a temporary working
directory where you can do tricky stuff with it.
+
+ Maven itself uses maven-verifier to run its core integration tests. For more
information, please refer to
{{{http://docs.codehaus.org/display/MAVEN/Creating+a+Maven+Integration+Test}Creating
a Maven Integration Test}}.
+
++-----+
+public class TrivialMavenVerifierTest extends TestCase
+{
+ public void testMyPlugin ()
+ throws Exception
+ {
+ // Check in your dummy Maven project in /src/test/resources/...
+ // The testdir is computed from the location of this
+ // file.
+ File testDir = ResourceExtractor.simpleExtractResources( getClass(),
"/my-dummy-maven-project" );
+
+ Verifier verifier;
+
+ /*
+ * We must first make sure that any artifact created
+ * by this test has been removed from the local
+ * repository. Failing to do this could cause
+ * unstable test results. Fortunately, the verifier
+ * makes it easy to do this.
+ */
+ verifier = new Verifier( testDir.getAbsolutePath() );
+ verifier.deleteArtifact( "org.apache.maven.its.itsample", "parent",
"1.0", "pom" );
+ verifier.deleteArtifact( "org.apache.maven.its.itsample",
"checkstyle-test", "1.0", "jar" );
+ verifier.deleteArtifact( "org.apache.maven.its.itsample",
"checkstyle-assembly", "1.0", "jar" );
+
+ /*
+ * The Command Line Options (CLI) are passed to the
+ * verifier as a list. This is handy for things like
+ * redefining the local repository if needed. In
+ * this case, we use the -N flag so that Maven won't
+ * recurse. We are only installing the parent pom to
+ * the local repo here.
+ */
+ List cliOptions = new ArrayList();
+ cliOptions.add( "-N" );
+ verifier.executeGoal( "install" );
+
+ /*
+ * This is the simplest way to check a build
+ * succeeded. It is also the simplest way to create
+ * an IT test: make the build pass when the test
+ * should pass, and make the build fail when the
+ * test should fail. There are other methods
+ * supported by the verifier. They can be seen here:
+ * http://maven.apache.org/shared/maven-verifier/apidocs/index.html
+ */
+ verifier.verifyErrorFreeLog();
+
+ /*
+ * Reset the streams before executing the verifier
+ * again.
+ */
+ verifier.resetStreams();
+
+ /*
+ * The verifier also supports beanshell scripts for
+ * verification of more complex scenarios. There are
+ * plenty of examples in the core-it tests here:
+ * http://svn.apache.org/repos/asf/maven/core-integration-testing/trunk
+ */
+ }
++-----+
+
+ <<Note>>: maven-verifier and maven-verifier-plugin sound similar, but are
totally different unrelated pieces of code. maven-verifier-plugin simply
verifies the existence/absence of files on the filesystem. You could use it
for functional testing, but you may need more features than
maven-verifier-plugin provides.
- These tests require the artifact, so they should be run before the
<<<install>>> phase, in the <<<integration-test>>>
- phase.
+* maven-invoker-plugin
+
+ You can use
{{{https://svn.apache.org/repos/asf/maven/sandbox/plugins/maven-invoker-plugin/}maven-invoker-plugin}}
+ to invoke Maven and to provide some BeanShell tests. Tests written in this
way don't run under JUnit/TestNG; instead, they're run by Maven itself.
+
++-----+
+<project
+ xmlns="http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
+ http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ ...
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-invoker-plugin</artifactId>
+ <version>1.0-SNAPSHOT</version>
+ <configuration>
+ <debug>true</debug>
+ <projectsDirectory>src/it</projectsDirectory>
+ <pomIncludes>
+ <pomInclude>**/pom.xml</pomInclude>
+ </pomIncludes>
+ <postBuildHookScript>verify.bsh</postBuildHookScript>
+ </configuration>
+ <executions>
+ <execution>
+ <phase>integration-test</phase>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ ...
+ </plugins>
+ </build>
+ ...
+</project>
++-----+
+
+* shitty-maven-plugin
+
+ The {{{http://mojo.codehaus.org/shitty-maven-plugin/}shitty-maven-plugin}}
(Super Helpful Integration Testing ThingY) provides a simple way to run
integration tests for a project (single module or multi-module).
+
+ shitty-maven-plugin does many of the same things as the maven-invoker-plugin
(it supports Groovy tests instead of BeanShell tests), though it has some
features that aren't available in maven-invoker-plugin. Notably, it provides
some advanced setup steps to automatically install your plugin using a special
version name ("TESTING"), so your dummy projects can depend on that version
explicitly.
* maven-it-plugin
+ <<Note>>: maven-it-plugin is not at 1.0 yet. Use it at your own risk.
+
The
{{{https://svn.apache.org/repos/asf/maven/sandbox/plugins/maven-it-plugin/}maven-it-plugin}}
is used directly
- in the <<<integration-test>>> phase.
+ in the <<<integration-test>>> phase.
<<Note>>: this it plugin can not be used to test a plugin that is being built
for the first time (i.e. with no
release). In this case, you could, for instance, defined an it-snapshot of
the plugin with <<<maven-install-plugin>>>.
@@ -221,7 +353,7 @@
* maven-plugin-test-plugin
The
{{{https://svn.apache.org/repos/asf/maven/sandbox/plugins/maven-plugin-test-plugin/}maven-plugin-test-plugin}}
- is attended to staging/unstaging plugin into local repository for
pre/post-integration-test.
+ is to stage/unstage a plugin into the local repository for
pre/post-integration-test.
You need to configure the <<<maven-plugin-test-plugin>>> and the
<<<maven-surefire-plugin>>>:
@@ -350,48 +482,4 @@
<<Note>>: The
{{{https://svn.apache.org/repos/asf/maven/sandbox/plugins/maven-plugin-management-plugin/}maven-plugin-management-plugin}}
is similar to <<<maven-plugin-test-plugin>>>.
-
-Functional testing
-
-* maven-invoker
-
- The
{{{https://svn.apache.org/repos/asf/maven/sandbox/plugins/maven-invoker-plugin/}maven-invoker-plugin}}
- is attended to invoke maven and to provide some BeanShell tests.
-
-+-----+
-<project
- xmlns="http://maven.apache.org/POM/4.0.0";
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
- http://maven.apache.org/xsd/maven-4.0.0.xsd";>
- ...
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-invoker-plugin</artifactId>
- <version>1.0-SNAPSHOT</version>
- <configuration>
- <debug>true</debug>
- <projectsDirectory>src/it</projectsDirectory>
- <pomIncludes>
- <pomInclude>**/pom.xml</pomInclude>
- </pomIncludes>
- <postBuildHookScript>verify.bsh</postBuildHookScript>
- </configuration>
- <executions>
- <execution>
- <phase>integration-test</phase>
- <goals>
- <goal>run</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- ...
- </plugins>
- </build>
- ...
-</project>
-+-----+
