[
http://jira.codehaus.org/browse/SUREFIRE-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Fabulich updated SUREFIRE-457:
----------------------------------
Attachment: testNGListenerTest.zip
Jason Chaffee claims that you can get the right thing with onStart/onFinish
notifications. I've attached a sample project that seems to show that this is
false.
http://markmail.org/message/in2iosogjpjev5yk
http://markmail.org/message/yhbzjbhxmcydvqna
I've attached a sample project that simply attaches an ITestListener that logs
to system.out whenever it gets called, specifically during onStart and
onFinish. It's got two test classes: Foo and Bar, each of which have one
method "foo()" and "bar()" respectively. So you'd expect:
onStart [Foo]
onTestStart [foo()]
onTestFinish [foo()]
onFinish [Foo]
onStart [Bar]
onTestStart [bar()]
onTestFinish [bar()]
onFinish [Bar]
But here's what you get instead:
onStart
onTestStart [foo()]
onTestFinish [foo()]
onTestStart [bar()]
onTestFinish [bar()]
onFinish
... not too useful, is it? In fact, onStart is called at the start/end of the
"test" which is TestNG's very silly term for a collection of tests that isn't a
"group" or a "suite".
> TestNG should log on the console before/after every test class
> --------------------------------------------------------------
>
> Key: SUREFIRE-457
> URL: http://jira.codehaus.org/browse/SUREFIRE-457
> Project: Maven Surefire
> Issue Type: Wish
> Components: TestNG support
> Affects Versions: 2.4, 2.4.1, 2.4.2
> Reporter: Dan Fabulich
> Priority: Minor
> Fix For: Future
>
> Attachments: testNGListenerTest.zip
>
>
> In Surefire 2.3.x, when running the tests in "directory mode" (i.e. without a
> suite.xml file) we used to run each TestNG class as a separate suite. As a
> result, we logged to the console before/after every test class.
> This behavior totally broke tests that had rich interdependencies (which is a
> big part of the point of TestNG), so in Surefire 2.4 we handed more control
> over to TestNG. As a result, we let TestNG run the entire directory at once,
> and we aren't notified when classes start/end. Since we aren't notified, we
> can't log to the console before/after every test class. But this looks like
> a regression to those who were used to the 2.3.x behavior.
> (This is trickier than it looks, because TestNG can/will run test methods
> entirely out of order, running part of class X, and then part of class Y,
> then back to class X, then a bit of class Z, etc. And that's not even
> considering parallelized testing.)
> I argue that to fix this "right" we'd need TestNG to notify us when classes
> start/end; we should pressure the TestNG team to provide this functionality.
> Benjamin has argued that there should be an option to run each test in its
> own suite; I think that option is confusing and error-prone.
> http://www.nabble.com/Re%3A-Test-Suites%2C-Ant%2C-Surefire%2C-and-JunitReport-p15094586.html
> If you think you want a one-class-per-suite option, I offer this remark: Many
> TestNG tests aren't safe to run in one-class-per-suite mode. If your tests
> are safe to run in that mode, then they're also safe to run in JUnit 4. (In
> fact, if your tests are that simple, you probably aren't using any of
> TestNG's unique features, and you should prefer to use JUnit 4.) So, as a
> workaround, don't use TestNG; use JUnit 4 instead.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
