[
https://issues.apache.org/jira/browse/SUREFIRE-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16985711#comment-16985711
]
Falko Modler commented on SUREFIRE-1728:
----------------------------------------
{quote}
you want to express an error or failure message in the XML or TXT report.
{quote}
Not necessarily. I mean yes, this would be one option but I was thinking more
whether the plugin could just cause a build failure like _without_
{{maven.test.failure.ignore}}.
In short:
# {{maven.test.failure.ignore=true}} + timeout: build failure, test results up
to the point of the timeout can be parsed from the reports
# {{maven.test.failure.ignore=true}} + test failure: _no_ build failure, all
results can be parsed from the reports
# {{maven.test.failure.ignore=false}} + timeout: same as 1.
# {{maven.test.failure.ignore=false}} + test failure: build failure, test
results can be parsed from the reports
In case scenario/variant 1. is too much of a "breaking change", the value for
{{maven.test.failure.ignore}} could be extended to something like {{true/all}}
| {{failures}} | {{false/none}}.
Just a thought...
> maven.test.failure.ignore: differentiate between test failure and timeout
> -------------------------------------------------------------------------
>
> Key: SUREFIRE-1728
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1728
> Project: Maven Surefire
> Issue Type: Improvement
> Affects Versions: 2.22.2, 3.0.0-M3
> Environment: Maven 3.6.2
> Reporter: Falko Modler
> Assignee: Tibor Digana
> Priority: Major
> Fix For: 3.0.0-M5
>
>
> On a build server like Jenkins, people typically set
> {{-Dmaven.test.failure.ignore=true}} to get the maximum number of test
> results instead of failing the build after the first module with test
> failures.
> Unfortunately, timeouts are also ignored when this property is activated,
> leaving the Jenkins JUnit plugin no chance to detect that something went
> wrong (because a timeout is not reported in a txt or xml report).
> See also [JENKINS-46553|https://issues.jenkins-ci.org/browse/JENKINS-46553].
> The two cases should be differentiated.
> Due to backward compatibility reasons, I am not sure whether it would be wise
> to simply exclude timeout cases.
> One backward compatible solution might be to extend the value range of
> {{maven.test.failure.ignore}} from just {{true}} XOR {{false}} to something
> like:
> {{true}}/{{all}} XOR {{failure}} XOR {{false}}.
> The alternative would be to introduce yet another property...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
