[
https://issues.apache.org/jira/browse/SUREFIRE-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029745#comment-17029745
]
Albert Johnston edited comment on SUREFIRE-1748 at 2/4/20 10:51 AM:
--------------------------------------------------------------------
Third time's the charm, I know what the issue is. It's pretty late, so I'll
keep this _relatively_ short (meaning no PR and such, this is a long-ish
explanation) and come back to it tomorrow. The issue appears to be that when
the AssertionError message is the empty string (not null), then it messes up
the encoding/decoding process (specifically
ForkedChannelEncoder/ForkedChannelDecoder). This actually goes for both JUnit 5
and JUnit 4, the difference being that JUnit 5 coerces a null message into an
empty string, whereas JUnit 4 does not. So to trigger the issue in JUnit 4, you
have to explicitly pass in an empty string, it cannot simply be null.
Anyways, when you pass in a null message, the message is translated into "-"
during the encoding process. Totally fine. However, when you pass in an empty
string message, it stays as an empty string when encoded. This causes the
surrounding encoding for the message to look like "::". That seems correct,
however this leads to the important part. As a result of some part of this
process (I'm thinking it's the way it's tokenized when decoded), the smart
stack trace is decoded into the message, the stack trace is decoded into the
smart stack trace, and the stack trace is left as null. In other words, the
message is "skipped", causing everything to be read into the property that
comes before it, leaving the stack trace with nothing to fill it. Which, in
turn, leads to the null stack trace writer I mentioned in an earlier comment.
Assuming the issue is something with the decoder's tokenization of the encoded
failure, it's potentially a relatively easy fix. I already attempted to fix it
by simply having the empty string be encoded as "-" as if it was null, but that
breaks other tests, and I suppose is technically incorrect (it isn't actually
null, after all).
Relevant code:
-
[https://github.com/apache/maven-surefire/blob/master/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/booterclient/output/ForkedChannelDecoder.java#L261-L275]
-
[https://github.com/apache/maven-surefire/blob/master/surefire-api/src/main/java/org/apache/maven/surefire/booter/ForkedChannelEncoder.java#L292-L309]
was (Author: ajohnston):
Third time's the charm, I know what the issue is. It's pretty late, so I'll
keep this _relatively_ short (meaning no PR and such, this is a pretty long
explanation) and come back to it tomorrow. The issue appears to be that when
the AssertionError message is the empty string (not null), then it messes up
the encoding/decoding process (specifically
ForkedChannelEncoder/ForkedChannelDecoder). This actually goes for both JUnit 5
and JUnit 4, the difference being that JUnit 5 coerces a null message into an
empty string, whereas JUnit 4 does not. So to trigger the issue in JUnit 4, you
have to explicitly pass in an empty string, it cannot simply be null.
Anyways, when you pass in a null message, the message is translated into "-"
during the encoding process. Totally fine. However, when you pass in an empty
string message, it stays as an empty string when encoded. This causes the
surrounding encoding for the message to look like "::". That seems correct,
however this leads to the important part. As a result of some part of this
process (I'm thinking it's the way it's tokenized when decoded), the smart
stack trace is decoded into the message, the stack trace is decoded into the
smart stack trace, and the stack trace is left as null. In other words, the
message is "skipped", causing everything to be read into the property that
comes before it, leaving the stack trace with nothing to fill it. Which, in
turn, leads to the null stack trace writer I mentioned in an earlier comment.
Assuming the issue is something with the decoder's tokenization of the encoded
failure, it's potentially a relatively easy fix. I already attempted to fix it
by simply having the empty string be encoded as "-" as if it was null, but that
breaks other tests, and I suppose is technically incorrect (it isn't actually
null, after all).
Relevant code:
-
[https://github.com/apache/maven-surefire/blob/master/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/booterclient/output/ForkedChannelDecoder.java#L261-L275]
-
[https://github.com/apache/maven-surefire/blob/master/surefire-api/src/main/java/org/apache/maven/surefire/booter/ForkedChannelEncoder.java#L292-L309]
> JUnit 5 Assertions.fail() breaks reporting
> ------------------------------------------
>
> Key: SUREFIRE-1748
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1748
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support, Maven Surefire Plugin
> Affects Versions: 3.0.0-M4
> Reporter: Albert Johnston
> Priority: Major
>
> Associated versions on my end at time of testing:
> * JUnit Jupiter 5.3.2
> * JDK 1.8u172
> * Maven 3.6.3
> Demonstration available at
> [https://github.com/ajohnstonTE/surefire-jira-examples/tree/master/junit-assertions-fail]
> If the following JUnit 5 code is run through Surefire, then the reporting
> breaks for that class, resulting in no reported errors/failures from that
> class (though any failures will still be reported overall in the counts):
> {code:java}
> public class JUnit5Test {
> @Test
> void failWithNoParameters() {
> Assertions.fail();
> }
> }
> {code}
> Resulting output:
>
> {noformat}
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.016
> s <<< FAILURE! - in JUnit5Test
> [ERROR] JUnit5Test.failWithNoParameters Time elapsed: 0.008 s <<< FAILURE!
> [ERROR] Failures:
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test)
> on project junit-assertions-fail: Execution default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.:
> NullPointerException -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace...{noformat}
> And with the -e flag, trimmed to the relevant part:
>
> {noformat}
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.016
> s <<< FAILURE! - in JUnit5Test
> [ERROR] JUnit5Test.failWithNoParameters Time elapsed: 0.007 s <<< FAILURE!
> [ERROR] Failures:
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test)
> on project junit-assertions-fail: Execution default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.:
> NullPointerException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test
> (default-test) on project junit-assertions-fail: Execution default-test of
> goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.
> Caused by: java.lang.NullPointerException
> at
> org.apache.maven.plugin.surefire.report.DefaultReporterFactory.printTestFailures
> (DefaultReporterFactory.java:398)
> at
> org.apache.maven.plugin.surefire.report.DefaultReporterFactory.runCompleted
> (DefaultReporterFactory.java:198)
> at org.apache.maven.plugin.surefire.report.DefaultReporterFactory.close
> (DefaultReporterFactory.java:171)
> at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run
> (ForkStarter.java:254)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider
> (AbstractSurefireMojo.java:1217)
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked
> (AbstractSurefireMojo.java:1063)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute
> (AbstractSurefireMojo.java:889){noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
