winglam opened a new pull request, #560: URL: https://github.com/apache/maven-surefire/pull/560
This PR is a new implementation for [PR#348](https://github.com/apache/maven-surefire/pull/348) and [PR#495](https://github.com/apache/maven-surefire/pull/495). The changes in this pull request enable Surefire to run test classes and test methods in any order specified by a newly added [<runOrder>](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder) (testorder). [PR#348](https://github.com/apache/maven-surefire/pull/348) attempted to implement this feature but @Tibor17 [suggested](https://github.com/apache/maven-surefire/pull/348#issuecomment-1022724116) that such a feature should be implemented differently. [PR#495](https://github.com/apache/maven-surefire/pull/495) supports only test class ordering, while this PR supports test class ordering and test method ordering within each test class. Overall, the changes include - A new [<runOrder>](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder) called testorder that would make Surefire run test classes and test methods in the order specified by [<test>](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#test). Regex and include/exclude are supported as is the case for other runOrders - Tests for the newly added features The newly added testorder [<runOrder>](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder) is supported for JUnit 3 and JUnit 4 tests. We have also implemented support for JUnit 5, but we are unsure on how we should get and sort all the test methods before sorting them for JUnit 5. Unlike JUnit 4, JUnit5 does not provide a interface like `org.junit.runner.Request#sortWith()` in JUnit4 to help us sort the methods. To help with this issue, we current rely on Java reflection to get the methods and then sort them. For this reason, we created another [PR](https://github.com/amamiya-yuuko-1225/maven-surefire/pull/3) to show that change, but we left the changes out of this PR. Please let us know if you have any suggestions for supporting JUnit 5. **Simple (no regex) example using [Http-Request](https://github.com/kevinsawicki/http-request):** ``` mvn test -Dsurefire.runOrder=testorder \ -Dtest=com.github.kevinsawicki.http.HttpRequestTest#getUrlEncodedWithPercent,\ com.github.kevinsawicki.http.HttpRequestTest#uploadProgressSendReader,\ com.github.kevinsawicki.http.HttpRequestTest#malformedStringUrlCause,\ com.github.kevinsawicki.http.EncodeTest#encode ``` Output from Maven should say that the test class `HttpRequestTest` ran before `EncodeTest` ``` Tests run: 3 … in com.github.kevinsawicki.http.HttpRequestTest Tests run: 1 … in com.github.kevinsawicki.http.EncodeTest ``` The file `HttpRequestTest.xml` (`http-request/lib/target/surefire-reports/TEST-com.github.kevinsawicki.http.HttpRequestTest.xml`) should say ``` <testcase name="getUrlEncodedWithPercent"...> <testcase name="uploadProgressSendReader"...> <testcase name="malformedStringUrlCause...> ``` **Regex example using [Http-Request](https://github.com/kevinsawicki/http-request):** ``` mvn test -Dsurefire.runOrder=testorder \ -Dtest=com.github.kevinsawicki.http.EncodeTest*,\ com.github.kevinsawicki.http.HttpRequestTest#getUrl* ``` Output from Maven should say ``` Tests run: 2 … in com.github.kevinsawicki.http.EncodeTest Tests run: 4 … in com.github.kevinsawicki.http.HttpRequestTest ``` The file `HttpRequestTest.xml` (`http-request/lib/target/surefire-reports/TEST-com.github.kevinsawicki.http.HttpRequestTest.xml`) should say ``` <testcase name=”getUrlEncodedWithPercent”...> <testcase name="getUrlEncodedWithSpace"...> <testcase name="getUrlEncodedWithUnicode”...> <testcase name="getUrlEmpty”...> ``` **Include/Exclude example using [Http-Request](https://github.com/kevinsawicki/http-request):** ``` mvn test -Dsurefire.runOrder=testorder \ -Dtest=com.github.kevinsawicki.http.HttpRequestTest#getUrl*,\ \!com.github.kevinsawicki.http.HttpRequestTest#getUrlEmpty ``` Output from Maven should say ``` Tests run: 3 … in com.github.kevinsawicki.http.HttpRequestTest ``` The file `HttpRequestTest.xml` (`http-request/lib/target/surefire-reports/TEST-com.github.kevinsawicki.http.HttpRequestTest.xml`) should say ``` <testcase name=”getUrlEncodedWithPercent”...> <testcase name="getUrlEncodedWithSpace"...> <testcase name="getUrlEncodedWithUnicode”...> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
