Aslak Hellesøy created SUREFIRE-2041:
----------------------------------------
Summary: Order test classes according to -Dtest
Key: SUREFIRE-2041
URL: https://issues.apache.org/jira/browse/SUREFIRE-2041
Project: Maven Surefire
Issue Type: New Feature
Components: Maven Surefire Plugin
Reporter: Aslak Hellesøy
I'm working on a Test Case Prioritization tool that uses machine learning to
predict what tests are most likely to fail (based on historic changes and test
results). The output from this tool is a list of test names, ordered by
decreasing probability of failure. This tool can provide the order as test
classes or as test methods. For example:
Class order:
* testing.Y
* testing.X
Method order:
* testing.Y#a
* testing.X#c
* testing.Y#b
Ideally, it would be possible to launch surefire with a test order that uses a
list of tests to indicate what order they should be run in, e.g:
{{# class order}}
{{mvn test -Dsurefire.runOrder=test -Dtest=testing.Y,testing.X}}
{{# method order}}
{{mvn test -Dsurefire.runOrder=test -Dtest=testing.Y#a,testing.X#c,testing.Y#b}}
>From what I can tell, the current design of Surefire wouldn't allow the kind
>of method ordering illustrated above. It cannot run {*}testing.Y#a{*}, then
>{*}testing.X#c{*}, then {*}testing.Y#b{*}. There is an inherent assumptions in
>*TestsToRun* that the overall ordering is by class.
I would therefore be ok with just class ordering for now.
Previous relates issues and pull requests:
* https://issues.apache.org/jira/browse/SUREFIRE-1405
* https://github.com/apache/maven-surefire/pull/169
* https://github.com/apache/maven-surefire/pull/348
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
