Re: Extend surefire to rerun failing tests

2014-06-24 Thread Andreas Gudian
Hi, You introduced the idea a couple of ago on this list already, and IIRC, one main feedback question was why you don't use JUnit's Rules concept for this. I can think of a couple of arguments to rather use Rules instead of the extension in surefire, but I'd like to know your reasoning to go this

Extend surefire to rerun failing tests

2014-06-24 Thread Qingzhou Luo
Hi, I am an intern at Google. The first step of my internship project is to add the ability to Maven to automatically rerun failing tests a few times, to see if they ever pass in any of the reruns. It is useful because in many cases a test fails because it is flaky, not because there is a bug in t

Re: Extend surefire to rerun failing tests

2014-06-01 Thread Andreas Gudian
My first approach would also be to handle this in the test framework, e.g. using the JUnit Rules as Kristian already suggested. There you can decide on a much more fine-grained level what you want to do in case of a failing test: re-run all the tests of the class? Or only those test cases that fai

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Kristian Rosenvold
I would really like to see and understand properly how this would collaborate with JUnit rules for retries. I suspect there might be some interesting issues regarding reporting (i.e. can the existing logic handle reporting of n different executions of a single test). How does this explode i paralle

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Jason van Zyl
I personally don't see an issue with those feature additions, but Kristian does the lions share of the work on the Surefire plugin so it would be his call. I would definitely support the additions of those features and would help test. On May 28, 2014, at 1:46 PM, Qingzhou Luo wrote: > Hi Jaso

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Qingzhou Luo
Hi Jason, Thanks for your reply. We plan to first extend surefire, so users can put a boolean parameter rerunFailintTests and an int parameter rerunFailintTestsCount, to tell surefire whether to rerun failing tests immediately after they fail. If a test passes in any of those reruns if will be mar

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Kristian Rosenvold
2014-05-28 14:41 GMT+02:00 Jason van Zyl : > Maybe a simple mechanism where you store the failed tests in a file and then > have a flag to run only the failed tests. I think this would be very useful. > Then possible a small, configurable loop around the main execution of tests > if you wanted

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Alexander Kriegisch
I have not tried, Kristian, but it looks promising. Both your link and the randomised testing one I have forwarded to my old Scrum team (I am no longer coaching them, thus unable to access their code base and try for myself). Thank you. :-) -- Alexander Kriegisch > Am 28.05.2014 um 09:50 sch

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Jason van Zyl
On May 27, 2014, at 1:14 PM, Qingzhou Luo wrote: > Hi, > > I am an intern at Google. The first step of my internship project is to add > the ability to Maven to automatically rerun failing tests a few times, to > see if they ever pass in any of the reruns. It is useful because in many > cases a

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Kristian Rosenvold
Does this fit the bill for you ? http://stackoverflow.com/questions/8295100/how-to-re-run-failed-junit-tests-immediately Kristian 2014-05-28 9:14 GMT+02:00 Alexander Kriegisch : > > > -- > Alexander Kriegisch > http://scrum-master.de > > Am 28.05.2014 um 05:13 schrieb Benson Margulies : > >> On

Re: Extend surefire to rerun failing tests

2014-05-28 Thread Alexander Kriegisch
-- Alexander Kriegisch http://scrum-master.de Am 28.05.2014 um 05:13 schrieb Benson Margulies : > On Tue, May 27, 2014 at 1:14 PM, Qingzhou Luo wrote: > >> I am an intern at Google. The first step of my internship project is to add >> the ability to Maven to automatically rerun failing tests

Re: Extend surefire to rerun failing tests

2014-05-27 Thread Benson Margulies
On Tue, May 27, 2014 at 1:14 PM, Qingzhou Luo wrote: > Hi, > > I am an intern at Google. The first step of my internship project is to add > the ability to Maven to automatically rerun failing tests a few times, to > see if they ever pass in any of the reruns. It is useful because in many > cases

Extend surefire to rerun failing tests

2014-05-27 Thread Qingzhou Luo
Hi, I am an intern at Google. The first step of my internship project is to add the ability to Maven to automatically rerun failing tests a few times, to see if they ever pass in any of the reruns. It is useful because in many cases a test fails because it is flaky, not because there is a bug in t