On Tue, May 27, 2014 at 1:14 PM, Qingzhou Luo <[email protected]>wrote:
> Hi, > > I am an intern at Google. The first step of my internship project is to add > the ability to Maven to automatically rerun failing tests a few times, to > see if they ever pass in any of the reruns. It is useful because in many > cases a test fails because it is flaky, not because there is a bug in the > new source code change. > I entirely disagree. 99.9999% of programs are completely deterministic. If you feed them the same inputs, you get the same output. The exceptions are tests that use threads, and it's quite hard to do a good job writing them. Just 'rerunning a few times' does not lead to useful information in my view. if you want random testing, there's a fine alternative at http://labs.carrotsearch.com/randomizedtesting.html, and it has a fine maven plugin. It can already 'rerun your tests a few times' very easily. I wonder what the more involved surefire maintainers think, but as a PMC member I'd be -1 on this proposal. > > We think the right way to achieve this is to modify surefire plugin of > maven. We want to add it as a part of the configuration of surefire, so > users can decide whether they want to enable this feature, and how many > times they want to rerun failing tests. We plan to open-source our > contribution, and hopefully can merge our code into surefire master branch > in the end. Therefore, we are wondering do you have any > comments/suggestions/opinions regarding this? We appreciate any input. > > Thank you very much! > > Regards, > > Qingzhou >
