On Mon, Nov 04, 2019 at 11:40:08AM +0100, Thomas Goirand wrote: > Hi Evgeny, > > As elixir-lang is going to be AUTORM if nobody takes care of it, I > already added a bit of code in debian/rules to make it retry unit tests > if they fail. It will retry 3 times, meaning that if it used to fail 10% > of the time, making it retry 3 times makes the unit tests fail 0.01% of > the times, which is IMO acceptable. The recent failures on some arch > shows that my debian/rules hack works well! :)
Please don't do that sort of thing. It makes things gratuitously painful for people like me doing QA. Either a unit test is to be trusted or it's not (by "trusted" I mean here that a failure in the test means the package is unsuitable for release). If the test is trusted but it fails, accept that it fails and do not let the package to propagate to testing. If it's not trusted, there is no point in making it to fail with any probability. Not 10%, and not 0.01%. Just disable the test. Thanks.
