On 11/4/19 12:11 PM, Santiago Vila wrote: > On Mon, Nov 04, 2019 at 11:40:08AM +0100, Thomas Goirand wrote: >> Hi Evgeny, >> >> As elixir-lang is going to be AUTORM if nobody takes care of it, I >> already added a bit of code in debian/rules to make it retry unit tests >> if they fail. It will retry 3 times, meaning that if it used to fail 10% >> of the time, making it retry 3 times makes the unit tests fail 0.01% of >> the times, which is IMO acceptable. The recent failures on some arch >> shows that my debian/rules hack works well! :) > > Please don't do that sort of thing. It makes things gratuitously painful > for people like me doing QA. > > Either a unit test is to be trusted or it's not (by "trusted" I mean > here that a failure in the test means the package is unsuitable for > release). > > If the test is trusted but it fails, accept that it fails and do not > let the package to propagate to testing. > > If it's not trusted, there is no point in making it to fail with any > probability. Not 10%, and not 0.01%. Just disable the test. > > Thanks.
I don't agree with this view. Yes, this makes your work harder. But running tests improve the quality of the package. Removing them is in no way an improvement. Thomas Goirand (zigo)
