Simon McVittie writes ("Bug#901804: autopkgtest: consider using exit status 8
("no tests found") if every test was ignored"):
> On the Gitlab MR, Paul Gevers wrote:
> > And maybe a regression from 0 to 2 (without changes to the autopkgtest
> > that runs) should also be a regression for britney (if it is not caused
> > by a flaky test :( ).
>
> I'm not so sure about this. If debci was previously giving tests
> a particular capability bounding set in their LXC containers, and a
> security fix to lxc removes a capability that turns out to be exploitable,
> then tests that were exercising that capability will have to be skipped
> (ideally they'd already be 'skippable', and exit 77 if they don't have
> it), so they'd appear to "regress". That seems undesirable.
Despite what I wrote in my other mail in the pseudo-restriction
thread, I agree with your analysis.
The *point* of SKIP is that it isn't a regression or a failure. If a
feature is removed, and its absence causes SKIP, and this ought to be
treated as a regression, then the feature ought to have its own test
(perhaps, somewhere entirely else than the package whose test is now
SKIPped).
Ian.
--
Ian Jackson <ijack...@chiark.greenend.org.uk> These opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
