On Thu, Nov 17, 2016 at 2:16 AM, Michael Palimaka <kensing...@gentoo.org> wrote:
>
> In cases where all USE flags combinations are not being tested, it is
> still recommended to test:
> * with all USE flags enabled
> * with all USE flags disabled
> * the default USE flag settings
>
I imagine that in practice only the last of these really tends to get tested.
> * A leaf package such as {{package|kde-apps/kcalc}} may not require any
> runtime testing at all
I'm not really a big fan of this, but if we REALLY didn't want to do
any runtime testing on a package then we should have some way to tag
the package as such, and then have some way to do automated
build-test-only stabilization. If you aren't doing runtime testing
then manual stabilization adds zero value.
Overall though the writeup was good and maybe it will trigger some
discussion. I tend to think that if we want to do things like testing
permutations and such then automated build-only tools might be the way
to address this. Manual effort should be focused on things like
runtime testing where it adds the most value. This also strikes me as
the sort of thing that could probably be assigned out to volunteers
who do not have commit access.
It really seems like the sort of thing that could be managed by
something other than bugzilla. Some tool finds out about packages
that ought to be stabilized (probably via multiple methods), then it
triggers the automated build tests/etc that do a lot of the low-level
QA, and if the package looks good it gets queued for runtime testing.
Then volunteers report in on status and when whatever criteria we
establish is met then the tool stabilizes the package, probably in a
dependency-aware fashion. Obviously this would require some care for
coordinated packages like xorg/DEs/etc, and it might not be the
preferred approach for many system packages.
--
Rich
