On Thu, 17 Nov 2016 18:16:27 +1100 Michael Palimaka <kensing...@gentoo.org> wrote:
> ==== Runtime testing ====
>
> Consider the level of runtime testing that is required for the target
> package. Remember, the focus of stabilisation is to integrate a testing
> ebuild into the stable tree and not to identify routine bugs or
> regressions - that is the purpose of the package's waiting time in ~arch.
>
> The level of runtime testing required will vary wildly based on a
> variety of factors. Consider the following examples:
>
> * Multiple days of "normal use" testing may be appropriate for a new
> version of {{package|sys-libs/glibc}}
> * Basic functionality testing, such as browsing some web pages, may make
> sense for a new version of {{package|www-client/firefox}}
> * Passing tests might be enough for {{package|dev-python/yenc}}
> * A leaf package such as {{package|kde-apps/kcalc}} may not require any
> runtime testing at all
Could we maybe include some place (metadata.xml?) to state what is
the best way to test a package? I'm thinking it could include things
like:
- whether the test of the package are reliable,
- whether runtime testing is required and what kind of,
- how likely it is that revdeps need to be checked.
For example, in LLVM I would like to ask arch testers to always check
a few common clang calls.
--
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>
pgpVj3PEJEKHh.pgp
Description: OpenPGP digital signature
