On Sun, Nov 28, 2021 at 11:43:13PM -0500, Sandro Tosi wrote: > Package: autopkgtest > Version: 5.16 > Severity: normal > X-Debbugs-Cc: mo...@debian.org > > Hello, > There are at least 2 main places where source packages declare tests: > > 1. in debian/rules, to be executed at build-time > 2. in debina/tests/*, to be executed post-build via autopkgtests > > Usually 1) are written first, because you need to get them to pass during > build-time; this activity also require to add additional dependencies to > debian/control, which are only required by tests (and not by the package > building commands). > > And then there's 2), where often you're required to duplicate the steps from > 1) > and that could lead to misalignment, forgotten dependencies (and so > failures), > etc. > > In general, it's preferred to reduce duplication to the minimum, just to avoid > the problems listed above, but autopkgtest kinda requires you to have exactly > this duplication. > > F.e., i make sure to mark <!nocheck> all the b-d only needed for tests, but > there's no selector in debian/tests/control to only pull in those packages, > and > sometimes the quickest way is to get all @builddeps@, even if that has the > risk > to include extra packages in the mix. > > There's also the problem of duplicating how we run tests; while autopkgtest > is less restrictive than the buildd environment (f.e. via allow-network), > there > could be substantial duplication on how test runners are setup in d/rules and > in autopkgtest. > > I dont know if you ever thought about the duplication problem, i feel it would > be really helpful if that could be substantially reduced.
This is usually solved outside of autopkgtest itself.
For example Ruby packages have a single place where tests are specified
(debian/ruby-tests.*{rb,rake,yaml}, and the appropriate
debian/tests/control file is autogenerated by autodep8. I would say 99%
of Ruby packages require no duplication, or even explicitly specifying
commands to run tests at all.
In general, this is most efficiently solved by package type (e.g.
programming language), because the way the tests are run at build-time
usually is tied to the build system. You then usually need some test
runner, probably extracted from, or provided by, the build system, to
also provide the same funcionality in an autopkgtest-compatible way.
All the package types that are well supported in autodep8 have their
specialized test runner tools that avoid this type of duplication.
signature.asc
Description: PGP signature
