Hello, Paul Wise [2016-04-10 17:13 +0800]: > > Assuming this is just a trivial "run target", wouldn't it be even better > > if autopkgtests added support for this directly like it did with > > pkg-perl tests (i.e. adding a "Testsuite: autopkgtest-automake" or so)? > > I don't know enough about DEP-8 to answer this. Thoughts Martin? > > I also note that autodep8 exists and may be the place to put this?
In principle autodep8 could grow an automatic test for this, such as Test-Command: make installcheck-local Restrictions: needs-build, needs-root, isolation-container, breaks-testbed However, this has some caveats: * You always have to build the package as part of the test to even be able to run "make installcheck-local". Depending on the package, this can range anywhere from "trivial" over "nuisance" to "prohibitively expensive". * There is no way to predict whether the test can be run as user, thus this could only default to running as root. * There is no way to predict what kind of features the test expects from the testbed, i. e. whether it is safe to run in a schroot, or won't work in a container, thus again you can only assume the worst and restrict this to "isolation-machine" (i. e. QEMU or bare metal). However, it would then not actually run on our Debian infrastructure, thus we'd rather default to isolation-container, and accept a few failures. * For the same reason we have to assume the worst and declare it to break the testbed, so that we don't run this e. g. via the ssh runner on a non-throwaway machine. * It's not a given that the package's binaries are sufficient to run the test, so it could fail on missing test dependencies. Thus, while possible, this would have the maximum inefficiency. Doing this on a per-package basis means knowing what the installcheck will actually do and then the restrictions can become much softer and also the test dependencies can be adjusted. This of course should also get tried for the packages which have installcheck-local to see what else breaks, before we enable this in production. Martin -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
signature.asc
Description: PGP signature
