Package: autopkgtest,ftp.debian.org
Severity: wishlist
The Debian ftp team have ruled that they consider packages in main whose
automated tests download unpackaged programs from outside main to be a
REJECTable problem (reference:REJECT-FAQ.html ยง(Non-Main II)). However,
for some packages both inside and outside main, the ability to download
executable code from outside Debian is a large part of their purpose,
which should ideally be covered by tests.
For example, flatpak, snapd and docker.io are in main, because they
don't depend on third-party programs (it's more like the other way round,
the third-party programs depend on flatpak, snapd or docker.io); but if
we accidentally ship a version of one of those packages that is unable
to download those third-party programs, then its usefulness to our users
would be quite limited, and that's a failure mode that should be
straightforward to detect.
To prevent that sort of thing, I'd like to be able to add an autopkgtest
test-case that asserts that Flatpak *can* download and install third-party
programs, possibly with an appropriate Restriction that means official
Debian infrastructure won't run that test unless the operators of that
infrastructure are happy to do so. But I don't want that potential for
increased test coverage to result in removal from Debian, which would be
quite counterproductive.
For background, in case ftp team members are not familiar with autopkgtest
Restrictions: each test-case can be marked with "Restrictions" flags,
which are things like needs-root and isolation-container. Implementations
of the spec, like autopkgtest itself and devscripts' sadt, will not run a
test that has restrictions they don't understand unless specifically told
to do so, so marking a test with a newly-invented Restrictions flag will
cause that test to not be run by default, while leaving it straightforward
for a maintainer to run it locally before upload if they want to.
Some questions for the ftp team:
1. If flatpak (a package in main) had an autopkgtest that was basically
this, marked with Restrictions: needs-internet:
#!/bin/sh
set -e
flatpak remote-add flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install flathub org.gnome.Recipes
(which will download third-party executable code but not actually
run it), would the ftp team consider that to be valid?
2. Would the answer to (1) change if the test additionally had some new
restriction defined by the autopkgtest spec, perhaps
Restrictions: downloads-third-party-programs or something similar?
3. Are the answers to (1) and (2) different if the test is downloading
works from outside Debian that are not executable code, such as text,
graphics or sound files? (I'm intentionally trying to avoid the word
"software" here to avoid the ambiguity highlighted by GRs in 2004.)
4. Would the answer to (1) change if the test downloaded third-party
executable code, but unlike in (1), the test also *ran* the third-party
executable code? (For example if I added
"flatpak run org.gnome.Recipes --help" to the end of that test, which
ends up running "gnome-recipes --help" in the downloaded container.)
5. Would the answer to (4) change if the test had a restriction like
Restrictions: runs-third-party-programs?
6. For a package in contrib (like game-data-packager) or non-free, are the
ftp team's rules about autopkgtests different? Is it considered more
valid for g-d-p to have an autopkgtest that downloads and/or runs
third-party programs than it would be for flatpak, because g-d-p is
in contrib?
When the ftp team has clarified what they will and will not allow, I would
like to add whatever new Restrictions flags would be useful into
the autopkgtest specification
<https://salsa.debian.org/ci-team/autopkgtest/blob/master/doc/README.package-tests.rst>,
but there doesn't seem to be much point in me preemptively defining new
restrictions if uses of those restrictions would cause packages to be
removed from Debian.
Thanks,
smcv
