Package: autopkgtest
Version: 4.3
Severity: wishlist
Tags: patch
There is some controversy over the extent to which it is OK for
packages to touch the network at build-time, in terms of privacy
(leaking information), determinism of builds, robustness against network
services failing and so on. (See also #833503, #850988). autopkgtest's
Restrictions and Features mechanisms give it the opportunity to do better:
one interpretation can be the default, and the other can be selected
by selectively ignoring requirements or requiring features.
As background for this, I recently implemented tests in libnss-mdns[1]
which add and remove packages, which was necessary to test the postinst
and its interactions with libnss-resolve. #786039 suggests that this is
considered to be OK in autopkgtest.
[1] https://anonscm.debian.org/cgit/collab-maint/nss-mdns.git/tree/debian/tests
I attach some initial patches for discussion, based on the
assumption that Restrictions are the right way to do it. This could be
used in conjunction with the patches on #850494, for example running
"autopkgtest --ignore-restrictions=uses-network ..." on CI infrastructure
whose privacy is unimportant.
The uses-network restriction could potentially be supplemented with
less "powerful" uses-dns and uses-web restrictions, if people think
that's valuable. The nss-mdns tests would only need uses-dns and
reconfigures-apt in that case.
If you think it is normally OK for a test to access the network or
reconfigure apt, then this could be recast in terms of Features,
with a corresponding --require-features command-line option - but
I'd like it to be explicit one way or the other.
S
>From a774945fe7d0c9a3ddc0c7e32ef1f25cbbaf18b6 Mon Sep 17 00:00:00 2001
From: Simon McVittie <s...@debian.org>
Date: Mon, 16 Jan 2017 00:29:19 +0000
Subject: [PATCH 1/2] doc: Define new uses-network restriction
nss-mdns' tests should have this restriction, because it asserts that
nss-mdns-test-$(dbus-uuidgen).nonexistent.invalid doesn't exist.
---
doc/README.package-tests.rst | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/doc/README.package-tests.rst b/doc/README.package-tests.rst
index 4775a07..3117f3d 100644
--- a/doc/README.package-tests.rst
+++ b/doc/README.package-tests.rst
@@ -210,6 +210,13 @@ needs-recommends
Enable installation of recommended packages in apt for the test
dependencies. This does not affect build dependencies.
+uses-network
+ The test might contact other machines, so is not suitable to
+ be run in an environment where privacy is essential.
+ Tests should avoid relying on remote network services, but in
+ some cases a test attempts to contact the network as a side-effect,
+ for example resolving a hostname in DNS.
+
Defined features
----------------
--
2.11.0
>From 36edf0678a7d3d8490fb44059221d4de10f40ed4 Mon Sep 17 00:00:00 2001
From: Simon McVittie <s...@debian.org>
Date: Mon, 16 Jan 2017 00:34:33 +0000
Subject: [PATCH 2/2] doc: Add restrictions for using or reconfiguring apt
nss-mdns should use this in its autopkgtests, which exercise
different orderings for installation of libnss-mdns vs. libnss-resolve,
and make sure that upgrading from jessie's libnss-mdns works.
---
doc/README.package-tests.rst | 15 +++++++++++++++
runner/autopkgtest | 4 +++-
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/doc/README.package-tests.rst b/doc/README.package-tests.rst
index 3117f3d..bac0a91 100644
--- a/doc/README.package-tests.rst
+++ b/doc/README.package-tests.rst
@@ -217,6 +217,21 @@ uses-network
some cases a test attempts to contact the network as a side-effect,
for example resolving a hostname in DNS.
+needs-apt
+ The test needs to use apt to install or remove packages during
+ testing. The configured ``Depends`` will be installed initially,
+ but other packages might be added or removed during testing.
+ The test should also declare ``needs-root``.
+
+reconfigures-apt
+ The test reconfigures the apt sources, for example to enable an
+ older suite and test an upgrade. This restriction implies
+ ``needs-apt``. The test should also declare ``needs-root``.
+ A test with this restriction may assume that the configured
+ apt mirror has the other suites that would normally be available
+ from the same distro: for example, a test running in Debian
+ unstable might rely on the same mirror having Debian stable.
+
Defined features
----------------
diff --git a/runner/autopkgtest b/runner/autopkgtest
index 4bed5cb..12edc1d 100755
--- a/runner/autopkgtest
+++ b/runner/autopkgtest
@@ -163,7 +163,9 @@ def run_tests(tests, tree):
opts.build_parallel)
if not t.result:
errorcode |= 4
- if 'breaks-testbed' in t.restrictions:
+ if ('breaks-testbed' in t.restrictions or
+ 'needs-apt' in t.restrictions or
+ 'reconfigures-apt' in t.restrictions):
testbed.needs_reset()
testbed.needs_reset()
--
2.11.0
