Package: sbuild
Version: 0.78.1-2
Severity: wishlist
Here's my initial version of the cleaned up patch for adding a
--chroot-mode=systemd-nspawn. Some things I'm not sure about:
- Should we maybe ping upstream and/or Debian maintainers on
https://github.com/systemd/systemd/issues/13297 to see how hard it
would be to get it fixed so I could remove that whole ugly workaround?
(The workaround also only handles bind mount settings at present -
and for example, I've found that a lot of package builds will require
SystemCallFilter=@memlock due to a lot of crypto libraries and
utilities giving errors if they're denied access to mlock. So I would
probably want to add that to my
/etc/systemd/nspawn/unstable-amd64-sbuild.nspawn config file.)
- It currently requires giving sudo access for systemd-run, which
essentially would open up execution of anything desired. And the fact
that it requires NOPASSWD (because some of the commands redirect
stdin/stdout) makes things even worse. And even if you restrict it to
e.g. "systemd-run -M unstable-amd64-sbuild*" it still seems it would
be possible to fool that with something like "sudo systemd-run -M
unstable-amd64-sbuild -M .host ~/myevilcmd".
- If you want to ignore the small patch in lib/Sbuild/Chroot.pm that's
fine. It's not really related to the systemd-nspawn chroot mode.
- It does add a dependency on libipc-run-perl.
- It would be nice (as a future enhancement) if it would be possible
to configure this backend to start the container in --network-veth
mode and set up the host's side of the veth to forward only traffic
to/from apt-cacher-ng on localhost:3142. Not sure how hard that would
be to accomplish.
--
Daniel Schepler
Description: Implement systemd-nspawn chroot mode
Adds a chroot mode using systemd-nspawn to start a container to use
for builds.
Author: Daniel Schepler <dschep...@gmail.com>
--- sbuild-0.78.1.orig/lib/Sbuild/Build.pm
+++ sbuild-0.78.1/lib/Sbuild/Build.pm
@@ -53,6 +53,7 @@ use Sbuild::ChrootInfoSchroot;
use Sbuild::ChrootInfoUnshare;
use Sbuild::ChrootInfoSudo;
use Sbuild::ChrootInfoAutopkgtest;
+use Sbuild::ChrootInfoNspawn;
use Sbuild::ChrootRoot;
use Sbuild::Sysconfig qw($version $release_date);
use Sbuild::Sysconfig;
@@ -422,6 +423,8 @@ sub run_chroot_session {
$chroot_info = Sbuild::ChrootInfoAutopkgtest->new($self->get('Config'));
} elsif ($self->get_conf('CHROOT_MODE') eq 'unshare') {
$chroot_info = Sbuild::ChrootInfoUnshare->new($self->get('Config'));
+ } elsif ($self->get_conf('CHROOT_MODE') eq 'systemd-nspawn') {
+ $chroot_info = Sbuild::ChrootInfoNspawn->new($self->get('Config'));
} else {
$chroot_info = Sbuild::ChrootInfoSudo->new($self->get('Config'));
}
--- sbuild-0.78.1.orig/lib/Sbuild/Chroot.pm
+++ sbuild-0.78.1/lib/Sbuild/Chroot.pm
@@ -244,10 +244,8 @@ sub get_read_file_handle {
my $dir = "/";
$dir = $options->{'DIR'} if defined $options->{'DIR'};
- my $escapedsource = shellescape $source;
-
my $pipe = $self->pipe_command({
- COMMAND => [ "sh", "-c", "cat $escapedsource" ],
+ COMMAND => [ "cat", "--", $source ],
DIR => $dir,
USER => $user,
PIPE => 'in'
--- /dev/null
+++ sbuild-0.78.1/lib/Sbuild/ChrootInfoNspawn.pm
@@ -0,0 +1,68 @@
+package Sbuild::ChrootInfoNspawn;
+
+use Sbuild::ChrootInfo;
+use Sbuild::ChrootNspawn;
+
+use strict;
+use warnings;
+
+BEGIN {
+ use Exporter ();
+ our (@ISA, @EXPORT);
+
+ @ISA=qw(Exporter Sbuild::ChrootInfo);
+
+ @EXPORT = qw();
+}
+
+sub new {
+ my $class = shift;
+ my $conf = shift;
+
+ my $self = $class->SUPER::new($conf);
+ bless($self, $class);
+
+ return $self;
+}
+
+sub get_info_all {
+ my $self = shift;
+
+ my $chroots = {};
+
+ open CHROOTS, '-|', $self->get_conf('MACHINECTL'), 'list-images'
+ or die 'Can\'t run machinectl list-images';
+
+ # skip header line
+ <CHROOTS>;
+
+ while (($_ = <CHROOTS>) ne "\n") {
+ chomp;
+ my @fields = split /\s+/, $_;
+ my $chroot = $fields[0];
+ $chroots->{'chroot'}->{$chroot} = 1;
+ $chroots->{'source'}->{$chroot} = 1;
+ }
+
+ # skip "N images listed"
+ <CHROOTS>;
+
+ close CHROOTS or die "Can't close machinectl list-images pipe";
+
+ $self->set('Chroots', $chroots);
+}
+
+sub _create {
+ my $self = shift;
+ my $chroot_id = shift;
+
+ my $chroot = undef;
+
+ if (defined($chroot_id)) {
+ $chroot = Sbuild::ChrootNspawn->new($self->get('Config'), $chroot_id);
+ }
+
+ return $chroot;
+}
+
+1;
--- /dev/null
+++ sbuild-0.78.1/lib/Sbuild/ChrootNspawn.pm
@@ -0,0 +1,238 @@
+package Sbuild::ChrootNspawn;
+
+use strict;
+use warnings;
+
+use IPC::Run qw(run start finish);
+use IO::Handle;
+use File::Basename qw(basename);
+
+BEGIN {
+ use Exporter ();
+ use Sbuild::Chroot;
+ our (@ISA, @EXPORT);
+
+ @ISA = qw(Exporter Sbuild::Chroot);
+
+ @EXPORT = qw();
+}
+
+sub new {
+ my $class = shift;
+ my $conf = shift;
+ my $chroot_id = shift;
+
+ my $self = $class->SUPER::new($conf, $chroot_id);
+ bless($self, $class);
+
+ return $self;
+}
+
+sub begin_session {
+ my $self = shift;
+ my $chroot = $self->get('Chroot ID');
+
+ return 0 if !defined $chroot;
+
+ my $namespace = undef;
+ if ($chroot =~ m/^(chroot|source):(.+)$/) {
+ $namespace = $1;
+ $chroot = $2;
+ }
+ my @cmd = ($self->get_conf('SUDO'), $self->get_conf('SYSTEMD_NSPAWN'), '-b',
+ '-D', "/var/lib/machines/$chroot",
+ '--console=passive',
+ '--slice=machine-sbuild.slice');
+ push @cmd, '-x' if defined $namespace && $namespace eq 'chroot';
+
+ if ($self->get_conf('DEBUG')) {
+ printf STDERR "running @cmd\n";
+ }
+ my $CONTAINER_STDERR = IO::Handle->new();
+ my $h = start \@cmd,
+ '</dev/null',
+ '>/dev/null',
+ '2>pipe', $CONTAINER_STDERR;
+ my $session_id, $location;
+ if (($_ = <$CONTAINER_STDERR>) =~ m/^Spawning container (\S*) on (.*)\.$/) {
+ $session_id = $1;
+ $location = $2;
+ }
+ else {
+ die "Failed to find expected output from systemd-nspawn";
+ }
+ <$CONTAINER_STDERR>; # consume "press ^] three times" message
+
+ $self->set('Session ID', $session_id);
+ print STDERR "Setting up chroot $chroot (session id $session_id)\n"
+ if $self->get_conf('DEBUG');
+ $self->set('Location', $location);
+ $self->set('Session Purged', 1);
+
+ $self->set('Container Handler', $h);
+ $self->set('Container Process Stderr', $CONTAINER_STDERR);
+
+ # wait for container to be ready to process commands
+ while (1) {
+ print STDERR "Checking for container's system bus availability...\n"
+ if $self->get_conf('DEBUG');
+
+ run [$self->get_conf('SUDO'), $self->get_conf('SYSTEMD_RUN'),
+ '-M', $session_id,
+ '--pipe', '--quiet', '/bin/true'],
+ '</dev/null', '>/dev/null', '2>/dev/null';
+ if ($?) {
+ sleep(1);
+ }
+ else {
+ last;
+ }
+ }
+
+ # work around https://github.com/systemd/systemd/issues/13297:
+ # apply bind mounts (and early in boot process in case some units
+ # within the container have dependencies on them)
+ if ($namespace eq 'chroot') {
+ return 0 if !$self->apply_bind_mounts($chroot);
+ }
+
+ # wait for container to be fully booted before running setup commands
+ print STDERR "Waiting for container to be fully booted...\n"
+ if $self->get_conf('DEBUG');
+ system($self->get_conf('SUDO'), $self->get_conf('SYSTEMD_RUN'),
+ '-M', $session_id,
+ '--pipe', '--quiet', '--property=After=multi-user.target',
+ '/bin/true');
+ if ($?) {
+ print STDERR "systemd-run is failing to connect\n";
+ return 0;
+ }
+
+ return 0 if !$self->_setup_options();
+
+ return 1;
+}
+
+sub apply_bind_mounts {
+ my $self = shift;
+ my $chrootBaseName = shift;
+
+ foreach my $searchdir ('/etc/systemd/nspawn', '/run/systemd/nspawn') {
+ if (-e "$searchdir/$chrootBaseName.nspawn") {
+ open(my $FH, '<', "$searchdir/$chrootBaseName.nspawn");
+ while ($_ = <$FH>) {
+ chomp;
+ if (m/^\s*Bind\s*=\s*(.*)/) {
+ my $bindDir = $1;
+ chomp $bindDir;
+ print STDERR "Bind mounting $bindDir\n"
+ if $self->get_conf('DEBUG');
+ system($self->get_conf('SUDO'), $self->get_conf('MACHINECTL'), 'bind',
+ $self->get('Session ID'), $bindDir);
+ if ($?) {
+ print STDERR "machinectl bind failed\n";
+ close($FH);
+ return 0;
+ }
+ }
+ elsif (m/^\s*BindReadOnly\s*=\s*(.*)/) {
+ my $bindDir = $1;
+ chomp $bindDir;
+ print STDERR "Bind mounting $bindDir read-only\n"
+ if $self->get_conf('DEBUG');
+ system($self->get_conf('SUDO'), $self->get_conf('MACHINECTL'), 'bind',
+ '--read-only', $self->get('Session ID'), $bindDir);
+ if ($?) {
+ print STDERR "machinectl bind --read-only failed\n";
+ close($FH);
+ return 0;
+ }
+ }
+ }
+ close($FH);
+ return 1;
+ }
+ }
+
+ # no config file found
+ return 1;
+}
+
+sub end_session {
+ my $self = shift;
+
+ return if $self->get('Session ID') eq "";
+
+ print STDERR "Cleaning up chroot (session id " . $self->get('Session ID') . ")\n"
+ if $self->get_conf('DEBUG');
+ system($self->get_conf('SUDO'), $self->get_conf('MACHINECTL'), 'poweroff',
+ $self->get('Session ID'));
+ $self->set('Session ID', "");
+ if ($?) {
+ print STDERR "Chroot cleanup failed\n";
+ return 0;
+ }
+
+ my $CONTAINER_STDERR = $self->get('Container Process Stderr');
+ <$CONTAINER_STDERR>; # consume "Container has been shut down" message
+ my $h = $self->get('Container Handler');
+ finish $h;
+ close($CONTAINER_STDERR);
+
+ $self->set('Container Handler', undef);
+ $self->set('Container Process Stderr', undef);
+
+ return 1;
+}
+
+sub get_command_internal {
+ my $self = shift;
+ my $options = shift;
+
+ return if $self->get('Session ID') eq "";
+
+ # Command to run. If I have a string, use it. Otherwise use the list-ref
+ my $command = $options->{'INTCOMMAND_STR'} // $options->{'INTCOMMAND'};
+
+ my $user = $options->{'USER'};
+ my $dir; # Directory to use (optional)
+ $dir = $self->get('Defaults')->{'DIR'} if
+ (defined($self->get('Defaults')) &&
+ defined($self->get('Defaults')->{'DIR'}));
+ $dir = $options->{'DIR'} if
+ defined($options->{'DIR'}) && $options->{'DIR'};
+
+ if (!defined $user || $user eq "") {
+ $user = $self->get_conf('USERNAME');
+ }
+
+ if (!defined($dir)) {
+ $dir = '/';
+ }
+
+ my @cmdline = ($self->get_conf('SUDO'), $self->get_conf('SYSTEMD_RUN'),
+ '-M', $self->get('Session ID'), '--quiet', '--pipe');
+ push @cmdline, "--uid=$user" if $user ne 'root' and $user ne '0';
+ push @cmdline, "--working-directory=$dir" if $dir ne '/';
+ push @cmdline, "--property=PrivateNetwork=yes"
+ if (defined($options->{'DISABLE_NETWORK'}) && $options->{'DISABLE_NETWORK'});
+ foreach my $envvar (sort keys %ENV) {
+ my $envvalue = $ENV{$envvar};
+ push @cmdline, "--setenv=$envvar=$envvalue";
+ }
+ if (ref $command) {
+ push @cmdline, '/usr/bin/env' unless $command->[0] =~ m:^/:;
+ push @cmdline, @$command;
+ } else {
+ push @cmdline, ('/bin/sh', '-c', $command);
+ $command = [split(/\s+/, $command)];
+ }
+
+ $options->{'USER'} = $user;
+ $options->{'COMMAND'} = $command;
+ $options->{'EXPCOMMAND'} = \@cmdline;
+ $options->{'CHDIR'} = undef;
+ $options->{'DIR'} = $dir;
+}
+
+1;
--- sbuild-0.78.1.orig/lib/Sbuild/Conf.pm
+++ sbuild-0.78.1/lib/Sbuild/Conf.pm
@@ -338,6 +338,54 @@ sub setup ($) {
HELP => 'Additional command-line options for autopkgtest-virt-*',
CLI_OPTIONS => ['--autopkgtest-virt-server-opt', '--autopkgtest-virt-server-opts']
},
+ 'MACHINECTL' => {
+ TYPE => 'STRING',
+ GROUP => '__INTERNAL',
+ CHECK => sub {
+ my $conf = shift;
+ my $entry = shift;
+ my $key = $entry->{'NAME'};
+
+ # Only validate if needed.
+ if ($conf->get('CHROOT_MODE') eq 'systemd-nspawn') {
+ $validate_program->($conf, $entry);
+ }
+ },
+ DEFAULT => 'machinectl',
+ HELP => 'Path to machinectl binary'
+ },
+ 'SYSTEMD_NSPAWN' => {
+ TYPE => 'STRING',
+ GROUP => '__INTERNAL',
+ CHECK => sub {
+ my $conf = shift;
+ my $entry = shift;
+ my $key = $entry->{'NAME'};
+
+ # Only validate if needed.
+ if ($conf->get('CHROOT_MODE') eq 'systemd-nspawn') {
+ $validate_program->($conf, $entry);
+ }
+ },
+ DEFAULT => 'systemd-nspawn',
+ HELP => 'Path to systemd-nspawn binary'
+ },
+ 'SYSTEMD_RUN' => {
+ TYPE => 'STRING',
+ GROUP => '__INTERNAL',
+ CHECK => sub {
+ my $conf = shift;
+ my $entry = shift;
+ my $key = $entry->{'NAME'};
+
+ # Only validate if needed.
+ if ($conf->get('CHROOT_MODE') eq 'systemd-nspawn') {
+ $validate_program->($conf, $entry);
+ }
+ },
+ DEFAULT => 'systemd-run',
+ HELP => 'Path to systemd-run binary'
+ },
# Do not check for the existance of fakeroot because it's needed
# inside the chroot and not on the host
'FAKEROOT' => {
@@ -699,10 +747,10 @@ sub setup ($) {
die "Bad chroot mode \'" . $conf->get('CHROOT_MODE') . "\'"
if !isin($conf->get('CHROOT_MODE'),
- qw(schroot sudo autopkgtest unshare));
+ qw(schroot sudo autopkgtest unshare systemd-nspawn));
},
DEFAULT => 'schroot',
- HELP => 'Mechanism to use for chroot virtualisation. Possible value are "schroot" (default), "sudo", "autopkgtest" and "unshare".',
+ HELP => 'Mechanism to use for chroot virtualisation. Possible value are "schroot" (default), "sudo", "autopkgtest" and "unshare" and "systemd-nspawn".',
CLI_OPTIONS => ['--chroot-mode']
},
'CHROOT_SPLIT' => {
--- sbuild-0.78.1.orig/lib/Sbuild/Makefile.am
+++ sbuild-0.78.1/lib/Sbuild/Makefile.am
@@ -33,12 +33,14 @@ MODULES = \
ChrootSudo.pm \
ChrootAutopkgtest.pm \
ChrootUnshare.pm \
+ ChrootNspawn.pm \
ChrootSetup.pm \
ChrootInfo.pm \
ChrootInfoSchroot.pm \
ChrootInfoSudo.pm \
ChrootInfoAutopkgtest.pm \
ChrootInfoUnshare.pm \
+ ChrootInfoNspawn.pm \
Exception.pm \
ResolverBase.pm \
AptitudeResolver.pm \
--- sbuild-0.78.1.orig/man/sbuild.1.in
+++ sbuild-0.78.1/man/sbuild.1.in
@@ -28,7 +28,7 @@ sbuild \- build debian packages from sou
.RB [ \-\-archive=\fIarchive\fP ]
.RB [ \-d \[or] \-\-dist=\fIdistribution\fP ]
.RB [ \-c \[or] \-\-chroot=\fIchroot\fP ]
-.RB [ \-\-chroot-mode=\fIschroot|sudo|autopkgtest|unshare\fP ]
+.RB [ \-\-chroot-mode=\fIschroot|sudo|autopkgtest|unshare|systemd\-nspawn\fP ]
.RB [ \-\-arch=\fIarchitecture\fP ]
.RB [ \-\-arch\-any " \[or] " \-\-no\-arch\-any ]
.RB [ \-\-build=\fIarchitecture\fP ]
@@ -274,12 +274,13 @@ This command line option sets the \fBCHR
.BR sbuild.conf (5)
for more information.
.TP
-.BR "\-\-chroot-mode=\fIschroot|sudo|autopkgtest|unshare\fP"
+.BR "\-\-chroot-mode=\fIschroot|sudo|autopkgtest|unshare|systemd\-nspawn\fP"
Select the desired chroot mode. Four values are possible: schroot (the
default), sudo (which uses sudo to execute chroot in a directory from
/etc/sbuild/chroot or ./chroot), autopkgtest which uses the autopkgtest-virt-* binaries
-(selectable via the \-\-autopkgtest-virt-server option) and unshare (which uses linux
-namespaces for chroot and doesn't require superuser privileges).
+(selectable via the \-\-autopkgtest-virt-server option), unshare (which uses linux
+namespaces for chroot and doesn't require superuser privileges), and
+systemd\-nspawn (which uses systemd\-nspawn to start a temporary container).
See the section
.BR "CHROOT MODES"
for more information.
@@ -1204,7 +1205,7 @@ environment. Provisioning and managing t
sbuild itself but by multiple backends. The default backend (or chroot mode) is
schroot which is an suid binary that allows regular users to enter a chroot
environment. But sbuild also allows one to build packages in a qemu virtual
-machine, lxc, lxd or on a remote host reached by ssh using the autopkgtest
+machine, lxc, lxd, systemd\-nspawn or on a remote host reached by ssh using the autopkgtest
backend. The backend can be chosen using the \f[CB]--chroot-mode\fP command
line argument or the \fI$chroot_mode\fP configuration parameter.
.TP
@@ -1261,6 +1262,17 @@ arbitrary tarballs containing chroot env
building. The default tarball location is in ~/.cache/sbuild/. The expected
names are resolved in the same order as for the schroot chroot mode and can be
overridden using the \-c or \-\-chroot options.
+.TP
+.BR systemd\-nspawn
+This is an experimental backend that uses systemd-nspawn to start an
+ephemeral container. The base chroot images are expected to be in
+subdirectories of /var/lib/machines; the expected names are resolved in the
+same order as for the schroot chroot mode and can be overridden using the
+\-c or \-\-chroot options. Each image needs to have systemd and dbus installed
+(e.g. via the \-\-include=systemd,dbus argument to debootstrap). The process
+of creating the ephemeral cloned image will be somewhat faster if the base
+image is a BTRFS subvolume. The build user needs to have sudo access to
+execute machinectl, systemd\-nspawn, and systemd\-run.
.SH BUILD ARTIFACTS
Sbuild is meant to be used to build architecture specific binary packages from
a given source package. In addition, sbuild is also able to generate
