On Fri, 20 Apr 2018 at 11:39:49 +0100, Simon McVittie wrote:
> On Sun, 04 Mar 2018 at 11:00:19 +0100, SZALAY Attila wrote:
> > <VirtSubproc>: failure: timed out on client shared directory setup
>
> I've been seeing this a lot recently. The attached patches are not a
> full solution (it still sometimes fails, particularly under load) but
> they seem to help.
I hacked in some extra debug logging and it looks like this is caused
by the base image (/dev/vdb) being used as the root filesystem instead of
the intended main image (/dev/vda). This is basically #862362, for which
I attached a patch last year. In principle the base image should be
something that we only set up in ways that can work reliably, but since
the reality seems to be different, I think it should be opt-in (open-iscsi
seems to be the only package that benefits from it).
Logs below, proposed patch attached. My earlier patches on this bug might
still be beneficial: reviews welcome.
Regards,
smcv
During a boot where the filesystem was set up successfully:
Apr 25 09:28:24 host kernel: vda: vda1
...
Apr 25 09:28:24 host kernel: EXT4-fs (vda1): re-mounted. Opts: errors=remount-ro
Apr 25 09:28:24 host systemd-fsck[157]: /dev/vda1: clean, 18906/610800 files,
220285/2440960 blocks
During a boot where it was not:
Apr 25 09:29:48 host kernel: vda: vda1
Apr 25 09:29:48 host kernel: vdb: vdb1
...
Apr 25 09:29:48 host systemd-fsck[182]: ^[[0;1;31mfsck failed with error code
8.^[[0m
Apr 25 09:29:48 host systemd-fsck[182]: ^[[0;1;39mIgnoring error.^[[0m
Apr 25 09:29:48 host systemd-remount-fs[189]: ^[[0;1;31m/bin/mount for / exited
with exit status 32.^[[0m
Apr 25 09:29:48 host systemd-fsck[182]: fsck.ext4: Operation not permitted
while trying to open /dev/vdb1
Apr 25 09:29:48 host systemd-fsck[182]: You must have r/w access to the
filesystem or be root
Apr 25 09:29:48 host systemd-remount-fs[189]: mount: /: cannot remount
/dev/vdb1 read-write, is write-protected.
...
Apr 25 09:29:48 host systemd-tmpfiles[214]:
^[[0;1;31msymlink(../proc/self/mounts, /etc/mtab) failed: Read-only file
system^[[0m
...
Apr 25 09:29:48 host systemd-udevd[209]: ^[[0;1;31mcould not open moddep file
'/lib/modules/4.15.0-3-amd64/modules.dep.bin'^[[0m
5e431b1d-7f238568# PS1="6a5f613a"-"4c3784c3# "; mkdir -p -m 1777
/run/autopkgtest/shared &&
mount -t 9p -o trans=virtio,access=any autopkgtest /run/autopkgtest/shared &&
chmod 1777 /run/autopkgtest/shared &&
touch /run/autopkgtest/shared/done_shared
> > > mount: /run/autopkgtest/shared: unknown filesystem type '9p'.
6a5f613a-4c3784c3#
>From 06f687aba87ba07f2295503298b07755c4b47b47 Mon Sep 17 00:00:00 2001
From: Simon McVittie <s...@collabora.com>
Date: Thu, 11 May 2017 17:26:36 +0100
Subject: [PATCH] Only set up base image device if requested
Adding a second virtual disk with the same UUIDs, partition labels,
filesystem labels etc. can interfere with anything that mounts filesystems
post-boot, and the attempts to address this in #842299 do not seem to
have been completely successful. Nested virtualization is a rare thing
to need, so let's make this opt-in.
Signed-off-by: Simon McVittie <s...@collabora.com>
---
virt/autopkgtest-virt-qemu | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/virt/autopkgtest-virt-qemu b/virt/autopkgtest-virt-qemu
index d89536c..b7c50e7 100755
--- a/virt/autopkgtest-virt-qemu
+++ b/virt/autopkgtest-virt-qemu
@@ -88,6 +88,11 @@ def parse_args():
help='Enable debugging output')
parser.add_argument('--qemu-options',
help='Pass through arguments to QEMU command.')
+ parser.add_argument('--baseimage', action='store_true', default=False,
+ help='Provide a read-only copy of the base image at /dev/baseimage')
+ parser.add_argument('--no-baseimage', dest='baseimage',
+ action='store_false',
+ help='Provide a read-only copy of the base image at /dev/baseimage')
parser.add_argument('image', nargs='+',
help='disk image to add to the VM (in order)')
@@ -567,7 +572,8 @@ def hook_open():
# files; let QEMU run with the deleted inode
os.unlink(overlay)
setup_shell()
- setup_baseimage()
+ if args.baseimage:
+ setup_baseimage()
setup_shared(shareddir)
setup_config(shareddir)
make_auxverb(shareddir)
@@ -606,11 +612,12 @@ def hook_cleanup():
def hook_prepare_reboot():
- # Remove baseimage drive again, so that it does not break the subsequent
- # boot due to the duplicate UUID
- monitor = VirtSubproc.get_unix_socket(os.path.join(workdir, 'monitor'))
- monitor.send(b'device_del virtio-baseimage\n')
- VirtSubproc.expect(monitor, b'(qemu)', 10)
+ if args.baseimage:
+ # Remove baseimage drive again, so that it does not break the subsequent
+ # boot due to the duplicate UUID
+ monitor = VirtSubproc.get_unix_socket(os.path.join(workdir, 'monitor'))
+ monitor.send(b'device_del virtio-baseimage\n')
+ VirtSubproc.expect(monitor, b'(qemu)', 10)
def hook_wait_reboot():
@@ -620,7 +627,8 @@ def hook_wait_reboot():
wait_boot()
setup_shell()
setup_shared(shareddir)
- setup_baseimage()
+ if args.baseimage:
+ setup_baseimage()
def hook_capabilities():
--
2.17.0
