Hi, Quoting Johannes Schauer Marin Rodrigues (2022-05-12 22:16:16) > On Fri, 08 Nov 2019 23:59:48 +0100 "Francesco Poli (wintermute)" > <invernom...@paranoici.org> wrote: > > Hello and thanks for maintaining autopkgtest! > > > > I wanted to give the QEMU/KVM testbed a try. Hence I tried to create > > a VM image by using autopkgtest-build-qemu. Its man page states: > > > > [...] > > | Note that you need to call this as root. > > [...] > > > > And indeed the command > > > > $ autopkgtest-build-qemu unstable ~/var/cache/autopkgtest/sid.img > > > > fails, when issued by a regular user, as it cannot even find parted > > in the search PATH. > > > > I guess this is because of vmdb2, which requires superuser privileges. > > But why? > > Is there any hope to improve vmdb2 or to use another tool, in order > > to create a KVM testbed without requiring superuser privileges? > > since you reported #944485 against mmdebstrap you already know that the answer > is "Yes" but I just stumbled across this bug and wanted to share one possible > solution with the readers here as well. > > How to do it is described in the mmdebstrap man page under "Use as replacement > for autopkgtest-build-qemu and vmdb2" and the TLDR is: > > $ mmdebstrap --variant=important --include=linux-image-amd64 \ > --customize-hook='chroot "$1" passwd --delete root' \ > --customize-hook='chroot "$1" useradd --home-dir /home/user > --create-home user' \ > --customize-hook='chroot "$1" passwd --delete user' \ > --customize-hook='echo host > "$1/etc/hostname"' \ > --customize-hook='echo "127.0.0.1 localhost host" > "$1/etc/hosts"' \ > --customize-hook=/usr/share/autopkgtest/setup-commands/setup-testbed \ > unstable debian-unstable.tar > $ cat << END > extlinux.conf > > default linux > > timeout 0 > > > > label linux > > kernel /vmlinuz > > append initrd=/initrd.img root=/dev/vda1 console=ttyS0 > END > $ guestfish -N debian-unstable.img=disk:8G -- \ > part-disk /dev/sda mbr : \ > part-set-bootable /dev/sda 1 true : \ > mkfs ext2 /dev/sda1 : mount /dev/sda1 / : \ > tar-in debian-unstable.tar / xattrs:true : \ > extlinux / : copy-in extlinux.conf / : \ > sync : umount / : shutdown > $ qemu-img convert -O qcow2 debian-unstable.img debian-unstable.qcow2 > > Unfortunately, extlinux is only available on amd64 and i386. To support more > architectures we have to boot using grub. Because this becomes a bit more > complicated I put the whole thing into a script: > > https://salsa.debian.org/debian/mmdebstrap/-/blob/master/mmdebstrap-autopkgtest-build-qemu > > This now works on amd64, arm64, armhf, i386 and ppc64el. But it only works for > building *native* qemu images because guestfish cannot handle foreign > architecture emulation. I see five options that allow avoiding guestfish for > building foreign architecture bootable images without being root. > > Option 1 is to run guestfish inside another qemu instance. This is extremely > slow though. Since it's also easiest to set-up I won't go further into this > option here. > > Option 2 is to do the partition setup and bootloader installation as an > initramfs hook script: > > main script: http://paste.debian.net/1235312/ > initramfs-hook: http://paste.debian.net/1235313/ > initramfs-script: http://paste.debian.net/1235314/ > > Option 3 is to use debian-installer and use pre-seeding to make the whole > process unattended. I managed to get this working for amd64, i386, arm64 and > ppc64el. Here is how to do it for ppc64el: > > $ qemu-img create -f qcow2 disk.qcow 4G > $ curl > https://deb.debian.org/debian/dists/bullseye/main/installer-ppc64el/current/images/netboot/debian-installer/ppc64el/initrd.gz > > initrd.gz > $ curl > https://deb.debian.org/debian/dists/bullseye/main/installer-ppc64el/current/images/netboot/debian-installer/ppc64el/vmlinux > > linux > $ qemu-system-ppc64le -no-reboot -machine pseries,graphics=off -serial > stdio -display none -monitor none -m 1G -initrd initrd.gz -kernel linux > -append 'console=hvc0 auto-install/enable=true debconf/priority=critical > preseed/url=http://www.debian.org/releases/stable/example-preseed.txt > netcfg/get_hostname=hostname netcfg/get_domain=domain > passwd/root-password=r00tme passwd/root-password-again=r00tme > passwd/user-fullname=user passwd/username=user passwd/user-password=insecure > passwd/user-password-again=insecure pkgsel/run_tasksel=false > popularity-contest:popularity-contest/participate=false > grub-installer/bootdev=/dev/sda1' -hda disk.qcow > > Option 4 is to use u-boot-qemu like this (in case of riscv64): > > $ mmdebstrap --arch=riscv64 --variant=apt \ > --include=linux-image-riscv64,u-boot-menu,systemd-sysv > --customize-hook='printf "U_BOOT_PARAMETERS=\"rw noquiet > root=/dev/vda1\"\nU_BOOT_FDT_DIR=\"noexist\"\n" > "$1"/etc/default/u-boot' \ > --customize-hook='chroot "$1" u-boot-update' \ > unstable chroot.tar \ > "deb http://deb.debian.org/debian-ports/ sid main" \ > "deb http://deb.debian.org/debian-ports/ unreleased main" > $ genext2fs --block-size 1024 --size-in-blocks 2097152 --bytes-per-inode > 16384 --tarball chroot.tar root.img > $ dd if=root.img of=disk.img seek=1 bs=4194304 > $ dd if=/dev/zero bs=512 count=1 >> disk.img > $ /sbin/parted -s disk.img "mklabel msdos" > $ printf debi | dd of=disk.img seek=440 bs=1 conv=notrunc > $ /sbin/parted -s disk.img "mkpart primary ext4 4MiB 2052MiB" > > Option 5 is the same as above but will work for architectures without > u-boot-qemu support. Without a bootloader, the kernel and initrd can be passed > to qemu manually via -kernel and -initrd arguments. > > While the last two options seem to be fragile, they have the advantage of > producing bit-by-bit reproducible output. The last option is also the only > option that works for all our architectures including mips*. > > So now we have seven different option of creating a (native or foreign) > bootable Debian image without being root. Locally I have scripts for all of > these and I'm tempted to put any of these into a script so that it gets easier > to create images for autopkgtest-virt-qemu without superuser privileges. > > On the other hand, I'm really not keen on adding yet another tool to this long > list: > > https://wiki.debian.org/SystemBuildTools#General_tools
I've pondered more about this because I also want this tool for sbuild-qemu (putting ckk into CC). I now have a plan and a proof-of concept implementation for all of the following: guestfish (option 0 from above): - fastest when building native qemu images with grub initramfs (option 2 from above) - same output as the last option but doesn't require guestfish. Installing guestfish requires more than 1.7 GB of disk space (574 packages) including systemd, dbus, poppler, python3, libllvm, linux-image, pocketsphinx and many other heavy packages that should not be necessary on a CI machine, for example - can create foreign architecture grub images which guestfish cannot as guestfish will only always emulate the native architecture debian-installer (option 3 from above) - useful because this creates a system that is as close to a system as users would install it as possible -- but it takes longer to create than the others kernel (option 5 from above) - works for all architectures (the others only work on i386, amd64, armhf, arm64 or ppc64el due to grub only being available on those platforms) - is bit-by-bit reproducible - but doesn't install a bootloader and has the kernel and initramfs on the outside, so the image cannot be upgraded and has to be re-created instead I am not considering option 1 from above because it is too slow and it feels wrong to solve the problem just by adding another layer. I'm not considering option 4 from above because it's mostly useful for riscv64 which is not a release architecture. Once it is, we can easily add it on later. I am offering to either rewrite autopkgtest-build-qemu with a --method option defaulting either to "guestfish" if a native image is requested, "initramfs" if a foreign architecture is requested and "kernel" if the image is neither i386, amd64, armhf, arm64 or ppc64el. If the user requires a system as close to a real system as possible, they can manually use --method=debian-installer. Why can existing tools not do the job: - vmdb2 needs root and this will not change: https://gitlab.com/larswirzenius/vmdb2/-/issues/62 - debos will only run on amd64 and not on the other arches - mkosi needs root - dqib only implements the "kernel" option Would you appreciate a MR implementing this in autopkgtest-build-qemu? Thanks! cheers, josch
signature.asc
Description: signature
