Hi Francesco,

Quoting Francesco Poli (2022-03-23 00:09:21)
> Do I understand correctly that mmdebstrap-autopkgtest-build-qemu is currently
> [included] in the Debian source package, but not shipped in the Debian binary
> package?
> 
>   $ dpkg -L mmdebstrap | grep qemu
> 
> gives no output.
> 
> [included]: 
> <https://salsa.debian.org/debian/mmdebstrap/-/blob/master/mmdebstrap-autopkgtest-build-qemu>
> 
> Are you going to ship the script in the next version of the binary package?

short answer: no.

A bit longer: The perfect is again the enemy of the good.

Full: the problem with the current version of mmdebstrap-autopkgtest-build-qemu
is, that it can only build qemu images for the native architecture. This is
because it relies on guestfish. Guestfish will never be able to operate on
foreign architecture qemu guests. This is because:

 - guestfish sets architecture specific options at compile time. This means
   that every guestfish binary can only be used for qemu guests of the same
   architecture as that binary and this cannot be changed at runtime

 - guestfish relies on another program called supermin. Essentially, supermin
   assembles a minimal chroot which is then loaded as qemu boots and then
   carries out the guestfish operations. Since supermin just copies binaries
   from the host, it cannot create foreign chroots either.

This means we have to replace guestfish by something else. I'm not aware that
this already exists so I wrote a proof-of-concepts that does what we need. It
works by first building a kernel and initramfs and then booting qemu with both.
The initramfs contains scripts that partition the disk, copy the rootfs and
install the bootloader:

main script: http://paste.debian.net/1235312/
initramfs-hook: http://paste.debian.net/1235313/
initramfs-script: http://paste.debian.net/1235314/

This works for both, foreign and native architectures. But since running
mmdebstrap to build the initramfs is far slower than supermin, I'd like to
fallback to using guestfish in the native case. So I have to combine above
script with mmdebstrap-autopkgtest-build-qemu.

I'm yet unsure whether I want to make these more general so that they can be
used for other purposes or whether the script should specifically build
autopkgtest qemu images.

A disadvantage is, that this only works for architectures for which qemu knows
how to boot them without kernel and initrd from the outside. This means that
mips* and s390x are not supported and neither are most of the Debian ports
architectures. I don't know how to solve this other than by teaching
autopkgtest-virt-qemu that now it needs three input files: the kernel, the
initrd and the rootfs.

Another disadvantage is, that the output can never be bit-by-bit reproducible
because grub-install is unreproducible.

I've worked on this in context with replacing vmdb2 in autopkgtest-build-qemu
so that sbuild-qemu-create (maintained by Christian Kastner) can be run without
superuser privileges.  We've tried to approach the vmdb2 author but Lars is
reluctant to include such drastic changes:
https://gitlab.com/larswirzenius/vmdb2/-/issues/62

So I'll probably release yet another https://wiki.debian.org/SystemBuildTools
because the existing solutions don't do what I want. The closest is probably
debos which can be run without being root because everything is done inside
qemu. But it also doesn't solve the hard problem of installing a bootloader,
leaving it to the user: https://github.com/go-debos/debos/issues/137

I also still need a name for this tool.

Thanks!

cheers, josch

Attachment: signature.asc
Description: signature

Reply via email to