Hi Pirate, Evangelos & Jonathon, Quoting Pirate Praveen (2024-12-15 19:37:57) > On 12/15/24 8:36 PM, Johannes Schauer Marin Rodrigues wrote: > > Quoting Pirate Praveen (2024-12-15 14:12:28) > >> Would it be posisble to use apt/dpkg from host architecture to resolve > >> dependencies and unpacking which does not need to run on the same > >> architecture > >> (avoiding qemu and possibly speeding up the process). Possibly using --root > >> option of dpkg to install into target chroot. postinst would still need to > >> use > >> qemu as it will need run in the target arch. > > I'm a bit confused what you are asking for because if I understand your > > query one way then the answer is: yes it already does that and if not then > > what you want to do looks like you want chrootless mode installations? > Currently Librem 5 crimson images are being built by debootstrap. So when I > tried running the scripts, I noticed the steps were very very slow. I was > wondering if switching to mmdebstrap can improve the situation.
it will, but not because mmdebstrap runs apt and dpkg on the outside. The reason that mmdebstrap will be a bit faster is because it uses apt to resolve dependencies which is both faster and correctly resolves dependencies as a bonus. If you want to switch to mmdebstrap, i think a better motivation is its multi-mirror support and the fact that you avoid the incomplete bash-based dependency "solver" of debootstrap. I would suspect that the majority of the slowness still comes from emulated maintainer scripts and the tools the scripts call. And that will be the same with mmdebstrap *unless* you use chrootless mode. The other advantages of mmdebstrap over debootstrap are irrelevant in your case, because you are interested in having it called by debos. > debos seems to support only debootstrap at the moment, there is an open issue > for a long time to support mmdebstrap in debos > https://github.com/go-debos/debos/issues/127 Yes, I don't think there is much of a point in including mmdebstrap into debos other than for a small speed boost. The things you get with mmdebstrap like bit-by-bit reproducible images, coverage of *all* Debian release architectures and no requirement for superuser privileges and no requirement for kvm support (unless you want a *really* slow build in fakemachine) run directly opposite to what debos is doing. You can probably switch it out but there would not be much point other than a few dozen seconds of improved runtime which i do not think is much of a motivation. > >> Context for building arm64 images from an amd64 machine > >> https://source.puri.sm/pureos/infra/pureos-image-recipes/-/issues/4#note_288559 > > > > Right, mmdebstrap can build bit-by-bit reproducible and bootable ext4 system > > image for arm64 on amd64 without superuser privileges. I maintain a script > > which does that for the MNT Reform which is also using the imx8mq just like > > the Librem 5 does. > > > > I think I first need to understand what you want to do and what you think > > mmdebstrap cannot do. > > I was just trying to test the current pureos crimson image building scripts. > > Can you share those scripts? It could be useful for crimson too, though > I'm not driving this task. My motivation was just to test the current > scripts and another unrelated bug caused by difference in behavior > between debootstrap and apt trigger and interest to see if mmdebstrap > can be used here instead of debootstrap. Sure, here you go: https://source.mnt.re/reform/reform-system-image/-/blob/main/mkimage.sh The vast majority of the script fills the shell array "$@" which is used to call mmdebstrap with the right arguments. The script is very long because it supports Debian unstable as well as bookworm, because it inlines scripts that are copied into the chroot and because it produced an image that has /boot on its own partition. But the script is stand-alone. All the magic is part of this script. If you'd like to start with something simpler, open the man page of mmdebstrap: https://manpages.debian.org/unstable/mmdebstrap/mmdebstrap.1.en.html And Ctrl+F for "Create a bootable USB Stick that boots into a full Debian desktop". If you would like to see how it is possible to create a bootable GNU Hurd image without *any* emulation at all (so this image is created in native speed even though I'm using an arm64 notebook and Hurd is i386), then read this mail: https://lists.debian.org/debian-hurd/2024/12/msg00003.html > > From what I read in the issue you cite, your goal is to create those > > images on amd64 (why don't you create them directly on a arm64 machine?) > > in a way that is fast and you want to avoid slowness by avoiding qemu as > > much as possible. Did you already benchmark how much runtime gets lost by > > apt and dpkg running inside the chroot? I did and the main problem in > > terms of slowness is not apt or dpkg but the decompression process. So > > technically you do not need to run apt or dpkg on the outside, you just > > need to run a native-arch decompressor. That being said, you know that the > > initial package installation is already done with apt and dpkg from the > > outside, right? The problem is the installation of more packages than the > > essential set later on. And for those, the big runtime wins are not from > > avoiding emulation of decompression but by avoiding emulated maintainer > > scripts. Especially running mkinitramfs & friends is horribly slow when > > you emulate it. You can also avoid qemu completely by using chrootless > > mode. This is how we currently are able to create bootable GNU Hurd images > > on linux (qemu does not allow you to run hurd binaries on linux). Nothing > > from the foreign architecture chroot is run. All the tools are from the > > outside. Note though, that not all packages support chrootless mode (also > > known as DPKG_ROOT support). > These scripts are eventually expected to run on arm64 in a ci. I was just > testing these locally on my amd64 machine. May be the solution is just > building these on arm64 directly. So what I was exploring has not much value > beyond the initial development as once the scripts are finalized they will > run an arm64. I can recommend the MNT Reform open hardware laptop to you. It has been my daily driver and only laptop for 2 years and it would let you create arm64 images without emulating anything. :) > > Can you clarify what exactly you are trying to do? This sounds like an X/Y > > problem to me so far. :) > > May be I can just wait for someone else to finish the scripts and > directly test the built images. This was not really an area where I have much > expertise anyway :) If you have any more questions, please go ahead and ask. If you feel this wishlist bug has served its purpose, please close it. Thanks! cheer, josch
signature.asc
Description: signature
