Re: Reviving schroot as used by sbuild

On Fri, 13 Sep 2024 at 11:15:55 +0200, Helmut Grohne wrote: > My initial experiments indicate that we're in > for a factor two [slowdown] whereas we could get this down significantly > by using an overlayfs approach that we cannot shoehorn into podman. Er, podman does use overlayfs, in at least so

Re: Reviving schroot as used by sbuild

Hi Sam and others, On Fri, Jun 28, 2024 at 07:08:20AM -0600, Sam Hartman wrote: > I'll be honest, I think building a new container backend makes no sense > at all. I looked hard at this as it was voiced by many. I have to say, I remain unconvinced of the arguments brought forward. > There's a lo

Re: Reviving schroot as used by sbuild

Hi Philipp, Let me go into some detail that is tangential to the larger discussion. On Mon, Jul 01, 2024 at 09:18:19AM +0200, Philipp Kern wrote: > How well does this setup nest? I had a lot of trouble trying to run the > unshare backend within an unprivileged container as setup by systemd-nspawn

Re: Reviving schroot as used by sbuild

On Mon, Jul 1, 2024 at 11:59 AM Simon McVittie wrote: > On Mon, 01 Jul 2024 at 09:18:19 +0200, Philipp Kern wrote: > One use-case that I'm familiar with is bwrap (bubblewrap, as used by > flatpak) nested inside podman. bwrap is a relatively limited container > technology with relatively "light" r

Re: Reviving schroot as used by sbuild

On Mon, 01 Jul 2024 at 09:18:19 +0200, Philipp Kern wrote: > Specifically I'm concerned about what [advocating use of podman] > means for tests and if they > should be able to use unprivileged containers themselves to test things. tl;dr: There's no regression here, because you already can't run th

Re: Reviving schroot as used by sbuild

Hi, On 2024-06-29 22:21, Christian Kastner wrote: At the moment, rootless Podman would seem like the obvious choice. As far as I'm aware, it has the same user namespaces requirements as the unshare backends in mmdebstrap, autopkgtest and schroot (user namespaces enabled, setuid newuidmap, 6553

Re: Builds that pass locally but fail on sbuild? (Re: Reviving schroot as used by sbuild)

> "Richard" == Richard Lewis writes: Richard> Otto Kekäläinen writes: >> Could you point me to some Debian Bug # or otherwise share >> examples of cases when a build succeeded locally but failed on >> official Debian builders due to something that is specific for >> sbuil

Re: Reviving schroot as used by sbuild

On 2024-06-25 15:02, Simon McVittie wrote: > Could we use a container framework that is also used outside the Debian > bubble, rather than writing our own from first principles every time, and > ending up with a single-maintainer project being load-bearing for Debian > *again*? I had hoped that a

Re: Builds that pass locally but fail on sbuild? (Re: Reviving schroot as used by sbuild)

Otto Kekäläinen writes: > Could you point me to some Debian Bug # or otherwise share examples of > cases when a build succeeded locally but failed on official Debian > builders due to something that is specific for sbuild/schroot? I believe both these uploads https://tracker.debian.org/news/128

Re: Reviving schroot as used by sbuild

> "Helmut" == Helmut Grohne writes: Helmut> In this work, limitations with --chroot-mode=unshare became Helmut> apparent and that lead to Johannes, Jochen and me sitting Helmut> down in Berlin pondering ideas on how to improve the Helmut> situation. That is a longer story, but

Re: Builds that pass locally but fail on sbuild? (Re: Reviving schroot as used by sbuild)

On Thu, 27 Jun 2024 at 19:56:43 -0700, Otto Kekäläinen wrote: > Could you point me to some Debian Bug # or otherwise share examples of > cases when a build succeeded locally but failed on official Debian > builders due to something that is specific for sbuild/schroot? I can't easily point you to a

Builds that pass locally but fail on sbuild? (Re: Reviving schroot as used by sbuild)

Hi Simon! > There are lots of options for doing this, some of which are listed in > . > > All of these have the same problem as cowbuilder, pbuilder, and any > other solution that is not sbuild + schroot: it isn't (currently) what > the

Re: Reviving schroot as used by sbuild

Simon, Quoting Simon McVittie (2024-06-27 19:16:54) > On Thu, 27 Jun 2024 at 17:26:20 +0200, Johannes Schauer Marin Rodrigues wrote: > > But, if everybody is so excited about this, where are the sbuild > > contributors > > implementing this? > > I'm sorry, consider it added it to my list. As usu

Re: Reviving schroot as used by sbuild

On Wed, 26 Jun 2024 at 18:05:15 -0400, Reinhard Tartler wrote: > I imagine that one could whip up some kind of wrapper > that is building a container either from a tarball created via mmboostrap or > similar > using buildah, have it install all necessary build dependencies, and then use > podman to

Re: Reviving schroot as used by sbuild

On Thu, 27 Jun 2024 at 17:26:20 +0200, Johannes Schauer Marin Rodrigues wrote: > But, if everybody is so excited about this, where are the sbuild contributors > implementing this? I'm sorry, consider it added it to my list. As usual, there's no guarantee that I will get there within my lifetime, b

Re: Reviving schroot as used by sbuild

Hi, Quoting Simon McVittie (2024-06-27 15:59:01) > On Thu, 27 Jun 2024 at 11:46:51 +0200, Helmut Grohne wrote: > > I don't quite understand the need for a Dockerfile here. I suspect that > > this is the obvious way that works reliably, but my impression was that > > using podman import would be ea

Re: Reviving schroot as used by sbuild

On Thu, 27 Jun 2024 at 11:46:51 +0200, Helmut Grohne wrote: > I am concerned about behavioural differences due to the reimplementation > from first principles aspect though. Jochen and Aurelien will know more > here, but I think we had a fair number of ftbfs due to such differences. > None of them

Re: Reviving schroot as used by sbuild

On Thu, Jun 27, 2024 at 7:45 AM Helmut Grohne wrote: > Please allow for another podman question (and more people than Simon > know the answer). Every time I run a podman container (e.g. when I run > autopkgtest) my ~/.local/share/containers grows. I think autopkgtest > manages to clean up in the

Re: Reviving schroot as used by sbuild

Hi Simon, Thanks for having taken the time to do another extensive writeup. Much appreciated. On Wed, Jun 26, 2024 at 06:11:09PM +0100, Simon McVittie wrote: > On Tue, 25 Jun 2024 at 18:55:45 +0200, Helmut Grohne wrote: > > The main difference to how everyone else does this is that in a typical >

Re: Reviving schroot as used by sbuild

On 25.06.24 15:02, Simon McVittie wrote: I have to ask: Could we use a container framework that is also used outside the Debian bubble, rather than writing our own from first principles every time, and ending up with a single-maintainer project being load-bearing for Debian *again*? I had hoped

Re: Reviving schroot as used by sbuild

On Wed, Jun 26, 2024 at 1:11 PM Simon McVittie wrote: > > I don't think podman can do this within a single run. It might be feasible > to do the setup (installing build-dependencies) with networking enabled; > leave the root filesystem of that container intact; and reuse it as the > root filesyst

Re: Reviving schroot as used by sbuild

On Tue, 25 Jun 2024 at 18:55:45 +0200, Helmut Grohne wrote: > At least for me, building container images locally is a requirement. I > have no interest in using a container registry. I expected you'd say that. podman --rootfs is one way to use it without a registry; a trivially short Dockerfile li

Re: Reviving schroot as used by sbuild

Hi Helmut (2024.06.25_16:55:45_+) > lxd/incus also was on my list, Personally, I have been using LXD (and now Incus, as it made it into Debian, yay) for my experimentation and local package builds, for a number of years now. They have native support for btrfs snapshots, locally built images, a

Re: Reviving schroot as used by sbuild

On Tue, 25 Jun 2024 at 18:47:49 +0200, Guillem Jover wrote: > I manage my chroots with schroot (but not via sbuild, for dog fooding > purposes :), and use type=directory and union-type=overlay so that I > get a fast and persistent base, independent of the underlying filesystem, > with fresh instanc

reproducible Debian containers exists today (Re: Reviving schroot as used by sbuild)

hi, On Tue, Jun 25, 2024 at 02:02:11PM +0100, Simon McVittie wrote: > I have to ask: > > Could we use a container framework that is also used outside the Debian > bubble, rather than writing our own from first principles every time, and > ending up with a single-maintainer project being load-bear

Re: autopkgtest + podman user experience (Was: Re: Reviving schroot as used by sbuild)

Hi. On 25-06-2024 8:18 p.m., Gioele Barabucci wrote: I'd like to take this chance to suggest, instead of writing more documentation, changing the autopkgtest packaging so that it is split into various per-backend packages, each of which provides a ready-to-go pre-configured environment. See

Re: Reviving schroot as used by sbuild

Hi On 25-06-2024 6:55 p.m., Helmut Grohne wrote: This is very cool. Running autopkgtests in system containers without being root (or incus-admin) very much is what I'd like to do. And it's much better if I don't have to write my own container framework for doing it. I couldn't get it to work loc

autopkgtest + podman user experience (Was: Re: Reviving schroot as used by sbuild)

On 25/06/24 18:55, Helmut Grohne wrote: For systemd-as-pid-1 specifically, `autopkgtest-build-podman --init=systemd` and `autopkgtest-virt-podman --init` demonstrate how this can be done, and last time I tried, it was possible to run them unprivileged (other than needing access to the setuid newu

Re: Reviving schroot as used by sbuild

On Tue, Jun 25, 2024 at 7:13 AM Helmut Grohne wrote: > > Hi, > > sbuild is our primary tool for constructing a build environment to build > Debian packages. It is used on all buildds and for a long time, the > backend used with sbuild has always been schroot. More recently, a > number of buildds h

Re: Reviving schroot as used by sbuild

> Ah, thank you, I didn't realize that existed. That sounds like a nice > generalization of the file system snapshot approach. I think that this how the sbuild-debian-developer-setup script, setup chroots Fred

Re: Reviving schroot as used by sbuild

PICCA Frederic-Emmanuel writes: >> Ah, thank you, I didn't realize that existed. That sounds like a nice >> generalization of the file system snapshot approach. > I think that this how the > sbuild-debian-developer-setup > script, setup chroots Yeah, I think all that my contribution to this

Re: Reviving schroot as used by sbuild

On Tue, Jun 25, 2024 at 10:24:12AM -0700, Russ Allbery wrote: > Guillem Jover writes: > > > I manage my chroots with schroot (but not via sbuild, for dog fooding > > purposes :), and use type=directory and union-type=overlay so that I get > > a fast and persistent base, independent of the underly

Re: Reviving schroot as used by sbuild

Guillem Jover writes: > I manage my chroots with schroot (but not via sbuild, for dog fooding > purposes :), and use type=directory and union-type=overlay so that I get > a fast and persistent base, independent of the underlying filesystem, > with fresh instances per session. (You can access the

Re: Reviving schroot as used by sbuild

On Jun 25, Guillem Jover wrote: > I manage my chroots with schroot (but not via sbuild, for dog fooding > purposes :), and use type=directory and union-type=overlay so that I > get a fast and persistent base, independent of the underlying filesystem, > with fresh instances per session. (You can a

Re: Reviving schroot as used by sbuild

Hi Simon, On Tue, Jun 25, 2024 at 02:02:11PM +0100, Simon McVittie wrote: > Could we use a container framework that is also used outside the Debian > bubble, rather than writing our own from first principles every time, and > ending up with a single-maintainer project being load-bearing for Debian

Re: Reviving schroot as used by sbuild

On Tue, 25 Jun 2024 at 09:32:21 -0700, Russ Allbery wrote: > Simon McVittie writes: > > I think the > > only model that should be used in new systems is to have some concept of > > a session (like schroot type=file, but unlike schroot type=directory) > > I'm not entirely sure that I'm following t

Re: Reviving schroot as used by sbuild

Hi! On Tue, 2024-06-25 at 09:32:21 -0700, Russ Allbery wrote: > Simon McVittie writes: > > Persisting a container root filesystem between multiple operations comes > > with some serious correctness issues if there are "hooks" that can modify > > it destructively on each operation: see

Re: Reviving schroot as used by sbuild

Simon McVittie writes: > Persisting a container root filesystem between multiple operations comes > with some serious correctness issues if there are "hooks" that can modify > it destructively on each operation: see > and . As a res

Re: Reviving schroot as used by sbuild

On Tue, Jun 25, 2024 at 02:02:11PM +0100, Simon McVittie wrote: > On Tue, 25 Jun 2024 at 10:16:20 +0200, Helmut Grohne wrote: > > In this work, limitations with --chroot-mode=unshare became apparent and > > that lead to Johannes, Jochen and me sitting down in Berlin pondering > > ideas on how to im

Re: Reviving schroot as used by sbuild

On Tue, Jun 25, 2024 at 02:02:11PM +0100, Simon McVittie wrote: > > In this work, limitations with --chroot-mode=unshare became apparent and > > that lead to Johannes, Jochen and me sitting down in Berlin pondering > > ideas on how to improve the situation. That is a longer story, but > > eventuall

Re: Reviving schroot as used by sbuild

On Tue, Jun 25, 2024 at 02:02:11PM +0100, Simon McVittie wrote: > Could we use a container framework that is also used outside the Debian > bubble, rather than writing our own from first principles every time, and > ending up with a single-maintainer project being load-bearing for Debian > *again*?

Re: Reviving schroot as used by sbuild

On Tue, 25 Jun 2024 at 10:16:20 +0200, Helmut Grohne wrote: > In this work, limitations with --chroot-mode=unshare became apparent and > that lead to Johannes, Jochen and me sitting down in Berlin pondering > ideas on how to improve the situation. That is a longer story, but > eventually Timo Röhli

Re: Reviving schroot as used by sbuild

Hi, Quoting Helmut Grohne (2024-06-25 10:16:20) > In this work, limitations with --chroot-mode=unshare became apparent and that > lead to Johannes, Jochen and me sitting down in Berlin pondering ideas on how > to improve the situation. That is a longer story, but eventually Timo Röhling > asked th

Reviving schroot as used by sbuild

Hi, sbuild is our primary tool for constructing a build environment to build Debian packages. It is used on all buildds and for a long time, the backend used with sbuild has always been schroot. More recently, a number of buildds have been moved away from schroot towards --chroot-mode=unshare than