On Thu, 19 Aug 2021 at 10:06:27 +0200, Helmut Grohne wrote: > On Tue, Aug 17, 2021 at 05:19:06PM +0100, Simon McVittie wrote: > > If we want to make buildd chroots merged-/usr any time soon, then I > > think we need to say this class of bugs is RC for bookworm. > > For a moment, let us assume perfection of this plan: Reproducible builds > identify all of the cases that need to be fixed. We bump them to RC > bugs. We fix them all. Then we change unstable to be merged-/usr and > we're done. Are we? We still need to support partial upgrades from > bullseye to bookworm, so - as you pointed out earlier - all packages in > bookworm will have to keep this support property. However, our > defaulting to merged-/usr makes it impossible for reproducible builds to > test it as producing an unmerged chroot is no longer possible. > Effectively, the unmerged case will be untested and as a consequence > will be broken.
Is this the scenario you're concerned about? * We reach a point where every package in bookworm builds identically (or with only non-/usr-related variation) in non-merged-/usr and merged-/usr chroots, i.e. the bugs falling into classification https://tests.reproducible-builds.org/debian/issues/unstable/paths_vary_due_to_usrmerge_issue.html have all been fixed * We deploy an automatic transition mechanism that makes all bookworm installations, including chroots, convert to merged-/usr (for example, making usrmerge transitively Essential would be one possible implementation, although perhaps not the best one) * Now our buildd chroots are all merged-/usr, and additionally we can no longer easily test for /usr-related reproducibility * The foo maintainer uploads foo_1.0 that has a regression, introducing a *new* /usr-related unreproducibility (for example it might end up hard-coding /usr/bin/sh when built in a merged-/usr environment, rather than the /bin/sh that is correct for traditional environments, even though earlier versions of foo did not have that bug) * The regression is not detected because we can't easily test for it any more * A user carries out a partial upgrade in which foo_1.0 is upgraded before whatever part of the system is responsible for carrying out the transition mechanism * Now foo doesn't work on that user's system I agree that's a potential failure mode, although the window for regression is not necessarily very long (foo would have to regress after the automatic transition is already in place). > For this plan to work, we will have to support unmerged chroots until > the end of bookworm, which appears to contradict the premise we started > with. One way we could escape from this would be to observe that the earliest point at which it is OK for package maintainers to rely on merged-/usr is when the bookworm+1 development cycle opens, therefore it is OK to have a mechanism for bookworm chroots/containers to be special-cased to opt out from the merged-/usr transition, as long as those chroots/containers will never be upgraded to bookworm+1 (because in practice we tend to discard and re-bootstrap chroots/containers rather than upgrading them). Conceptually, those chroots/containers are stuck in the "upgrade from bullseye to bookworm almost but not quite complete" state, forever. If desired, we could declare this to be formally unsupported, but make reproducible-builds do it anyway, purely as a way to get test coverage. This would be conceptually similar to the way people regularly do archive rebuilds with a gcc that is not supported as the default yet, in order to get the information we need before we can make it the default - but in reverse, with the "old" configuration being the one that is formally unsupported. reproducible-builds could do something like this: * for build1: do a merged-/usr chroot/container as (will become) usual * for build2: bootstrap a chroot/container with --no-merged-usr, and set whatever flag opts-out from the transition or (matching what they do for buster/bullseye): * bootstrap a chroot/container with --no-merged-usr * for build1: use it as-is (opting out from the transition if necessary) * for build2: enable the transition and let it happen, or install usrmerge manually Or, if the transition ends up being done during boot, either in the initramfs or in normal user-space, then chroots/containers would "naturally" not transition without manual action - similar to the way chroots/containers don't pick up kernel or initramfs behaviour changes like the changed default for /proc/sys/kernel/unprivileged_userns_clone in bullseye, because their kernel is not their own. smcv
