Your message dated Fri, 22 Sep 2023 23:42:25 +0200
with message-id <[email protected]>
and subject line migrate from merged-/usr to new alternative filesystem layout
has caused the Debian Bug report #1050001,
regarding migrate from merged-/usr to new alternative filesystem layout
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1050001: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050001
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: tech-ctte
Background and current status
In 2019 the TC decided[1] that Debian would achieve the largely-agreed
goal of having only one place to put most files, /usr, by using
symlinks to alias from /bin etc. to e.g. /usr/bin.
At the time, concerns were raised that package management systems and
other software would malfunction but the set of malfunctions was not
enumerated; proponents of the aliasing approach characterised them as
FUD. In the absence of the results of the research work which has now
been done, that characterisation seems to have been implicitly
accepted as true by the then TC.
Thanks to work funded by Freexian we now have a list of many of these
malfunctions[2] (although new ones keep popping up (eg [3]).
The most serious malfunction is the disappearing files bug, which has
prompted a seriously inconvenient file move moratoriam which has now
been in place for many years. After 4 years, we still don't have a
clear path forward to resolving that and other problems [4].
It should now be clear to most observers that the decision to go down
this path was a mistake.[5]
Fixing the mistake
I think we can probably fix it by backing out this change, and then
doing usrmerge the traditional Debian way by making changes to
debhelper, so that we move the files package by package, in the .debs.
But given the atmosphere, such a proposal would need clear political
backing from the TC. It wouldn't be worth anyone's time or emotional
energy to attempt to make a concrete and detailed plan if the TC is
not minded to consider it.
So I would like to pose the following questions for the TC:
* Given the information that the Committee has now, that it didn't
have in 2019, does the Commitee agree that the decision in 2019
was questionable ?
* Is the Committee open to the idea of a plan being developed which
reverses the mistake, rather than merely "mitigating" the problems ?
Would such a plan be considered on its merits ?
I appreciate that I'm asking the TC to revisit a previous and
controversial decision. That this isn't a step we should take
lightly, but I think in this case it's clearly warranted.
Timeline for a fix
Any plan to solve this would probably take a few releases (ie, many
more years) to sort out, sadly. We would probably need to wait with
shipping packages that move files in a conventional-for-Debian way,
until we have our userbase's systems restored to a non-aliased state.
So I think we would need trixie to undo the aliasing, and in trixie+1
we could move the files.
This delay is unfortunate, but - unlike pressing forward - it has
relatively low levels of risk, most of which is front-loaded. I think
we can develop tools which will reliably de-alias a system; and, once
users' systems have been de-aliased, the actual file movement is
routine work that Debian knows how to do.
I can see that there could possibly be ways of going straight to the
desired state (un-aliased systems but with nothing much in /), but I
haven't given them much thought them through because they seem risky
to me and involve some grievous hacks.
Protecting my mental health
I will try to avoid regularly reading this thread. I hope that now
that I have made the suggestion, others will be able to carry the
conversation. I will be configuring my mail client to disregard my
personal copies of messages sent to this bug; when I want to read
the thread I will look at the mailing list.
Also, if you disagree with my decision to raise this now, please don't
email me. If you feel I have overstepped a boundary, please contact
the Community Team or DAM.
If the Comittee gives a positive indication, I will be happy to
re-engage at the level of technical work to try to make it happen.
Thanks,
Ian.
[1] https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html
[2] https://subdivi.de/~helmut/dep17.html
[3] https://lists.debian.org/debian-devel/2023/05/msg00311.html
[4] https://lists.debian.org/debian-devel/2023/06/msg00353.html
[5]
Perhaps merged-usr via directory aliasing worked well in some other
distros with less sophisticated package management systems.
But we obtained almost all the practical benefits of abolishing the
distinction between / and /usr very early, by deciding that the
initramfs would mount /usr too. We have inflicted all this pain and
confusion on ourselves simply to do some tidying up. The result has
been the opposite.
If we had just moved the files rather than trying to rush things
with the directory alias symlinks, we would have been finished by
now.
--- End Message ---
--- Begin Message ---
Dear Ian,
thank you for reaching out to the Technical Committee and engaging in a
dialogue despite the very flammable nature of the topic.
We believe the major points of your argument are as follows: Directory
symlinks ("aliasing symlinks") are inherently less robust than file
symlinks. Aliasing symlinks violate a key assumption of dpkg, namely
that every managed filesystem object can be uniquely determined by its
path name. Given the pervasive nature of this assumption in the dpkg
code base, aliasing symlinks will remain a persistent source of
difficult-to-track bugs, which will negatively impact the robustness of
dpkg and Debian as a whole. Given that only a small fraction of binaries
actually need to be reachable through both / and /usr, the desired level
of compatibility with other distributions can also be achieved by using
a carefully curated list of file-to-file symlinks.
Therefore, you have asked the TC if it has changed its stance on the
2019 decision and whether or not it would be open to alternative
technical solutions.
The TC recognizes your standing as subject matter expert on dpkg and
agrees that aliasing symlinks expose a limitation of dpkg, which needs
to be dealt with. However, the TC disagrees with your risk assessment
and the conclusions you draw from DEP-17. Given the scope of your
proposed change, the current transition state, and the general fatigue
that has set in with regard to merged-/usr discussions, we find it
justified to set a high standard of proof before we impose more change
on the project. The presented arguments were not strong enough to let us
conclude that continuing on the present course will result in a much
worse bug situation. Formally, your questions do not propose any
immediately actionable steps either; on those grounds, the TC declines
to rule, which effectively upholds previous rulings on the matter.
Informally, I believe it is fair to say that we have gained a
significant understanding in the past few months on how to complete the
merged-/usr transition. While DEP-17 does enumerate several issues, none
of them are show-stoppers; they can be mitigated with temporary
workarounds for the trixie release and do not create a permanent
maintenance burden. Many TC members currently believe that the benefits
of a filesystem layout with aliasing symlinks outweigh its drawbacks.
Therefore, it is very unlikely that the TC will overturn the current
transition plan, let alone revert to the old split-/usr layout, unless
confronted with concrete and compelling evidence that the plan
results in irrecoverable breakage. Any remaining pain points, if
they persist beyond the trixie release and cannot be resolved by
tweaking the transition itself, should rather be addressed under the
assumption that the forky release cycle begins in a post DEP-17
world.
You have probably anticipated this response (and maybe hoped for a
different one). Although the TC may disagree with your conclusions, we
do appreciate your technical insight and are very thankful for your
continued constructive participation in the discussion. Personally, I
think this is a situation where the Perfect is the enemy of the Good. I
hope that despite our disagreements, you will continue to help turning
Good into Better.
Cheers
Timo
for the Technical Committee
--
⢀⣴⠾⠻⢶⣦⠀ ╭────────────────────────────────────────────────────╮
⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │
⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄⠀⠀⠀⠀ ╰────────────────────────────────────────────────────╯
signature.asc
Description: PGP signature
--- End Message ---
