Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Marco d'Itri
On Jul 19, Marc Haber  wrote:

> I am NOT looking forward having to manually convert legacy systems to
> merged /usr and I do sincerely hope that Debian will choose a way to
> get away without throwing away systems that have just a small /, still
> supporting a dedicated /usr as long as it's mounted by initramfs. I am
They cannot be supported without keeping a lot of unneeded complexity 
around, but there is no reason to reinstall them: you can move / inside 
/usr instead. You may use either sash or a live CD image.

> I also believe that smaller file systems are unlikely to break and
> that a system that can boot up to a ssh-able state even with a broken
> file system is way easier to fix. We have taken a huge step back in
And "apt install grml-rescueboot" is even better.
Also, with merged-/usr you may keep your *whole* OS in a read only file 
system.

-- 
ciao,
Marco


signature.asc
Description: PGP signature


Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Stephan Lachnit
On Mon, Jul 19, 2021 at 3:37 AM Guillem Jover  wrote:
> What I've also said multiple times, is that
> merged-usr-via-moves-and-symlink-farms could have been implemented in
> a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> all with full cooperation and managed by dpkg, with .debs shipping
> actual tracked pathnames, if it had not been for the mess required
> by merged-usr-via-aliased-dirs. :/

Maybe I get this wrong, but I don't think this conflicts with the
decision from the TC.

Until Debian 12 gets released, we have a lot of time for a transition.
Maybe we should start discussing the transition and less whether or
not we do it, as this has been decided now anyway.

We could start with collecting the packages that install to /bin*
instead of /usr/bin, and adjust the packaging so that they don't do
that. Of course, we would need to add a maintainer script that detects
un-merged usr and creates a symlink. Actually, I think it would be
enough to just let debhelper detect if a package installs to /bin, and
adjust the package automatically. For packages not using debhelper,
lintian can add a warning if the package installs to /bin. After all
packages that installed to /bin have been rebuilt, nothing would
install to /bin except for symlinks. At this point, it shouldn't
matter if you run a merged usr system or not, or am I forgetting
something? IMHO it would make way more sense to discuss how to merge
usr once the packages are fixed.

Anyway, I think the discussion made clear that we shouldn't
immediately start with merging usr once bullseye is released, and I
wouldn't interpret the TC decision as such.

Regards,
Stephan

* using /bin as an example, same goes for /lib etc



Re: merged /usr

2021-07-19 Thread Simon McVittie
On Mon, 19 Jul 2021 at 11:33:49 +0200, Stephan Lachnit wrote:
> We could start with collecting the packages that install to /bin*
> instead of /usr/bin, and adjust the packaging so that they don't do
> that. [...] At this point, it shouldn't
> matter if you run a merged usr system or not, or am I forgetting
> something?

This would have part of the practical effect of merged-/usr, but not all
of it. It could still be a useful step forwards, but we should not view
it as being entirely equivalent to merged-/usr.

What we have now on unmerged-/usr systems, using /bin/bash and
/usr/bin/perl as examples of Essential programs that use the two paths:

 bashperl
/bin/foo   physical location   (does not exist)
/usr/bin/foo   (does not exist)physical location

What we would have on unmerged-/usr systems if we do as you suggest:

 bash perl
/bin/foo   exists via symlinks  (does not exist)
/usr/bin/foo   physical locationphysical location

Merged-/usr for comparison:

 bash perl
/bin/foo   exists via symlinks  exists via symlinks
/usr/bin/foo   physical locationphysical location

As you can see from those tables, a package that hard-codes /usr/bin/bash
is currently considered broken, but would work with either your proposal
or merged-/usr. Conversely, a package that hard-codes /bin/perl would
still be considered broken, would *not* work with your proposal, but
would work on merged-/usr systems.

(Obviously the same applies when considering hard-coded paths in /sbin,
/lib, etc. instead of /bin, in particular the ELF interpreters like
/lib64/ld-linux-x86-64.so.2 that are hard-coded into every
dynamically-linked executable)

Meanwhile, various shared libraries are also moving from /lib to
/usr/lib.  One potentially serious problem with that on non-merged-/usr
systems is that we still don't understand why the bugs discussed
in https://bugs.debian.org/911225 and https://bugs.debian.org/949395
happened, and a similar thing could potentially happen to /lib libraries
other than GLib. The script that GLib uses to work around this is generic,
and could be used in other affected packages if required, but it's larger
and more fragile than I'm really comfortable with.

(Merged-/usr systems cannot suffer from bugs like the ones discussed in
#911225, because the paths involved are the same directory.)

smcv



Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Michael Biebl

Hi Guillem

Am 19.07.21 um 03:36 schrieb Guillem Jover:

What I've also said multiple times, is that
merged-usr-via-moves-and-symlink-farms could have been implemented in
a fully automated way, by debhelper, w/o requiring any maintainer scripts,
all with full cooperation and managed by dpkg, with .debs shipping
actual tracked pathnames
I'm convinced this view is way too naive and not implementable in 
practice (and yes, openSUSE is a data point that confirms that)


What you propose is, that each and every package does its /usr-merge 
transition on its own. This only works, if packages are independent 
(enough) so this actually works.
Unfortunately this is not the case. Take PAM for example. You can't just 
recompile src:pam and have debhelper automatically move all files to 
/usr. This would break all packages that install a PAM plugin. You have 
a transition here, involving many packages.
Same is true for udev rules, systemd service files, basically every 
package that provides interfaces/hooks to other packages is affected.

So it's not that simple unfortunately. You can't fully automate that.

According to
apt-file search -x '^/(lib|bin|sbin)'
on my Debian sid/amd64 system, we have 1747 packages shipping 24583 
files in those directories. There are *many* such entangled transitions 
hidden in there, so I fear this is not manageable.
As Luca pointed out, even distros with a much stricter governance model 
were not able to do that.


The /usr-merge transition as described and decided on in the TC bug, 
seems to me is the only viable way forward.


Yes, it does break dpkg -S, but your idea of using a list of mapped 
paths as in [1] seems like an entirely reasonable approach to solve this.


Once we have this global switch to merged-usr, packages can bit by bit, 
completely independent, update their debian/rules to use --prefix=/usr 
and after a few years, we don't have any packages anymore installing any 
files in /. We could aid this process with a lintian check that flags 
packages that install files in /(sbin|bin|lib)/.





Regards,
Michael


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858331#33




OpenPGP_signature
Description: OpenPGP digital signature


Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Michael Biebl

Am 19.07.21 um 07:23 schrieb Marc Haber:


I am NOT looking forward having to manually convert legacy systems to
merged /usr and I do sincerely hope that Debian will choose a way to
get away without throwing away systems that have just a small /, still
supporting a dedicated /usr as long as it's mounted by initramfs. I am
not sure whether we ever issued a clear statement about that.


I think this is a misunderstanding. Files from / would be moved to /usr. 
So the only way this could fail is, if your /usr partition was too 
small.That's still a possibility for existing systems, but a much 
smaller one then moving files from /usr to /. Typically a separate /usr 
partition is larger then /.



There are some technical reasons to separate /boot if you are using a file
system for other partitions that isn't suitable for early boot (or if
you're using cryptsetup or other file system layers).  /boot/efi is always
a separate partition because of how it works.  Apart from those two
special cases, the only reason to put something on a separate file system
is if you have a clear and compelling reason why you expect a given file
system to run out of space and you want to ensure that it cannot take
space from other parts of the system.


I also believe that smaller file systems are unlikely to break and
that a system that can boot up to a ssh-able state even with a broken
file system is way easier to fix. We have taken a huge step back in
that regard with systemd since the systemd rescue mode requiring the
"real" root password even for minor startup failures is way more
unfriendly than what we had before.


I assume you are referring to the sulogin issue here [1], i.e. whether 
we require a root password on an emergency failure or not.

Fwiw, this is mostly me being paranoid and not handing out root shells.
This has nothing to do with merged-/usr.


Michael


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802211



OpenPGP_signature
Description: OpenPGP digital signature


Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Johannes Schauer Marin Rodrigues
Quoting Michael Biebl (2021-07-19 15:10:42)
> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames
> I'm convinced this view is way too naive and not implementable in 
> practice (and yes, openSUSE is a data point that confirms that)
> 
> What you propose is, that each and every package does its /usr-merge 
> transition on its own. This only works, if packages are independent 
> (enough) so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just 
> recompile src:pam and have debhelper automatically move all files to 
> /usr. This would break all packages that install a PAM plugin. You have 
> a transition here, involving many packages.
> Same is true for udev rules, systemd service files, basically every 
> package that provides interfaces/hooks to other packages is affected.
> So it's not that simple unfortunately. You can't fully automate that.
> 
> According to
> apt-file search -x '^/(lib|bin|sbin)'
> on my Debian sid/amd64 system, we have 1747 packages shipping 24583 
> files in those directories.

more precisely, on amd64 unstable:

/bin 247 files
/lib{32,64,x32} 83 files
/lib/firmware 2379 files
/lib/live 115 files
/lib/modules 17500 files
/lib/systemd 2221 files
/lib/udev 614 files
/lib/x86_64-linux-gnu 343 files
/lib/* 441 files
/sbin 547 files

So most files come from /lib/modules, where only 14 packages are involved,
/lib/systemd which will be fixed by an update to dh_installsystemd, and
/lib/firmware where only 36 packages are involved. The remainder then doesn't
sound so scary anymore as it only involves 656 unique packages and not 1747.
And again many of those remaining packages will be fixed by an update to
debhelper, correct? Given that 90% of source packages use dh, that would reduce
the number to a very manageable size.

> There are *many* such entangled transitions 
> hidden in there, so I fear this is not manageable.
> As Luca pointed out, even distros with a much stricter governance model 
> were not able to do that.
> 
> The /usr-merge transition as described and decided on in the TC bug, 
> seems to me is the only viable way forward.
> 
> Yes, it does break dpkg -S, but your idea of using a list of mapped 
> paths as in [1] seems like an entirely reasonable approach to solve this.
> 
> Once we have this global switch to merged-usr, packages can bit by bit, 
> completely independent, update their debian/rules to use --prefix=/usr 
> and after a few years, we don't have any packages anymore installing any 
> files in /. We could aid this process with a lintian check that flags
> packages that install files in /(sbin|bin|lib)/.

So what what is actually the roadmap after the bullseye release? What is the
way forward? Should I rather file bugs with patches against individual packages
to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
already have a debhelper patch to do that move for us?

This would mean that we only have to bear with the problems mentioned by
guillem until that move is complete, correct?

And another question: once there are no more files shipped by any package in
/(sbin|bin|lib)/ we can let base-file create the symlinks?

Thanks!

cheers, josch

signature.asc
Description: signature


Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Marc Haber
On Mon, 19 Jul 2021 15:19:32 +0200, Michael Biebl 
wrote:
>Am 19.07.21 um 07:23 schrieb Marc Haber:
>> I am NOT looking forward having to manually convert legacy systems to
>> merged /usr and I do sincerely hope that Debian will choose a way to
>> get away without throwing away systems that have just a small /, still
>> supporting a dedicated /usr as long as it's mounted by initramfs. I am
>> not sure whether we ever issued a clear statement about that.
>
>I think this is a misunderstanding. Files from / would be moved to /usr. 
>So the only way this could fail is, if your /usr partition was too 
>small.That's still a possibility for existing systems, but a much 
>smaller one then moving files from /usr to /. Typically a separate /usr 
>partition is larger then /.

Right, that sounds much easier. It's still an Open Heart Operation,
especially for systems that I don't have out of band access for, which
is rather common for smaller installations.

>I assume you are referring to the sulogin issue here [1], i.e. whether 
>we require a root password on an emergency failure or not.

Yes. From my point of view, this is taking away a freedom from the
local admin.

In an ideal world, boot failure behavior would be locally
configurable.

A mis-booted sysv system, if I remember correctly (I have been a
mostly happy systemd user for already quite some time), could be told
to "just try to continue and show me how far you get", which in the
vast majority of cases led to a regular login prompt from which the
user-login-plus-sudo routine just worked. This didn't hand out any
more root shells than the current way of stopping dead and refusing to
do anything without the "real" root password, as far as I understand.
I probably don't have enough experience to have the final call on
that. But it's just a pet peeve of mine that I can easily live with.

>Fwiw, this is mostly me being paranoid and not handing out root shells.

It's good to be paranoid by default. It's bad to force that paranoia
on the local admin who might have a choice to move to a different
distribution. But alas, the others do it the same way. So it's just
freedom lost.

>This has nothing to do with merged-/usr.

I never said it has.

Greetings
Marc
-- 
-- !! No courtesy copies, please !! -
Marc Haber |   " Questions are the | Mailadresse im Header
Mannheim, Germany  | Beginning of Wisdom " | 
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834



Re: merged /usr

2021-07-19 Thread Simon McVittie
On Mon, 19 Jul 2021 at 16:41:42 +0200, Johannes Schauer Marin Rodrigues wrote:
> Should I rather file bugs with patches against individual packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/

As discussed in previous iterations of the ongoing merged-/usr megathread,
I don't think this can be the whole solution, because some packages have
paths outside /usr that are part of their Essential functionality.
Prominent examples include /bin/bash and /lib64/ld-linux-x86-64.so.2.
See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118 for
full details.

smcv



Re: Steam Deck: good news for Linux gaming, bad news for Debian :(

2021-07-19 Thread Simon McVittie
On Mon, 19 Jul 2021 at 02:25:16 +, Paul Wise wrote:
> BTW, the Valve Arch Linux overlay thing appears to be here:
> 
> https://repo.steampowered.com/arch/valveaur/

FYI, that's an overlay for "upstream" Arch, for Arch users who want to
try out Mesa and kernel patches that are not yet mainline - analogous
to the various PPAs provided by https://launchpad.net/~kisak. The name is
analogous to AUR, the repository for unofficial addon packages for Arch.

When SteamOS 3 becomes available to the public, I would expect it to be
more likely to appear in some other directory on repo.steampowered.com.

smcv



Re: Need help with Multi-Arch in systemd

2021-07-19 Thread Joerg Jaspert

On 16194 March 1977, Simon McVittie wrote:


Would it be feasible for dak to have a list of binary package name
regexes mapped to a source package and a section/priority, and 
auto-accept

packages from the given source package that match the regex, assigning
the given section/priority, without manual action? That would let the
ftp team pre-approve src:systemd to ship /^libsystemd-shared-[0-9]+$/
in libs/optional, for example.


At some point somewhere I think it was already said, but in general: We 
love MRs, we are at https://salsa.debian.org/ftp-team/dak/ and something 
doing kind of auto-NEW processing is NOT out of the question.


Exact details have to be hashed out, but this is the wrong thread for 
that. While sometimes NEW can be annoying, it regularly catches bugs 
even if "only a small change in packaging" happened, so it does have 
some reason why its there. But yes, there are some candidates (hello 
kernel) that can make use of something with less humans involved.


Your example above doesn't sound too bad as a starter, though haven't 
put any further thought into it yet.


--
bye, Joerg



Re: Steam Deck: good news for Linux gaming, bad news for Debian?

2021-07-19 Thread Simon McVittie
(Disclosure: I work for Collabora, and I'm currently working on the
Steam Runtime.)

On Sat, 17 Jul 2021 at 13:48:32 +0100, Samuel Henrique wrote:
> As some of you already seem, we have very good news for the Linux
> gaming community, although somewhat bad for Debian:
...
> https://www.steamdeck.com/en/tech
> Operating System: SteamOS 3.0 (Arch-based)

I think this is still good for Debian, although arguably less good for
Debian than it might have been if it was directly based on Debian like
the earlier-generation Steam Machines were.

Rather than thinking "ugh, this isn't Debian", I'm thinking of this
as "good, this is modern Linux", and in particular not Windows!

SteamOS 2 is stuck on Debian 8, which is a long way out of support at
this point, so SteamOS needed a significant amount of work to catch up
with the last ~ 6 years of development somehow. You might notice from
https://repo.steampowered.com/steamos/dists/clockwerk/ that packaging
metadata for a Debian-9-based version appeared at one point, although
I don't think any packages appeared in that repository.

Obviously, as a Debian developer, the route I would have tried first for
that work would be to rebase onto a newer version of Debian - either
a stable release, or testing, or their own testing-like snapshots of
unstable like Ubuntu does - but Valve have chosen to rebase on Arch
instead. I am not the right person to say why, sorry.

Arch and Debian are not actually a million miles apart: they're both
package-based binary distributions in a nearly-FHS directory layout
(unlike for example NixOS), using glibc and GNU userspace (unlike for
example Alpine), booting with systemd by default (unlike for example
Devuan), community-maintained rather than driven by a single corporate
sponsor, with divergence from upstream generally being treated as a bug
but tolerated if there's a good enough reason why it's necessary.
(Of course, there are other distros that I could say similar things about,
Debian's threshold for what is a good enough reason for divergence from
upstream tends to be lower than Arch's, and we make different technical
decisions in various places.)

In terms of the upstream versions involved, Debian 11 is more like Arch
than it is like Debian 8 (although obviously the packaging and OS layout
are rather different!) so this still seems better for Debian in terms
of having the games that run successfully on the Steam Deck also run
successfully on Debian (and Ubuntu, and Fedora, and other modern
distributions).

I would hope that if Valve later decide that they need to be basing a
future SteamOS release on something that has periodic stable releases
and a security update story other than "upgrade everything", the obvious
choice would be Debian - but we'll have to see what happens.

> the gaming side of Linux still
> receives major improvements with new releases of things like Proton

You might have noticed that https://www.steamdeck.com/en/developers
emphasizes Proton over native Linux games at the moment, as a way
to get a broader range of games available, and the publicity videos
seem to be mostly (entirely?) things like Jedi: Fallen Order that are
Windows(-and-Proton)-only. Valve have said they're looking into getting
a solution for Windows "anticheat" mechanisms under Proton, which are
one of the biggest gaps in what Proton can do at the moment. Anything
they do to improve Proton is going to benefit Arch and Debian equally.

Current versions of Proton run in a container that is mostly Steam
Runtime 2 (+ graphics stack from the host system). Steam Runtime 2 is very
heavily based on Debian 10, with selected packages like SDL backported.
A Steam Runtime 3 prototype based on Debian 11 exists, although there's
no public release of it at the moment; if Proton starts requiring newer
versions of something, it would be natural to assume that the priority
of getting that prototype released will suddenly go up :-)

Similarly, it is possible to set native Linux games to be run in a
container via the "Steam Linux Runtime" compatibility tool, although
it isn't the default. Until recently, this was Steam Runtime 1 (based
on Ubuntu 12.04) plus the graphics stack from the host system. As of a
recent update, it's changed to Steam Runtime 2, with a few libraries taken
from Steam Runtime 1, and the host system's graphics stack as before -
so a combination of a Debian derivative, an old Ubuntu derivative and the
host system. I'm hoping that will result in better compatibility for games
that implicitly assumed they were actually running on something newer than
Steam Runtime 1.

These benefit ordinary Linux systems like Debian just as much as the
Steam Deck - perhaps more, because if the Steam Deck is running an
Arch derivative, it will always need to be up-to-date (because that's
the only way to get security updates in a rolling release), whereas
non-rolling-release systems can easily have libraries that are older than
those in a runtime container.

I'm also hopi

Re: Steam Deck: good news for Linux gaming, bad news for Debian?

2021-07-19 Thread Samuel Henrique
Hello Simon,

That's an awesome reply, thank you very much for having the time to
write all of this and adding the links, I have found the Steam runtime
bit particularly interesting.

> (Disclosure: I work for Collabora, and I'm currently working on the
> Steam Runtime.)

If you're ever feeling like it, I would love to see a talk from you
about the inner workings of Steam on Linux/Debian.

> Anyway, enough email-writing, time to get back to Assassin's Creed
> Odyssey, under Proton, in a Debian-10-based Steam Runtime 2 container,
> on a Debian 11 machine :-)

Nice, I loved Odyssey and Origins, although I had a little bit of
issues on Odyssey because I didn't go for the hard difficulty when I
knew I was gonna try and do all the side quests... I ended up too
powerful at the end xD.

Cheers,

-- 
Samuel Henrique 



Re: XFCE4 notes

2021-07-19 Thread Wookey
On 2021-07-14 20:05 +0200, Michael Biebl wrote:
> Am 14.07.21 um 19:54 schrieb Paul Sutton:
> > Hi All
> > 
> > Am I right in thinking that xfce4-notes has been removed?  I have
> > installed xfce4-goodies and it isn't installed.
> 
> From unstable/testing that is correct.
> 
> See
> https://tracker.debian.org/pkg/xfce4-notes-plugin
> specifically
> https://tracker.debian.org/news/1114283/removed-181-3-from-unstable/

Hmm. That's very annoying. I use it on a daily basis.

I see that the reason is given in 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955349
'uses unmaintained libunique'.

Any further comments on what would be required to get this back?  Was
libunique actually broken/a problem, or merely unmaintained? 

Is there something else one should use to do the same job, in which
case maybe the best thing is to update xfce-notes-plugin to use it?
Assuming that's not too difficult I don't mind doing some work to get
it back in a ship-able state.

Or it there something else that is still around that does the same job
as the notes plugin?  It's quite unusual in the way it works.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature


Re: XFCE4 notes

2021-07-19 Thread Paul Sutton



On 20/07/2021 00:09, Wookey wrote:

On 2021-07-14 20:05 +0200, Michael Biebl wrote:

Am 14.07.21 um 19:54 schrieb Paul Sutton:

Hi All

Am I right in thinking that xfce4-notes has been removed?  I have
installed xfce4-goodies and it isn't installed.


 From unstable/testing that is correct.

See
https://tracker.debian.org/pkg/xfce4-notes-plugin
specifically
https://tracker.debian.org/news/1114283/removed-181-3-from-unstable/


Hmm. That's very annoying. I use it on a daily basis.

I see that the reason is given in 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955349
'uses unmaintained libunique'.

Any further comments on what would be required to get this back?  Was
libunique actually broken/a problem, or merely unmaintained?

Is there something else one should use to do the same job, in which
case maybe the best thing is to update xfce-notes-plugin to use it?
Assuming that's not too difficult I don't mind doing some work to get
it back in a ship-able state.

Or it there something else that is still around that does the same job
as the notes plugin?  It's quite unusual in the way it works.

Wookey




Hi

gnotes was suggested, however I think the issue for users will be that 
while the files that store data are in plain text, if they are backed up 
before reinstalling debian,  they can just be dropped back in place and 
you can carry on with your notes.


For me xrce4-notes are stored in

~/.local/share/notes/Notes

Using a new program means work flow is impacted for people (I a thinking 
of others here)


I am happy to switch to gnotes, however would be good to have 
xfce4-notes working again.



Paul



--
Paul Sutton, Cert Cont Sci (Open)
https://personaljournal.ca/paulsutton/
Pronoun : him/his/he
OpenPGP : 4350 91C4 C8FB 681B 23A6 7944 8EA9 1B51 E27E 3D99

21st Debian Conference August 22 to August 29, 2021.
DebCamp from August 15 to August 21, 2021

https://debconf21.debconf.org/



OpenPGP_signature
Description: OpenPGP digital signature


Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-19 Thread Niels Thykier
Johannes Schauer Marin Rodrigues:
> Quoting Michael Biebl (2021-07-19 15:10:42)
>> [...]
>>
>> According to
>> apt-file search -x '^/(lib|bin|sbin)'
>> on my Debian sid/amd64 system, we have 1747 packages shipping 24583 
>> files in those directories.
> 
> more precisely, on amd64 unstable:
> 
> /bin 247 files
> /lib{32,64,x32} 83 files
> /lib/firmware 2379 files
> /lib/live 115 files
> /lib/modules 17500 files
> /lib/systemd 2221 files
> /lib/udev 614 files
> /lib/x86_64-linux-gnu 343 files
> /lib/* 441 files
> /sbin 547 files
> 
> So most files come from /lib/modules, where only 14 packages are involved,

and debhelper's dh_installmodules that will need to be tweaked to look
into /usr/lib/modules as well.
  But I have no idea if/when we would be ready for that and I will not
be changing debhelper until we are ready to move these bits to /usr.

Likewise for udev, here dh_installudev still uses /lib/udev and will
continue to do so until I know that udev also checks /usr/lib/udev.


If you are involved in udev or/and /lib/modules and know that they are
ready to move to the relevant directory under /usr, then please feel
free to file a bug against debhelper requesting that the /usr directory
will be used in the future.
  Please do also note in that bug report whether you also want debhelper
to automatically migrate existing files from /lib to /usr/lib.  This
should only be the case where we expect (almost) zero breakage from an
automatic "unordered" transition.

(NB: Please use a bug report as I will first be implementing this after
the bullseye release and rely to this thread is likely to be lost in my
inbox)

> /lib/systemd which will be fixed by an update to dh_installsystemd, and

Indeed, I have heard this request and the systemd maintainers confirmed
to me that systemd in bullseye checks both /lib/systemd and
/usr/lib/systemd.

I have a branch lying around trying to support this.  The main key
feature missing is the migration of /lib/systemd -> /usr/lib/systemd
(which needs to handle that both exist and merge them into one somehow).

> /lib/firmware where only 36 packages are involved. The remainder then doesn't
> sound so scary anymore as it only involves 656 unique packages and not 1747.
> And again many of those remaining packages will be fixed by an update to
> debhelper, correct? Given that 90% of source packages use dh, that would 
> reduce
> the number to a very manageable size.
> 

Indeed, about 90% of all packages uses dh according to trends.

Though for good measure I thought I would explicitly answer the proposal
(from another mail in this thread) that debhelper could move everything
from /lib to /usr/lib:

  No, I will not support an unconditional move from /lib to /usr/lib
  via debhelper during bookworm.

There are already two distinct examples in this thread for how this
could break things. I will be sticking to "targeted" migrations where
the major consumers have informed me that they are ready to support the
migration.


>> [...]
>>
>> Once we have this global switch to merged-usr, packages can bit by bit, 
>> completely independent, update their debian/rules to use --prefix=/usr 
>> and after a few years, we don't have any packages anymore installing any 
>> files in /. We could aid this process with a lintian check that flags
>> packages that install files in /(sbin|bin|lib)/.
> 
> So what what is actually the roadmap after the bullseye release? What is the
> way forward? Should I rather file bugs with patches against individual 
> packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
> already have a debhelper patch to do that move for us?
> 

For some, debhelper will your problem but there will packages that will
need manual migration. As I see it, there will *not* be "the debhelper
patch to fix them all" - even if there will /some/ debhelper patchers
that will fix "most of them".

Related, I have no intention of supporting / maintaining a rewrite of
"--prefix/PREFIX" parameters to configure/make/cmake or whatever.  (Not
saying anyone proposed it - but mentioning it as another "there will
definitely be manual clean up" data point).

~Niels