Hello Theodore Ts'o, This mail dedicated to the potential confusion about Essentialness and Priority.
On Sat, Aug 19, 2017 at 06:51:11PM -0400, Theodore Ts'o wrote: > On Fri, Aug 18, 2017 at 01:21:57PM +0200, Andreas Henriksson wrote: > > I would like to breath some life into this bug report. I've been > > collecting information/brainstorming-ideas about minimal debian > > installation (debootstrap --variant=minbase) on > > https://wiki.debian.org/BusterPriorityRequalification > > > > The e2fsprogs was one of the packages that where questioned if > > it really needs to be Essential: yes / Priority: required. > > It is also one of the largest packages in the minbase set. > > There is one thing I can very quickly and easily do to to improve the > minbase set. We can significantly reduce the size of e2fsprogs (by > more half) by simply moving all of its locale files to a new package, > e2fsprogs-l10n. That shrinks installed size of e2fsprogs from 2825k > to 1330k. > > > step 1: Switch from Essential: yes to Important: yes > > > > This will not be much of a change in practise. Higher level tools > > like apt etc will still not want to uninstall the package, but > > you can now uninstall it using dpkg without getting the > > 'are you really sure you know what you're doing' question. > > I'd say the most important difference is that according to > > debian-policy (which doesn't know anything about Important: yes) > > the package is now no longer Essential and thus other packages > > are now allowed to add dependencies where needed. > > I think you mean "don't know anything about Required: yes" above, but No, AFAIK there's no such this as "Required: yes". Only "Essential: yes" and (the newish) "Important: yes". I guess you're mixing this up with the "Priority: ..." field because of the somewhat confusing naming. > yes, it seems largely safe to do this. Given that Debian policy > states that packages should not be made "Essential" without a > discussion on debian-devel, I'd probably want to also raise this as a > proposal on debian-devel before changing "Essential: yes" to > "Essential: no". Not sure the reverse of making something Essential automatically applies but sure a debian-devel discussion might be useful, if not only to increase actual debian development related mail on the list. (Fwiw, I recently sent out a discussion related to src:util-linux and lowering the Essential-ness of the packages which resulted in very little useful discussion. Doesn't seem too many people in genreral are interested in the core stuff.) > > > step 2: Dowgrade from Priority: required to Priority: important > > > > This means e2fsprogs will still be part of every normal install, > > just not part of 'debootstrap --variant=minbase' (e.g. buildd chroots > > and other specialized systems). > > Pitfalls with this change would be that every package who needs > > something from e2fsprogs (eg. lsattr, chattr) during build phase > > must have a build-dependency on e2fsprogs or they will FTBFS. > > Given we're still early in the Buster development cycle, I would > > personally think this change is pretty safe to do now but depends > > on how much fuzz you're willing to stir up. I'd like to offer > > my help in fixing this (by filing bug reports and doing non-maintainer > > uploads where needed). > > It's not just the builders; it will also be the Debian Docker image. I haven't checked, but I would assume the (normal) debian docker images are built from a normal default debootstrap, which would include "Priority: important" packages and thus e2fsprogs would still be there. I would expect only the 'slim' variant to be built from --variant=minbase and anyone using slim would ofcourse expect that it contains less software than the normal variant. > And while it might be fair to say that containers won't need "mount", > it's somewhat more likely that there might be build containers that > will assume the existence of mke2fs. > > I'm not sure how to address that, since at least with debian packages > it's a relatively closed universe so we would discover problems as the > reproducible build system tried to rebuild various packages, for > example. But I could imagine so Docker container designed to rebuild > cyanogen or AOSP (for example) that would be assuming e2fsprogs to be > present, and would blow up as a result. Apart from my above comment, the best we can do is to document the change in a high-level understandable way in the Release Notes. While not everyone reads the release notes, hopefully atleast the people making downstream redistributions does.... Then they can relay the information to their users in their most suitable way. > > This is one of the reasons why, if the goal is to shrink the size of > minbase shrinking e2fsprogs by removing all of the locale files is > probably the biggest bang for our buck that we can do right away, and > is guaranteed to be 100% safe --- since there won't be any users of > the e2fsprogs locale files except for e2fsprogs itself, and most of > the time they aren't used at all (at least in the English speaking > world :-). (And of course, in build chroots for reproducible-build > reasons.) Shrinking the size is mostly a nice bonus. Shrinking the number of packages in Essential is the real win! By making a package Essential: yes you (via debian-policy) forbit packages to declare a dependency and thus the relationships between packages can not easily be tracked. At the same time deinstalling an Essential: yes package is just one additional step (answering one additional confirmation question). Consider the case where I build my own minbase chroot/container and then wants to shrink it further. I go uninstall everything (except maybe apt just because) that I can because of cruft in the minbase installation (ie. caused by the previous version of debian-policy now fixed in 4.0.1 that said packages where not allowed to depend on packages with lower priority so people bumped priorities of libraries etc and then left them as 'Priority: required' even after other packages that caused this priority bump dropped the dependencies again). After that I investigate what's still installed and gets rid of this I "obviously don't need in a container" like e2fsprogs and confirm the extra question about me really wanting to do this.... After all my host will take care of actual disk management so why would I need e2fsprogs installed?! My generic base is now done and I archive it for reuse. Later on I use the previously created base for building and application container. I use apt to install 'foorandombar' package which happens to use chattr in its postinst maintainerscript. The package installation explodes and like most users I don't understand what happened, just throws some random 'apt-get update; apt-get install -f' commands around to try to fix my now "completely broken" system which "can't even install packages anymore" (since it's stuck on a half-installed one). This would have been solved if 'foorandombar' where allowed to simply declare a dependency on e2fsprogs! Using the 'Important: yes' field instead of 'Essential: yes' allows this, with no other real practical difference! No ill side effects comes from this change alone! (It's first when we start altering priority that things might become more sensitive.) Another example is that helmut has occationally thanked me for the changes I've requested since they positively affect bootstrapping debian. > > By the way, the other program that other packages may be using is > logsave(8) -- the sysvinit and initramfs-tools packages use logsave at > the very least. > > > Bonus step: consider chattr/lsattr move to util-linux? > > > > As I understand it, chattr and lsattr used to be ext-specific. Nowadays > > more filesystems seems to have implemented the same interface and thus > > made the tools useful in a broader scope. > > Yeah, mumble. I'd have to think about that. We're still adding new > ext4-specific flags to chattr and lsattr, so I'm less enthusiastic > about moving them. Ok, just a thought... Maybe they should indeed stay. Lets forget about this. Another possibly stupid idea might be to split these two into a separate binary package (still built from src:e2fsprogs) and mark that one as Essential: yes. > > The other libraries/packages that were moved to util-linux where the > uuid and blkid libraries (and associated userspace utilities). Those > were largely no longer changing in any ext4-specific way, so it was a > lot easier to be willing to move it to util-linux. The logsave binary > falls in that category, and so I would have no objections to moving > logsave to util-linux. It's stayed in e2fsprogs mostly out of > inertia. Since logsave is only used by 2 packages in the archive I think it's easier to just add dependencies to those two. (Keeping the chattr/lsattr binaries in the Essential set sounded more useful as we'd need to fix hundreds of packages, which I still think is doable though but just a bit tiresome.) I already followed up on this in the separate 'blocker' bug report at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=619785#13 Regards, Andreas Henriksson
