Re: /usr-merge: continuous archive analysis
Hi Luca, On Thu, Jul 13, 2023 at 01:38:16AM +0100, Luca Boccassi wrote: > On Wed, 12 Jul 2023 at 14:35, Helmut Grohne wrote: > > "risky" ones from becoming practically relevant). There is one kind of > > issue that may be actionable right now and that's the class of "empty > > directory" issues. If you notice that such an empty directory actually > > is not necessary (which probably is the case in the majority of cases), > > please go ahead and delete it from your package[2] or a file a bug with > > a patch. Also, please declare Conflicts or Replaces as usual as some > > forms of undeclared file conflicts also show up in the analysis. Another > > thing that helps now is cleaning up really old and unversioned Replaces > > as those may result in false negative detections. ... > > [2] List of packages that *may* be actionable: fwupd, gcc-snapshot, > > gretl, lib32lsan0, libjte-dev, libmpeg3-dev, libswe-dev, libx32lsan0, > > pcp, pkg-config, pkgconf, pkgconf-bin, python3-expeyes, systemd > > Does this mean I can nuke that empty directory from the systemd > package right now, without waiting for the rest of your proposal to be > implemented? Thanks for asking. We have empty directories in binary packages in lots of cases. Some of them are there for some technical need. In other cases, the empty directory is a oversight and entirely unnecessary. I am fairly convinced that the lib*-dev packages shipping an empty /usr/lib/pkgconfig fall in that latter category and can delete their /usr/lib/pkgconfig at no loss of functionality. On the flip side, my understanding is that at least pkgconf and systemd ship their empty directories on purpose as intentional integration points and that these are not unused and should therefore not be deleted. So while dumat can tell whether a particular empty directory participates in a /usr-merge issue, it cannot tell you what the right action is. In some of the cases, the action is delete and that can happen at any time while in other cases the action is to protect them from accidental deletion, which is something we have to defer until there is consensus on the chosen mitigation. Helmut
Re: /usr-merge: continuous archive analysis
Hi Ted, On Wed, Jul 12, 2023 at 10:23:08PM -0400, Theodore Ts'o wrote: > For those packages that are likely to be backported, would ti be > possible provide some tools so that the package maintainers can make > it easy to have the debian/rules file detect whetther it is being > built on a distro version that might have split-/usr, or not, or > whether we the package needs to do various mitigations or not? Please allow me to go into a little more detail as to why we get into a problem for backports and then circle back to your question. I currently imagine (and this has been vaguely circulated on d-devel a number of times) to facilitate the canonicalization using debhelper. We have minor disagreements on how exactly that should work. Let me give my preferred version while keeping in mind that this is not yet consensus: debhelper gains a new addon. It could be called usrmerge or something. If you enable usrmerge, debhelper would perform the path canonicalization for you. Your dh_auto_install could install to canonical and aliased paths, but the .deb would be canonicalized. Thus, you can easily opt into it by saying Build-Depends: dh-sequence-usrmerge. We may also add this addon to a new compat level as we expect that most packages in trixie will need it. Thus we're changing it from opt-in to opt-out. While you can merge like that, a number of packages will notice that you can simplify your packaging by e.g. changing --prefix=/ to --prefix=/usr or something similar and doing that canonicalization at dh_auto_install time. In doing this, they loose the information about how files were previously being split to / and /usr. For instance systemd needs extra effort to support the split layout and that support is going to be deleted soon. I expect this to happen for most packages. And this is the part that makes backporting hard in a way that honours the moratorium for bookworm-backports. I'm sorry for not having considered the use case of using a single debian/ directory tree for multiple distributions and releases, but it is fairly obvious in hindsight. Is checking for the presence of usr-is-merged good enough for your case? What I imagine you doing here is generally supporting split-/usr in e2fsprogs (for as long as you want to support building e2fsprogs on any system that needs such support) and then telling debhelper to enable the usrmerge addon whenever you don't need to support split-/usr. A fairly obvious candidate check would be checking for the presence of usr-is-merged, but while bookworm always contains that, we effectively want it to support split-/usr to facilitate upgrades. Some of the mitigations require the addition of a usrmerge-support package whose preinst will unconditionally reject unmerged systems. Would that be a suitable condition? > The point is before we lift the freeze, perhaps we can provide some > tools that make it easier for package maintainer to only "make > split-/usr support vanish" conditionally, so as to make life easier > for people who are doing the bookworm and bullseye backports? If going with the debhelper addon and keeping split-/usr support in the particular package otherwise, the one backporting can simply pass --without usrmerge to dh and be done. If using the usrmerge-support package as condition (could even be done inside debhelper), that would become automatic. > I don't mind keeping some buster and bullseye and bookworm schroots > around, and doing test-builds of the packages I build, and then making > minor adjustments as necessary to make sure things still work. > Combined with some test automation so that we can test to see whether > a package about to be uploaded to bullseye-backports won't break on a > split-/usr machine, and this should be quite doable. The real problem I see with such backports is a different one though. Consider the case where we reorganize a package (move files between packages) during the trixie cycle. In the normal scheme of things we have this sequence: * bookworm v1: split-/usr + original file layout upgrade * trixie in progress v2: merged-/usr + original file layout upgrade * trixie in progress v3: merged-/usr + reorganized package upgrade * trixie in progress v4: merged-/usr + reorganize again That reorganization may trigger the need for applying a mitigation and the main plan is to only apply such mitigations as-needed. Now when you backport this, you'd revert the merged-/usr part, so instead you end up with this: * bookworm v1: split-/usr + original file layout upgrade * bookworm-backports v2~bpo: split-/usr (reverted) + original file layout upgrade * bookworm-backports v3~bpo: split-/usr (reverted) + reorganized once package upgrade * trixie v4: merged-/usr + reorganized again This upgrade sequence may require a different mitigation, because we swapped the order of canonicalization and reorganization. I have not yet come up with an actual test case where this breaks, so maybe I'm really wrong worr
Re: /usr-merge: continuous archive analysis
On Wed, 12 Jul 2023 at 22:23:08 -0400, Theodore Ts'o wrote: > On Wed, Jul 12, 2023 at 03:34:38PM +0200, Helmut Grohne wrote: > > There is one major issue where I don't have a good answer: > > bookworm-backports. When this originally surfaced, Luca Boccassi > > suggested that we do not canonicalize in backports. That's easier said > > than done as the support for split-/usr will soon vanish from packages... > > ... detect whetther it is being > built on a distro version that might have split-/usr, or not I think it's important that we be clear about what does and doesn't need to be supported in each suite, because it's a bit confusing, and I think I can already see some of that confusion in this thread. For bullseye and older, there were two supported layouts, which we could call split-/usr and merged-/usr. In both of these layouts, it was the package's choice whether the canonical path for a "/usr-like" file inside the package's data.tar.* is in /usr (like /usr/bin/env) or not (like /bin/bash). In split-/usr, the physical path on disk matches the path that dpkg considers to be canonical. In merged-/usr, the aliasing symlinks like /bin -> usr/bin exist, so the physical path on disk is always in /usr, regardless of whether the canonical path is. For bookworm, as resolved in #978636, merged-/usr is officially the only layout that is supported. However, as resolved in #994388, we recommend that all packages in bookworm keep canonical paths in data.tar.* the same as in bullseye, for example /bin/bash. What we are now doing is trying to find a safe way to lift that restriction for trixie. Also, as resolved in #994388, we expect packages in bookworm to work at least minimally in a split-/usr environment, because the upgrade from bullseye to bookworm happens in an unspecified order and the system should work (at least well enough for recovery) at all points through the upgrade. In an effort to make sure that still happens, bookworm buildds still build packages in a split-/usr environment (because that way round has been observed to cause fewer bugs than building in a merged-/usr environment and installing in a split-/usr environment). For trixie and newer, the strategy Helmut proposes involves gradually changing the canonical path of all "/usr-like" files inside packages' data.tar.*, so that by the time we release trixie it is below /usr in all cases (for example /usr/bin/bash), which will result in the package only working as intended if the aliasing symlinks are in place. I think the question that Ted needs an answer to, for backportable packaging, is not actually whether the distro version has split-/usr as a supported configuration or not, but more like: whether it is safe or unsafe to make all /usr-like paths in data.tar.* be below /usr on this distro version. In bookworm or older, that is not safe; in trixie, our goal is that it becomes safe, and indeed recommended. (We are not there yet!) Because bookworm-backports is an extension of bookworm and users are expected to upgrade to bookworm before enabling bookworm-backports, split-/usr is not considered to be a supported situation there, so it's reasonable for packages that are outside the critical path for bootstrap and compilation to assume merged-/usr; but it is (probably) still not safe for those package versions to canonicalize all their /usr-like paths into /usr. > I had *one* > set of debian files that would successfully build on Debian stable, > Debian testing, Debian oldstable, Debian oldoldstable [etc.] This is sometimes a nice thing to be able to do, and we shouldn't break it for no reason, but it's easy for it to over-constrain the solution space to an extent that makes distro-wide changes harder or impossible. Intentionally keeping packaging files (especially d/rules) compatible with older distro versions comes with a risk of ossifying packaging and holding back adoption of desirable changes. For instance, I think we can see echoes of this in the way multiarch library paths have been best-practice in Debian for at least a decade, but unstable *still* has libraries in /usr/lib. (Perhaps it's time for some mass-bug filing or a Lintian check...) The main things that Debian delivers are the current stable release and the next stable release, and I think maximizing the quality of those should be our priority. smcv
sbuild and trivial local repository
Hello, I try to write a really simple script in perl which allows me to rebuild a bunch of packages using a file with a really simple syntax backport hkl git haskell-hkl https://repo.or.cz/hkl.git contrib/haskell ... I setup a chroot with the sbuild-debian-developer-setup -> ok Now I can build packages with the sbuild command from a checkout package -> ok Now I need to keep the generated packages to build the next one. So I want to setup a local repository own by the user `/home//repository` I use apt-ftparchive in order to maintain the Packages, Sources and Release file between each package build. -> ok My problem is: how can I bind the local repository into the chroot via sbuild. I understand that I can put this configuration in the /etc/schroot/sbuild/fstab configuration. /home/user/repository /tmp/repository none rw,bind 0 0 but the user repository path is a moving target. So my question is how can I do this mount from the sbuild command line thanks Frederic
Re: sbuild and trivial local repository
Hi, 在 2023-07-13星期四的 15:50 +0200，PICCA Frederic-Emmanuel写道： > Hello, I try to write a really simple script in perl which allows me to > rebuild a bunch of packages using a file with a really simple syntax > > backport hkl > git haskell-hkl https://repo.or.cz/hkl.git contrib/haskell > ... > > I setup a chroot with the sbuild-debian-developer-setup -> ok > > Now I can build packages with the sbuild command from a checkout package -> ok > > Now I need to keep the generated packages to build the next one. > So I want to setup a local repository own by the user > `/home//repository` > I use apt-ftparchive in order to maintain the Packages, Sources and Release > file between each package build. -> ok > > My problem is: how can I bind the local repository into the chroot via sbuild. > > I understand that I can put this configuration in the > /etc/schroot/sbuild/fstab configuration. > > /home/user/repository /tmp/repository none rw,bind 0 0 > > but the user repository path is a moving target. > > So my question is > > how can I do this mount from the sbuild command line Putting the set up of local repository aside, I believe that sbuild provides useful cmdline parameters like --extra-package=/path/to/a/dir , --extra-repository=spec and --add-depends=dependency , which may already meet you needs even without a local repository. Check sbuild(1) man page for more information. Thanks, Boyuan Yang signature.asc Description: This is a digitally signed message part
Re: systmd-analyze security as a release goal
Hi! On Thu, 2023-07-06 at 18:41:38 +1000, Trent W. Buck wrote: > "Trent W. Buck" writes: > > e.g. I expect "SystemCallArchitectures=native" to break for a lot of > > people (anyone doing dpkg --add-architecture) Yes, see #982456. > Short version: > > • SystemCallArchitectures=native + debianutils:i386 doesn't break > dpkg-db-backup.service. > • Probably savelog simply never calls an ARCHITECTURE-SPECIFIC syscall. That's because the only thing used from debianutils:i386 is savelog which is a shell script, and as long as you have dash:amd64 then that would work. The same would apply to any other interpreted program, as long as the interpreter matches the systemd native architecture. > • SystemCallArchitectures=native + nginx:i386 DOES break nginx.service. > • Neither journalctl nor coredumpctl makes it obvious this is WHY nginx > crashed. Right. > So I guess a program like savelog doesn't trigger it, because > it's so simple it never hits an architecture-specific syscall? Any binary program, will always at least call 1 syscall (either the implicit exit() from the libc CRT function calling main(), or an explicit exit() or raise() or similar to be able to terminate itself. So there's no escape from this. So SystemCallArchitectures=* seems fine as a local admin tool when they control their deployment or for very controlled distribution builders. Otherwise for a general purpose distribution where you can install foreign packages, it has always seemed like a bad idea to include them as part of the .deb, because it's depending on external implementation details, and this is not helped with the error failure mode. As I mentioned now on that report above, this could be made safe and be enabled for I guess all installed services (for free!) by "something" generating overrides based on the current list of dpkg architectures (could perhaps be added as a hook on dpkg for the add and remove architecture actions, or maybe as a generator, dunno). Thanks, Guillem
Re: systmd-analyze security as a release goal
* Guillem Jover [2023-07-13 19:36]: The same would apply to any other interpreted program, as long as the interpreter matches the systemd native architecture. This, by the way, includes the following scenario: * Trent W. Buck [2023-07-06 10:41]: dpkg --add-architecture arm64 apt update apt install mg:arm64 qemu-user-static systemctl edit dpkg-db-backup qemu is basically an interpreter for foreign machine code. If your threat model allows access to qemu-user-static for an attacker, they can run pretty much any binary is if it were native, and the whole SystemCallArchitectures hardening becomes meaningless. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Fwd: Wayland and NVidia driver conflict
-- Forwarded message - Från: inkrm Date: tors 13 juli 2023 kl 23:48 Subject: Wayland and NVidia driver conflict To: Hi I just installed Debian 12 'Bookworm' with KDE Plasma 5.27.5 , and I installed NVidia driver on my machine, following the official guide (https://wiki.debian.org/NvidiaGraphicsDrivers#Debian_12_.22Bookworm.22). When I rebooted my machine, I logged in Wayland, but it threw me back into login manager. I honestly prefer Wayland because it has better compatibility with 4K HiDPI displays (which I have), but I can't use it properly, and instead I have to use X11. I searched on web but didn't find anything, so I am writing this email, to ask what do I do. I'm gonna be very pleased to read your response. Kind regards, inkrm -- Sent with Tutanota
