Re: proposal: dhcpcd-base as standard DHCP client starting with Trixie
On Fri, Jul 07, 2023 at 06:07:58PM -0600, Sam Hartman wrote: > > "Bastian" == Bastian Blank writes: > Bastian> Why do we need to have the priority adjusted instead of fix > Bastian> d-i to install what it knows the user needs? > Because it's not just D-I, it's bootstrapping in general. > For that reason I support raising the priority. No, it isn't. Bootstrapping does not magically enable ifupdown to do dhcp, this requires explicit config, as done by d-i. And you can expect the tool doing the config to make sure the appropriate packages are installed. Bastian -- It is undignified for a woman to play servant to a man who is not hers. -- Spock, "Amok Time", stardate 3372.7
Re: Second take at DEP17 - consensus call on /usr-merge matters
Hi Sam,
* Sam Hartman [2023-07-07 08:50]:
TL;DR:
It looks like if we do not want to encode merged /usr into the bootstrap
protocol, we must keep some aliases and cannot move all files in
data.tar.
I think removing all aliasing is important and so I am firmly in the
camp of doing usrmerge in the bootstrap protocol.
We seem to agree on the goal (if by "removing all aliasing" you mean
stop shipping any files in /bin and /lib), but I firmly disagree
with your premise.
Let me elaborate.
So, my understanding of the specific proposal is that:
We in are in category 1:
1. Don't move. We just keep those files that require a particular
location (such as /bin/sh or the dynamic loader) in their
non-canonical location. As such, maintainer scripts will be able to
run and perform the conversion to symbolic links afterwards.
In Helmut's mail to which you are referring, he later explains that
the bootstrap categories come from two orthogonal questions:
Q1. Who is responsible for the top-level symlinks?
Q2. Are there any packages which install files through a top-level
symlink, i.e, an aliased path?
We want the answer of Q2 to be No. The answer to Q1 has, as far as I
can tell, three alternatives:
Q1a. Symlinks are created by the bootstrap tool early on
Q1b. Symlinks are shipped in a package (e.g. base-files) and will
be created in the initial unpack phase
Q1c. Symlinks are created by a usrmerge conversion, either in a
maintainer script in the configuration phase or in the final
cleanup phase by the bootstrap tool
Q1a puts us in Category 4 in Helmut's taxonomy (or as I am going to
call it from now on, HC-4); I'll skip that discussion for now.
Q1c defers the symlink creation to such a late stage that we run
into the ABI problems outlined in other mails; basically, this is
the answer that may force us to ship the dynamic loader in /lib{64,}
and the POSIX shell in /bin forever, and firmly entrench the
usrmerge conversion into our bootstrap process. In my opinion, we
can rule this out completely, because it does not finish the
usrmerge transition as much as hide it away in a dark place where
nobody dares to look. We would end up in HC-1 or HC-3.
Q1b is where it gets interesting, because the way our bootstrapping
protocol currently works, *all* packages will be unpacked first, and
only *then* the configuration phase begins and maintainer scripts
are executed. This means that we *can* move all files to their
canonical paths, because the top-level symlinks will be created
juuust early enough. Depending on the answer to Q2, we end up in
HC-1 or HC-2.
Q1b is my favorite, because it allows us to move the files, finish
the transition, and regain the property that a Debian system is
fully described by package metadata (HC-2).
Q1b has one important failure mode while there is any package left
in the essential set which has not moved its files to the canonical
location: as the initial unpack order is undefined, it is possible
that such a package gets unpacked early enough to create real /bin
or /lib directories, which would then conflict with the subsequently
unpacked top-level symlinks. I think this is the "fragility" other
people keep referring to. Any other usrmerge transition
problem will affect Q1a as much as Q1b, because after the unpack
phase, both are in the exact same state.
The simplest solution for the unpack problem would be to amend the
bootstrap protocol and have base-files be unpacked first. As soon as
all packages have properly transitioned and moved their files to
/usr, we are free to remove that requirement again, because the
unpack order will no longer matter.
Alternatively, we could move all files except for the few critical
ones (/bin/sh, dynamic loader), and then have a coordinated flag day
upload where we add the base-files symlinks *and* move all remaining
files, but this feels quite a bit more fragile to me.
Cheers
Timo
--
⢀⣴⠾⠻⢶⣦⠀ ╭╮
⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │
⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄ ╰╯
signature.asc
Description: PGP signature
Re: Second take at DEP17 - consensus call on /usr-merge matters
Hi Sam,
On Fri, Jul 07, 2023 at 08:50:49AM -0600, Sam Hartman wrote:
>
> TL;DR:
> It looks like if we do not want to encode merged /usr into the bootstrap
> protocol, we must keep some aliases and cannot move all files in
> data.tar.
Reading both of your recent mails left me very confused. It is now
obvious to me that we have a misunderstanding (at least one) and I am
not exactly sure how we can get to a point where we talk about the same
things.
In your second mail, you classify 3C (the one about not changing the
bootstrap protocol by adding aliasing symlinks to base-files) as a
category 1 solution (where we leave some files in their aliased location
to facilitate bootstrap). In reality, 3C is fully incompatible with
category 1 as the premise of 3C is that every essential package has all
of its files moved. This directly contradicts your TL;DR here.
Let me try to ignore much of the past conversation and instead explain
the bootstrap-relevant part of the transition plan that I see as
favourable (i.e. 3C). Consider this an opinionated presentation for one
of multiple ways to move forward.
We first move towards a category 1 solution where we move files to their
canonical locations as much as possible without breaking the current way
of bootstrapping that relies on either pre-creating the symlinks
(debootstrap) or usrmerge.postinst (mmdebstrap, cdebootstrap). This
process has lots of non-obvious details that I'll skip for the sake of
the bootstrap topic here. Let us for a moment assume that we'd manage to
get to this category 1 solution where most files (in essential packages)
have moved their files to canonical locations and we're left with some
packages (e.g. libc6, dash, util-linux, ...) that could not move their
files.
There is one aspect, I want to get into more detail as it is partially
relevant to bootstrapping and this is protecting the aliasing symlinks
(DEP17-P9). Here, I am selecting DEP17-M4 as the relevant mitigation.
That amounts to adding a usrmerge-support package that creates
protective diversions for the aliasing symbolic links and assigns them
to base-files. Both base-files and any package that moves files out of
aliased locations has to gain Pre-Depends: usrmerge-support in order for
this method to be effective. So we'll likely see debhelper's
${misc:Pre-Depends} that we added for multiarch-support return.
Until this point, we can still decide whether we do 3B or 3C and I am
explaining what happens in the 3C choice now.
Concurrently with the earlier changes, we modify debootstrap.
debootstrap requires uploads to bookworm and bullseye anyway, because we
will have to change --variant=buildd to become merged for trixie and
forward while currently debootstrap always creates --variant=buildd as
unmerged. On top of this necessary change, we add a change relevant to
3C. Currently, debootstrap creates the aliasing symbolic links prior to
the initial package unpack. I want to swap these operations. In doing it
afterwards, debootstrap cannot just create the symlinks but may have to
perform an actual merge in much the same way that usrmerge does now
except for dropping the atomicity requirement (as we don't need that in
the bootstrap setting). Other than that, this change is fully backwards
compatible (since bootstrapping as unmerged and the installing usrmerge
also works) and will continue to debootstrap a merged bullseye, bookworm
and trixie in the same way as before. I'll get into why we do this in a
moment.
Then, we assume that updating debootstrap has happened, usrmerge-support
exists in trixie and the number of essential packages shipping files in
aliased locations is about 10. We now move away from category 1. At this
point my original categorization becomes a little difficult, so I defer
explaining what we move to. We prepare changes to canonicalize all those
remaining packages (only essential ones) to canonicalize their paths and
also prepare a change to base-files to change the type of the aliasing
symlinks from directories to symlinks. While it technically is a
directory-to-symlink conversion from a dpkg point of view, that
conversion has already happened on the filesystem level, so this
practically is a change to the dpkg database only. Now we upload all of
these packages concurrently. This is when mmdebstrap and cdebootstrap
temporarily break. Once all of the binary packages have been built and
installed into the archive, things should work again.
Due to having modified deboostrap, we arguably have moved into category
4. Since that change is backwards-compatible and has been uploaded to
bullseye and bookworm, we can kinda pretend that the bootstrap protocol
never was different and therefore say that we move into category 2, but
without the reasons originally given for why this cannot work. Had we
skipped that change to debootstrap, unpacking base-files would make
debootstrap fail, because it passes -k to tar and when tar would try to
create the /bin -> usr/bin symlink include
Re: Second take at DEP17 - consensus call on /usr-merge matters
On 08/07/23 09:44, Timo Röhling wrote: Alternatively, we could move all files except for the few critical ones (/bin/sh, dynamic loader) Allow me to add some hard data to this discussion. In essential (proper) there are 153 files that need to be moved, distributed across 15 packages (+ base-files). Packages and number of files to be moved: 1 libc-bin 1 login 1 sed 1 tar 2 bash 3 dash 3 debianutils 3 dpkg 3 grep 5 hostname 7 sysvinit-utils 14 gzip 28 coreutils 36 util-linux 45 ncurses-base Even the most convoluted and lock-stepped procedure can surely be carried out over a single day in an all-hands-on-deck effort. Especially if the files of non-critical packages are moved before the flag day. Regards, -- Gioele Barabucci
Re: Bug#1040590: ITP: tecla -- keyboard layout viewer for the GNOME desktop
By the way, there is an existing old unrelated libtecla package in Debian. Upstream for new Tecla does not believe there will be a name conflict. https://gitlab.gnome.org/GNOME/tecla/-/issues/6 Thank you, Jeremy Bícha
Re: Second take at DEP17 - consensus call on /usr-merge matters
Hi Gioele, * Gioele Barabucci [2023-07-08 10:53]: Even the most convoluted and lock-stepped procedure can surely be carried out over a single day in an all-hands-on-deck effort. Especially if the files of non-critical packages are moved before the flag day. I agree. Maybe my wording was too harsh; as a matter of fact, I think the flag day will work just fine, and even if we stretch the upload over more than a dinstall cycle, the absolute worst that will happen is that bootstrapping will not work for a few hours or days. I initially thought that the unpack order would be simpler to implement without breaking compatibility with older releases, but after Helmut's sibling reply in this thread, I don't think there is much of a difference, if at all. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Re: Second take at DEP17 - consensus call on /usr-merge matters
> "Helmut" == Helmut Grohne writes: Helmut> Hi Sam, Helmut> On Fri, Jul 07, 2023 at 08:50:49AM -0600, Sam Hartman wrote: >> >> TL;DR: It looks like if we do not want to encode merged /usr into >> the bootstrap protocol, we must keep some aliases and cannot move >> all files in data.tar. Helmut> Reading both of your recent mails left me very confused. It Helmut> is now obvious to me that we have a misunderstanding (at Helmut> least one) and I am not exactly sure how we can get to a Helmut> point where we talk about the same things. Helmut> In your second mail, you classify 3C (the one about not Helmut> changing the bootstrap protocol by adding aliasing symlinks Helmut> to base-files) as a category 1 solution (where we leave some Helmut> files in their aliased location to facilitate bootstrap). In Helmut> reality, 3C is fully incompatible with category 1 as the Helmut> premise of 3C is that every essential package has all of its Helmut> files moved. This directly contradicts your TL;DR here. Yes, we definitely have a misunderstanding. I am at a conference today without much focus for Debian. I did try to read your mail, but mostly my brain rebelled and argued that's too complicated. I'll try again tomorrow evening US time or Monday when I can give it real focus. Here's how we got to the understanding. * I asked for a specific proposal to evaluate for 3C * you responded: >> So I think that to argue for 3C you need a specific proposal >> about what happens. Helmut> https://lists.debian.org/20230517093036.ga4104...@subdivi.de >Unfortunately, we're not yet done here. When (for instance) dash is the >last package to move /bin/sh and such to /usr and you upgrade dash, dpkg >notices that no package owns /bin anymore. Thus it helpfully deletes >/bin (the symlink). You're not happy when this happens. We remember the >silver bullet: diversions! So dash.preinst could dpkg-divert --no-rename >--divert /bin.usrmerged -add /bin and dash.postinst could revert that. >Unfortunately, such a diversion affects every package but dash and we >want it exactly the other way round. So what we could do is pass >--package dash-usrmerged (which must not exist). Then it'll actually >keep /bin safe. Unfortunately, we don't know whether dash or bash will >be the last package owning /bin, so both of them need this diversion and >this is a conflict. >And as if that wasn't enough, we also run into issues around hard links. >As dpkg unpacks a canonicalized gzip, it notices that /bin/gunzip (which >is scheduled for deletion) has the same inode as the new >/usr/bin/uncompress (because gunzip and uncompress are hard linked). As >far as I can see, all we get here is a warning and both files survive >the unpack. It is not clear to me whether this happens by chance or >reliably. >dpkg: warning: old file '/bin/uncompress' is the same as several new >files! (both '/usr/bin/gunzip' and '/usr/bin/uncompress')^ >dpkg: warning: old file '/bin/gunzip' is the same as several new files! >(both '/usr/bin/gunzip' and '/usr/bin/uncompress') >Given what I've seen, I'm fairly convinced that I haven't reached the >bottom of it and I'm ready to conclude that this approach is fragile - a >property that is most unwelcome when we deal with the essential set. >So what's left is category 1. I looked into what the minimum set of So, I assumed we were in a category 1 solution. Because I read your message that you pointed to as a proposal for how 3C worked, and before the concluding paragraph you said we were in a category 1 solution. I at least understand a category 1 solution. It's fairly easy to reason about; I even think it's safe. I don't like it for the architectural reasons I explained. But well, if you tell me to go read a message that will contain a proposal for what you want to do, and the last thing I read there--the only thing that your analysis claims works--is category 1, that's going to be left in my mind. I really am trying to understand here, but this is exceeding the level of complexity I can keep in my head. That in and of itself is concerning: I'm well above average at being able to work through this sort of thing. I'll take a stab at your most recent email later. Things I noticed already though: * You are describing things in terms of changing. Bootstrapping happens against a static archive. It might be easier to analyze if you describe what the end state looks like in terms of bootstrapping rather than changes to get there. * You do not talk much about upgrades. Upgrades happen against a moving archive. There, talking about the changes, and why the changes are always safe is important. So for me, a 3C proposal would have two components: 1) An explanation of what the archive looks like at time of bootstrap (and changes to any bootstrap programs) so I can reason about whether bootstrap works. 2) An argument of safety o
Bug#1040645: ITP: ruby-circuitbox -- Robust circuit breaker that manages failing external services
Package: wnpp Severity: wishlist Owner: Vivek K J X-Debbugs-Cc: debian-devel@lists.debian.org, vive...@disroot.org * Package name: ruby-circuitbox Version : 2.0.0 Upstream Contact: Fahim Ferdous * URL : https://github.com/yammer/circuitbox * License : Apache-2.0 Programming Lang: Ruby Description : Robust circuit breaker that manages failing external services It protects your application from failures of its service dependencies. It wraps calls to external services and monitors for failures in one minute intervals. Using a circuit's defaults once more than 5 requests have been made with a 50% failure rate, Circuitbox stops sending requests to that failing service for 90 seconds. This helps your application gracefully degrade. This package will be maintained by Ruby Packaging team.
Bug#1040647: ITP: ruby-neighbor -- Nearest neighbor search for Rails and Postgres
Package: wnpp Severity: wishlist Owner: Vivek K J X-Debbugs-Cc: debian-devel@lists.debian.org, vive...@disroot.org * Package name: ruby-neighbor Version : 0.2.3 Upstream Contact: Andrew Kane * URL : https://github.com/ankane/neighbor * License : Expat Programming Lang: Ruby Description : Nearest neighbor search for Rails and Postgres Ruby package used for finding nearest neighbor search for Rails and Postgres. This package will be maintained by Ruby Packaging team.
Bug#1040664: ITP: warp -- securely send files between machines
Package: wnpp Severity: wishlist Owner: Matthias Geiger X-Debbugs-Cc: debian-devel@lists.debian.org, jbi...@debian.org, matthias.geiger1...@tutanota.de, Debian GNOME Maintainers -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 * Package name: warp Version : 0.5.4 Upstream Contact: Fina Wilke * URL : https://gitlab.gnome.org/World/warp * License : GPL-3+ Programming Lang: Rust Description : securely send files between machines Warp a GUI cient for magic-wormhole. It allows to securely send files between machines. It's fully compatible with the CLI client and supports android apps on top. As of today it's still missing a few dependencies; I will continue to work on those. warp will be maintained within the Debian GNOME team. regards, werdahias -BEGIN PGP SIGNATURE- iQJUBAEBCgA+FiEEwuGmy/3s5RGopBdtGL0QaztsVHUFAmSptTcgHG1hdHRoaWFz LmdlaWdlcjEwMjRAdHV0YW5vdGEuZGUACgkQGL0QaztsVHU1uA//We94Oqxmib0W eo4Jo7aSMPnhcZdhBd3t4rKbK1VTAaUM59fTD3qqwAnSLuUfHWOjGsT+ZkjDI8Tl 3xCOGf5L84jeyHNsif1E88V8yhsu+UVjqtjtYWZsiz58TqLV7TZNwn3HHcqU/vG0 B/nB+3uHviHtF97Ijnm1a51DANRIbF6pHnXAhueDSvMnjLXpYS6NFLcFe6LsocSu CqjUFqZlk8dRvoNxV+1wOaWmobMPLqAo4F2DWmx68pwlyoAn/MkcRQ+1V7vMX/RV Bi4MI5dKdNEFszHPoC0Nx/Wuf9RCFI9eJvggEfRXs7C/bqrAB5EBRLx0JiXwgJi+ cEJPbnFrXX9j6obqrCKr7DrIRe2Fa8JptGvTTs8mgimFMzqxAbuHFPfQR5Xu4pIT H7JTbtDGpTbfG1q4cXaEWozTD+Gkg1l/S0g7dCz1ITDvv4UYLFMKCvR8Bt08UX9R GbTS0jCDp968tFHjzUVX3vxK1JHtUGcxfgshMbOkji2MQbgxN+rK9//W44DeyRFc /v2sqbhdLB0PcBysXfI42mjsZZzm4NZx4zJ0PXl/sbYh7Zau93H/t7UyoW5lIyzO kBqavh+8mCxTv3EPP1p38COOo8XObv/c8qp9hP93mwZurFPR1kBp32PnlzIuxWiC mWB3leIhH3P+BMit2x+NCAI3RUWw1mw= =BCZX -END PGP SIGNATURE-
