Re: Supporting alternative zlib implementations
On Wed, Sep 25, 2024 at 01:55:17AM +0200, Guillem Jover wrote: > Hi! > > On Wed, 2024-09-25 at 00:39:10 +0200, Fay Stegerman wrote: > > * Guillem Jover [2024-09-24 17:45]: > > > Personally, I think fully migrating from zlib to zlib-ng would sound > > > great (even for trixie), but I guess we can take it slow if you do not > > > feel confident or have concerns over this. > > > > As using an alternative zlib implementation could impact Reproducible Builds > > [1], I would recommend taking that into consideration before deciding on > > this > > kind of change. > > Ah, this is related to something I wanted to mention too and forgot. > > I don't think the specific case you mention is in itself a concern for > Debian, because we only guarantee reproducibility given the same inputs, > which includes the set of packages and their versions that were used > when building the binaries. So if there was a switch, those would end up > being recorded as well, and used when reproducing the outputs. And this > could also happen with a newer version of zlib itself. > > The problem though is, that because the compressed stream is going to > change, that can make certain test suites fail if we perform this > switch, which I think would be the main fallout that we'd see from > this and would need manual fixing, although I assume Fedora has probably > handled most of these already. For example when I added explicit > zlib-ng support to dpkg, I had to fix its test suite to parametrize > sizes for test artifacts. As someone who recently tested a rust port of zlib-ng, another factor to take into account is that while zlib-ng is faster, it also looks like it compresses less at the same compression level. Using higher compression levels with the same compression rate is also usually faster too, but without touching the compression levels, you end up with something that compresses in less time, but also with a bigger output. Mike
Re: Supporting alternative zlib implementations
* Guillem Jover [2024-09-24 17:45]: [...] > Personally, I think fully migrating from zlib to zlib-ng would sound > great (even for trixie), but I guess we can take it slow if you do not > feel confident or have concerns over this. As using an alternative zlib implementation could impact Reproducible Builds [1], I would recommend taking that into consideration before deciding on this kind of change. - Fay [1] https://lists.reproducible-builds.org/pipermail/rb-general/2024-September/003526.html
Re: Supporting alternative zlib implementations
Hi! On Wed, 2024-09-25 at 00:39:10 +0200, Fay Stegerman wrote: > * Guillem Jover [2024-09-24 17:45]: > > Personally, I think fully migrating from zlib to zlib-ng would sound > > great (even for trixie), but I guess we can take it slow if you do not > > feel confident or have concerns over this. > > As using an alternative zlib implementation could impact Reproducible Builds > [1], I would recommend taking that into consideration before deciding on this > kind of change. Ah, this is related to something I wanted to mention too and forgot. I don't think the specific case you mention is in itself a concern for Debian, because we only guarantee reproducibility given the same inputs, which includes the set of packages and their versions that were used when building the binaries. So if there was a switch, those would end up being recorded as well, and used when reproducing the outputs. And this could also happen with a newer version of zlib itself. The problem though is, that because the compressed stream is going to change, that can make certain test suites fail if we perform this switch, which I think would be the main fallout that we'd see from this and would need manual fixing, although I assume Fedora has probably handled most of these already. For example when I added explicit zlib-ng support to dpkg, I had to fix its test suite to parametrize sizes for test artifacts. I think it would be pretty easy to at least see the extent of this fallout by performing a mass rebuild for packages build-depending on zlib1g-dev with a zlib-ng version. Thanks, Guillem
Re: Supporting alternative zlib implementations
Le Tue, Sep 24, 2024 at 03:58:10PM +0200, Mark Brown a écrit : > >zlib-ng: https://github.com/zlib-ng/zlib-ng Hi Mark, just out of curiosity, would the carbon footprint of Debian be lower or higher after replacing zlib with zlib-ng? Have a nice day, Charles -- Charles Plessy Nagahama, Yomitan, Okinawa, Japan Debian Med packaging team http://www.debian.org/devel/debian-med Tooting from home https://framapiaf.org/@charles_plessy - You do not have my permission to use this email to train an AI -
Reproducibility across different hardware
Hi, On 9/25/24 08:55, Guillem Jover wrote: So if there was a switch, those would end up being recorded as well, and used when reproducing the outputs. And this could also happen with a newer version of zlib itself. I have a POWER9 box, which includes a NX-GZIP coprocessor, which is currently not used for anything, but it makes even -9 compression very cheap (9 GB/s). If I were to use it (probably through the kernel subsystem), I should probably record it somewhere. Thinking about it, I'd expect POWER to generate different output than x86 when building an arch:all package with the result of some floating point computation, at least when -ffast-math is active (x86 being wrong). Simon
Re: Supporting alternative zlib implementations
Hi! On Tue, 2024-09-24 at 15:58:10 +0200, Mark Brown wrote: > In the past I've pushed back on doing anything here since zlib is > essential and it seemed better to be consistent over the ecosystem than > to use a more niche implementation, and some of the early optimisation > efforts had not worked well on CPUs other than their immediate targets. > However given the user feedback and looking at the Fedora experience I > think it might be time to reevaluate that. Great! I'm happy to hear that. > Obviously it's far too late to do anything with the default for trixie, > we might want to evaluate doing something after the release but for now > it's too late. Personally I don't think it's too late, there should be several months until the freeze, and I think if we wanted to switch we could perhaps do a staged transition and see how it goes and only do the final replacement if everything seems fine. > There's been some ongoing discussion (which sadly I wasn't looped into > most of) of zlib-ng in WNPP: > >https://bugs.debian.org/1002056 > > with some packaging done, but not AIUI building the zlib compatible ABI > for zlib-ng yet which would allow it to be used as a replacement. > Adding support for the compatible ABI allowing it to be an alternative > for standard zlib seems to me like an obvious step we could take, it > would need a lot of care given that zlib is essentially but would let > people get zlib-ng if they wanted, and if there are problems it can be > held in unstable (or experimental) to avoid impact on trixie. This > would allow people to kick the tires. Sorry, I've been meaning to bring this up again to your attention, given that as you mention zlib-ng has seen steady development and buy in from the community at large. But at the same time, I've been both a bit reluctant to upload anything to avoid the impression of some kind of attempt to a hostile takeover, and to bring this up to you as from your earlier push back I thought that would require some (perhaps) exceeding changed circumstances. But given your mail, I'm happy to work on this again and start with say uploading some initial stuff into experimental for example, after this thread settles a bit? (I'll start by refreshing the packaging first though.) > Does anyone have thoughts on this? Personally, I think fully migrating from zlib to zlib-ng would sound great (even for trixie), but I guess we can take it slow if you do not feel confident or have concerns over this. Also if you'd prefer to take over the zlib-ng ITP, as a continuation of zlib, that'd seem fine with me too. Thanks, Guillem
Supporting alternative zlib implementations
A recurrning question with the zlib package in Debian is interest in the various alternative zlib implementations that are out there. There was a long period where upstream zlib development seemed very stalled, during that period people who wanted improvements started forking their own projects. The main ones I'm aware of are: zlib-ng: https://github.com/zlib-ng/zlib-ng chromium: https://chromium.googlesource.com/chromium/src/third_party/zlib zlib-ng seems pretty healthy, the chromium fork is less generally active but is used by Chrome/ChromeOS which is a big userbase. The main thing people seem excited about is performance work for modern platforms though both projects have been doing other work on the code. Unfortunately it looks like there is little interest in bringing these forks together in spite of zlib's upstream development having picked up a bit again. Fedora did a transition to zlib-ng relatively recently, in version 40: https://fedoraproject.org/wiki/Changes/ZlibNGTransition https://packages.fedoraproject.org/pkgs/zlib/zlib/ https://packages.fedoraproject.org/pkgs/zlib-ng/zlib-ng/ In the past I've pushed back on doing anything here since zlib is essential and it seemed better to be consistent over the ecosystem than to use a more niche implementation, and some of the early optimisation efforts had not worked well on CPUs other than their immediate targets. However given the user feedback and looking at the Fedora experience I think it might be time to reevaluate that. Obviously it's far too late to do anything with the default for trixie, we might want to evaluate doing something after the release but for now it's too late. There's been some ongoing discussion (which sadly I wasn't looped into most of) of zlib-ng in WNPP: https://bugs.debian.org/1002056 with some packaging done, but not AIUI building the zlib compatible ABI for zlib-ng yet which would allow it to be used as a replacement. Adding support for the compatible ABI allowing it to be an alternative for standard zlib seems to me like an obvious step we could take, it would need a lot of care given that zlib is essentially but would let people get zlib-ng if they wanted, and if there are problems it can be held in unstable (or experimental) to avoid impact on trixie. This would allow people to kick the tires. Does anyone have thoughts on this? signature.asc Description: PGP signature
Re: GnuPG 2.4 before Trixie freeze
Il 24/09/2024 14:43, Stephan Verbücheln ha scritto: Hello everyone Debian Trixie and Sid still have GnuPG 2.2.x. GnuPG 2.4.5 is available in Experimental. GnuPG 2.4.0 was released on December 20, 2022. The 2.4.x series has various useful new features such as TPM support. Is it realistic to get GnuPG 2.4 into Trixie before the freeze? What is missing for acceptance into unstable/testing? Regards Stephan You can search bugs related to 2.4 and looking into them for possible information, for example from a very fast look this seems one: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1022702 OpenPGP_signature.asc Description: OpenPGP digital signature
Re: proposal: Hybrid network stack for Trixie
Hi, On 9/22/24 19:22, Chris Hofstaedtler wrote: The "server" group supposedly wants (and I agree) networkd, but they also want the configuration interface of networkd. I'm not sure about that -- I'd expect the "server" group to be split into - "pets": their IP address doesn't change often anyway, anything beyond "set IP address during boot" is bloat. ifupdown is bloat because it requires a python interpreter to do what a one-line shell script can do, and networkd is bloat because it runs an entire service to do nothing. Nobody cares about either kind of bloat, because resources are cheap -- what people do care about is interface stability, which is why switching the interface naming scheme was so controversial back then. - "cattle": the IP configuration comes from a central place, and is integrated with asset management. If it's a small operation, they use DHCP, but anything more sophisticated brings their own management solution, and whatever system we provide needs to be able to go out of the way. - "container": the IP address is managed from outside. The OS installation we generate should not interfere. The "laptop" group supposedly wants (and I agree) NetworkManager, but they also want the configuration interface of NetworkManager. They specifically want the user interface. The configuration interface is less of a contract and more of a guideline, but that doesn't matter to the users, because they will only ever use the integrated solution where system and UI components are provided by the same package, and are kept consistent that way. Providing a typesafe interface for passing the privilege-separation boundary is hard, but it is much more difficult if that interface also needs to be long-term stable. Anything that wants to replace N-M is either a rewrite with the same basic structure (tightly coupled interfaces on both sides of the privilege separation boundary, upgraded in lock step) or a massive sink of manpower that, frankly, no one has. Who actually wants the configuration interface of netplan, especially by default? I can see it in the server space, because we need the integration with other tools that contribute fragments of network configuration without wanting to take over (libvirt and docker), and with tools that take over. Integrating these tools into a consistent whole would be the core task of a distribution. ifupdown defines a policy that works reasonably well on servers: "do not interfere." That is, if libvirt or docker change something because they need it (like moving the default route into a bridge), then ifupdown does not revert that change. It's shit, but it works. systemd-networkd, for good reasons, deviates from this, but this means further integration work is required from the distribution because the end result needs to be consistent again. If netplan can provide the "consistent whole", then it makes sense not to reinvent the wheel. My expectation as a user is that I should be able to install libvirt and docker on a server, and configure bridged networking in both without losing connectivity. Right now, it works by accident, which is bad, but breaking it in the name of correctness will make a lot of people very angry, like renaming all the interfaces or requiring "nofail" in the fstab to continue booting did. Simon
GnuPG 2.4 before Trixie freeze
Hello everyone Debian Trixie and Sid still have GnuPG 2.2.x. GnuPG 2.4.5 is available in Experimental. GnuPG 2.4.0 was released on December 20, 2022. The 2.4.x series has various useful new features such as TPM support. Is it realistic to get GnuPG 2.4 into Trixie before the freeze? What is missing for acceptance into unstable/testing? Regards Stephan signature.asc Description: This is a digitally signed message part
Bug#1082655: ITP: ocaml-stdlib-random -- versioned Random module from the OCaml standard library
Package: wnpp Severity: wishlist Owner: Stéphane Glondu X-Debbugs-Cc: debian-devel@lists.debian.org, debian-ocaml-ma...@lists.debian.org * Package name: ocaml-stdlib-random Version : 1.2.0 Upstream Contact: Florian Angeletti * URL : https://github.com/ocaml/stdlib-random * License : LGPL-2.1 with linking exception Programming Lang: C, OCaml Description : versioned Random module from the OCaml standard library The stdlib-random package provides a stable and compiler-independent implementation of all the PRNGs used in the Random module. Those PRNGs are available in the various libraries: . * stdlib-random.v3: OCaml 3.07 to 3.11 PRNG * stdlib-random.v4: OCaml 3.12 to 4.14 PRNG * stdlib-random.v5: current OCaml 5.0 PRNG * stdlib-random.v5o: pure OCaml version of the OCaml 5 PRNG . All those libraries can be used together and the signature of their Random$n module has been extended to the latest signature whenever possible. This package is a test dependency of camomile. It will be maintained in the OCaml team.
Salsa CI documentation updated
Hi! We've overhauled the README.md at https://salsa.debian.org/salsa-ci-team/pipeline to be as complete as possible, yet clear and to the point. If you are not yet using Salsa CI for pre-upload quality assurance for your package, you might want to check out what Salsa CI offers, or just review that your current usage is optimal. We also added a RUNNERS.md for those who want to experiment with customer runners or donate CI capacity to Debian. Feedback and improvement suggestions are also welcome as Merge Requests are via email. - Otto
Re: proposal: Hybrid network stack for Trixie
On 23.09.24 13:33, Richard Lewis wrote: Lukas Märdian writes: On 23.09.24 12:27, Ansgar 🙀 wrote: On Mon, 2024-09-23 at 12:22 +0200, Lukas Märdian wrote: On 22.09.24 15:58, Ansgar 🙀 wrote: On Fri, 2024-09-20 at 13:12 +0200, Lukas Märdian wrote: The benefit that Netplan would provide in such cases is that debian-installer installs a /etc/netplan/01-network-manager-all.yaml config file, reading: network: version: 2 renderer: NetworkManager So on desktop installations including NetworkManager, netplan will be configured to do nothing? Why install netplan at all on desktop systems then? Because it allows to add configuration in a way that is common with server, cloud and other instances of Debian Could you give an example of why this is useful to unify? For example: is there a scenario in which someone is using systemd-networkd but then finds they need to do X, which they cant essily do using systend but which nm support is better--- therefore if they are using netplan they can simply install network-manager, change a netplan setting and gain X with no need to understand the differences between the network-manager and systemd configuration languages? My ideas was not so much about switching from one networking daemon to another. In most cases users will probably stick to the network stack of their chosen environment. With systemd-networkd and NetworkManager being good candidates for their corresponding scenarios. But knowledge builds up over the years in our community and spreads around forums, stack overflow, etc. Those are the places where we figure out "How to setup a bond on Debian?", "How to connect a headless embedded board to the WiFi network" or "How to change the embedded-switch mode on SR-IOV physical functions?". We find solutions and help each other out, which is great! But with fragmented network-configuration tooling those solutions will not work in many cases, as they depend on a specific context of the base-image in use (e.g. server vs desktop vs embedded vs cloud). With Netplan I'm hoping to avoid such frustration by providing a way to configure the network that works independently of the base-image context. Of course without forcing people to use it or impacting the lower layer to be configured directly. -- Lukas
Re: proposal: Hybrid network stack for Trixie
Chris Hofstaedtler wrote on 23/09/2024 at 12:25:15+0200: > * Pierre-Elliott Bécue [240923 11:34]: >> Lukas Märdian wrote on 20/09/2024 at 13:12:36+0200: >> > # Why >> > The ifupdown package is a Debian only solution that is becoming a >> > maintenance >> > burden. We've had plenty of discussions over the years and consensus is >> > that we >> > want to get rid of it. >> >> I like ifupdown. It's simple and just works. > > I find this quite funny, given a recent discussion about IPv6 dad > issues with ifupdown on #debian-admin. Still looking for the funny bit, except if you're implying that as a DSA member, I should dislike ifupdown because Philipp had to manually intervene on a server managed by it. In that case it'd indeed be really funny, but probably not for the reason you'd think it is. > It's certainly limited in what it can do within reasonable > configuration effort, and it often works. I think both are true for > almost all of the discussed options :-) Well, netplan has no added value in my environments, contrary to some big changes we had in the past (systemd, journald - which still forwards logs to rsyslog, …), so I'd rather not being force-fed this tool. I'd react the same if someone were to try making me adopt resolved or timesyncd. But in the whole, I don't care that much, I'm perfectly fine with apt purge netplan.io after bootstrapping a server if the "consensus" is to force-feed it. -- PEB signature.asc Description: PGP signature
Re: ifupdown-ng source stanza (Was: proposal: Hybrid network stack for Trixie)
On mån, 2024/09/23 at 10:57:22 +0200, Daniel Gröber wrote:
> Hi Sirius,
>
> Thanks for taking ifupdown-ng for a spin.
No problem at all. Thank you for being patient for my response, I have
been working out some kinks around finit to get the system spitting out a
graphical session. :-D
> On Mon, Sep 23, 2024 at 08:22:51AM +0200, Sirius wrote:
> > I did switch to ifupdown-ng (as it seems the regular ifupdown is on its
> > way out) and the one thing I noticed was that ifupdown-ng does not handle
> > the include directive for /etc/network/interfaces.d/.
>
> I think you're using the version in stable? I upstreamed a patch for
> 'source' stanza glob pattern support as we use for interfaces.d a while
> ago. We've had it in Debian since 0.12.1-1, but its not in stable. I'll
> upload a backport as soon as I feel -ng is ready.
I am. It really is no problem, it arrives when it arrives. In the
meantime, having the stanzas directly in /etc/network/interfaces works. No
problem at all.
> > iwd and ifupdown{,-ng} works, and works well, but what I like about them
> > most is that they have lean dependencies and focus on one thing in true
> > unix fashion. I can sidestep both systemd and Network-Manager entirely
> > with a few rebuilt packages. That works for me.
>
> When using ifupdown with wpa_supplicant we have per-SSID configuration
> support (cf. wpa_action.8) I wonder if a similar integration would be
> possible with iwd?
With iwd, you run iw and then find the SSID and give it the secret for the
SSID and it then stores that in its database. As long as system dbus and
iwd is running when you ifup the interface, it works. No wpa_supplicant
needed.
And while it may be an Intel piece of software, it works with any wifi
card. My system has a MediaTek and it works just as well as the Intel card
on my laptop.
--
Kind regards,
/S
