Re: Mandatory LC_ALL=C.UTF-8 during package building

2025-02-23 Thread Chris Hofstaedtler
Hi, > I think a way forward could be to make the sanitizing the default, and > finally drop debian/rules as a supported (user) build entry point, I had in > mind re-proposing this already, but the above kind of gives it more urgency, > so > I'll try to do that soon. Given I ran into this discrep

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-08-03 Thread Simon McVittie
On Sat, 03 Aug 2024 at 19:16:30 +0200, Guillem Jover wrote: > I'm not sure the current state is ideal, because we are back to > packages being able to rely on some stuff on build daemons, that are > not guaranteed by default for our supported build entry points This was already true, though: the o

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-08-03 Thread Guillem Jover
Hi! On Sat, 2024-07-06 at 13:13:48 +0200, Alexandre Detiste wrote: > Le mar. 2 juil. 2024 à 14:37, Guillem Jover a écrit : > > On Tue, 2024-07-02 at 09:52:05 +0100, Simon McVittie wrote: > > > On Tue, 02 Jul 2024 at 03:47:29 +0200, Guillem Jover wrote: > > > > On Fri, 2024-06-07 at 15:40:07 +0200

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-08-03 Thread Guillem Jover
Hi! [ Mostly trying to clarify some of my earlier comments. ] On Fri, 2024-06-07 at 17:20:29 +0100, Simon McVittie wrote: > On Fri, 07 Jun 2024 at 14:32:14 +0200, Guillem Jover wrote: > > I'm a non-native speaker, who has been involved > > in l10n for a long time, while at the same time I've pret

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-07-06 Thread Alexandre Detiste
Hi, Thank you both. One thing that could be fixed quite quickly is fixing the few remaining official buildd workers that does not yet run with an UTF-8 locale. If one is unlucky the build will mysteriously fail. Adding export {LC_ALL|LANG|LC_CTYPE}=C.UTF-8 in every single d/rules by fear of th

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-07-02 Thread Guillem Jover
Hi! On Tue, 2024-07-02 at 09:52:05 +0100, Simon McVittie wrote: > On Tue, 02 Jul 2024 at 03:47:29 +0200, Guillem Jover wrote: > > On Fri, 2024-06-07 at 15:40:07 +0200, Alexandre Detiste wrote: > > > Maybe a compromise would be to at least mandate some UTF-8 locale. > > > > dpkg-buildpackage: Requi

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-07-02 Thread Simon McVittie
On Tue, 02 Jul 2024 at 03:47:29 +0200, Guillem Jover wrote: > On Fri, 2024-06-07 at 15:40:07 +0200, Alexandre Detiste wrote: > > Maybe a compromise would be to at least mandate some UTF-8 locale. > > dpkg-buildpackage: Require an UTF-8 (or ASCII) locale when > building packages Allowing ASCII seem

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-07-01 Thread Guillem Jover
Hi! On Fri, 2024-06-07 at 15:40:07 +0200, Alexandre Detiste wrote: > Maybe a compromise would be to at least mandate some UTF-8 locale. Ah, good thinking! That would actually seem acceptable. I've prepared the attached preliminary patch (missing better commit message, etc), as a PoC for how this

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-17 Thread Adrian Bunk
Sorry for being late to this discussion, but there are a few points and a suggestion I'd like to make: 1. Reproducibility is not a big concern Quoting policy: Packages should build reproducibly, which for the purposes of this document means that given ... - a set of environment variab

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-11 Thread Simon Richter
Hi, On 6/11/24 00:26, Simon McVittie wrote: Reproducibility outside of sterile environments is however a problem for us as a distribution, because it affects how well people are able to contribute to packages they are not directly maintaining If our package-building entry point sets up aspec

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-10 Thread Simon McVittie
On Sat, 08 Jun 2024 at 02:14:36 +0900, Simon Richter wrote: > Reproducibility outside of sterile environments is however a problem for us > as a distribution, because it affects how well people are able to contribute > to packages they are not directly maintaining If our package-building entry poi

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Simon Richter
Hi, On 6/8/24 00:42, Simon McVittie wrote: Having an UTF-8 locale available would be a good thing, but allowing packages to rely on the active locale to be UTF-8 based reduces our testing scope. I'm not sure I follow. Are you suggesting that we should build each package *n* times (in a UTF-8

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Simon McVittie
On Fri, 07 Jun 2024 at 14:32:14 +0200, Guillem Jover wrote: > I'm a non-native speaker, who has been involved > in l10n for a long time, while at the same time I've pretty much > always run my systems with either LANG=C.UTF-8 or before that LANG=C, > LC_CTYPE=ca_ES.UTF-8 and LC_COLLATE=ca_ES.UTF-8.

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Simon McVittie
On Fri, 07 Jun 2024 at 23:22:46 +0900, Simon Richter wrote: > On 6/7/24 22:40, Alexandre Detiste wrote: > > Maybe a compromise would be to at least mandate some UTF-8 locale. > > Having an UTF-8 locale available would be a good thing, but allowing > packages to rely on the active locale to be UTF-

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Simon Richter
Hi, On 6/7/24 22:40, Alexandre Detiste wrote: Maybe a compromise would be to at least mandate some UTF-8 locale. Having an UTF-8 locale available would be a good thing, but allowing packages to rely on the active locale to be UTF-8 based reduces our testing scope. Basically, we need to de

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Gioele Barabucci
On 07/06/24 14:32, Guillem Jover wrote: Related to this, dpkg-buildpackage 1.20.0 gained a --sanitize-env, which for now on Debian and derivatives sets LC_COLLATE=C.UTF-8 and umask=0022. That's great news! But _iff_ we end up with dpkg-buildpackage being declared the only supported entry poin

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Alexandre Detiste
Maybe a compromise would be to at least mandate some UTF-8 locale.

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Holger Levsen
On Fri, Jun 07, 2024 at 02:32:14PM +0200, Guillem Jover wrote: > And I think forcing a locale on buildds makes perfect sense, because > we want easy access to build logs. But forcing LC_ALL from the build > tools implies that no tool invoked will get translated messages at > all, and means that use

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Guillem Jover
Hi! On Thu, 2024-06-06 at 15:31:55 +0100, Simon McVittie wrote: > On Thu, 06 Jun 2024 at 13:32:27 +0300, Hakan Bayındır wrote: > > C, or C.UTF-8 is not a universal locale which works > > for all. > > Sure, and I don't think anyone is arguing that you or anyone else should > set the locale for you

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-07 Thread Holger Levsen
On Thu, Jun 06, 2024 at 07:11:46PM +0200, Michael Biebl wrote: > I would prefer that dpkg-buildpackage provides a "sane" build environment by > default (which I think includes a LC_ setting pointing at a .UTF-8 locale) > and fewer packages explicitly setting those things via debian/rules. same her

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Michael Biebl
Am 06.06.24 um 18:52 schrieb Andrey Rakhmatullin: On Thu, Jun 06, 2024 at 12:08:17PM +0200, Johannes Schauer Marin Rodrigues wrote: Or whether we should switch the default and require that d/rules is run in an environment (for example as set-up by dpkg-buildpackage) where these variables are se

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Andrey Rakhmatullin
On Thu, Jun 06, 2024 at 12:08:17PM +0200, Johannes Schauer Marin Rodrigues wrote: > Or whether we should switch the default and require that d/rules is run in an > environment (for example as set-up by dpkg-buildpackage) where these variables > are set? (a previous discussion on this: https://list

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Marco d'Itri
On Jun 06, Johannes Schauer Marin Rodrigues wrote: > A question that goes in a similar direction is whether every d/rules that > needs > it should have to do this: > > export DPKG_EXPORT_BUILDFLAGS=y > include /usr/share/dpkg/buildflags.mk > > Or whether we should switch the default and requir

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Simon McVittie
On Thu, 06 Jun 2024 at 12:56:10 +0200, Johannes Schauer Marin Rodrigues wrote: > If we imagine a hypothetical switch to LC_ALL=C.UTF-8 for all source > packages by default, then there will be bugs. Do you mean: there will be bugs that break the build of certain packages, which previously built suc

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Jeremy Bícha
I believe debhelper already sets LC_ALL=C.UTF-8 for the cmake, meson, and ninja buildsystems; therefore many but definitely not all packages are already built with LC_ALL=C.UTF-8. Thank you, Jeremy Bícha

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Simon McVittie
On Thu, 06 Jun 2024 at 13:32:27 +0300, Hakan Bayındır wrote: > C, or C.UTF-8 is not a universal locale which works > for all. Sure, and I don't think anyone is arguing that you or anyone else should set the locale for your interactive terminal session, your GUI desktop environment, or even your se

Re: Locale sanitizer (Was: Mandatory LC_ALL=C.UTF-8 during package building)

2024-06-06 Thread Simon McVittie
On Thu, 06 Jun 2024 at 14:40:23 +0200, Daniel Gröber wrote: > On Thu, Jun 06, 2024 at 11:32:33AM +0200, Simon Richter wrote: > > If your package is not reproducible without it, then your package is > > broken. > > At build-time, if a program doesn't call setlocale before using locale > dependent s

Locale sanitizer (Was: Mandatory LC_ALL=C.UTF-8 during package building)

2024-06-06 Thread Daniel Gröber
Hi, On Thu, Jun 06, 2024 at 11:32:33AM +0200, Simon Richter wrote: > If your package is not reproducible without it, then your package is > broken. It can go in with the workaround, but the underlying problem > should be fixed at some point. It's easy to say "should be fixed" but finding the sour

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Simon Richter
Hi, On 6/6/24 19:56, Johannes Schauer Marin Rodrigues wrote: At the same time though, I also get annoyed of copy-pasting d/rules snippets from one of my packages to the next instead of making use of a few more defaults in our package build environment. Same here -- I just think that such a wo

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Johannes Schauer Marin Rodrigues
Hi, Quoting Hakan Bayındır (2024-06-06 12:32:27) > On 6.06.2024 ÖS 1:08, Johannes Schauer Marin Rodrigues wrote: > > Quoting Simon Richter (2024-06-06 11:32:33) > >>> Would it be possible to set in stone that packages are supposed to always > >>> be built in an environment where LC_ALL=C.UTF-8, or

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Hakan Bayındır
Hi, On 6.06.2024 ÖS 1:08, Johannes Schauer Marin Rodrigues wrote: Hi, Quoting Simon Richter (2024-06-06 11:32:33) Would it be possible to set in stone that packages are supposed to always be built in an environment where LC_ALL=C.UTF-8, or, in other words, that builders must set LC_ALL=C.UTF-8

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Johannes Schauer Marin Rodrigues
Hi, Quoting Simon Richter (2024-06-06 11:32:33) > > Would it be possible to set in stone that packages are supposed to always > > be built in an environment where LC_ALL=C.UTF-8, or, in other words, that > > builders must set LC_ALL=C.UTF-8? > > This would be the opposite of the current rule. >

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Simon Richter
Hi, > Would it be possible to set in stone that packages are supposed to > always be built in an environment where LC_ALL=C.UTF-8, or, in other > words, that builders must set LC_ALL=C.UTF-8? This would be the opposite of the current rule. Setting LC_ALL=C in debian/rules is an one-liner. If yo

Re: Mandatory LC_ALL=C.UTF-8 during package building

2024-06-06 Thread Luca Boccassi
On Thu, 6 Jun 2024 at 09:07, Gioele Barabucci wrote: > > Hi, > > setting LC_ALL=C.UTF-8 in d/rules is a common way to fix many > reproducibility problems. It is also, in general, a more sane way to > build packages, in comparison to using whatever locale settings happen > to be set during a build.

Mandatory LC_ALL=C.UTF-8 during package building

2024-06-05 Thread Gioele Barabucci
Hi, setting LC_ALL=C.UTF-8 in d/rules is a common way to fix many reproducibility problems. It is also, in general, a more sane way to build packages, in comparison to using whatever locale settings happen to be set during a build. However, sprinkling a variant of `export LC_ALL=C.UTF-8` in e