Re: debian names

2020-11-09 Thread Bjørn Mork
Jonas Smedegaard  writes:

> I would consider it highly unlikely that (Disney would claim and a court 
> would agree with them, that) Pixar customers could confuse some Pixar 
> products with a Debian release.

I believe Disney lawyers are world famous for their ability to construct
a legal problem from thin air.  Or maybe they don't even need air if it
is licensed?  Evil tongues have claimed that Carl Barks modelled
Sylvester J. Sharky after one or more Disney lawyer:
https://en.wikipedia.org/wiki/List_of_Donald_Duck_universe_characters#Lawyer_Sharky

In any case, the problem will be solved when Disney buys Debian.  That's
how The Simpsons finally got away with all their Disney jokes.

Or you might want to reference Don Rosas drawing of "the hardest,
toughest, most impervious substances known to mankind" at the bottom of
https://duck.neamar.fr/DUCK/the_universal_solvent



Bjørn



Re: restarting instanced systemd services on upgrade

2020-11-09 Thread Simon McVittie
On Mon, 09 Nov 2020 at 12:24:03 +0900, Norbert Preining wrote:
[Paul wrote]:
> > These are not "systemd user services" as described in your email subject, 
> > they are
> > per-user instances of systemd system services.

How to tell: run systemd-cgls and look at where they appear in the tree.
Everything below user@${uid}.service is a user service. Everything below
session-${n}.service is a non-systemd-managed user process. The rest are
system services or non-systemd-managed system processes.

Normally, the only system service that is instanced per-user is
user@.service, which is the `systemd --user` that is started by
systemd-logind on behalf of each user.

It is unusual to instance other system services on a per-user basis,
because there is already `systemd --user` available to manage per-user
services (like dbus-daemon --session, pulseaudio, gpg-agent).

When you have an instanced service, systemd doesn't know what the
instance represents: that's private to the service itself. It's often
an arbitrary instance name (the Debian packages for openvpn, postfix,
tor and openarena-server are examples of this), but it could be some
related resource like a virtual console (getty@tty2.service).

> > package, it seems you need a onedrive.service in addition to
> 
> We ship also one onedrive.service. This is used when a user starts
>   systemctl --user onedrive

That's a user-service, which is unrelated (it's managed by a different
per-user instance of systemd, and is not visible to the system-wide
instance). I think Paul means you might need a *system service* named
onedrive.service.

For example, have a look at postfix and openvpn. They have a non-instanced
service that just runs /bin/true, and an instanced service that runs
the actual daemon. The instanced service postfix@.service is marked
as PartOf=postfix.service, which means that when you stop or restart
postfix.service, systemd will automatically stop or restart all the
instances.

However, I think systemd upstream would probably say that this is a hack,
because a no-op service that runs /bin/true is not really a service,
and that the non-instanced service should be replaced by a non-instanced
target (for example postfix@.service PartOf=postfix.target).

Unlike services, targets are specifically designed to be used to group
units together, without starting daemons or other processes themselves.
For example, /lib/systemd/system/systemd-nspawn@.service has
PartOf=machines.target, which means that `systemctl restart machines.target`
will restart all services matching systemd-nspawn@foo.service; and
similarly the cron-* services, targets, etc. installed by the systemd-cron
package are PartOf=cron.target.

I think most Debian package maintainers have used no-op non-instanced
services because that gives them the most obvious migration path
from sysv-rc services to systemd services (sysv-rc has no concept of
instances, targets or user services, only an equivalent of systemd
non-instanced system services), or because debhelper historically did
not have full support for non-service units. It should be possible to
use targets instead since at least dh compat level 11 (the point at which
responsibility for handling systemd services moved from dh_installinit,
dh_systemd_enable and dh_systemd_start to dh_installsystemd).

> What about the following code in postinst?
> restart_units="`systemctl list-units --no-legend --type=service 
> --state=running 'onedrive*' | awk '{print$1}'`"
> if ! [ "x$restart_units" = "x" ] ; then
>   echo -n "Restarting running Onedrive processes ... "
>   for unit in $restart_units ; do
> echo -n "$unit "
> systemctl restart "$unit"
>   done
>   echo "."
> fi
> Would this be against any policy?

It would be against the policy of "if you write less code, you'll have
fewer bugs"? :-)

(In addition to the units you intended, this would restart an unrelated
onedriveperchild.service or onedrivetorulethemall.service, which could
result in data loss.)

It would be better to give onedrive@.service PartOf=onedrive.target, and
have the postinst restart onedrive.target - which I think dh_installsystemd
will set up automatically, so you might not have to write any code for it
at all, other than creating the .target unit.

smcv



Re: per-user instanced system services vs systemd user services

2020-11-09 Thread Simon McVittie
On Mon, 09 Nov 2020 at 09:25:25 +0900, Norbert Preining wrote:
> Onedrive ships
>   /lib/systemd/system/onedrive@.service
> and for example on my system root has started
>   onedrive@norbert
> service (this is better than an actual user starting, since systemd
> tears down the service after log out, which I don't want).

As said elsewhere in the thread, this is an instanced system service
that happens to have a username as its instance identifier (but systemd
doesn't know that, because instance identifiers are opaque to everyone
except the service and the sysadmin). I've said more about how to manage
instanced systemd services elsewhere in the thread, which hopefully answers
your original question.

A major disadvantage of this is that users cannot manage their own onedrive
services: if the onedrive service runs as their own uid, then maybe they
can send POSIX signals to it, but they cannot use `systemctl --user restart`
to restart it, or use `systemctl --user edit` to alter its launch parameters,
or use `systemctl --user disable` to stop it from running. Only the sysadmin
can control whether and how onedrive runs on their behalf.

When you say "an actual user starting", a systemd user service managed
by `systemd --user` would be an example of this. This is how
`dbus-daemon --session`, pulseaudio, gpg-agent, gvfs-daemon, Tracker
and various other per-user services are managed on systemd systems, and
is increasingly used by GUI environments like KDE and GNOME to manage
desktop components.

If you would like the onedrive service for a particular uid to persist,
*without* having to move it to system level, there are a few ways you can
achieve that. Choose one:

* Enable "lingering" for the user: `loginctl enable-linger norbert`.
  This will run user@${uid}.service on system boot, and not shut it
  down until system shutdown. Any user service that is enabled in
  default.target will start on boot and not be killed until shutdown.

* Or, make sure the user's processes are not killed when the user logs out,
  and then make sure at least one process from the user's login session
  continues to run in the "abandoned scope" even after logout. Killing
  processes is controlled by KillUserProcesses, KillOnlyUsers,
  KillExcludeUsers in logind.conf(5). The upstream default is
  KillUserProcesses=yes, but Debian overrides it to KillUserProcesses=no.

* Or, have a PAM stack that includes pam_systemd.so, and a system service
  that "logs in" with that PAM stack for the duration of the time in which
  you want background services like onedrive to persist.

smcv



Re: debian names

2020-11-09 Thread Jonas Smedegaard
Hi Bjørn,

Quoting Bjørn Mork (2020-11-09 10:33:55)
> Jonas Smedegaard  writes:
> 
> > I would consider it highly unlikely that (Disney would claim and a court 
> > would agree with them, that) Pixar customers could confuse some Pixar 
> > products with a Debian release.
> 
> I believe Disney lawyers are world famous for their ability to 
> construct a legal problem from thin air.  Or maybe they don't even 
> need air if it is licensed?  Evil tongues have claimed that Carl Barks 
> modelled Sylvester J. Sharky after one or more Disney lawyer: 
> https://en.wikipedia.org/wiki/List_of_Donald_Duck_universe_characters#Lawyer_Sharky
> 
> In any case, the problem will be solved when Disney buys Debian.  
> That's how The Simpsons finally got away with all their Disney jokes.
> 
> Or you might want to reference Don Rosas drawing of "the hardest, 
> toughest, most impervious substances known to mankind" at the bottom 
> of https://duck.neamar.fr/DUCK/the_universal_solvent

True, but do courts equally commonly confirm Disney claims?

Debian is shapen by common law, not fear of potential lawsuits.  
Otherwise many things in Debian would look different from how they do 
today.

e.g. we avoid discussing patents because common law says awareness of 
them can be used against us, not because we are afraid of patents - see 
https://www.debian.org/legal/patent before discussing specific patents.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc
Description: signature


Re: MBF proposal: remaining users of fltk1.1 (vs. fltk1.3).

2020-11-09 Thread Simon McVittie
On Sun, 08 Nov 2020 at 21:07:03 -0500, Aaron M. Ucko wrote:
> Debian has had two versions of the FLTK GUI toolkit with nearly
> identical APIs for over nine years: 1.1 and 1.3.  The only breaking
> change in the 1.3 series is that it expects text to be in UTF-8 rather
> than legacy national encodings, which Debian has generally deprecated
> for quite some time; meanwhile, 1.1 is thoroughly dead upstream.

I would say it's always a good time to be doing *non-RC* mass bug filing
for use of deprecated dependencies, but it might be a bit late in the
cycle to be seriously trying to drop FLTK 1.1 at this stage unless you
are prepared to be quite aggressive about providing patches, making bugs
RC, and if necessary doing NMUs (transition freeze is only 2 months away).

In an ideal world the non-RC mass-bug-filing would have happened at the
earliest point when the FTLK maintainers considered 1.1 to be deprecated,
with bugs escalated to RC when it becomes feasible to consider removing it
altogether.

> As such, I would like to try to drop 1.1, or at least its development
> package, in time for bullseye.

You probably can't drop the -dev package unless/until you're ready to
drop the shared library, because it would be a RC bug for dependendent
packages to be unbuildable (consider what would happen if there was a
security update to the dependent package during the bullseye cycle).

>   Subject: #PACKAGE#: Please migrate to FLTK 1.3
> 
>   #PACKAGE# still builds against FLTK 1.1, for which it is long past
>   time for Debian to drop support.  Please migrate to 1.3, which is
>   generally as simple as adjusting #PACKAGE#'s build dependencies and
>   ensuring that it uses UTF-8 rather than a legacy text encoding.  (That
>   said, please also make sure that you're linking FLTK dynamically,
>   e.g. via fltk-config --ldflags rather than fltk-config --ilbs.)

I think you mean "--libs", not "--ilbs".

This is an unusual calling convention and it might be worth mentioning
explicitly that these options don't do what you would expect from their
names. Typically, in an Autotools-style build system, "ldflags" would
refer to options like -L/path/to/foo, and "libs" would refer to the
libraries, whose position in the command-line is significant (-lfoo or
/path/to/foo.a). But it looks as though in FLTK,
"fltk-config --ldflags" is the equivalent of "pkg-config --libs foo"
or "sdl2-config --libs", while "fltk-config --libs" is something that
normally shouldn't be used by Debian packages.

Tangentially related to this, it would be great if you could get FLTK
upstream to provide pkg-config metadata (.pc file). The fltk-config script
can't be made "Multi-Arch: same" without considerable extra hacking, and
pkg-config is generally preferred by distros.

smcv



Bug#974047: TAG: devuan-keyring -- GnuPG archive key of the devuan repository

2020-11-09 Thread Ivan J .
Package: wnpp
Severity: ITP
X-Debbugs-CC: debian-devel@lists.debian.org

Greetings.

I was wondering if it would be possible to start maintaining the
devuan-keyring package on salsa, and have it being built upstream in
Debian?

This way it would be possible to unify our efforts in debootstrap
maintenance, and provide a separate script for Devuan releases, much
like what is happening with Ubuntu and the gutsy target right now.

If this is possible, following it, I would submit a patch to debootstrap
where an extra Devuan target would be added, instead of the current
symlinks to "sid".

Thank you for your consideration,
Ivan



Bug#974048: ITP: devuan-keyring -- GnuPG archive key of the devuan repository

2020-11-09 Thread Ivan J .
Package: wnpp
Severity: ITP
X-Debbugs-CC: debian-devel@lists.debian.org

Greetings.

I was wondering if it would be possible to start maintaining the
devuan-keyring package on salsa, and have it being built upstream in
Debian?

This way it would be possible to unify our efforts in debootstrap
maintenance, and provide a separate script for Devuan releases, much
like what is happening with Ubuntu and the gutsy target right now.

If this is possible, following it, I would submit a patch to debootstrap
where an extra Devuan target would be added, instead of the current
symlinks to "sid".

Thank you for your consideration,
Ivan



Re: restarting instanced systemd services on upgrade

2020-11-09 Thread Peter Pentchev
On Mon, Nov 09, 2020 at 10:02:46AM +, Simon McVittie wrote:
> On Mon, 09 Nov 2020 at 12:24:03 +0900, Norbert Preining wrote:
> [Paul wrote]:
> > > These are not "systemd user services" as described in your email subject, 
> > > they are
> > > per-user instances of systemd system services.
> 
[snip]
> > > package, it seems you need a onedrive.service in addition to
> > 
> > We ship also one onedrive.service. This is used when a user starts
> > systemctl --user onedrive
> 
> That's a user-service, which is unrelated (it's managed by a different
> per-user instance of systemd, and is not visible to the system-wide
> instance). I think Paul means you might need a *system service* named
> onedrive.service.

[snip the rest of the explanation about the various benefits and
 drawbacks of having a no-op service vs a target]

(looks up from taking notes for the long-long-long-overdue conversion of
 stunnel4 to systemd, being held back mainly by myself not having a very
 clear idea which of the options to choose)

Thanks *a lot* for this awesome concise yet complete explanation!

G'luck,
Peter

-- 
Peter Pentchev  r...@ringlet.net r...@debian.org p...@storpool.com
PGP key:http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13


signature.asc
Description: PGP signature


Re: MBF proposal: remaining users of fltk1.1 (vs. fltk1.3).

2020-11-09 Thread Aaron M. Ucko
Simon McVittie  writes:

> In an ideal world the non-RC mass-bug-filing would have happened at the
> earliest point when the FTLK maintainers considered 1.1 to be deprecated,
> with bugs escalated to RC when it becomes feasible to consider removing it
> altogether.

Yeah, I've been sitting on this notion entirely too long; reverse
dependencies' maintainers have generally been proactive about migrating
(thanks, everyone!), and continued maintenance of 1.1 hasn't been much
of a burden.  Per that second point, I'm open to carrying 1.1 in full
through one more release if necessary, so I'll leave the bugs as non-RC
for now.

> You probably can't drop the -dev package unless/until you're ready to
> drop the shared library, because it would be a RC bug for dependendent
> packages to be unbuildable (consider what would happen if there was a
> security update to the dependent package during the bullseye cycle).

Understood.  I was thinking more in terms of dependent software from
outside the archive, though that is of course a secondary concern.

>>   Subject: #PACKAGE#: Please migrate to FLTK 1.3
>> 
>>   #PACKAGE# still builds against FLTK 1.1, for which it is long past
>>   time for Debian to drop support.  Please migrate to 1.3, which is
>>   generally as simple as adjusting #PACKAGE#'s build dependencies and
>>   ensuring that it uses UTF-8 rather than a legacy text encoding.  (That
>>   said, please also make sure that you're linking FLTK dynamically,
>>   e.g. via fltk-config --ldflags rather than fltk-config --ilbs.)
>
> I think you mean "--libs", not "--ilbs".

Yeah, oops.

> This is an unusual calling convention and it might be worth mentioning
> explicitly that these options don't do what you would expect from their
> names.

How about replacing that parenthetical remark with a separate paragraph
reading

  Also, please note that fltk-config (in both branches) differs from
  typical *-config scripts in having --libs yield paths to *static*
  libraries.  If your build has been making use of this syntax, please
  substitute --ldflags to obtain proper dynamic linkage.  (FLTK does not
  yet offer pkg-config metadata, sorry.)

> Tangentially related to this, it would be great if you could get FLTK
> upstream to provide pkg-config metadata (.pc file). The fltk-config script
> can't be made "Multi-Arch: same" without considerable extra hacking, and
> pkg-config is generally preferred by distros.

Yeah, I'm well aware of that, but it would need multiple .pc files, and
upstream hasn't historically showed much interest in supplying them:

  https://www.fltk.org/str.php?L2180

However, it looks like they may finally be coming around:

  https://github.com/fltk/fltk/issues/22

In general, thanks for weighing in!

-- 
Aaron M. Ucko, KB1CJC (amu at alum.mit.edu, ucko at debian.org)
http://www.mit.edu/~amu/ | http://stuff.mit.edu/cgi/finger/?a...@monk.mit.edu



Bug#974080: ITP: lua-binaryheap -- Binary heap implementation in lua

2020-11-09 Thread Jakub Ružička
Package: wnpp
Severity: wishlist
Owner: Jakub Ružička 

* Package name: lua-binaryheap
  Version : 0.4
  Upstream Author : Thijs Schreijer 
* URL : https://github.com/Tieske/binaryheap.lua
* License : MIT
  Programming Lang: lua
  Description : binary heap implementation in lua

binaryheap.lua is a binary heap (binary tree) implementaion in lua.

- lua-binaryheap package is a missing requirement of lua-http package which is
  already included in Debian as described in bug #958665
- this package is an indirect requirement of Knot Resolver which I intend to
  package for Debian. upstream Knot Resolver package repos currntly use custom
  lua-binaryheap package which I use as a base for the Debian package.
- binaryheap.lua is a small and focused project which started in 2015 with
  last activity in 2019 so I assume it's stable/mature with changes unlikely
  thus it should be easy to maintain.
- I'm not a debian Dev but I try to become Maintainer with active sponsor as
  a part of Debian DNS team. I'm willing to maintain the package in forseeable
  future, especially as I expect minimal or no changes.


Re: restarting instanced systemd services on upgrade

2020-11-09 Thread Norbert Preining
Hi Simon,

thanks for your detailed answers, the are **spectacularly** good!!!
That was the best explanation in ages about something sytemd related.

I will go with your suggestion:

On Mon, 09 Nov 2020, Simon McVittie wrote:
> It would be better to give onedrive@.service PartOf=onedrive.target, and
> have the postinst restart onedrive.target - which I think dh_installsystemd
> will set up automatically, so you might not have to write any code for it
> at all, other than creating the .target unit.

and I will try to improve the upstream documentation of onedrive to
include other options like lingering etc.

Again, big big thanks, and at least 3 vitual beer!!

Norbert

--
PREINING Norbert  https://www.preining.info
Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13



Re: Updating dpkg-buildflags to enable reproducible=+fixfilepath by default

2020-11-09 Thread Vagrant Cascadian
On 2020-10-27, Vagrant Cascadian wrote:
> The dpkg-buildflags feature reproducible=+fixfilepath was added to dpkg
> in 2018.
...
> The result of enabling this feature by default will be to make several
> hundred packages reproducible with varying build-path and reduce the
> differences in many other packages, making it easier to identify other
> more nuanced reproducibility issues.
>
> It would be great to see the reproducible=+fixfilepath feature enabled
> by default in dpkg-buildflags, and we would like to proceed forward with
> this soon unless we hear any major concerns or other outstanding issues.

Given no objections or concerns of any kind raised in the last two
weeks, I've submitted a bug against dpkg:

  https://bugs.debian.org/974087


live well,
  vagrant


signature.asc
Description: PGP signature


Re: doubt with autopkgtest for javascript packages

2020-11-09 Thread Sudip Mukherjee
Hi Utkarsh and Xavier,

On Mon, Nov 9, 2020 at 5:08 AM Utkarsh Gupta  wrote:
>
> Hi Sudip,
>
> On Mon, Nov 9, 2020 at 4:21 AM Sudip Mukherjee
>  wrote:
> > While going through packages to find autopkgtests which are
> > superficial I noticed that some of of the javascript packages are
> > doing just:
> > nodejs -e "require('foo.js');"
> >
> > After reading the documentation at [1], it seems that it will just
> > load the object in cache so that the exported properties and methods
> > of the object can be used in the code.
> > Can you please confirm if my understanding is correct and the tests
> > are indeed superficial in nature as it is just loading the object and
> > not actually using any of the exported methods or properties.
>
> You're indeed correct and these are just superficial test which just
> helps in the preliminary check that things are in order (syntax wise
> and that the module can be used and loaded).
> From what I remember, most of these (at least from a bunch of those
> that I have touched) are already marked as superficial. In case we can
> get a list of tests that aren't yet superficial, fixing them should be
> trivial! :)
>
> Thanks for all your work on this! \o/

Thanks to both of you for the confirmation. I have attached the list.
Do you want me to add them in my MBF list and raise bug reports for
them?


-- 
Regards
Sudip


js_pkg_list
Description: Binary data


Re: doubt with autopkgtest for javascript packages

2020-11-09 Thread Xavier
Hi,

Yes, do it, thanks!

Cheers,
Xavier


Le 9 novembre 2020 22:08:46 GMT+01:00, Sudip Mukherjee 
 a écrit :
>Hi Utkarsh and Xavier,
>
>On Mon, Nov 9, 2020 at 5:08 AM Utkarsh Gupta  wrote:
>>
>> Hi Sudip,
>>
>> On Mon, Nov 9, 2020 at 4:21 AM Sudip Mukherjee
>>  wrote:
>> > While going through packages to find autopkgtests which are
>> > superficial I noticed that some of of the javascript packages are
>> > doing just:
>> > nodejs -e "require('foo.js');"
>> >
>> > After reading the documentation at [1], it seems that it will just
>> > load the object in cache so that the exported properties and methods
>> > of the object can be used in the code.
>> > Can you please confirm if my understanding is correct and the tests
>> > are indeed superficial in nature as it is just loading the object and
>> > not actually using any of the exported methods or properties.
>>
>> You're indeed correct and these are just superficial test which just
>> helps in the preliminary check that things are in order (syntax wise
>> and that the module can be used and loaded).
>> From what I remember, most of these (at least from a bunch of those
>> that I have touched) are already marked as superficial. In case we can
>> get a list of tests that aren't yet superficial, fixing them should be
>> trivial! :)
>>
>> Thanks for all your work on this! \o/
>
>Thanks to both of you for the confirmation. I have attached the list.
>Do you want me to add them in my MBF list and raise bug reports for
>them?
>
>
>-- 
>Regards
>Sudip

-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma 
brièveté.

Re: doubt with autopkgtest for javascript packages

2020-11-09 Thread Julien Puydt
Le lundi 09 novembre 2020 à 21:08 +, Sudip Mukherjee a écrit :
> I have attached the list.
> Do you want me to add them in my MBF list and raise bug reports for
> them?

If I remember well, I'm the main culprit behind those, and I'll refresh
them -- probably at the end of the week:

node-ast-types
node-es5-shim
node-esprima-fb
node-mocha-lcov-reporter
node-posix-getopt
node-regenerate
node-stringmap
node-unicode-canonical-property-names-ecmascript
node-unicode-property-aliases-ecmascript
node-unicode-property-value-aliases

Cheers,

JP



Re: Updating dpkg-buildflags to enable reproducible=+fixfilepath by default

2020-11-09 Thread Graham Inggs
Hi

On Mon, 9 Nov 2020 at 22:06, Vagrant Cascadian
 wrote:
> Given no objections or concerns of any kind raised in the last two
> weeks, I've submitted a bug against dpkg:
>
>   https://bugs.debian.org/974087

There was a query from one of our upstreams in #972294 to which I have
not seen a response.

Regards
Graham



Bug#974078: ITP: ruby-azure-storage-common -- ruby package that supports service client libraries for azure storage

2020-11-09 Thread Abraham Raji
Package: wnpp
Severity: wishlist
Owner: Abraham Raji 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name    : ruby-azure-storage-common
  Version : 2.0.2
  Upstream Author : 2015 Microsoft Corporation
* URL : https://github.com/azure/azure-storage-ruby
* License : Expat
  Programming Lang: Ruby
  Description : ruby package that supports service client libraries for
  azure storage
  Official Ruby client library to consume Azure Storage Common service.
  Microsoft Azure Storage Common Client Library for Ruby.
 
This package is necessary for the gitlab 13.4.4 update.

I am part of the Debian Ruby team and with their help I will maintain this
package myself.

Abraham Raji
-- 
Mea navis aëricumbens anguillis abundat.


Split Packages files based on new section "buildlibs"

2020-11-09 Thread Joerg Jaspert

Hi everybody,

Short Reason: Too many packages of no use to our users.

Longer reason: Many packages get added to Debian that are of no (direct)
use to our users. Each package adds metadata to the indices that needs
to be downloaded, processed by tools and also clutters up the whole
package list for no practical benefit. A split out packages file will
allow us to minimise the effect on users.

More and more packages are being uploaded into the Debian archive which
are only ever used for building packages. These are not only never
intended to be installed onto an end-user's system, they are even
actively discouraged from being used directly by a user. The two
currently most notable examples are packages used by the Go and Rust
programming languages and their ecosystem, but there well may be
others[1]. While we need their library packages to build the
applications that use them, they are entirely statically compiled and
none of the libraries will ever be installed on a normal user's system.

Moreover, the language ecosystem in Debian actively discourages users
from installing them for anything other than rebuilding a Debian
package. If you do general (non-Debian-specific) development using Go or
Rust on your machine, the expectation is that you will use the
language-specific tools to install your dependencies [2].

Currently however, all of those packages end up in the indices we
generate, which users have to download and package managers have to read
and deal with. Each of those packages therefore slightly increases the
size of these indices for little reason and while many users have access
to large bandwidth connections and fast CPUs, that is not the case for
many other users and does not benefit global warming.

For the Rust ecosystem, those sizes increase even more, as each of their
libraries can provide multiple features. For example, a TLS library can
link against GnuTLS or OpenSSL or some other random TLS implementation.
Such features may even be combined in various different ways, resulting
in an excess number of possible feature combinations for one Rust
package, called "crate". Those are "mapped" to the Debian package world
by creating something we call *feature packages*, with one such feature
package per feature or combination thereof (usually grouped by common
dependencies).

Those feature packages are empty packages, only containing a symlink for
their /usr/share/doc/… directory. Their size is smaller than the
metadata they will produce. Adding new features means one more trip
through the NEW queue each time such new binary packages are introduced.

The FTPTeam disagree with the feature-package solution[3], so currently
there is a workaround. By collapsing the features into the main library
package and declaring the features using the Provides header similar
functionality is achieved. However this doesn’t work in all situations,
for example:

   Tools can generate really long Provides: lines, with the current
   record being around 250kb. That's long enough that a tool (not dak
   itself) broke on it already. And those lines may grow larger in
   future.

   Some features may need different (build-)dependencies (say, GnuTLS
   vs OpenSSL), should those conflict with each other, you cannot
   combine them into one package and must fall back to the feature
   package solution.

   Generally, the workaround involves changing upstream's dependency
   structure in order to fit it into the aforementioned Debian
   constraints, and so of course this may not always play nicely with
   other packages that expect the unchanged upstream dependency
   structure. The feature-package solution is a 1-to-1 mapping.

There have been multiple discussions between the FTPTeam and the Rust
package maintainers. The FTPTeam does not want those feature packages in
the part of main downloaded by users and currently rejects them from
NEW, while the Rust maintainers see them as needed and the workaround as
just that. Both sides agree that this is not a productive and
sustainable solution and that we need to agree on something better.

The current proposal is to reduce the main Packages.xz files size by
splitting[4] out all of the packages that are not intended for users,
writing those into an own file. Those packages would have a section of
"buildlibs", independent of their other properties. That section should
only be activated on buildds and in situations that need
build-dependencies available (say, an archive rebuild, a user rebuilding
packages that need Build-Dependencies from there), but not by default
anywhere else. This section will allow feature packages and *may* even
let them bypass binary-NEW if they only add new feature (empty)
packages.

The exact way of how this gets implemented, both in dak and also apt, is
still being discussed between the ftpteam and the apt maintainers. We
have ideas from writing out section based packages files to presenting
it as a subcomponent to main, and we think we will have som

Re: Split Packages files based on new section "buildlibs"

2020-11-09 Thread Paul Wise
On Mon, Nov 9, 2020 at 10:37 PM Joerg Jaspert wrote:

> More and more packages are being uploaded into the Debian archive which
> are only ever used for building packages. These are not only never
> intended to be installed onto an end-user's system, they are even
> actively discouraged from being used directly by a user. The two
> currently most notable examples are packages used by the Go and Rust
> programming languages and their ecosystem, but there well may be
> others[1].

Does this include the -dev packages for C/etc libraries?

I guess it also applies to Haskell and other statically-linked languages.

https://wiki.debian.org/StaticLinking

> The current proposal is to reduce the main Packages.xz files size by
> splitting[4] out all of the packages that are not intended for users,
> writing those into an own file. Those packages would have a section of
> "buildlibs", independent of their other properties.

Should (almost?) everything in the existing libdevel section move to
the new buildlibs section?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise