Bug#934733: ITP: webots -- Webots is robot simulator providing a complete development environment to model, program and simulate robots, vehicles and biomechanical systems.

[Olivier Michel]

Package: wnpp
Severity: wishlist
Owner: Olivier Michel 

* Package name    : webots
  Version : R2019b
  Upstream Author : Olivier Michel 
* URL : https://github.com/omichel/webots
* License : Apache 2.0
  Programming Lang: C++
  Description : Webots is robot simulator providing a complete development 
environment to model, program and simulate robots, vehicles and biomechanical 
systems.

Webots is an open-source robot simulator.
It is widely used in industry, education and research.
The Webots project started in 1996, initially developed by Dr. Olivier Michel 
at the Swiss Federal Institute of
Technology (EPFL) in Lausanne, Switzerland.
Since December 2018, it is released under the Apache 2 license.

Webots uses a fork of the ODE (Open Dynamics Engine) for detecting of 
collisions and simulating rigid body dynamics.
The ODE library allows one to accurately simulate physical properties of 
objects such as velocity, inertia and friction.

A large collection of freely modifiable robot models comes in the software 
distribution.
In addition, it is also possible to build new models from scratch.
When designing a robot model, the user specifies both the graphical and the 
physical properties of the objects.
The graphical properties include the shape, dimensions, position and 
orientation, colors, and texture of the object.
The physical properties include the mass, friction factor, as well as the 
spring and damping constants.
Simple fluid dynamics is present in the software.

Webots includes a set of sensors and actuators frequently used in robotic 
experiments, e.g. proximity sensors,
light sensors, touch sensors, GPS, accelerometers, cameras, emitters and 
receivers, servo motors (rotational & linear),
position and force sensor, LEDs, grippers, gyros, compass, etc.

The robot controller programs can be written in C, C++, Python, ROS, Java and 
MATLAB using a simple API.

Webots offers the possibility to take PNG screen shots and to record the 
simulations as MPEG (Mac/Linux) and AVI
(Windows) movies. Webots worlds are stored in cross-platform .wbt files which 
format is based on the VRML language.
It is also possible to import and export Webots worlds or objects in the VRML 
format.
Another useful feature is that the user can interact with a running simulation 
at any time, i.e.,
it is possible to move the robots and other object with the mouse.
Webots can stream a simulation on web browsers using WebGL.

Webots is used in several online robot programming competitions including 
https://robotbenchmark.net

This package is useful as it is often recognized as the best tool for 
simulating mobile robots.
It is used by thousands of research labs worldwide in both academia and 
industry.
Another package providing a similar functionality is Gazebo.
Comparing to Gazebo, Webots is easier-to-use and more powerful on a large 
number of features:
3D rendering, sensor modeling, physics simulation acuracy, transfer to real 
robots, etc.

We plan to maintain it on the long term as we are receiving European funding 
for that purpose.
We do not need co-maintenainers, we would welcome volunteer :).
We need a sponsor to get into debian.

Re: salsa.debian.org partially down

[Ian Jackson]

Alexander Wirt writes ("Re: salsa.debian.org partially down"):
> It is already recovered. We will investigate where we can extend the
> ressources. But some misusages (like requesting >1300 merge requests via API
> on a big project, that in consequence run >1300 ci jobs, that...) can't be
> solved regardless on how many resources we add. 

Thanks for the reports from you and Bastian.  Thanks also for having
the energy and effort to deal with this kind of thing.  It's annoying
when a thing you're responsible for breaks because of foolish user
action, and then you have to scramble to fix it.

Maybe I'm teaching my grandmother to such eggs, but your message made
me want to suggest possible solutions/mitigations for the problem you
mention above.  Please feel free to disregard what follows.


I think the problem can be summarised/generalised as "someone makes
more requests to salsa than it has capacity to fulfil".

Traditional approaches to this include (mentioning all that I can
think of, even inappropriate or already-done ones; and, not knowing
what features gitlab has for this):

 * Per-user quotas.  (The kind of user who submits 1300 MRs might well
   react to a limit by creating more guest accounts...)

 * Per-project quotas.  (This avoids the above problem.  It
   ring-fences problems with poor contributor behaviour to the
   projects whose contributors are behaving poorly.)

 * Queuing jobs, so that the effect is contained (eg to the CI
   subsystem) until an administrator can cancel some jobs.  I think
   maybe earlier when Bastian wrote "It turns out that the configured
   amount of concurrency in CI builds can't be handled by the current
   available system resources" he was referring to a tuneable which
   would have the effect of queueing things, next time.  I guess
   you've adjusted this already.

 * Restricting resource-intensive actions to certain users.
   In our context this would seem to involve asking project
   maintainers to manually trigger CI on MRs.  That seems like it
   would be annoying and best avoided if we can.

 * Balkanising the system into multiple instances (perhaps with
   different configurations) so that each instance is exposed to a
   much smaller userbase.  I doubt we have the effort for this even if
   we could come up with a sensible division, and liked the idea.
   (One way to test the waters in this direction would be for someone
   to set up a competitor to salsa based on an entirely different
   management stack.)

 * Documentation, deterrence and punishment.  I mention this for
   completeness; given that we have so many users, and also offer
   guest accounts, this is not an appropriate strategy for salsa.

I hope that you find this message useful, rather than just a statement
of things which are mostly obvious and/or irrelevant.

Regards,
Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

[Simon McVittie]

On Tue, 13 Aug 2019 at 22:01:34 -0400, Theodore Y. Ts'o wrote:
> That's just a matter of having sysvinit (and other non-systemd init
> systems) have an init script which runs as soon as the root file
> system is remounted read/write to initialize /etc/machine-id if it
> doesn't exist or if it is a zero-length file, right?

Yes ish, although it isn't *necessarily* an init system responsibility.
Somehow describing which containers and chroots should have a machine ID,
which ones should share the host's machine ID and which ones don't need
either is a gap in my proposal.

init is no longer Essential, so Debian chroots and containers will often
have neither systemd nor sysvinit (or any of the other alternatives),
but perhaps they should have a machine-id anyway - or perhaps container
managers that don't run a full init system, like schroot, should be
responsible for that? Or perhaps this requirement isn't necessary
for containers that don't provide either system services or user
logins? (The elephant in the room here is that Docker doesn't arrange to
have a machine-id, and also doesn't set the $container_uuid proposed in
.)

systemd-nspawn already sets up a machine ID for its containers, and lxc
(presumably also lxd) normally runs init, but schroot and Docker don't
normally run init and also don't take any particular steps to have a
machine ID.

Flatpak copies the machine ID from the host system into its containers,
and I would assume that other frameworks with "app containers" that are
conceptually part of the host machine rather than their own machine,
like Snap and AppImage, probably do the same.

An implementation of this should copy the dbus machine ID if it exists
(if the dbus machine ID differs from machine-id(5) then for historical
reasons various libraries will disagree on which is more important)
and the other subtleties described in systemd-machine-id-setup(1) are
probably also a good idea. On Linux systems dbus-uuidgen is not required,
because `tr -d - < /proc/sys/kernel/random/uuid` is suitable. I'm sure
kFreeBSD and Hurd have an equivalent, but I don't know what it is.

smcv

Re: salsa.debian.org partially down

[Thomas Goirand]

On 8/13/19 1:59 PM, Alexander Wirt wrote:
> It is already recovered. We will investigate where we can extend the
> ressources. But some misusages (like requesting >1300 merge requests via API
> on a big project, that in consequence run >1300 ci jobs, that...) can't be
> solved regardless on how many resources we add. 
> 
> Alex

It'd be nice if Gitlab could be plugged into something like Zuul [1],
and then we'd use nodepool, which would handle this type of load.

Cheers,

Thomas Goirand (zigo)

[1] https://zuul-ci.org/docs/zuul/

Re: salsa.debian.org partially down

[Bastian Blank]

On Wed, Aug 14, 2019 at 10:53:56AM +0200, Thomas Goirand wrote:
> On 8/13/19 1:59 PM, Alexander Wirt wrote:
> > It is already recovered. We will investigate where we can extend the
> > ressources. But some misusages (like requesting >1300 merge requests via API
> > on a big project, that in consequence run >1300 ci jobs, that...) can't be
> > solved regardless on how many resources we add. 
> It'd be nice if Gitlab could be plugged into something like Zuul [1],
> and then we'd use nodepool, which would handle this type of load.

Please describe the problem to solve first.  Because formorer only
talked about what triggered the problem, not what part of the whole
system actually broke.

(Hint: it's not the number of jobs, or at least not directly.  And even
a different CI platform could not fix an overload that a different
system brought to it's knees.)

Regards,

-- 
Conquest is easy. Control is not.
-- Kirk, "Mirror, Mirror", stardate unknown

Building GTK programs without installing systemd-sysv?

[Simon Richter]

Hi,

I have a few users who do test builds of kicad on my server, so I'd like to
provide the necessary build dependencies, but since a few days, the
dependency chain

libwxgtk3.0-gtk3-dev
  libwxgtk3.0-gtk3-0v5
libgtk-3-0
  libgtk-3-common
dconf-gsettings-backend
  dconf-service
dbus-user-session
  libpam-systemd
systemd-sysv

stops gtk upgrades from happening because I have pinned systemd-sysv to
-100 to avoid repeating the last unfortunate incident where I had to drive
to the colo facility.

Does anyone with more insight into which of these dependencies are actually
hard requirements have a suggestion if we could demote one of them to a
Recommends?

And in the long run: would it make sense to require that packages that are
build dependencies of something can be installed without starting any
service?

   Simon

Re: Building GTK programs without installing systemd-sysv?

[Holger Levsen]

On Wed, Aug 14, 2019 at 12:40:24PM +0200, Simon Richter wrote:
> And in the long run: would it make sense to require that packages that are
> build dependencies of something can be installed without starting any
> service?

why not build in your favorite container?


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Building GTK programs without installing systemd-sysv?

[Philipp Kern]

On 8/14/2019 12:40 PM, Simon Richter wrote:
> I have a few users who do test builds of kicad on my server, so I'd like to
> provide the necessary build dependencies, but since a few days, the
> dependency chain
> 
> libwxgtk3.0-gtk3-dev
>   libwxgtk3.0-gtk3-0v5
> libgtk-3-0
>   libgtk-3-common
> dconf-gsettings-backend
>   dconf-service
> dbus-user-session
>   libpam-systemd
> systemd-sysv
> 
> stops gtk upgrades from happening because I have pinned systemd-sysv to
> -100 to avoid repeating the last unfortunate incident where I had to drive
> to the colo facility.

You want dbus-x11 instead of dbus-user-session then, I think.

Kind regards
Philipp Kern

Re: Why keep upstream sources in Git at salsa.d.o?

[Daniel Leidert]

Am Dienstag, den 13.08.2019, 22:10 +0200 schrieb Alexis Murzeau:

[..]
> I'm curious about keeping only debian/ in git.
> How to build such git repository ?
> 
> I've tried one of yours but failed like this:
> ```
> $ gbp clone https://salsa.debian.org/ruby-team/ruby-html-proofer
> gbp:info: Cloning from
> 'https://salsa.debian.org/ruby-team/ruby-html-proofer'
> $ cd ruby-html-proofer/
> $ gbp buildpackage "--git-builder=sbuild -v -As -d unstable" \
> > --git-export-dir=../build-area
> gbp:error: upstream/3.11.1 is not a valid treeish
> ```
> 
> I guess I'm missing either a part of git history or just the orig
> tarball, but what's the normal way to get it ?

You need the tarball of course, which you can obtain via origtargz. The tarball
is expected to be found at the location specified in tarball_dir in /etc/git-
buildpackage/gbp.conf, ~/.gbp.conf or given via command line paramter. You can
also use the command line to run origtargz:

gbp clone https://salsa.debian.org/ruby-team/ruby-html-proofer
cd ruby-html-proofer
gbp buildpackage --git-preexport="origtargz -dt" --git-tarball-dir=../ \
 --git-builder="sbuild -v -As -d unstable" \
 --git-export-dir=../build-area/

> (maybe there is a doc or wiki somewhere about this ?)

It is not yet very well documented.

HTH and regards, Daniel


signature.asc
Description: This is a digitally signed message part

Re: Building GTK programs without installing systemd-sysv?

[Adam Borowski]

On Wed, Aug 14, 2019 at 01:47:47PM +0200, Philipp Kern wrote:
> On 8/14/2019 12:40 PM, Simon Richter wrote:
> > I have a few users who do test builds of kicad on my server, so I'd like to
> > provide the necessary build dependencies, but since a few days, the
> > dependency chain
> > 
> > libwxgtk3.0-gtk3-dev
> >   libwxgtk3.0-gtk3-0v5
> > libgtk-3-0
> >   libgtk-3-common
> > dconf-gsettings-backend
> >   dconf-service
> > dbus-user-session
> >   libpam-systemd
> > systemd-sysv
> > 
> > stops gtk upgrades from happening because I have pinned systemd-sysv to
> > -100 to avoid repeating the last unfortunate incident where I had to drive
> > to the colo facility.
> 
> You want dbus-x11 instead of dbus-user-session then, I think.

For this and related issues, I wonder if it would be better to replace the
virtual package pair of {default-,}dbus-session-bus with a real dummy
package that has:
Depends: dbus-x11 | systemd-sysv, dbus-user-session | dbus-x11

This would pull in dbus-user-session for systemd users, dbus-x11 for any
other init.

(The idea might be bogus -- I haven't slept since Monday morning; anyone up
for beersigning near Frankfurt airport until late evening?)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian is one big family.  Including that weird uncle
⢿⡄⠘⠷⠚⠋⠀ and ultra-religious in-laws.
⠈⠳⣄

Re: Building GTK programs without installing systemd-sysv?

[Simon Richter]

Hi,

On Wed, Aug 14, 2019 at 01:47:47PM +0200, Philipp Kern wrote:

> You want dbus-x11 instead of dbus-user-session then, I think.

Ah, that works, and aptitude is able to resolve that automatically (but apt
gets confused). So the immediate solution works for me and I'll file a bug
against apt. Thanks a lot!

The long term question remains though -- I dimly remember that we once had
the same discussion about a library pulling in rpcbind, and that made a lot
of people very unhappy at the time.

   Simon

Re: Why keep upstream sources in Git at salsa.d.o?

[Maximiliano Curia]

¡Hola Daniel!

El 2019-08-13 a las 01:17 +0200, Daniel Leidert escribió:

`gbp pq` doesn't work with this layout.


Actually, it does, you just have to add the option --pq-from=TAG.

Happy hacking,
--
"The most important thing in the programming language is the name. A language 
will not succeed without a good name. I have recently invented a very good 
name and now I am looking for a suitable language."

-- Donald Knuth
Saludos /\/\ /\ >< `/


signature.asc
Description: PGP signature

Re: Building GTK programs without installing systemd-sysv?

[Andrey Rahmatullin]

On Wed, Aug 14, 2019 at 02:24:26PM +0200, Simon Richter wrote:
> The long term question remains though -- I dimly remember that we once had
> the same discussion about a library pulling in rpcbind, and that made a lot
> of people very unhappy at the time.
There was also http://bugs.debian.org/658541 for libimobiledevice and
usbmuxd (it is now Recommends).

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Bug#934756: ITP: ruby-omniauth-salesforce -- OmniAuth strategy for salesforce.com

[Sruthi Chandran]

Package: wnpp
Severity: wishlist
Owner: Sruthi Chandran 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name: ruby-omniauth-salesforce
  Version : 1.0.5
  Upstream Author : Richard Vanhook
* URL : https://github.com/realdoug/omniauth-salesforce
* License : Expat
  Description : OmniAuth strategy for salesforce.com
 OmniAuth is a Ruby library that standardizes multi-provider
authentication for web applications. It was created to be powerful,
flexible, and do as little as possible. Any developer can create
strategies for OmniAuth that can authenticate users via disparate
 systems. OmniAuth strategies have been created for everything from
Facebook to LDAP.
 .
 This package contains the OmniAuth strategy for salesforce.com



signature.asc
Description: OpenPGP digital signature

Bug#934757: ITP: ycm-cmake-modules -- Extra CMake Modules for YARP and friends

[Daniele E. Domenichelli]

Package: wnpp
Severity: wishlist
Owner: "Daniele E. Domenichelli" 

* Package name: ycm-cmake-modules
  Version : 0.10.4
  Upstream Author : Istituto Italiano di Tecnologia (IIT)
* URL : https://github.com/robotology/ycm/ 
* License : BSD-3-Clause
  Programming Lang: CMake
  Description : Extra CMake Modules for YARP and friends

The package YCM contains a set of CMake files that support creation and
maintenance of repositories and software packages. YCM has been written
with the aim of solving some of the problems encountered while managing
large research projects but it can be used outside it initial context.

YCM is not a replacement to CMake or yet another build system. YCM is
just a thin layer over CMake. It does not add extra dependencies like
other build systems, it does not use Python, Ruby, or other scripting
languages, it is written using CMake language. It is a collection of
CMake modules, most of them candidate for being contributed upstream.

Re: Why keep upstream sources in Git at salsa.d.o?

[Daniel Leidert]

Am Mittwoch, den 14.08.2019, 09:48 -0300 schrieb Maximiliano Curia:
> El 2019-08-13 a las 01:17 +0200, Daniel Leidert escribió:
> > `gbp pq` doesn't work with this layout.
> 
> Actually, it does, you just have to add the option --pq-from=TAG.

How so? There is no upstream branch. So independent of the setting in --pq-from 
the command fails here. Are you referring to a layout, where an upstream branch
exists, but is not merged with the debian branch?

Regards, Daniel


signature.asc
Description: This is a digitally signed message part

Bug#934759: ITP: ruby-sassc-rails -- Integrate SassC-Ruby into Rails

[Sruthi Chandran]

Package: wnpp
Severity: wishlist
Owner: Sruthi Chandran 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name: ruby-sassc-rails
  Version : 2.1.2
  Upstream Author : Ryan Boland
* URL : https://github.com/sass/sassc-rails
* License : Expat
  Description : Integrate SassC-Ruby into Rails
 Compilation of Sass can take quite a long time for larger codebases.
This gem integrates the C implementation of Sass, LibSass, into the
asset pipeline. In one larger project, this made compilation 4x faster.
 .
 This should essentially be a drop in alternative to sass-rails.



signature.asc
Description: OpenPGP digital signature

Re: Building GTK programs without installing systemd-sysv?

[Philipp Kern]

On 8/14/2019 2:24 PM, Simon Richter wrote:
> The long term question remains though -- I dimly remember that we once had
> the same discussion about a library pulling in rpcbind, and that made a lot
> of people very unhappy at the time.

As Holger said: Then use a chroot. With policy-rc.d you can deny service
startup, which is also what the builders do.

Kind regards
Philipp Kern

Re: Building GTK programs without installing dconf-service?

[Simon McVittie]

On Wed, 14 Aug 2019 at 14:24:26 +0200, Simon Richter wrote:
> The long term question remains though -- I dimly remember that we once had
> the same discussion about a library pulling in rpcbind, and that made a lot
> of people very unhappy at the time.

I think this is a bit of a lose/lose situation: if we downgrade the
Depends to Recommends, as long as there is an (IMO unwise) meme that
globally disabling Recommends is the right thing to do for "efficient"
and "minimal" systems, people will install GTK programs on their desktops
with Recommends disabled, and be surprised (and/or open high-severity
bugs) when settings aren't saved as a result.

 libwxgtk3.0-gtk3-dev
   libwxgtk3.0-gtk3-0v5
 libgtk-3-0

None of the dependencies above here can be broken because of how shared
libraries work.

   libgtk-3-common

This is an implementation detail of libgtk-3-0, containing its /usr/share.
GTK won't work without at least the GSettings schemas (the GSettings
API is such that installing the code without the settings schemas is
considered to be invalid / a packaging error), so Depends is right.

 dconf-gsettings-backend | gsettings-backend

This dependency is generated by dh_installgsettings as a result of
libgtk-3-common containing GSettings schemas. It *could* be downgraded
to a Recommends, but then the user of GSettings (in this case GTK)
would fall back to the in-memory settings backend, which doesn't
actually persist any settings. Not so good. These days there's also a
GKeyFile-based backend, which is used inside Flatpak containers; but I
think that has problems with multiple writers overwriting each other's
changes (like they would tend to for anything else based on a simple
text configuration file written by more than one user of a library),
which makes it undesirable outside constrained single-app situations
like Flatpak. There's also no migration path between that and dconf, so
installing dconf-gsettings-backend would result in your previous settings
apparently disappearing, which would probably also be considered to be
a high-severity bug.

   dconf-service

dconf-gsettings-backend cannot save any settings without the service that
deals with change-notification and arbitration between multiple writes
(that's how dconf works), so this should probably be a hard dependency.

 dbus-user-session | dbus-x11 (effectively)

dconf is a D-Bus session service which won't work, at all, without a
D-Bus session bus. In a more opinionated distro, maybe we would leave out
this dependency and just assume that working D-Bus is part of a standard
GUI installation (the same way we have that assumption for working X11),
but my understanding is that in Debian that would be considered to be a
bug (and in a more opinionated distro, we'd probably also have gone for
systemd 100%, rather than remaining in a superposition of states). So
this needs to be a hard dependency.

dbus-user-session is the preferred option on any machine that boots with
systemd, because it uses a real service manager, gives the session bus
a well-defined lifetime, and works for Wayland, X11 and non-graphical
sessions; whereas dbus-x11 (dbus-launch) only works for X11 sessions,
results in the dbus-daemon being a child of an arbitrary process with
an unpredictable execution environment, and is constrained by the need
to avoid breaking assumptions that are not necessarily valid or relevant
this decade.

   libpam-systemd
 systemd-sysv

dbus-user-session relies on `systemd --user`, which needs systemd-logind
*and* systemd as pid 1 (a systemd-logind clone like elogind is not
enough), so this really is a hard dependency.

If someone implements a sufficiently capable non-systemd user-service
manager to provide the dbus-user-session semantics without systemd[1],
I'd be willing to consider hooking up dbus-user-session to that as
an alternative dependency.

 smcv

[1] briefly: one dbus-daemon per "user-session", already listening before
ordinary user processes can run, where a user-session is a contiguous
overlapping set of login sessions under the same uid (this is the
same as the lifetime of XDG_RUNTIME_DIR in the freedesktop.org
basedirs specification); and then use the dbus-daemon's socket
address "$XDG_RUNTIME_DIR/bus" to set $DBUS_SESSION_BUS_ADDRESS for
all ordinary user processes

Re: Bypassing the 2/3/4GB virtual memory space on 32-bit ports

[Aurelien Jarno]

On 2019-08-09 16:26, Luke Kenneth Casson Leighton wrote:
> ---
> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
> 
> On Fri, Aug 9, 2019 at 1:49 PM Ivo De Decker  wrote:
> >
> > Hi Aurelien,
> >
> > On 8/8/19 10:38 PM, Aurelien Jarno wrote:
> >
> > > 32-bit processes are able to address at maximum 4GB of memory (2^32),
> > > and often less (2 or 3GB) due to architectural or kernel limitations.
> >
> > [...]
> >
> > Thanks for bringing this up.
> >
> > > 1) Build a 64-bit compiler targeting the 32-bit corresponding
> > > architecture and install it in the 32-bit chroot with the other
> > > 64-bit dependencies. This is still a kind of cross-compiler, but the
> > > rest of the build is unchanged and the testsuite can be run. I guess
> > > it *might* be something acceptable. release-team, could you please
> > > confirm?
> >
> > As you noted, our current policy doesn't allow that. However, we could
> > certainly consider reevaluating this part of the policy if there is a
> > workable solution.
> 
> it was a long time ago: people who've explained it to me sounded like
> they knew what they were talking about when it comes to insisting that
> builds be native.
> 
> fixing binutils to bring back the linker algorithms that were
> short-sightedly destroyed "because they're so historic and laughably
> archaic why would we ever need them" should be the first and only
> absolute top priority.
> 
> only if that catastrophically fails should other options be considered.
> 
> with the repro ld-torture code-generator that i wrote, and the amount
> of expertise there is within the debian community, it would not
> surprise me at all if binutils-ld could be properly fixed extremely
> rapidly.
> 
> a proper fix would also have the advantage of keeping linkers for
> *other* platforms (even 64 bit ones) out of swap-thrashing, saving
> power consumption for build hardware and costing a lot less on SSD and
> HDD regular replacements.

That would only fix ld, which is only a small part of the issue. Do you
also have ideas about how to fix llvm, gcc or rustc which are also
affected by virtual memory exhaustion on 32-bit architectures?

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Bug#934773: ITP: ruby-activerecord-explain-analyze -- ActiveRecord#explain with support for EXPLAIN ANALYZE

[Sruthi Chandran]

Package: wnpp
Severity: wishlist
Owner: Sruthi Chandran 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name: ruby-activerecord-explain-analyze
  Version : 0.1.0
  Upstream Author : Peter Graham
* URL : https://github.com/6/activerecord-explain-analyze
* License : Expat
  Description : ActiveRecord#explain with support for EXPLAIN ANALYZE
 Extends ActiveRecord#explain with support for EXPLAIN ANALYZE and
output formats of JSON, XML, and YAML.
 .
 What's EXPLAIN ANALYZE? PostgreSQL devises a query plan for each query
it receives. Choosing the right plan to match the query structure and
the properties of the data is absolutely critical for good performance,
so the system includes a complex planner that tries to choose good
plans. One can use the EXPLAIN command to see what query plan the
planner creates for any query. With EXPLAIN ANALYZE, EXPLAIN actually
executes the query, and then displays the true row counts and true run
time accumulated within each plan node.



signature.asc
Description: OpenPGP digital signature

Re: Building GTK programs without installing dconf-service?

[Jonas Smedegaard]

Quoting Simon McVittie (2019-08-14 17:59:00)
> On Wed, 14 Aug 2019 at 14:24:26 +0200, Simon Richter wrote:
> > The long term question remains though -- I dimly remember that we 
> > once had the same discussion about a library pulling in rpcbind, and 
> > that made a lot of people very unhappy at the time.
> 
> I think this is a bit of a lose/lose situation: if we downgrade the 
> Depends to Recommends, as long as there is an (IMO unwise) meme that 
> globally disabling Recommends is the right thing to do for "efficient" 
> and "minimal" systems, people will install GTK programs on their 
> desktops with Recommends disabled, and be surprised (and/or open 
> high-severity bugs) when settings aren't saved as a result.

Please follow Debian Policy, and let those misguided souls have their 
surprises.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Git Packaging Round 1: Hopefully Easy Stuff

[Sam Hartman]

Hi.

In my latest bits from the DPL, I said that I was going to start a
couple of consensus-forming discussions on debian-devel surrounding Git
and packaging.  These will run similarly to the Debhelper/Dh discussion.
The issues are more complicated, and so I want to spread things out a
bit.

In this message I want to cover a few things that I think are
non-controversial.

If you disagree with any of the following, please reply.

BTS for Patches
===

Regardless of anything they are doing with Git, maintainers of a package
are expected to process patches sent to the BTS.  You cannot respond to
a patch telling someone they need to file a merge request to have it
considered.
Today, the BTS is still very much an appropriate way to propose changes
to a Debian package.

Unmonitored Merge Requests Harmful
==

If you're going to set up a repository for Debian packaging on Salsa,
you need to either:

1) turn off merge requests

or

2) Monitor them and process them.

I don't want to get into how frequently you look at merge requests just
as I don't want to get into how responsive you need to be to the BTS.
But I hope we can agree that if you're going to have merge requests
enabled that they cannot go into a black hole.

I think the question of whether people should use merge requests is more
complex.  Sean pointed out some reasons why we might prefer the BTS.
And yet it's clear that many people do find merge requests useful.


VCS Packaging Info
==

If you have a public repo then you should use vcs-git and vcs-browser.
I'm reasonably sure this is even already well documented.



Thanks for your consideration,

--Sam


signature.asc
Description: PGP signature

Re: Building GTK programs without installing dconf-service?

[Simon McVittie]

On Wed, 14 Aug 2019 at 19:52:42 +0200, Jonas Smedegaard wrote:
> Quoting Simon McVittie (2019-08-14 17:59:00)
> > I think this is a bit of a lose/lose situation: if we downgrade the 
> > Depends to Recommends, as long as there is an (IMO unwise) meme that 
> > globally disabling Recommends is the right thing to do for "efficient" 
> > and "minimal" systems, people will install GTK programs on their 
> > desktops with Recommends disabled, and be surprised (and/or open 
> > high-severity bugs) when settings aren't saved as a result.
> 
> Please follow Debian Policy, and let those misguided souls have their 
> surprises.

Which point in the dependency chain do you think should be weakened from
Depends to Recommends? I think the dependency from libgtk-3-0-common
generated by dh_installgsettings is probably the most appropriate, or at
least, least inappropriate? (This would require debhelper changes to add
a dh_installgsettings option analogous to dh_shlibdeps -- -dRecommends,
so that the dependency could be moved to ${misc:Recommends}.)

When an angry user turns up on the BTS complaining that GTK has a
grave bug (configuration lost) or a serious bug (Policy §3.5, missing
dependencies), is there consensus that this should be considered to
be not-a-bug and closed?

I thought I remembered Policy having something to say about weakening
shared libraries' dependencies on services to Recommends or weaker
(e.g. libdbus-1-3 only Recommends dbus and does not depend on it, even
though it's of little use without dbus), but now I can't find it in
Policy, and I also can't find a bug asking for that. Does this exist,
or did I imagine it?

smcv

Re: Git Packaging Round 1: Hopefully Easy Stuff

[Jonathan Carter]

On 2019/08/14 20:02, Sam Hartman wrote:
> If you're going to set up a repository for Debian packaging on Salsa,
> you need to either:
> 
> 1) turn off merge requests

That would be quite horrible IMHO, this is the de facto method that
young (let's say under 35 years old) developers use to submit
improvements to other projects. We have all the infrastructure and tools
conveniently in place to accommodate this, it seems suboptimal to treat
MRs as a second class citizen.

> 2) Monitor them and process them.
> 
> I don't want to get into how frequently you look at merge requests just
> as I don't want to get into how responsive you need to be to the BTS.
> But I hope we can agree that if you're going to have merge requests
> enabled that they cannot go into a black hole.
> 
> I think the question of whether people should use merge requests is more
> complex.  Sean pointed out some reasons why we might prefer the BTS.
> And yet it's clear that many people do find merge requests useful.

The Debian QA DDPO pages will show you whether you have MRs on the same
page where you see how many open bugs, RC bugs, lintian errors, etc you
have. This makes it super easy to notice MRs when doing routine checks
of your general package health overview.

-Jonathan

-- 
  ⢀⣴⠾⠻⢶⣦⠀  Jonathan Carter (highvoltage) 
  ⣾⠁⢠⠒⠀⣿⡁  Debian Developer - https://wiki.debian.org/highvoltage
  ⢿⡄⠘⠷⠚⠋   https://debian.org | https://jonathancarter.org
  ⠈⠳⣄  Be Bold. Be brave. Debian has got your back.

Re: Building GTK programs without installing dconf-service?

[Adam Borowski]

On Wed, Aug 14, 2019 at 07:22:33PM +0100, Simon McVittie wrote:
> Which point in the dependency chain do you think should be weakened from
> Depends to Recommends? I think the dependency from libgtk-3-0-common
> generated by dh_installgsettings is probably the most appropriate, or at
> least, least inappropriate?

I believe that full Depends is ok.  The original reporter's problem wasn't
about bloat, but about libgtk-3's dependency chain pulling in system-sysv in
a way that makes apt refuse to install it otherwise without some obscure
knowledge on the part of the user, that eludes even some DDs.

> When an angry user turns up on the BTS complaining that GTK has a
> grave bug (configuration lost) or a serious bug (Policy §3.5, missing
> dependencies), is there consensus that this should be considered to
> be not-a-bug and closed?

Aye, good point.

> I thought I remembered Policy having something to say about weakening
> shared libraries' dependencies on services to Recommends or weaker
> (e.g. libdbus-1-3 only Recommends dbus and does not depend on it, even
> though it's of little use without dbus), but now I can't find it in
> Policy, and I also can't find a bug asking for that. Does this exist,
> or did I imagine it?

That proposal was about moving Depends:/Recommends: relationships, not about
removing them.  Since, as you describe, the programs would still require
*conf to save their data, it wouldn't fix this particular problem.


One idea I have would be to change the way dbus-session is chosen, with
a real metapackage that defaults to dbus-user-session if systemd is already
installed or dbus-x11 if not, but that's just brainstorming, and my brain
hasn't slept in 60 hours. 


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian is one big family.  Including that weird uncle
⢿⡄⠘⠷⠚⠋⠀ and ultra-religious in-laws.
⠈⠳⣄

Re: Git Packaging Round 1: Hopefully Easy Stuff

[Sam Hartman]

> "Jonathan" == Jonathan Carter  writes:

Jonathan> On 2019/08/14 20:02, Sam Hartman wrote:
>> If you're going to set up a repository for Debian packaging on
>> Salsa, you need to either:
>> 
>> 1) turn off merge requests

Jonathan> That would be quite horrible IMHO, this is the de facto
Jonathan> method that young (let's say under 35 years old)
Jonathan> developers use to submit improvements to other
Jonathan> projects. We have all the infrastructure and tools
Jonathan> conveniently in place to accommodate this, it seems
Jonathan> suboptimal to treat MRs as a second class citizen.
And yet, consider the following cases:

* You are simply mirroring from somewhere else to Salsa

* You buy Sean's argument that MRs aren't stable enough to capture
  discussion long-term and/or we're unlikely to port forward the
  database of merge requests if we move away from Gitlab.

I think that merge requests are quite valuable, but I'm quite certain
we're not going to get to "you must enable merge requests."

Re: Building GTK programs without installing dconf-service?

[Jonas Smedegaard]

Quoting Simon McVittie (2019-08-14 20:22:33)
> On Wed, 14 Aug 2019 at 19:52:42 +0200, Jonas Smedegaard wrote:
> > Quoting Simon McVittie (2019-08-14 17:59:00)
> > > I think this is a bit of a lose/lose situation: if we downgrade 
> > > the Depends to Recommends, as long as there is an (IMO unwise) 
> > > meme that globally disabling Recommends is the right thing to do 
> > > for "efficient" and "minimal" systems, people will install GTK 
> > > programs on their desktops with Recommends disabled, and be 
> > > surprised (and/or open high-severity bugs) when settings aren't 
> > > saved as a result.
> > 
> > Please follow Debian Policy, and let those misguided souls have 
> > their surprises.
> 
> Which point in the dependency chain do you think should be weakened 
> from Depends to Recommends? I think the dependency from 
> libgtk-3-0-common generated by dh_installgsettings is probably the 
> most appropriate, or at least, least inappropriate? (This would 
> require debhelper changes to add a dh_installgsettings option 
> analogous to dh_shlibdeps -- -dRecommends, so that the dependency 
> could be moved to ${misc:Recommends}.)

libgtk-3-0-common could even be relaxed to _suggest_ dconf/gconf since 
already the applications using dconf/gconf declare a dependency on those 
disk-based backends.


> When an angry user turns up on the BTS complaining that GTK has a 
> grave bug (configuration lost) or a serious bug (Policy §3.5, missing 
> dependencies), is there consensus that this should be considered to be 
> not-a-bug and closed?

Issues solely caused by disregarding recommends should be closed as 
not-a-bug indeed.  Why do you ask?  If you somehow disagree with that, 
then I can only take it that you disagree with Debian Policy and it 
makes sense to deal with that instead of cowardly putting it on me.


> I thought I remembered Policy having something to say about weakening 
> shared libraries' dependencies on services to Recommends or weaker 
> (e.g. libdbus-1-3 only Recommends dbus and does not depend on it, even 
> though it's of little use without dbus), but now I can't find it in 
> Policy, and I also can't find a bug asking for that. Does this exist, 
> or did I imagine it?

Sorry, I am not Paul Wise or Colin Watson ;-)


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Git Packaging Round 1: Hopefully Easy Stuff

[Jonas Smedegaard]

Quoting Jonathan Carter (2019-08-14 20:25:05)
> On 2019/08/14 20:02, Sam Hartman wrote:
> > If you're going to set up a repository for Debian packaging on Salsa,
> > you need to either:
> > 
> > 1) turn off merge requests
> 
> That would be quite horrible IMHO, this is the de facto method that
> young (let's say under 35 years old) developers use to submit
> improvements to other projects. We have all the infrastructure and tools
> conveniently in place to accommodate this, it seems suboptimal to treat
> MRs as a second class citizen.

Interesting!

I systematically turn off Gitlab MR support for projects I am involved 
in, because I am not confortable and efficient using it myself, it is 
additional work for me to tune into (there is more to it than getting 
notification about MRs pending!), and I already have a workflow that 
works well for me and the teams I work with.

This is similar to my turning off Gitlab issue tracking.  And not using 
webforums and Matrix and Jabber and Diaspora and and and all the places 
where potential helpers may be waiting for me to reach out to them.

I do want to collaborate.  Sincerely!  I would have wanted that Debian 
reached out to me on Jabber when many years ago - they didn't and I had 
to do more work to get a text-chat experience with my peers.

I try hard to turn off any and all features except core git hosting.  
Did same for Alioth, and am disappointed that features are enabled by 
default putting pressure on me to change working habits.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Git Packaging Round 1: Hopefully Easy Stuff

[Jonathan Carter]

On 2019/08/14 20:36, Sam Hartman wrote:
> And yet, consider the following cases:
> 
> * You are simply mirroring from somewhere else to Salsa
> 
> * You buy Sean's argument that MRs aren't stable enough to capture
>   discussion long-term and/or we're unlikely to port forward the
>   database of merge requests if we move away from Gitlab.

Ah yes, good points.

-Jonathan

Re: Building GTK programs without installing dconf-service?

[Simon McVittie]

On Wed, 14 Aug 2019 at 20:59:04 +0200, Jonas Smedegaard wrote:
> libgtk-3-0-common could even be relaxed to _suggest_ dconf/gconf since 
> already the applications using dconf/gconf declare a dependency on those 
> disk-based backends.

The dependency is not because applications store configuration via
GSettings, it is because GTK widgets themselves store configuration (which
is shared between all applications that use those widgets) via GSettings.
The most prominent of these widgets is probably the file chooser
(File -> Open or File -> Save As dialog). Whenever applications use those,
preferences like the sort order are stored via GSettings, even if the
application itself has no mention of GSettings in its source code and
no reason to add a dependency on a GSettings backend on its own behalf.

abiword and audacity are examples of applications that depend on GTK,
probably use file chooser dialogs, but do not depend on a GSettings
backend themselves because they do not use GSettings directly.

The preferences stored in this way are not vitally important, so perhaps
it would be OK for them to just not be propagated outside the application
or stored after it exits (with a warning on stderr) if the GSettings
backend is missing; but it isn't obvious to me that there would be
consensus about that not being a (RC?) bug, which is why I asked.

I suppose one way to represent that in the spirit of what you're saying
would be to drop libgtk-3-0-common's dependency on a GSettings backend,
and make GTK's shlibs/symbols generate a dependency like

libgtk-3-0 (>= x.y), dconf-gsettings-backend | gsettings-backend

instead of just libgtk-3-0? But I don't think that works, because
it would only move the problem up one level, and instead of "I
installed libgtk-3-dev and it pulled in dconf-service", we'd have "I
installed libgtk-vnc-2.0-dev and it pulled in dconf-service" (where
libgtk-vnc-2.0-dev Depends: libgtk-vnc-2.0-0 which is an arbitrarily
chosen library that Depends: libgtk-3-0).

> Issues solely caused by disregarding recommends should be closed as 
> not-a-bug indeed.

This is a good rule of thumb, but there must be some point on the scale
between "not required" and "is a hard dependency" where a Recommends
becomes a Depends. If libgtk-3-0 only had a Recommends on libatk1.0-0
(because libgtk-3-0 contains both GDK and GTK, and strictly speaking
you can use GDK without ATK, as long as you don't also link to GTK),
I think there would be consensus that it would be wrong to consider
"libgtk-3-0 should depend on libatk1.0-0" to be not-a-bug.

Obviously failure to store some preferences is less serious than "one
of the two libraries doesn't work at all", but neither is an absolute:
they're both points on a continuum.

smcv

Re: Building GTK programs without installing dconf-service?

[Jonas Smedegaard]

Quoting Simon McVittie (2019-08-14 22:20:05)
> On Wed, 14 Aug 2019 at 20:59:04 +0200, Jonas Smedegaard wrote:
> > libgtk-3-0-common could even be relaxed to _suggest_ dconf/gconf 
> > since already the applications using dconf/gconf declare a 
> > dependency on those disk-based backends.
> 
> The dependency is not because applications store configuration via 
> GSettings, it is because GTK widgets themselves store configuration 
> (which is shared between all applications that use those widgets) via 
> GSettings. The most prominent of these widgets is probably the file 
> chooser (File -> Open or File -> Save As dialog). Whenever 
> applications use those, preferences like the sort order are stored via 
> GSettings, even if the application itself has no mention of GSettings 
> in its source code and no reason to add a dependency on a GSettings 
> backend on its own behalf.
> 
> abiword and audacity are examples of applications that depend on GTK, 
> probably use file chooser dialogs, but do not depend on a GSettings 
> backend themselves because they do not use GSettings directly.

Ah, thanks for pointing that out.


> The preferences stored in this way are not vitally important, so 
> perhaps it would be OK for them to just not be propagated outside the 
> application or stored after it exits (with a warning on stderr) if the 
> GSettings backend is missing; but it isn't obvious to me that there 
> would be consensus about that not being a (RC?) bug, which is why I 
> asked.

Would it be fair to describe both widget and application needs for 
file-based config backend as required in "all but unusual 
installations."




> I suppose one way to represent that in the spirit of what you're 
> saying would be to drop libgtk-3-0-common's dependency on a GSettings 
> backend, and make GTK's shlibs/symbols generate a dependency like
> 
> libgtk-3-0 (>= x.y), dconf-gsettings-backend | gsettings-backend
> 
> instead of just libgtk-3-0? But I don't think that works, because it 
> would only move the problem up one level, and instead of "I installed 
> libgtk-3-dev and it pulled in dconf-service", we'd have "I installed 
> libgtk-vnc-2.0-dev and it pulled in dconf-service" (where 
> libgtk-vnc-2.0-dev Depends: libgtk-vnc-2.0-0 which is an arbitrarily 
> chosen library that Depends: libgtk-3-0).

No, I don't mean that libgtk-3-0 should tell all its consumers that they 
all should depend(!) on a file-based config backend.

I simply noticed that a bunch of applications currently declare a 
dependency on those backends, without knowing how they grew that package 
relationship.

Based on your enlightening me above, I'd say that libgtk-3-0 should 
recommend those backends, and consumers of the config API provided by 
libgtk-3-0 should also (ideally only by default, possible to 
relax/tighten by each concrete package) recommend those backends as 
well.


> > Issues solely caused by disregarding recommends should be closed as 
> > not-a-bug indeed.
> 
> This is a good rule of thumb, but there must be some point on the 
> scale between "not required" and "is a hard dependency" where a 
> Recommends becomes a Depends. If libgtk-3-0 only had a Recommends on 
> libatk1.0-0 (because libgtk-3-0 contains both GDK and GTK, and 
> strictly speaking you can use GDK without ATK, as long as you don't 
> also link to GTK), I think there would be consensus that it would be 
> wrong to consider "libgtk-3-0 should depend on libatk1.0-0" to be 
> not-a-bug.

Not sure I follow you above.

If an application links with libgtk-3-0, 

If libgtk-3-0 provides an API that links with both GDK and ATK and 
gracefully handles consumers calling the ATK-related parts while libatk 
is missing, then libgtk-3-0 should only recommend libatk.

If however using the parts of libgtk-3-0 API related to ATK causes the 
library to segfault if libatk is missing, then libgtk-3-0 should depend 
on libatk.

Or are you talking about some other kind of behaviour?


> Obviously failure to store some preferences is less serious than "one 
> of the two libraries doesn't work at all", but neither is an absolute: 
> they're both points on a continuum.

Question is not if a feature totally stops working when related package 
is missing but instead if said feature truly is optional by _any_ extend 
- i.e. is there some exotic situation where it is sensible to omit 
without causing the code to become unstable?


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Git Packaging Round 1: Hopefully Easy Stuff

[Holger Levsen]

On Wed, Aug 14, 2019 at 09:08:44PM +0200, Jonas Smedegaard wrote:
> I systematically turn off Gitlab MR support for projects I am involved 
> in, because I am not confortable and efficient using it myself, it is 

what helps me is having a note with this line:

git config alias.mr '!sh -c "git fetch $1 merge-requests/$2/head:mr-$1-$2 && 
git checkout mr-$1-$2" -'  

which I copy & paste into a shell running inside the directory of a git
repo, and after that "git mr origin 2" will checkout the merge request #2.

Thanks to Raphael Hertzog for pointing this out some time ago.


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

Dance like no one's watching. Encrypt like everyone is.


signature.asc
Description: PGP signature

Making mailcap optional ?

[Charles Plessy]

Hello everybody,

(please CC me, I am not subscribed)

The mailcap (metamail capability file) system, defined by RFC 1524,
associates media types (called MIME types at the time) to software
available to handle them.  It was created to open email attachements but
can also be used with downloaded files, etc.  A main database is found
in /etc/mailcap and user-specified entries can be added in their home
directories.

In Debian, the mailcap system is provided by the mime-support package,
which I adopted a couple years ago.  It has a Standard priority.  The
package contains a template for /etc/mailcap, a tool for interrogating
this database and launching programs (run-mailcap), and a few
debian-specific additions to handle Debian packages, audio files, etc.

Other Debian packages can provide entries for /etc/mailcap by depositing
files containing new entries in /usr/lib/mime/packages, which is
monitored by dpkg with a "noawait" trigger.

Modern desktop environments typically use the Desktop menu entry system
and the shared-mime-info database instead of the mailcap system.
Thereofore, I wonder if it is time to make the mailcap system optional,
for instance to help reducing the number of dpkg triggers active on
standard systems.

But the mime-support package has also another function, which is to
provide the /etc/mime.types file, that associates file suffixes to media
types.  This is of course essential to the mailcap system, but it is
also used by many other packages, as one can see from the
reverse-depends list of mime-support.  The contents of /etc/mime.types
mostly stems from the IANA (RFC 6838).

Therefore I am tempted to move the mailcap system into a new package
(which could be called "mailcap" for instance).  The mime-support
package would basically contain /etc/mime.types only.  In the future,
I would be happy to automate (or see other people automate) the update
of the /etc/mime.types file from information taken from the IANA
website, perhaps in coordination with other distributions.

Please let me know your thoughts (perhaps after waiting a day or two to
let your thoughts mature).

Have a nice day,

Charles

-- 
Charles Plessy
Akano, Uruma, Okinawa, Japan

Re: ML-Policy and tesseract-ocr

[Mo Zhou]

On 2019-08-13 00:44, Sam Hartman wrote:
>> "Mo" == Mo Zhou  writes:
> 
> 
> >> (result of running the training program with specific input data,
> >> if I understand correctly?)
> 
> Mo> Yes, correct.
> 
> >> The source package would need to Build-Depend on the training
> >> program and its inputs, but in general there would not need to be
> >> a normal Depends.
> 
> Mo> I see. The idea is that an ELF binary (ML model) doesn't have to
> Mo> Depend on it's compiler (training program) and source (input
> Mo> data).  This makes sense to me and the "Suggest:" restriction
> Mo> may be better.
> 
> Mo> The "Suggest:" relationship implicitly hints the user about the
> Mo> following questions: 1. what is the binary blob
> Mo> /usr/.../foobar.ml-model installed by the package foobar?
> Mo> 2. where did these digits come from?  3. how can I well
> Mo> understand how this model is created by the original author?
> Mo> 4. how do I obtain a similar model with my own dataset?  etc.
> 
> As a user, if I want to understand how some binary thing gets created,
> I'll
> apt source 
> 
> rather than looking at suggests.
> 
> In cases where the model is created in the build process, I think
> build-depends is better than suggests.
> 
> In cases where the model is not recreated, but where software in Debian
> could create the model, I think a README file is better than a package
> relationship.

How about these:

* A source package that produces binary package containing ML model
  must contain the corresponding training program/script, or
  Build-Depends on the package that ships the program/script.

  (The training program will be present if the user wanted to
investigate)

* A package that contains ML model should annotate the type of
  model it ships in README.Debian, and answer at least the following
  questions: (1) what is the binary blob / pile of digits? (2) where
  is the training program located, which binary package and which path?
  (3) on what dataset was the model trained? (4) what's the license
  of the original dataset?

  (These are necessary information for the user to study a model)

* In a complex system where several ML models interact with each
  other and contribute to the system output, any ML model in
  different type would taint Free Model. Non-Free model taints
  any other model when used together.

  (well, I need to update the definition of the model types)

Re: ML-Policy and tesseract-ocr

[Mo Zhou]

On 2019-08-13 01:00, Paul Wise wrote:
> On Tue, Aug 13, 2019 at 8:45 AM Sam Hartman wrote:
> 
>> In cases where the model is not recreated, but where software in Debian
>> could create the model, I think a README file is better than a package
>> relationship.
> 
> Personally, I'd like to see Policy specify a standard mechanism that
> people could use to indicate to debian/rules that they want to
> automatically rebuild *everything* from source, not matter what the
> cost is.

This makes sense as the official policy doesn't cover non-standard
d/rules targets. How about this one:

* For a source package that produce binary package containing ML models,
  it's encouraged to write a "reproduce-original-model" that may e.g.
  download a dataset from internet and redo the same procedure to
  produce the original model.

It's "encouraged" because some models involve complex manual steps
that a random developer can hardly understand, e.g.
https://github.com/tesseract-ocr/tesseract/wiki/TrainingTesseract

See my another mail sent out several minutes ago. In that mail
maintainers are required to provide necessary information about
the distributed ML model in README.Debian .

Re: ML-Policy and tesseract-ocr

[Paul Wise]

On Wed, 2019-08-14 at 17:44 -0700, Mo Zhou wrote:

> * For a source package that produce binary package containing ML models,
>   it's encouraged to write a "reproduce-original-model" that may e.g.
>   download a dataset from internet and redo the same procedure to
>   produce the original model.

Sounds good except for the name. I'd prefer something more generic that
can also be used in situations that don't involve ML but are still
resource intensive enough that we won't run them on every build.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: Bypassing the 2/3/4GB virtual memory space on 32-bit ports

[Luke Kenneth Casson Leighton]

On Wed, Aug 14, 2019 at 5:13 PM Aurelien Jarno  wrote:

> > a proper fix would also have the advantage of keeping linkers for
> > *other* platforms (even 64 bit ones) out of swap-thrashing, saving
> > power consumption for build hardware and costing a lot less on SSD and
> > HDD regular replacements.
>
> That would only fix ld, which is only a small part of the issue. Do you
> also have ideas about how to fix llvm, gcc or rustc which are also
> affected by virtual memory exhaustion on 32-bit architectures?

*deep breath* - no.  or, you're not going to like it: it's not a
technical solution, it's going to need a massive world-wide sustained
and systematic education campaign, written in reasonable and logical
language, explaining and advising GNU/Linux applications writers to
take more care and to be much more responsible about how they put
programs together.

a first cut at such a campaign would be:

* designing of core critical libraries to be used exclusively through
dlopen / dlsym.  this is just good library design practice in the
first place: one function and one function ONLY is publicly exposed,
returning a pointer to a table-of-functions (samba's VFS layer for
example [1]).
* compile-time options that use alternative memory-efficient
algorithms instead of performance-efficient ones
* compile-time options to remove non-essentlal resource-hungry features
* compiling options in Makefiles that do not assume that there is vast
amounts of memory available (KDE "developer" mode for example would
compile c++ objects individually whereas "maintainer" mode would
auto-generate a file that #included absolutely every single .cpp file
into one, because it's "quicker").
* potential complete redesigns using IPC/RPC modular architectures:
applying the "UNIX Philosophy" however doing so through a runtime
binary self-describing "system" specifically designed for that
purpose.  *this is what made Microsoft [and Apple] successful*.  that
means a strategic focus on getting DCOM for UNIX up and running [2].
god no please not d-bus [3] [4].

also, it's going to need to be made clear to people - diplomatically
but clearly - that whilst they're developing on modern hardware
(because it's what *they* can afford, and what *they* can get - in the
West), the rest of the world (particularly "Embedded" processors)
simply does not have the money or the resources that they do.

unfortunately, here, the perspective "i'm ok, i'm doing my own thing,
in my own free time, i'm not being paid to support *your* hardware" is
a legitimate one.

now, i'm not really the right person to head such an effort.  i can
*identify* the problem, and get the ball rolliing on a discussion:
however with many people within debian alone having the perspective
that everything i do, think or say is specifically designed to "order
people about" and "tell them what to do", i'm disincentivised right
from the start.

also, i've got a thousand systems to deliver as part of a
crowd-funding campaign [and i'm currently also dealing wiith designing
the Libre RISC-V CPU/GPU/VPU]

as of right now those thousand systems - 450 of them are going to have
to go out with Debian/Testing 8.  there's no way they can go out with
Debian 10.  why? because this first revision hardware - designed to be
eco-conscious - uses an Allwinner A20 and only has 2GB of RAM
[upgrades are planned - i *need* to get this first version out, first]

with Debian 10 requiring 4GB of RAM primarily because of firefox,
they're effectively useless if they're ever "upgraded"

that's a thousand systems that effectively go straight into landfill.

l.

[1] 
https://www.samba.org/samba/docs/old/Samba3-Developers-Guide/vfs.html#id2559133

[2] incredibly, Wine has had DCOM and OLE available and good enough to
use, for about ten years now.  it just needs "extracting" from the
Wine codebase. DCOM stops all of the arguments over APIs (think
"libboost".  if puzzled, add debian/testing and debian/old-stable to
/etc/apt/sources.lst, then do "apt-cache search boost | wc")

due to DCOM providing "a means to publish a runtime self-describing
language-independent interface", 30-year-old WIN32 OLE binaries for
which the source code has been irretrievably lost will *still work*
and may still be used, in modern Windows desktops today.  Mozilla
ripped out XPCOM because although it was "inspired" by COM, they
failed, during its initial design, to understand why Co-Classes exist.

as a result it caused massive ongoing problems for 3rd party java and
c++ users, due to binary incompatibility caused by changes to APIs on
major releases.  Co-Classes were SPECIFICALLY designed to stop EXACTLY
that problem... and Mozilla failed to add it to XPCOM.

bottom line: the free software community has, through "hating" on
microsoft, rejected the very technology that made microsoft so
successful in the first place.

Microsoft used DCOM (and OLE), Apple (thanks to Steve's playtime /
break doing NeXT) developed Objective-C / Objective-J / Objectiv