Re: isa-support -- exit strategy?

[Stephan Lachnit]

On Sat, Mar 26, 2022 at 2:36 AM M. Zhou  wrote:
>
> Indeed supporting number crunching programs on ancient
> hardware is not meaningful, but the demand on Debian's
> support for number crunching is not that strong according
> to my years of observation.
>
> For popular applications that can take advantage of above-baseline
> instruction sets, they will eventually write the dynamic code
> dispatcher and add the fallback.
>
> For applications seriously need performance, they will
> leave CPU and go to GPU or other hardware. If the user correctly
> write the code and fully leverage GPU, the non-optimal CPU
> code won't necessarily be a bottleneck.
>
> For applications seriously need CPU performance, they are
> possibly going to tell the users how to tweak compiling
> parameters and how to compile locally.

I have to disagree on this one. Yes, runtime detection and GPU
acceleration is great and all, but not every scientific library does
it and I think it's unrealistic for us to patch them all up.
Also I don't like the point "since there is low demand for number
crunching on Debian, so let's just continue to ignore this problem".
At least I know a decent amount of people that use Debian (or
downstream distros) for scientific number crunching. Compiling
optimized for large workloads will always be a thing no matter the
baseline, but when getting started distro packages are just one less
thing to care about.

On Sat, Mar 26, 2022 at 7:25 AM Andrey Rahmatullin  wrote:
>
> A partial arch (whatever that is, yeah) with the x86-64-v3 baseline, and
> optionally raise the main amd64 baseline to x86-64-v2?

+1

Re: isa-support -- exit strategy?

[Simon McVittie]

On Sat, 26 Mar 2022 at 09:43:32 +0800, Paul Wise wrote:
> It might be worth looking at how things like Steam and Flatpak/Snap
> solve this issue

In general they don't, or to put it another way, they "solve" it to the
same extent that Debian/apt/dpkg currently does. Each binary build has
a required instruction set (often the default for its SDK's compiler,
but sometimes higher via command-line options like -msse2) and won't
work on older CPUs.

It seems like many upstreams, notably Mozilla and Rust, have effectively
chosen i686 + MMX + SSE + SSE2 to be their baseline for 32-bit builds,
because that level of functionality is nearly 20 years old and lets them
avoid the quirks of the i387 FPU.

The binaries for Steam itself are documented to require x86_64 with
CMPXCHG16B and SSE3, so Steam is higher-than-baseline even on x86_64.
The developers of Steam have no interest in supporting early 2000s
hardware, or embedded 32-bit x86 variants.

For most native Linux Steam games, the compilers in the SDK are the
version of gcc from an old version of Ubuntu (gcc 4.6, 4.8 or 5), or one
of several approximately contemporary versions of clang, or a backport
of gcc 9 from Debian, and inherit whatever our baseline was at the time
(i586 or nearly-i686); but Steam game developers are encouraged to
compile for x86_64 anyway, so it's mostly older games that have 32-bit
binaries. For games that still have 32-bit binaries, I suspect that
using -msse2 is common.

For Proton and for native Linux Steam games that use the new soldier and
sniper container runtimes (not really supported yet, but a few games jump
through the necessary hoops to do it anyway), the compiler in the SDK
is the version of gcc or clang from the Debian release that the runtime
is based on (Debian 10 or 11), or in the case of the Debian-10-based
soldier runtime, a backport of gcc 9. Again, game developers are encouraged
to compile for x86_64 when targeting these runtimes.

Most Flatpak apps are directly or indirectly based on the freedesktop.org
runtimes available from Flathub (the GNOME and KDE runtimes are derived
from the fd.o runtime), and the non-EOL versions of those runtimes no
longer directly support 32-bit builds, only x86_64 and aarch64.

Flatpak apps that need biarch x86 (i386 and x86_64 in parallel), like
the Flatpak version of Steam, use the x86_64 version of the runtime and
add the "Compat.i386" runtime extension. That extension uses multiarch
paths, but is conceptually more similar to Debian packages like lib64z1
than it is to Debian's i386 architecture - it contains 32-bit code,
but it is only for use by x86_64 CPUs in 32-bit compatibility mode, so
it can safely assume the presence of all the CPU extensions that are in
the x86_64 baseline, and in particular SSE2.

> I expect games and proprietary apps often use CPU
> features unsupported on old systems.

Yes, for large values of "old": bear in mind that, for example, SSE2 was
introduced by Intel in 2000 and adopted by AMD in 2003. If Wikipedia is
to be believed, the last new releases of mainstream pre-SSE2 CPUs were
late models of the Athlon XP (2004) and Pentium III mobile variants (2007).

(I'm aware that embedded 32-bit x86 CPUs like the AMD Geode series and
Intel Quark do not have the same functionality as mainstream desktop/laptop
CPUs of a comparable age.)

> I also wonder if qemu could be used to emulate newer CPU features on
> older systems. That would probably be unusably slow though.

For opcodes that are used to improve performance, that would be
completely self-defeating.

For opcodes that are necessary for correctness of a particular sequence
of code (like CMPXCHG16B), I don't see how that would even work; at best
it would be really expensive, and at worst it would not provide the
required semantics.

smcv

Bug#1008304: ITP: http-nio -- http/s file system provider for Java NIO.2

[Pierre Gruet]

Package: wnpp
Severity: wishlist
Owner: Debian-med team 
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-...@lists.debian.org

* Package name: http-nio
  Version : 0.1.0
  Upstream Author : Daniel Gomez-Sanchez
* URL : https://github.com/lbergelson/http-nio
* License : BSD-3-Clause
  Programming Lang: Java
  Description : http/s file system provider for Java NIO.2

This package provides a http or https file system that can be used in
conjunction with Java NIO.2. This lightweight library provides a few classes
related to file systems and paths.

It is provided by the developers of gatk, which is a long-term packaging target
of Debian-med team. I am packaging it as a reverse dependency of gatk.
For this reason, it will be team-maintained inside Debian-med team.

Bug#1008307: ITP: sphinx-reredirects -- extension for Sphinx projects that handles redirects

[Georges Khaznadar]

Package: wnpp
Severity: wishlist
Owner: Georges Khaznadar 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: sphinx-reredirects
  Version : 0.0.0+git20220326
  Upstream Author : Matt from Documatt 
* URL : https://gitlab.com/documatt/sphinx-reredirects
* License : BSD-3
  Programming Lang: Python3
  Description : extension for Sphinx projects that handles redirects

 sphinx-reredirects is the extension for Sphinx documentation projects
 that handles redirects for moved pages. It generates HTML pages with
 meta refresh redirects to the new page location to prevent 404 errors
 if you rename or move your documents.
 .
 Good URLs are never changing URLs. But if you must,
 sphinx-reredirects helps you manage redirects with ease and from the
 single place in project's conf.py. For example, if you rename
 document start to intro, and tell it to sphinx-reredirects, it will
 generate HTML page start.html with
 .
 The extension supports wildcards and moving to different domain too.


-

This package has become a build dependency for the package sympy, which
I am maintaining, since the last upstream version. It must be included in
debian if we want to keep maintaining the package sympy.

-

The only change I made was to replace the file logo.png, whose licensing
seems to be unsafe, by a home-brew logo (debian/arrow.svg, exported to
logo.png)

Bug#1008310: ITP: uc-micro-py -- python port of uc.micro

[Georges Khaznadar]

Package: wnpp
Severity: wishlist
Owner: Georges Khaznadar 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: uc-micro-py
  Version : 1.03
  Upstream Author : Tsutsu3 
* URL : https://github.com/tsutsu3/uc.micro-py
* License : MIT
  Programming Lang: Python3
  Description : python port of uc.micro

 Micro subset of unicode data files for linkify-it-py projects
 .
 This package was developped for the package python3-linkify-it

-

This package has become a build dependency for the package sympy, which
I am maintaining, since the last upstream version. It must be included in
debian if we want to keep maintaining the package sympy.

Bug#1008311: ITP: linkify-it-py -- links recognition library with FULL unicode support

[Georges Khaznadar]

Package: wnpp
Severity: wishlist
Owner: Georges Khaznadar 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: linkify-it-py
  Version : 1.0.3
  Upstream Author : Tsutsu3 
* URL : https://github.com/tsutsu3/linkify-it-py
* License : MIT
  Programming Lang: Python3
  Description : links recognition library with FULL unicode support

 linkify-it-py is focused on high quality link patterns detection in plain
text.
 .
 Why it's awesome:
 .
  - Full unicode support, with astral characters!
  - International domains support.
  - Allows rules extension & custom normalizers.

-

This package has become a build dependency for the package sympy, which
I am maintaining, since the last upstream version. It must be included in
debian if we want to keep maintaining the package sympy.

Re: isa-support -- exit strategy?

[M. Zhou]

On Sat, 2022-03-26 at 11:42 +0100, Stephan Lachnit wrote:
> On Sat, Mar 26, 2022 at 2:36 AM M. Zhou  wrote:
> > 
> > Indeed supporting number crunching programs on ancient
> > hardware is not meaningful, but the demand on Debian's
> > support for number crunching is not that strong according
> > to my years of observation.
> > 
> > For popular applications that can take advantage of above-baseline
> > instruction sets, they will eventually write the dynamic code
> > dispatcher and add the fallback.
> > 
> > For applications seriously need performance, they will
> > leave CPU and go to GPU or other hardware. If the user correctly
> > write the code and fully leverage GPU, the non-optimal CPU
> > code won't necessarily be a bottleneck.
> > 
> > For applications seriously need CPU performance, they are
> > possibly going to tell the users how to tweak compiling
> > parameters and how to compile locally.
> 
> I have to disagree on this one. Yes, runtime detection and GPU
> acceleration is great and all, but not every scientific library does
> it and I think it's unrealistic for us to patch them all up.

Please note I wrote "they (i.e. the upstream)" will implement
the runtime detection or GPU acceleration, instead of us (Debian).

> Also I don't like the point "since there is low demand for number
> crunching on Debian, so let's just continue to ignore this problem".

If it was 6 years ago, I would disagree with what I've said in the
original post. Whether you like it or not, what I said is my
changed mind after closely working on numerical related libraries
for 6 years in Debian. And to be clear, I hold a negative opinion
on what we Debian could actually change besides the upstream.

If the upstream does not write runtime detection or GPU acceleration,
they are either not facing a wide range of audience, or the problem
does not matter, or simply the software isn't appropriate for
Debian packaging.

I mentioned infinite times that the eigen3 library which implements
the core numerical computation part of TensorFlow does not support
runtime detection -- because CPU acceleration does not matter for
most of the users. Sane users who really need CPU performance are
able to recompile tensorflow themselves.

> At least I know a decent amount of people that use Debian (or
> downstream distros) for scientific number crunching. Compiling
> optimized for large workloads will always be a thing no matter the
> baseline, but when getting started distro packages are just one less
> thing to care about.

I humbly believe over 1/3 of packages I (co-)maintained for Debian are
for number crunching. And I INSIST in my NEGATIVE opinion after
trying to do some experiments over the years. The number of people
who really care about the ISA baseline for Debian distributed package
is very likely less than you expected.

I appreciate people who speak for ISA baseline, and appreciate any
actual effort in this regard. But the lack of care eventually
changed my mind and make me hold a negative opinion.

If you think I was simply unsuccessful in promoting any solution for
the topics in this discussion, please go ahead and I will support
you in a visible way.

> On Sat, Mar 26, 2022 at 7:25 AM Andrey Rahmatullin  wrote:
> > 
> > A partial arch (whatever that is, yeah) with the x86-64-v3 baseline, and
> > optionally raise the main amd64 baseline to x86-64-v2?
> 
> +1

So again, that's possibly something like a partial debian archive with
a dpkg fork I mentioned.
That's probably the same idea as the ancient SIMDebian proposal.
See the example patch for dpkg:
https://github.com/SIMDebian/dpkg/commit/13b062567ac58dd1fe5395fb003d6230fd99e7c1
So that a partial archive with selected source packages can be
rebuilt automatically in bumped ISA baseline.

To be clear, the fact that tensorflow does not support runtime detection
while the baseline code sucks in performance is the direct reason
why I proposed SIMDebian. The project is abandoned, and patch is
only for reference.

Bug#1008472: ITP: jh7100-second-boot -- StarFive JH7100 boot loader

[Domenico Andreoli]

Package: wnpp
Severity: wishlist
Owner: Domenico Andreoli 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: jh7100-second-boot
  Version : 2021-11-02
  Upstream Author : StarFive Tech.
* URL : https://github.com/starfive-tech/JH7100_secondBoot
* License : GPL
  Programming Lang: C
  Description : StarFive JH7100 boot loader

This is the very first piece of software loaded by the CPU during boot.

Its purpose is to prepare the HW for the execution of OpenSBI and all
the subsequent stages of the boot, like U-Boot and Linux.

Depending on your board, it might be needed to build a bootable media
or just be programmed in an internal flash.

Bug#1008473: ITP: jh7100-ddrinit -- StarFive JH7100 DDR initialization stage

[Domenico Andreoli]

Package: wnpp
Severity: wishlist
Owner: Domenico Andreoli 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: jh7100-ddrinit
  Version : 2021-11-02
  Upstream Author : StarFive Tech.
* URL : https://github.com/starfive-tech/JH7100_ddrinit
* License : GPL
  Programming Lang: C
  Description : StarFive JH7100 DDR initialization stage

This firmware is loaded early by the boot loader, its purpose is to
configure the DDR controller before any later stage is executed.

Depending on your board, it might be needed to build a bootable media
or just be programmed in an internal flash.

Bug#1008475: ITP: jh71xx-tools -- StarFive JH71xx bootloader recovery and updater tool

[Domenico Andreoli]

Package: wnpp
Severity: wishlist
Owner: Domenico Andreoli 
X-Debbugs-Cc: debian-devel@lists.debian.org

* Package name: jh71xx-tools
  Version : 2021-07-16
  Upstream Author : Kali Prasad 
* URL : https://github.com/kprasadvnsi/JH71xx-tools
* License : MIT
  Programming Lang: C
  Description : StarFive JH71xx bootloader recovery and updater tool

This utility is used to download the bootloader recovery firmware,
the second stage bootloader and the ddrinit firmware to the CPU via UART.

It uses the xmodem protocol over UART, as expected by the JH71xx when
the boot flash is blank.

Results for Voting secrecy

[devotee]

Greetings,

This message is an automated, unofficial publication of vote results.
 Official results shall follow, sent in by the vote taker, namely
Debian Project Secretary

This email is just a convenience for the impatient.
 I remain, gentle folks,

Your humble servant,
Devotee (on behalf of Debian Project Secretary)

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Starting results calculation at Sun Mar 27 00:00:09 2022

Option 1 "Hide identities of Developers casting a particular vote"
Option 2 "Hide identities of Developers casting a particular vote and allow 
verification"
Option 3 "Reaffirm public voting"
Option 4 "None of the above"

In the following table, tally[row x][col y] represents the votes that
option x received over option y.

  Option
  1 2 3 4 
===   ===   ===   === 
Option 1   72   114   149 
Option 2144 142   185 
Option 3137   107 163 
Option 4 946168   



Looking at row 2, column 1, Hide identities of Developers casting a particular 
vote and allow verification
received 144 votes over Hide identities of Developers casting a particular vote

Looking at row 1, column 2, Hide identities of Developers casting a particular 
vote
received 72 votes over Hide identities of Developers casting a particular vote 
and allow verification.

Option 1 Reached quorum: 149 > 45.8421203698084
Option 2 Reached quorum: 185 > 45.8421203698084
Option 3 Reached quorum: 163 > 45.8421203698084


Dropping Option 1 because of Majority. 
(1.585106382978723404255319148936170212766)  1.585 (149/94) < 3
Option 2 passes Majority.   3.033 (185/61) >= 3
Option 3 passes Majority.   2.397 (163/68) >= 1


  Option 2 defeats Option 3 by ( 142 -  107) =   35 votes.
  Option 2 defeats Option 4 by ( 185 -   61) =  124 votes.
  Option 3 defeats Option 4 by ( 163 -   68) =   95 votes.


The Schwartz Set contains:
 Option 2 "Hide identities of Developers casting a particular vote and 
allow verification"



-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

The winners are:
 Option 2 "Hide identities of Developers casting a particular vote and 
allow verification"

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

-- 
The voters have spoken, the bastards... --unknown
DEbian VOTe EnginE
digraph Results {
  ranksep=0.25;
 "Hide identities of Developers casting a particular vote\n1.59" [ 
style="filled" , color="pink", shape=octagon, fontname="Helvetica", fontsize=10 
 ];
 "None of the above" -> "Hide identities of Developers casting a particular 
vote\n1.59" [ label="-55" ];
 "Hide identities of Developers casting a particular vote and allow 
verification\n3.03" [ style="filled" , color="powderblue", shape=egg, 
fontcolor="NavyBlue", fontname="Helvetica", fontsize=10  ];
 "Hide identities of Developers casting a particular vote and allow 
verification\n3.03" -> "Reaffirm public voting\n2.40" [ label="35" ];
 "Hide identities of Developers casting a particular vote and allow 
verification\n3.03" -> "None of the above" [ label="124" ];
 "Reaffirm public voting\n2.40" [ style="filled" , fontname="Helvetica", 
fontsize=10  ];
 "Reaffirm public voting\n2.40" -> "None of the above" [ label="95" ];
 "None of the above" [ style="filled" , shape=diamond, fontcolor="Red", 
fontname="Helvetica", fontsize=10  ];
}


pgpmTrwb7oX1c.pgp
Description: PGP signature