Re: Updated Sourceware infrastructure plans

[FX Coudert via Gcc]

> I regenerate auto* files from time to time for libgfortran. Regenerating
> them has always been very fragile (using --enable-maintainer-mode),
> and difficult to get right.

I have never found them difficult to regenerate, but if you have only a non 
maintainer build, it is a pain to have to make a new maintainer build for a 
minor change.

Moreover, our m4 code is particularly painful to use and unreadable. I have 
been wondering for some time: should we switch to simpler Python scripts? It 
would also mean that we would have fewer files in the generated/ folder: right 
now, every time we add new combinations of types, we have a combinatorial 
explosion of files.

$ ls generated/sum_*
generated/sum_c10.c generated/sum_c17.c generated/sum_c8.c  generated/sum_i16.c 
generated/sum_i4.c  generated/sum_r10.c generated/sum_r17.c generated/sum_r8.c
generated/sum_c16.c generated/sum_c4.c  generated/sum_i1.c  generated/sum_i2.c  
generated/sum_i8.c  generated/sum_r16.c generated/sum_r4.c

We could imagine having a single file for all sum intrinsics.

How do Fortran maintainers feel about that?

FX

Re: Updated Sourceware infrastructure plans

[Christophe Lyon via Gcc]

Hi,

On Thu, 18 Apr 2024 at 10:15, FX Coudert  wrote:
>
> > I regenerate auto* files from time to time for libgfortran. Regenerating
> > them has always been very fragile (using --enable-maintainer-mode),
> > and difficult to get right.
>
> I have never found them difficult to regenerate, but if you have only a non 
> maintainer build, it is a pain to have to make a new maintainer build for a 
> minor change.
>

FWIW, we have noticed lots of warnings from autoreconf in libgfortran.
I didn't try to investigate, since the regenerated files are identical
to what is currently in the repo.

For instance, you can download the "stdio" output from the
autoregen.py step in
https://builder.sourceware.org/buildbot/#/builders/269/builds/4373

Thanks,

Christophe


> Moreover, our m4 code is particularly painful to use and unreadable. I have 
> been wondering for some time: should we switch to simpler Python scripts? It 
> would also mean that we would have fewer files in the generated/ folder: 
> right now, every time we add new combinations of types, we have a 
> combinatorial explosion of files.
>
> $ ls generated/sum_*
> generated/sum_c10.c generated/sum_c17.c generated/sum_c8.c  
> generated/sum_i16.c generated/sum_i4.c  generated/sum_r10.c 
> generated/sum_r17.c generated/sum_r8.c
> generated/sum_c16.c generated/sum_c4.c  generated/sum_i1.c  
> generated/sum_i2.c  generated/sum_i8.c  generated/sum_r16.c generated/sum_r4.c
>
> We could imagine having a single file for all sum intrinsics.
>
> How do Fortran maintainers feel about that?
>
> FX

Re: Updated Sourceware infrastructure plans

[Janne Blomqvist via Gcc]

On Thu, Apr 18, 2024 at 11:15 AM FX Coudert  wrote:
>
> > I regenerate auto* files from time to time for libgfortran. Regenerating
> > them has always been very fragile (using --enable-maintainer-mode),
> > and difficult to get right.
>
> I have never found them difficult to regenerate, but if you have only a non 
> maintainer build, it is a pain to have to make a new maintainer build for a 
> minor change.
>
> Moreover, our m4 code is particularly painful to use and unreadable. I have 
> been wondering for some time: should we switch to simpler Python scripts? It 
> would also mean that we would have fewer files in the generated/ folder: 
> right now, every time we add new combinations of types, we have a 
> combinatorial explosion of files.
>
> $ ls generated/sum_*
> generated/sum_c10.c generated/sum_c17.c generated/sum_c8.c  
> generated/sum_i16.c generated/sum_i4.c  generated/sum_r10.c 
> generated/sum_r17.c generated/sum_r8.c
> generated/sum_c16.c generated/sum_c4.c  generated/sum_i1.c  
> generated/sum_i2.c  generated/sum_i8.c  generated/sum_r16.c generated/sum_r4.c
>
> We could imagine having a single file for all sum intrinsics.
>
> How do Fortran maintainers feel about that?

For the time being I'm not an active maintainer, so my opinion doesn't
per se have weight, but back when I was active I did think about this
issue. IMHO the best of my ideas was to convert these into C++
templates. What we're essentially doing with the M4 stuff and the
proposed in-house Python reimplementation is to make up for lack of
monomorphization in plain old C. Rather than doing some DIY templates,
switch the implementation language to something which has that feature
built-in, in this case C++.  No need to convert the entire libgfortran
to C++ if you don't want to, just those objects that are generated
from the M4 templates. Something like

template
void matmul(T* a, T* b, T* c, ...)
{
   // actual matmul code here
}

extern "C" {
  // Instantiate template for every type and export the symbol
  void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
  {
matmul(a, b, c, ...);
  }
  // And so on for other types
}


-- 
Janne Blomqvist

Generated files in libgfortran for Fortran intrinsic procedures (was: Updated Sourceware infrastructure plans)

[Tobias Burnus]

Hi Janne,

Janne Blomqvist wrote:

back when I was active I did think about this
issue. IMHO the best of my ideas was to convert these into C++
templates.


I think this will work – but we have to be super careful:

With C++, there is the problem that we definitely do not want to add 
dependency on libstdc++ nor to use some features which require special 
hardware support (like exceptions [always bad], symbol aliases, ...). — 
On some systems, a full C++ support might be not available, like 
embedded systems (including some odd embedded OS) or offloading devices.


The libstdc++ dependency would be detected by linking as we currently 
do. For in-language features, we have to ensure the appropriate flags 
-fno-exceptions (and probably a few more). And it should be clear what 
language features to use.


If we do, I think that would surely be an option.


What we're essentially doing with the M4 stuff and the
proposed in-house Python reimplementation is to make up for lack of
monomorphization in plain old C. Rather than doing some DIY templates,
switch the implementation language to something which has that feature
built-in, in this case C++.  No need to convert the entire libgfortran
to C++ if you don't want to, just those objects that are generated
from the M4 templates. Something like

template
void matmul(T* a, T* b, T* c, ...)
{
// actual matmul code here
}

extern "C" {
   // Instantiate template for every type and export the symbol
   void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
   {
 matmul(a, b, c, ...);
   }
   // And so on for other types
}


Cheers,

Tobias

Re: Generated files in libgfortran for Fortran intrinsic procedures (was: Updated Sourceware infrastructure plans)

[Martin Uecker via Gcc]

Am Donnerstag, dem 18.04.2024 um 14:01 +0200 schrieb Tobias Burnus:
> Hi Janne,
> 
> Janne Blomqvist wrote:
> > back when I was active I did think about this
> > issue. IMHO the best of my ideas was to convert these into C++
> > templates.

I haven't looked at libgfortran but I didn't find it problematic
at all to use C in similar numerical code and this helps
with portability. 

Either I use macros, which I keep short and then do not find
inferior to templates (having used C++ for years previously) or 
- if there is really a lot of code that needs to be specialized 
for a type - simply by using includes:

#define matmul_type double
#include "matmul_impl.c"

Martin


> 
> I think this will work – but we have to be super careful:
> 
> With C++, there is the problem that we definitely do not want to add 
> dependency on libstdc++ nor to use some features which require special 
> hardware support (like exceptions [always bad], symbol aliases, ...). — 
> On some systems, a full C++ support might be not available, like 
> embedded systems (including some odd embedded OS) or offloading devices.
> 
> The libstdc++ dependency would be detected by linking as we currently 
> do. For in-language features, we have to ensure the appropriate flags 
> -fno-exceptions (and probably a few more). And it should be clear what 
> language features to use.
> 
> If we do, I think that would surely be an option.
> 
> > What we're essentially doing with the M4 stuff and the
> > proposed in-house Python reimplementation is to make up for lack of
> > monomorphization in plain old C. Rather than doing some DIY templates,
> > switch the implementation language to something which has that feature
> > built-in, in this case C++.  No need to convert the entire libgfortran
> > to C++ if you don't want to, just those objects that are generated
> > from the M4 templates. Something like
> > 
> > template
> > void matmul(T* a, T* b, T* c, ...)
> > {
> > // actual matmul code here
> > }
> > 
> > extern "C" {
> >// Instantiate template for every type and export the symbol
> >void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
> >{
> >  matmul(a, b, c, ...);
> >}
> >// And so on for other types
> > }
> 
> Cheers,
> 
> Tobias

Re: Deprecation/removal of nios2 target support

[Marek Vasut via Gcc]

On 4/18/24 7:53 AM, Thomas Huth wrote:

On 18/04/2024 05.27, Sandra Loosemore wrote:
Tomorrow I plan to push patches to mark the nios2 target as obsolete 
in GCC 14.


Background: Intel has EOL'ed the Nios II processor IP and is now 
directing their FPGA customers to a RISC-V platform instead.


https://www.intel.com/content/www/us/en/content-details/781327/intel-is-discontinuing-ip-ordering-codes-listed-in-pdn2312-for-nios-ii-ip.html

The Nios II hardware on loan from Intel that we were using for testing 
at Mentor Graphics/Siemens was returned around the first of the year. 
For some time we had been using QEMU to test the nios2-elf target, but 
we never had a QEMU test harness set up that would boot the Linux 
kernel, and user-mode QEMU on this target is too buggy/unmaintained to 
use for primary testing. So the current situation is that none of the 
listed maintainers for any of the GNU toolchain components have access 
to a fully working test configuration any more, we have all moved on 
to new jobs and different projects, Intel has also moved on to a 
different platform, and our former contacts on Intel's Nios II team 
have moved on as well.  It seems like it's time to pull the plug.


Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and 
remove support from all toolchain components after the release is 
made.  I'm not sure there is an established process for 
obsoleting/removing support in other components; besides binutils, 
GDB, and GLIBC, there's QEMU, newlib/libgloss, and the Linux kernel.  
But, we need to get the ball rolling somewhere.


Thanks for the heads-up, Sandra! FWIW: QEMU already marked the nios2 
target as deprecated, too, and plans to remove it in version 9.1 (in 
autumn this year).


Thank you and sorry for being inactive for so long around nios2.

Re: Deprecation/removal of nios2 target support

[Joseph Myers via Gcc]

On Wed, 17 Apr 2024, Sandra Loosemore wrote:

> Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and remove
> support from all toolchain components after the release is made.  I'm not sure
> there is an established process for obsoleting/removing support in other
> components; besides binutils, GDB, and GLIBC, there's QEMU, newlib/libgloss,
> and the Linux kernel.  But, we need to get the ball rolling somewhere.

CC:ing Arnd Bergmann regarding the obsolescence in the Linux kernel.

-- 
Joseph S. Myers
josmy...@redhat.com

Re: Updated Sourceware infrastructure plans

[Joseph Myers via Gcc]

On Thu, 18 Apr 2024, Mark Wielaard wrote:

> But we like to get more feedback on what people really think a
> "pull-request" style framework should look like. We used to have a
> gerrit setup which wasn't really popular. And we already have a
> sourcehut mirror that can be used to turn your "pull-requests" into a
> git send-email style submission (without having to setup any
> email/smtp yourself): https://sr.ht/~sourceware/

The xz backdoor showed up one issue with some implementations of 
pull-request systems: GitHub removed access to the repository, and with it 
access to the past pull requests, so disrupting investigation into the 
sequence of bad-faith contributions.  I suggest that a basic principle for 
such a system is that it should be *easy* to obtain and maintain a local 
copy of the history of all pull requests.  That includes all versions of a 
pull request, if it gets rebased, and all versions of comments, if the 
system allows editing comments.  A system that uses git as the source of 
truth for all the pull request data and has refs through which all this 
can be located (with reasonably straightforward, documented formats for 
the data, not too closely tied to any particular implementation of a 
pull-request system), so that a single clone --mirror has all the data, 
might be suitable (people have worked on ensuring git scales well with 
very large numbers of refs, which you'd probably get in such a system 
storing all the data in git); a system that requires use of rate-limited 
APIs to access pull request data, not designed for maintaining such a 
local copy, rather less so.

There are some other considerations as well, such as ensuring the proposed 
commit message is just as much subject to review as the proposed code 
changes, and allowing both pull requests that propose a single commit 
(with subsequent fixups in the PR branch intended to be squashed) and pull 
requests that propose a series of commits (where fixups found in the 
review process need to be integrated into the relevant individual commit 
and the branch rebased before merge).

-- 
Joseph S. Myers
josmy...@redhat.com

Re: Deprecation/removal of nios2 target support

[Joel Sherrill]

On Thu, Apr 18, 2024 at 10:46 AM Joseph Myers  wrote:

> On Wed, 17 Apr 2024, Sandra Loosemore wrote:
>
> > Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and remove
> > support from all toolchain components after the release is made.  I'm
> not sure
> > there is an established process for obsoleting/removing support in other
> > components; besides binutils, GDB, and GLIBC, there's QEMU,
> newlib/libgloss,
> > and the Linux kernel.  But, we need to get the ball rolling somewhere.
>
> CC:ing Arnd Bergmann regarding the obsolescence in the Linux kernel.
>

Just an FYI that the RTEMS Project plans to drop NIOS II support based
on what happens with the tooling.

--joel

>
> --
> Joseph S. Myers
> josmy...@redhat.com
>
>

Re: Deprecation/removal of nios2 target support

[Jeff Law via Gcc]

On 4/18/24 9:57 AM, Joel Sherrill wrote:



On Thu, Apr 18, 2024 at 10:46 AM Joseph Myers > wrote:


On Wed, 17 Apr 2024, Sandra Loosemore wrote:

 > Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and
remove
 > support from all toolchain components after the release is made. 
I'm not sure

 > there is an established process for obsoleting/removing support
in other
 > components; besides binutils, GDB, and GLIBC, there's QEMU,
newlib/libgloss,
 > and the Linux kernel.  But, we need to get the ball rolling
somewhere.

CC:ing Arnd Bergmann regarding the obsolescence in the Linux kernel.


Just an FYI that the RTEMS Project plans to drop NIOS II support based
on what happens with the tooling.
ACK.  Just one more note to the wider audience.  I looked at QEMU's user 
mode support for nios2 on/off over the last couple years.  It never 
seemed to work well enough be able to run the GCC testsuite reliably.


As a result, my tester builds nios2 and will run the testsuite, but all 
the execution tests are only built, they're not actually run.  It's been 
fairly stable, but its not doing in-depth testing.


jeff

Re: Deprecation/removal of nios2 target support

[Sandra Loosemore]

On 4/18/24 10:06, Jeff Law wrote:


ACK.  Just one more note to the wider audience.  I looked at QEMU's user 
mode support for nios2 on/off over the last couple years.  It never 
seemed to work well enough be able to run the GCC testsuite reliably.


I looked at the problems with the nios2 user-mode support in QEMU in 
some detail a few years ago.  It looked like the problem was that it had 
copied the target syscall data structures from GLIBC and wasn't 
accounting for 32-bit target/64-bit host differences -- this 
particularly affected signal handling.  I'm pretty sure we asked Intel 
if they wanted this fixed and they were not interested in pursuing that. 
 The end result is that user-mode QEMU is not very useful for GLIBC or 
GDB testing.


As a result, my tester builds nios2 and will run the testsuite, but all 
the execution tests are only built, they're not actually run.  It's been 
fairly stable, but its not doing in-depth testing.


Yes, as I noted in my previous message, there is nothing seriously wrong 
with the nios2 GCC port at present; it just seems kind of pointless to 
invest time in continuing to maintain it as a hobby when the 
architecture is dead.  I think legacy customers generally would prefer 
to keep using the toolchains previously distributed by Altera/Intel or 
Mentor/Siemens instead of trying to build a new bleeding-edge toolchain 
from scratch, too.


-Sandra

Re: Updated Sourceware infrastructure plans

[Frank Ch. Eigler via Gcc]

Hi -

> [...]  I suggest that a basic principle for such a system is that it
> should be *easy* to obtain and maintain a local copy of the history
> of all pull requests.  That includes all versions of a pull request,
> if it gets rebased, and all versions of comments, if the system
> allows editing comments.  A system that uses git as the source of
> truth for all the pull request data and has refs [...]

Do you know of a system with these characteristics?

- FChE

Re: Updated Sourceware infrastructure plans

[Joseph Myers via Gcc]

On Thu, 18 Apr 2024, Frank Ch. Eigler via Gcc wrote:

> Hi -
> 
> > [...]  I suggest that a basic principle for such a system is that it
> > should be *easy* to obtain and maintain a local copy of the history
> > of all pull requests.  That includes all versions of a pull request,
> > if it gets rebased, and all versions of comments, if the system
> > allows editing comments.  A system that uses git as the source of
> > truth for all the pull request data and has refs [...]
> 
> Do you know of a system with these characteristics?

I haven't studied the features supported by different systems, though (for 
example) there are some discussions in comments on 
https://lwn.net/Articles/867956/ of various systems.

-- 
Joseph S. Myers
josmy...@redhat.com

Re: Updated Sourceware infrastructure plans

[Matt Rice via Gcc]

On Thu, Apr 18, 2024 at 5:38 PM Frank Ch. Eigler  wrote:
>
> Hi -
>
> > [...]  I suggest that a basic principle for such a system is that it
> > should be *easy* to obtain and maintain a local copy of the history
> > of all pull requests.  That includes all versions of a pull request,
> > if it gets rebased, and all versions of comments, if the system
> > allows editing comments.  A system that uses git as the source of
> > truth for all the pull request data and has refs [...]
>
> Do you know of a system with these characteristics?
>
> - FChE

The closest thing I know of which may have these characteristics is
alibaba's AGit-Flow described here:
https://git-repo.info/en/2020/03/agit-flow-and-git-repo/
It actually sends pull-requests through the git protocol using a
custom proc-receive hook.

I'm a bit uncertain how code-review comments are handled in their system,
And it isn't exactly something which can just be used off-the-shelf,
AFAIK their server
side implementation hasn't been released.

I had written a prototype-worthy implementation of the server-side git
hook here.
It basically allows sending a pull-request through git push, along
with a cover letter.
But i've never really used it in the full PR review cycle beyond that.

https://github.com/pullreqr/pullreqr_githook

But protocol-wise IMO it seems like a good basis for building a system
with these characteristics to me.

Re: Deprecation/removal of nios2 target support

[Arnd Bergmann via Gcc]

On Thu, Apr 18, 2024, at 17:44, Joseph Myers wrote:
> On Wed, 17 Apr 2024, Sandra Loosemore wrote:
>
>> Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and remove
>> support from all toolchain components after the release is made.  I'm not 
>> sure
>> there is an established process for obsoleting/removing support in other
>> components; besides binutils, GDB, and GLIBC, there's QEMU, newlib/libgloss,
>> and the Linux kernel.  But, we need to get the ball rolling somewhere.
>
> CC:ing Arnd Bergmann regarding the obsolescence in the Linux kernel.

We have not yet marked nios2 as deprecated in the kernel, but that
is mostly because the implementation does not get in the way too
much and Dinh Nguyen is still around as a maintainer and merging
bugfixes.

Almost all nios2 kernel changes I see in the past decade have been
done blindly without testing on hardware, either for treewide
changes, or by code inspection. The only notable exceptions I could
find are from Andreas Oetken and Bernd Weiberg at Siemens and
from Marek Vasut (all added to Cc in case they have something to add).

We should probably remove nios2 from the kernel in the near future,
but even if we decide not to, I think deprecating it from gcc is the
right idea: If there are a few remaining users that still plan
to update their kernels, gcc-14 will still be able to build new
kernels for several years.

 Arnd

Youl

[Delphine Lasota via Gcc]

Sent from my iPhone

gcc-11-20240418 is now available

[GCC Administrator via Gcc]

Snapshot gcc-11-20240418 is now available on
  https://gcc.gnu.org/pub/gcc/snapshots/11-20240418/
and on various mirrors, see https://gcc.gnu.org/mirrors.html for details.

This snapshot has been generated from the GCC 11 git branch
with the following options: git://gcc.gnu.org/git/gcc.git branch 
releases/gcc-11 revision ab39cab2a984a0f57162f2072cdc0d5f6f79064c

You'll find:

 gcc-11-20240418.tar.xz   Complete GCC

  SHA256=28b7a2b34d10ea4176a4d20f9fa9aa1a9d0beabaa88f85ff8126a39b0603db85
  SHA1=ae5b7651f842cda2186844658433752aeb8a8fa5

Diffs from 11-20240411 are available in the diffs/ subdirectory.

When a particular snapshot is ready for public consumption the LATEST-11
link is updated and a message is sent to the gcc list.  Please do not use
a snapshot before it has been announced that way.

Re: Deprecation/removal of nios2 target support

[Marek Vasut via Gcc]

On 4/18/24 8:41 PM, Arnd Bergmann wrote:

On Thu, Apr 18, 2024, at 17:44, Joseph Myers wrote:

On Wed, 17 Apr 2024, Sandra Loosemore wrote:


Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and remove
support from all toolchain components after the release is made.  I'm not sure
there is an established process for obsoleting/removing support in other
components; besides binutils, GDB, and GLIBC, there's QEMU, newlib/libgloss,
and the Linux kernel.  But, we need to get the ball rolling somewhere.


CC:ing Arnd Bergmann regarding the obsolescence in the Linux kernel.


We have not yet marked nios2 as deprecated in the kernel, but that
is mostly because the implementation does not get in the way too
much and Dinh Nguyen is still around as a maintainer and merging
bugfixes.

Almost all nios2 kernel changes I see in the past decade have been
done blindly without testing on hardware, either for treewide
changes, or by code inspection. The only notable exceptions I could
find are from Andreas Oetken and Bernd Weiberg at Siemens and
from Marek Vasut (all added to Cc in case they have something to add).


I might still have the 10M50 board, but it wasn't powered for a very 
long time.