Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 07:06:57PM -0700, Steve Kargl wrote: > On Sun, Jun 29, 2025 at 06:54:53PM -0700, Steve Kargl wrote: > > > > /usr/local/bin/ld: cannot find -lcaf_shmem: No such file or directory > > collect2: error: ld returned 1 exit status > > compiler exited with status 1 > > > > The freshly built gfortran cannot find the libcaf_shmem.a. > > > > % find . -name \*libcaf\* | wc -l >0 > % pwd > /home/kargl/gcc/obj > > % find . -name \*shmem\* > ./x86_64-unknown-freebsd15.0/libgfortran/caf/.deps/shmem.Plo > ./x86_64-unknown-freebsd15.0/libgfortran/caf/shmem > ./x86_64-unknown-freebsd15.0/libgfortran/caf/.libs/shmem.o > ./x86_64-unknown-freebsd15.0/libgfortran/caf/shmem.o > ./x86_64-unknown-freebsd15.0/libgfortran/caf/shmem.lo > I'm slowly solving the mystery. % gmake -j15 bootstrap | tee sgk.log % more sgk.log ... mv -f $depbase.Tpo $depbase.Plo libtool: compile: /home/kargl/gcc/obj/./gcc/xgcc -B/home/kargl/gcc/obj/./gcc/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/bin/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/lib/ -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/include -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/sys-include -fchecking=1 -DHAVE_CONFIG_H -I. -I../../../gcc/libgfortran -iquote../../../gcc/libgfortran/io -I../../../gcc/libgfortran/../gcc -I../../../gcc/libgfortran/../gcc/config -I../../../gcc/libgfortran/../libquadmath -I../.././gcc -I../../../gcc/libgfortran/../libgcc -I../libgcc -I../../../gcc/libgfortran/../libbacktrace -I../libbacktrace -I../libbacktrace -std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wextra -Wwrite-strings -Werror=implicit-function-declaration -Werror=vla -fcx-fortran-rules -ffunction-sections -fdata-sections -O -g -MT caf/shmem/allocator.lo -MD -MP -MF caf/shmem/.deps/allocator.Tpo -c ../../../gcc/libgfortran/caf/shmem/allocator.c -fPIC -DPIC -o caf/shmem/.libs/allocator.o libtool: compile: /home/kargl/gcc/obj/./gcc/xgcc -B/home/kargl/gcc/obj/./gcc/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/bin/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/lib/ -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/include -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/sys-include -fchecking=1 -DHAVE_CONFIG_H -I. -I../../../gcc/libgfortran -iquote../../../gcc/libgfortran/io -I../../../gcc/libgfortran/../gcc -I../../../gcc/libgfortran/../gcc/config -I../../../gcc/libgfortran/../libquadmath -I../.././gcc -I../../../gcc/libgfortran/../libgcc -I../libgcc -I../../../gcc/libgfortran/../libbacktrace -I../libbacktrace -I../libbacktrace -std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wextra -Wwrite-strings -Werror=implicit-function-declaration -Werror=vla -fcx-fortran-rules -ffunction-sections -fdata-sections -O -g -MT caf/shmem/collective_subroutine.lo -MD -MP -MF caf/shmem/.deps/collective_subroutine.Tpo -c ../../../gcc/libgfortran/caf/shmem/collective_subroutine.c -fPIC -DPIC -o caf/shmem/.libs/collective_subroutine.o libtool: compile: /home/kargl/gcc/obj/./gcc/xgcc -B/home/kargl/gcc/obj/./gcc/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/bin/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/lib/ -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/include -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/sys-include -fchecking=1 -DHAVE_CONFIG_H -I. -I../../../gcc/libgfortran -iquote../../../gcc/libgfortran/io -I../../../gcc/libgfortran/../gcc -I../../../gcc/libgfortran/../gcc/config -I../../../gcc/libgfortran/../libquadmath -I../.././gcc -I../../../gcc/libgfortran/../libgcc -I../libgcc -I../../../gcc/libgfortran/../libbacktrace -I../libbacktrace -I../libbacktrace -std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wextra -Wwrite-strings -Werror=implicit-function-declaration -Werror=vla -fcx-fortran-rules -ffunction-sections -fdata-sections -O -g -MT caf/shmem/counter_barrier.lo -MD -MP -MF caf/shmem/.deps/counter_barrier.Tpo -c ../../../gcc/libgfortran/caf/shmem/counter_barrier.c -fPIC -DPIC -o caf/shmem/.libs/counter_barrier.o libtool: compile: /home/kargl/gcc/obj/./gcc/xgcc -B/home/kargl/gcc/obj/./gcc/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/bin/ -B/home/kargl/work/x/x86_64-unknown-freebsd15.0/lib/ -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/include -isystem /home/kargl/work/x/x86_64-unknown-freebsd15.0/sys-include -fchecking=1 -DHAVE_CONFIG_H -I. -I../../../gcc/libgfortran -iquote../../../gcc/libgfortran/io -I../../../gcc/libgfortran/../gcc -I../../../gcc/libgfortran/../gcc/config -I../../../gcc/libgfortran/../libquadmath -I../.././gcc -I../../../gcc/libgfortran/../libgcc -I../libgcc -I../../../gcc/libgfortran/../libbacktrace -I../libbacktrace -I../libbacktrace -std=gnu11 -Wall -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wextra -Wwrite-strings -Werror=implicit-function-declaration -Werror=vla -fcx-fortran-
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 03:30:21PM -0700, Steve Kargl wrote: > On Sun, Jun 29, 2025 at 11:07:31AM -0700, Steve Kargl wrote: > > On Sun, Jun 29, 2025 at 10:35:39AM -0700, Steve Kargl wrote: > > > > > > === gfortran Summary === > > > > > > # of expected passes73149 > > > # of unexpected failures522 > > It seems that something is mucking with library paths. Many of the > unsigned testcases are failing, and the issue is that gfortran is > trying to link in libgfortran.so.5 from GCC 14 instead of the > freshly built libfotran. > > > Setting LD_LIBRARY_PATH to > .:/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./li > batomic/.libs:/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libquadmath/.libs > :/home/kargl/gcc/obj/gcc/testsuite/gfortran/../..:.:/home/kargl/gcc/obj/x86_64-u > nknown-freebsd15.0/./libatomic/.libs:/home/kargl/gcc/obj/x86_64-unknown-freebsd1 > 5.0/./libquadmath/.libs:/home/kargl/gcc/obj/gcc/testsuite/gfortran/../.. > Execution timeout is: 300 > spawn [open ...] > ld-elf.so.1: /usr/local/lib/gcc14/libgfortran.so.5: version GFORTRAN_15 > required > by /home/kargl/gcc/obj/gcc/testsuite/gfortran/unsigned_10.exe not found > FAIL: gfortran.dg/unsigned_10.f90 -O0 execution test > Deleting fort.10 > Rebuilt everything with clang as the bootstrap compiler instead of gcc14. Doesn't change matters. Something is breaking the library paths. Much more telling is ld-elf.so.1: /usr/local/lib/gcc14/libgfortran.so.5: version GFORTRAN_15 required by /home/kargl/gcc/obj/gcc/testsuite/gfortran6/alloc_comp_1.exe not found FAIL: gfortran.dg/coarray/alloc_comp_1.f90 -fcoarray=lib -O2 -lcaf_single execution test Executing on host: /home/kargl/gcc/obj/gcc/testsuite/gfortran6/../../gfortran -B/home/kargl/gcc/obj/gcc/testsuite/gfortran6/../../ -B/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libgfortran/ /home/kargl/gcc/gcc/gcc/testsuite/gfortran.dg/coarray/alloc_comp_1.f90 -fdiagnostics-plain-output -fdiagnostics-plain-output -fcoarray=lib -O2 -lcaf_shmem -L/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libatomic/.libs -L/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libquadmath/.libs -lm -o ./alloc_comp_1.exe(timeout = 300) spawn -ignore SIGHUP /home/kargl/gcc/obj/gcc/testsuite/gfortran6/../../gfortran -B/home/kargl/gcc/obj/gcc/testsuite/gfortran6/../../ -B/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libgfortran/ /home/kargl/gcc/gcc/gcc/testsuite/gfortran.dg/coarray/alloc_comp_1.f90 -fdiagnostics-plain-output -fdiagnostics-plain-output -fcoarray=lib -O2 -lcaf_shmem -L/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libatomic/.libs -L/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libquadmath/.libs -lm -o ./alloc_comp_1.exe /usr/local/bin/ld: cannot find -lcaf_shmem: No such file or directory collect2: error: ld returned 1 exit status compiler exited with status 1 The freshly built gfortran cannot find the libcaf_shmem.a. Which reminds, why is there an extra library? Cannot all of the functions be added to libgfortran?
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 06:54:53PM -0700, Steve Kargl wrote: > On Sun, Jun 29, 2025 at 03:30:21PM -0700, Steve Kargl wrote: > > On Sun, Jun 29, 2025 at 11:07:31AM -0700, Steve Kargl wrote: > > > On Sun, Jun 29, 2025 at 10:35:39AM -0700, Steve Kargl wrote: > > > > > > > > === gfortran Summary === > > > > > > > > # of expected passes73149 > > > > # of unexpected failures522 > > After forcefully adding '#include ', '#include ', and 'extern char **environ' where needed. I now see === gfortran Summary === # of expected passes73561 # of unexpected failures110 # of expected failures 343 # of unresolved testcases 78 # of unsupported tests 94 for 'gmake check-fortran', and if I use RUNTESTFLAGS='dg.exp=\*unsign\*' to test the unsigned issue reported earlier, I see === gfortran Summary === # of expected passes434 # of unsupported tests 6 /home/kargl/gcc/obj/gcc/gfortran version 16.0.0 20250629 (experimental) (GCC) I suspect we'll need to have #include "config.h" #indef HAVE_UNISTD_H #include #endif and similar for other headers files. I further suspect that this are going to be rough on CYWIN/MINGW. -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 06:54:53PM -0700, Steve Kargl wrote: > > /usr/local/bin/ld: cannot find -lcaf_shmem: No such file or directory > collect2: error: ld returned 1 exit status > compiler exited with status 1 > > The freshly built gfortran cannot find the libcaf_shmem.a. > % find . -name \*libcaf\* | wc -l 0 % pwd /home/kargl/gcc/obj % find . -name \*shmem\* ./x86_64-unknown-freebsd15.0/libgfortran/caf/.deps/shmem.Plo ./x86_64-unknown-freebsd15.0/libgfortran/caf/shmem ./x86_64-unknown-freebsd15.0/libgfortran/caf/.libs/shmem.o ./x86_64-unknown-freebsd15.0/libgfortran/caf/shmem.o ./x86_64-unknown-freebsd15.0/libgfortran/caf/shmem.lo -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
Yet, another head scratcher. ../../../gcc/libgfortran/caf/shmem/supervisor.c:235:27: error: 'environ' undeclared (first use in this function) 235 | for (char **e = environ; *e; ++e, ++n) | ^~~ I see 'extern char **environ;' in shmem.c. Is this suppose to be in a header or declared as extern in supervisor.c? -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
Hi Jerry, real 0m7.426s user 0m47.602s sys 0m42.707s Do you know if these numbers are for a shared-memory MPI implementation or a traditional one? Best regards Thomas
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Thu, Jun 26, 2025 at 10:15:01AM +0200, Andre Vehreschild wrote: > > I deem this library fit for educational and research use, > where small to medium sized problems are researched. I do > not expect it to support a long term running application, > because is does not join adjacent blocks in the shared memory > upon free. I.e. the shared memory will get fragmented and at > some time no shared memory can be allocated anymore. > Hi Andre, In reviewing the email thread to see if I missed a step in rebuilding gfortran, the above comment caught my eye. I'm a bit leary here with regard to the general user experience. After 25+ years of contributing to gfortran, I've come to recognize one truth: Gfortran users will use it in unexpected and demanding ways. This can lead to a negative user experience, which then get reported in blog posts, stackoverflow, fortran-lang discourse, etc, (see for example, parameterized derived types and finalization). With regard to memory fragmentation, how severe to you think this issue may be? I doubt that we can tell users to only use -fcoarray=shmem for small/medium codes that only run for a short time. Again, we'll hit the negative user experience. -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Tue, Jun 24, 2025 at 03:09:33PM +0200, Andre Vehreschild wrote: > > this series of patches (six in total) adds a new coarray backend library to > libgfortran. The library uses shared memory and processes to implement > running multiple images on the same node. The work is based on work started > by > Thomas and Nicolas Koenig. No changes to the gfortran compile part are > required > for this. > Just applied the 6 patch (with the update part 5). I'm seeing a significant increase in the number of failures in 'make check-fortran' testing. === gfortran Summary === # of expected passes73149 # of unexpected failures522 # of expected failures 343 # of unresolved testcases 78 # of unsupported tests 94 /home/kargl/gcc/obj/gcc/gfortran version 16.0.0 20250628 (experimental) (GCC) Without your patch, the number of unexpected failes is ~25. All of these failures are due to poorly written ASAN tests. -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On 6/29/25 7:52 AM, Thomas Koenig wrote: Hi Jerry, real 0m7.426s user 0m47.602s sys 0m42.707s Do you know if these numbers are for a shared-memory MPI implementation or a traditional one? Best regards Thomas I believe it is traditional. I was using OpenCoarrays with mpich. Jerry
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 10:35:39AM -0700, Steve Kargl wrote: > > Just applied the 6 patch (with the update part 5). I'm > seeing a significant increase in the number of failures > in 'make check-fortran' testing. > Just re-applied your patches in the top-level gcc/ directory. I was expecting most changes to appear in gcc/gcc/fortran and gcc/libgfortran. These files are dumped into gcc/ alloc.c alloc.h allocator.c allocator.h collective_subroutine.c collective_subroutine.h counter_barrier.c counter_barrier.h hashmap.c hashmap.h shared_memory.c shared_memory.h supervisor.c supervisor.h sync.c sync.h teams_mgmt.c teams_mgmt.h thread_support.c thread_support.h Am I doing something wrong? For the record, I do cd gcc/ patch < ../pr88076_v1_1.patch patch < ../pr88076_v1_2.patch patch < ../pr88076_v1_3.patch patch < ../pr88076_v1_4.patch patch < ../pr88076_v1_5.patch patch < ../pr88076_v1_6.patch -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
Hi Steve, I would do git am pr88076_v1_1.patch or patch -p1 each of the patch files. Then they go where they ought to be. Pro tip: create a new branch before doing the git am, then you can just switch back to master when desired. - Andre Andre Vehreschild * ve...@gmx.de Am 29. Juni 2025 20:29:21 schrieb Steve Kargl : On Sun, Jun 29, 2025 at 10:35:39AM -0700, Steve Kargl wrote: Just applied the 6 patch (with the update part 5). I'm seeing a significant increase in the number of failures in 'make check-fortran' testing. Just re-applied your patches in the top-level gcc/ directory. I was expecting most changes to appear in gcc/gcc/fortran and gcc/libgfortran. These files are dumped into gcc/ alloc.c alloc.h allocator.c allocator.h collective_subroutine.c collective_subroutine.h counter_barrier.c counter_barrier.h hashmap.c hashmap.h shared_memory.c shared_memory.h supervisor.c supervisor.h sync.c sync.h teams_mgmt.c teams_mgmt.h thread_support.c thread_support.h Am I doing something wrong? For the record, I do cd gcc/ patch < ../pr88076_v1_1.patch patch < ../pr88076_v1_2.patch patch < ../pr88076_v1_3.patch patch < ../pr88076_v1_4.patch patch < ../pr88076_v1_5.patch patch < ../pr88076_v1_6.patch -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 08:36:38PM +0200, Andre Vehreschild wrote: > > I would do git am pr88076_v1_1.patch or patch -p1 each of the patch files. Then they go where they ought to be. Thanks. I needed -p1 for v1_5 and v1_6. > Pro tip: create a new branch before doing the git am, then you can just > switch back to master when desired. No where near a pro with git and not paid to learn yet-another-RCS. My usage is fairly limited rm -rf gcc/ git clone URL_to_gcc gcc In gcc/ git reset --hard git diff > ../prX.diff and with 'git diff' if I add a new file, there is 30 or so minutes of google to learning why the new file does not appear in the diff and how to get it. -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 10:35:39AM -0700, Steve Kargl wrote: > > === gfortran Summary === > > # of expected passes73149 > # of unexpected failures522 > # of expected failures 343 > # of unresolved testcases 78 > # of unsupported tests 94 > /home/kargl/gcc/obj/gcc/gfortran version 16.0.0 20250628 (experimental) > (GCC) > > > Without your patch, the number of unexpected failes is ~25. All > of these failures are due to poorly written ASAN tests. > After a 'git reset --hard', I see === gfortran Summary === # of expected passes73487 # of unexpected failures31 # of expected failures 343 # of unsupported tests 92 -- Steve
Re: [Patch, Fortran, Coarray, PR88076, v1] 0/6 Add a shared memory multi process coarray library.
On Sun, Jun 29, 2025 at 11:07:31AM -0700, Steve Kargl wrote: > On Sun, Jun 29, 2025 at 10:35:39AM -0700, Steve Kargl wrote: > > > > === gfortran Summary === > > > > # of expected passes73149 > > # of unexpected failures522 It seems that something is mucking with library paths. Many of the unsigned testcases are failing, and the issue is that gfortran is trying to link in libgfortran.so.5 from GCC 14 instead of the freshly built libfotran. Setting LD_LIBRARY_PATH to .:/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./li batomic/.libs:/home/kargl/gcc/obj/x86_64-unknown-freebsd15.0/./libquadmath/.libs :/home/kargl/gcc/obj/gcc/testsuite/gfortran/../..:.:/home/kargl/gcc/obj/x86_64-u nknown-freebsd15.0/./libatomic/.libs:/home/kargl/gcc/obj/x86_64-unknown-freebsd1 5.0/./libquadmath/.libs:/home/kargl/gcc/obj/gcc/testsuite/gfortran/../.. Execution timeout is: 300 spawn [open ...] ld-elf.so.1: /usr/local/lib/gcc14/libgfortran.so.5: version GFORTRAN_15 required by /home/kargl/gcc/obj/gcc/testsuite/gfortran/unsigned_10.exe not found FAIL: gfortran.dg/unsigned_10.f90 -O0 execution test Deleting fort.10 -- steve -- Steve
