On 11/16/26 12:18 PM, Janne Blomqvist wrote:
On Mon, Nov 16, 2026 at 5:46 PM Suvarchal Kumar Cheedela
<[email protected]> wrote:

Hello there,

I would like to ask for comments on a change in how libgfortran performs
asynchronous WRITE: copy the output list into a runtime buffer when the
statement executes, so that the program can continue to modify its data
while the I/O thread writes it.

I believe this to be misguided. The Fortran standard is quite clear
that the programmer is not to meddle with variables that are
in-progress for asynchronous I/O. So what a feature like this would
accomplish would be to make some broken programs work correctly (for
some definition of work not defined in the Fortran standard), at the
cost of making standard-conforming programs slower and use more memory
by virtue of the extra buffer copies.

Hi Janne, good to hear from you here. I posted some brief comments in the Bugzilla reports. One thing to peak at is the benchmark runs I did here:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128722

I have not done a full review yet. I have taken note of your VOLATILE comment 
below.


The solution, clearly, is then to fix the broken programs to manage
the extra buffering themselves, if that makes sense for the particular
application.

   - -fno-async-volatile: ASYNCHRONOUS variables are currently compiled as
     VOLATILE, which stops vectorization of loops that use them (about 45%
     slower on a stencil kernel; ifx loses about 50% for the same reason).
     The option compiles them like other variables.  It also needs the
     READ transfer calls of an asynchronous statement to declare their data
     argument as escaping, which is part of the series.

I would be very wary of bugs with an approach like this. The volatile
is needed as the variables can, from the perspective of the compiler,
mysteriously change behind its back.

I have no idea if this is feasible, but I wonder if the volatile
attribute could be applied to the variable only when it's part of an
in-progress async I/O, and then removed when that I/O is waited on?

   - Optionally, CPU placement of the I/O thread
     (GFORTRAN_ASYNC_IO_CPUS) and several threads for large copies.

One idea I was thinking about myself many years ago would be to use
io_uring for async I/O rather than threads. But I never got as far as
implementing it. Food for thought, maybe.


6. Next step: compression on the I/O thread
-------------------------------------------

Once the data is copied, the I/O thread can also compress it without
cost to the program.  On the same runs, zstd level 1 on the I/O thread
cost nothing extra at the median and made the restarts 37% smaller (1.07
to 0.67 GB per restart on 32 nodes); compressing synchronously made it
the slowest restart case.  Writing compressed data in fixed-size chunks
would also allow aligning the writes with the stripes or objects of
parallel file systems.  A prototype (zstd, zlib, lz4 through c-blosc2,
loaded with dlopen) is on the async-compress branch, but it involves a
file format, optional libraries and an interface, so I would send a
separate RFC for it after this one.

Fortran I/O has many gnarly cases that need to be handled wrt to
jumping back and forth in a file, in-place updates of records, and
whatnot. Not sure how this would work in the face of compression where
the compressed size is unknown. This might be another of those cases
where it might be a better idea to implement in the application or as
a library rather than building into the compiler.



I would lean away from using compression in the libgfortran library. The right place for this would be in your own code base, compress the data before you write and decompress after you read.

Best regards,

Jerry



Reply via email to