On Mon, Oct 17, 2011 at 15:49, Tobias Burnus <bur...@net-b.de> wrote: > This patch adds a call to _commit() on _WIN32 for the FLUSH subroutine and > the FLUSH statement. It removes the _commit from gfortran's buf_flush.
Like I argued in this message http://gcc.gnu.org/ml/fortran/2011-10/msg00094.html , I think this is a gross mistake. libgfortran should not require _commit nor fsync in any situation. Those calls are useful for writing databases and other applications which must make data integrity guarantees, and are prepared to pay the performance cost associated with it. It's absolutely not something a language support library should do unless the language spec explicitly requires such data integrity guarantees. > Background: > * gfortran internally buffers the I/O, but it calls the nonbuffering > open/write/read functions (and not, e.g., fopen/fwrite/fread). On > Unix/Darwin/Linux system, the changes become immediately visible to all > processes after a "write()". > * On Windows, there seems to be a file-descriptor specific system buffer > such that the data only becomes available to other processes after either a > "_commit" or after closing the file. > * The Windows _commit() is a combination of a (system) buffer flush - as a > "write()" already implies on POSIX systems, but it also ensures that the > data ends up on the disk, which on POSIX systems is done via fsync(). I admit I'm somewhat confused by this issue, and despite extensive googling I haven't been able to come up with a clear explanation for the behavior seen in PR 44698. That write() would be buffered on windows makes no sense to me, and I have found no documentation supporting this view. The closest what I found was in the documentation for WriteFile and WriteFileEX (which are the win32 native calls, write() is a wrapper around one of these (the EX version if available, presumably)): http://msdn.microsoft.com/en-us/library/windows/desktop/aa365748%28v=vs.85%29.aspx "When writing to a file, the last write time is not fully updated until all handles used for writing have been closed. Therefore, to ensure an accurate last write time, close the file handle immediately after writing to the file." This suggests that metadata updates are not immediately visible to other processes. Or at least it makes sense that it would update the size at the same time it updates the last write time. Which would explain why stat("/path/to/file") would show an old size, as the size hasn't necessarily yet been updated to the directory, although the file data itself is already transferred to the OS. And, while I haven't checked it, it would make sense that fstat(fd,...) would return the up to date info, as that would just need to check the data via the process local file descriptor table rather than looking up the metadata via the directory. That is, I guess that the implementation is something along the lines of these calls not being buffered (thus no data lost if the process crashes after WriteFile(EX)), but the kernel maintains a per-handle metadata cache which is flushed to the filesystem during batched metadata updates (and close(), _commit(), or process ending), at which point it also becomes visible to other processes. Or something like that. That being said, this is just me speculating. If someone knows better, please feel free to share. And, while I'm at it, this kind of "relaxed consistency" is not unheard of in the unix world either. Consider NFS, where data and metadata may not be flushed to the server until fsync() or close() is called, or the attribute cache timeout forces the writeout(?), and thus it's possible for clients to have an inconsistent view of a file. In both cases the remedy is the same; if this kind of consistency matters, the user should close the file or fsync()/_commit() before expecting that the OS metadata is consistent. I think that's a better option than sprinkling _commit() all over the library. So I would rather prefer my own patch from the URL above. Also, I think it would be nice if we could get this fix into 4.6.2.. -- Janne Blomqvist
