[issue16212] mmap() dumps core upon resizing the underlying file

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue16212] mmap() dumps core upon resizing the underlying file

Change by Irit Katriel : -- versions: +Python 3.10, Python 3.8, Python 3.9 -Python 3.2, Python 3.3, Python 3.4 ___ Python tracker ___ _

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: > But I think we've talked enough... So do I. Let's go for it. I'll make a patch (which apparently takes some time) and post it here, we'll discuss it then. Thanks for your comments. Now I believe it's a bit harder than I thought initially, but still doable.

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: > SIGBUS as well as SIGFPE or SIGSEGV is a synchronous signal. It is delivered > to the thread that caused the trouble and the stack contents is well defined. Except that signal handlers are per-process, not per thread. So if another thread (which coul

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: Update: The correct link to the POSIX definition of longjmp exiting a signal handler as an ISO C extension is http://pubs.opengroup.org/onlinepubs/9699919799/functions/longjmp.html -- ___ Python tracker

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: > ...it's just impossible to guarantee that Python internal structures are > still consistent. In generic case, I completely agree. Here the things are much simpler (unless I missed something). We know perfectly well the code we need to protect (mmap access

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: > For your specific case, you should... There's nothing I should. As I said above, the bug doesn't trouble me. I just posted it as a generic contribution and don't really care whether it's going to be fixed or not. If decided so, I could help. Otherwise I'll

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: SIGBUS as well as SIGFPE or SIGSEGV is a synchronous signal. It is delivered to the thread that caused the trouble and the stack contents is well defined. > https://www.securecoding.cert.org/confluence/display/seccode/SIG35-C.+Do+not+return+from+SIGSEGV,+SIGI

[issue16212] mmap() dumps core upon resizing the underlying file

STINNER Victor added the comment: > I've found a few examples of handling non-restartable signals > with longjmps, but not that familiar with the codebase to estimate > how reliably it can be done on all supported platforms. > I don't really know how this code would behave, say, on Windows. I pr

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: >> You can't use longjmp from signal handlers. Well, you can, but 99% of the >> code that does it is broken, because you can only call async-safe functions >> from within a signal handler, and certainly can't run the intepreter. > > I don't see the rea

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: > You can't use longjmp from signal handlers. Well, you can, but 99% of the > code that does it is broken, because you can only call async-safe functions > from within a signal handler, and certainly can't run the intepreter. I don't see the reason to run th

[issue16212] mmap() dumps core upon resizing the underlying file

Antoine Pitrou added the comment: A pity Posix isn't smart enough to refuse truncate()ing when there's a mmap open on the affected pages. Python 3's buffer API is superior in that respect :-) -- nosy: +pitrou ___ Python tracker

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: > I know how to avoid the problem in my case, the bug does not really affect > me. I posted it because I thought that the possibility to crash the > interpreter is something to be avoided at all costs. I fully agree with you. However, that's a problem

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: I know how to avoid the problem in my case, the bug does not really affect me. I posted it because I thought that the possibility to crash the interpreter is something to be avoided at all costs. I've found a few examples of handling non-restartable signals

[issue16212] mmap() dumps core upon resizing the underlying file

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: > You can use file locks to be protected against such race condition. > See for example: Those are advisory locks, not mandatory locks. It's possible to do some mandatory locking on some systems, but on every file, and it's not portable. -- _

[issue16212] mmap() dumps core upon resizing the underlying file

STINNER Victor added the comment: "Well, there are some higly non-portable ways to try to escape this (see http://stackoverflow.com/questions/2663456/write-a-signal-handler-to-catch-sigsegv), but it won't fly in our case." There is also the libsigsegv library, but it's hard and unsafe to handle

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: > I think, handling the signal would do. You can't. Handling a signal like SIGSEGV or SIGBUS in any other way that exiting the process will result in an infinite loop (as soon as the signal handler returns the faulty instruction will be re-executed). We

[issue16212] mmap() dumps core upon resizing the underlying file

Vladimir Ushakov added the comment: I think, handling the signal would do. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: No, it's not. That's why I think there's nothing that can be done. -- ___ Python tracker ___ __

[issue16212] mmap() dumps core upon resizing the underlying file

Christian Heimes added the comment: The fstat() check isn't bullet proof, too. It has a race condition as another process could truncate the file between the fstat() check and the code lines that access the mmapped file. -- ___ Python tracker

[issue16212] mmap() dumps core upon resizing the underlying file

Charles-François Natali added the comment: That's normal. You're truncating the file after having it mapped, so touching the pages corresponding to the truncated part of the file will result in a segfault. See mmap's man page: """ Use of a mapped region can result in these signals:

[issue16212] mmap() dumps core upon resizing the underlying file

Christian Heimes added the comment: I'm able to reproduce the bug. Here is a stack trace: #0 0x005a650d in PyBytes_FromStringAndSize (str=0x7f44c127a000 , size=1) at Objects/bytesobject.c:82 82 (op = characters[*str & UCHAR_MAX]) != NULL) (gdb) bt #0 0x005a650

[issue16212] mmap() dumps core upon resizing the underlying file

New submission from Vladimir Ushakov: The following code crashes the interpreter on Linux: #!/usr/bin/python3 import mmap with open('test', 'wb') as f: f.write(bytes(1)) with open('test', 'r+b') as f: m = mmap.mmap(f.fileno(), 0) f.truncate() a = m[:] --- It's not specific t