[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: > To support 3.2 and 2.7: > (1) select.error must be caught as in the original patch, > (2) e.args[0] must be used - select.error doesn't have 'errno' > attribute. > Should I prepare the patch for 3.2 and 2.7? It shouldn't be necessary, I've just committed f

[issue7978] SocketServer doesn't handle syscall interruption

Roundup Robot added the comment: New changeset f8e7fcd581ff by Antoine Pitrou in branch '3.2': Fix the patch for issue #7978: select() raises select.error before 3.3, not OSError. http://hg.python.org/cpython/rev/f8e7fcd581ff New changeset 4298d6e79ecb by Antoine Pitrou in branch '2.7': Fix th

[issue7978] SocketServer doesn't handle syscall interruption

Jerzy Kozera added the comment: I forgot to mention my patch is 3.3-only, sorry - it depends on changes from #12555 (http://hg.python.org/cpython/rev/41a1de81ef2b#l18.21 to be precise). To support 3.2 and 2.7: (1) select.error must be caught as in the original patch, (2) e.args[0] must be us

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Antoine Pitrou : -- type: enhancement -> behavior ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: > You must recompute the timeout when select() is interrupted: see issue > #12338. Ah, right. It's a minor issue, though. I would suggest opening a separate issue for it. The patch is now committed to all 3 branches (I'm finally convinced this is a bug worth

[issue7978] SocketServer doesn't handle syscall interruption

STINNER Victor added the comment: You must recompute the timeout when select() is interrupted: see issue #12338. -- ___ Python tracker ___ ___

[issue7978] SocketServer doesn't handle syscall interruption

Roundup Robot added the comment: New changeset d941d1fcc6e6 by Antoine Pitrou in branch '2.7': Issue #7978: socketserver now restarts the select() call when EINTR is returned. http://hg.python.org/cpython/rev/d941d1fcc6e6 -- ___ Python tracker

[issue7978] SocketServer doesn't handle syscall interruption

Roundup Robot added the comment: New changeset 5493d44b56d8 by Antoine Pitrou in branch '3.2': Issue #7978: socketserver now restarts the select() call when EINTR is returned. http://hg.python.org/cpython/rev/5493d44b56d8 New changeset 97a0c6230ece by Antoine Pitrou in branch 'default': Issue #

[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: Jerzy's latest patch looks ok to me. This is a slight behaviour change so I'm not sure it should go in 3.2/2.7. -- stage: -> patch review versions: +Python 3.3 -Python 2.7, Python 3.1 ___ Python tracker

[issue7978] SocketServer doesn't handle syscall interruption

Jerzy Kozera added the comment: I've updated the patch according to suggestions from Gregory P. Smith. Thanks to a change from #12555 (PEP 3151) now just checking for OSError is enough. (I've decided to use mocked select() instead of calling alarm() to avoid depending on timing.) --

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Giampaolo Rodola' : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:/

[issue7978] SocketServer doesn't handle syscall interruption

Vetoshkin Nikita added the comment: I don't think we should block signals - we can sleep on select for about forever. -- ___ Python tracker ___ _

[issue7978] SocketServer doesn't handle syscall interruption

STINNER Victor added the comment: I think that #12224 is a duplicate of this issue. -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue7978] SocketServer doesn't handle syscall interruption

STINNER Victor added the comment: Using signalfd() (require Linux 2.6.22+), specified signals are written in a file and don't interrupt system calls (select). Bonus: we can wait for a signal using select! Using pthread_sigmask(), you can also block signals before calling select, but it's not

[issue7978] SocketServer doesn't handle syscall interruption

Gregory P. Smith added the comment: I've added some code comments on http://bugs.python.org/review/7978/show overall I think the patch is right, I pointed out one thing to clean up and I think the unittest can be greatly simplified by using stubbed out mock select.select() instead of fork+sig

[issue7978] SocketServer doesn't handle syscall interruption

Vetoshkin Nikita added the comment: Any news on this? Could we possibly apply patch as is? If I'm not mistaken timeout issue is the only one left unresolved. About timeout. The most elegant way, would be to use select's syscall timeout parameter, but man 2 select says: "On Linux, select()

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Kevin Chen : -- nosy: +kchen ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/m

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Terry J. Reedy : -- versions: -Python 2.5, Python 2.6 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7978] SocketServer doesn't handle syscall interruption

Yaniv Aknin added the comment: While bda's usecase is indeed fixed is recent versions, I want to point out the issue still exists in a recent py3k checkout. I've updated my patch to apply cleanly against recent py3k (and made less structural changes to test_socketserver.py; I still think it h

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Yaniv Aknin : Removed file: http://bugs.python.org/file16841/socketserver_eintr_py3k.diff ___ Python tracker ___ ___ Python-bugs-lis

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Christophe Simonis : -- nosy: +Christophe Simonis ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: > In BaseServer, a threading.Event is used in shutdown, so it can block > until server_forever is finished (after checking __serving). Since the > SIGTERM interrupts the select system call, the event set is never > reached, and shutdown hangs waiting on the even

[issue7978] SocketServer doesn't handle syscall interruption

Bryce Allen added the comment: I encountered this issue when trying to exit cleanly on SIGTERM, which I use to terminate background daemons running serve_forever. In BaseServer, a threading.Event is used in shutdown, so it can block until server_forever is finished (after checking __serving).

[issue7978] SocketServer doesn't handle syscall interruption

Yaniv Aknin added the comment: pitrou, re. test code: I actually started with the test code, so that part is done. I opted to use a forked SocketServer rather than threads. I'm not an expert on the low-level details of a multi-threaded process receiving threads, but it seems to me like there

[issue7978] SocketServer doesn't handle syscall interruption

Andrew Bennetts added the comment: Note that a trivial untilConcludes isn't correct for select if a timeout was passed. If a select(..., 60) was interrupted after 59 seconds, you probably want to restart it with a timeout of 1 second, not 60. The SocketServer_eintr.diff patch has this flaw.

[issue7978] SocketServer doesn't handle syscall interruption

Vetoshkin Nikita added the comment: I vote for making a global wrapper. As neologix pointed - many other modules can be (or are) affected. Futher found bugs could be fixed using that global wrapper. -- ___ Python tracker

[issue7978] SocketServer doesn't handle syscall interruption

Changes by Antoine Pitrou : -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/op

[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: Ok, I think the wrapper approach is a good one. Now remains to decide whether it should stay private to socketserver or be exposed as a generic method (for example in the "os" module as proposed). By the way, adding a test would be nice, too. I'm not sure it

[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: Ok, I think the wrapper approach is a good one. Now remains to decide whether it should stay private to socketserver or be exposed as a generic method (for example in the "os" module as proposed). By the way, adding a patch would be nice, too. I'm not sure it

[issue7978] SocketServer doesn't handle syscall interruption

Charles-Francois Natali added the comment: > If we agree so far, I believe that an implementation of untilConcludes > *should* be added to stdlib ("signal.restartable_call", anyone?). Definitely, it's better to have this handler written once and correct than having various implementations sca

[issue7978] SocketServer doesn't handle syscall interruption

Vetoshkin Nikita added the comment: According to "man 7 signal" select must be explicitely restarted, regardless of the SA_RESTART flag. -- ___ Python tracker ___ __

[issue7978] SocketServer doesn't handle syscall interruption

Gregory P. Smith added the comment: wrapping select in an eintr handler that loops like SocketServer_eintr.diff is fine. -- assignee: -> gregory.p.smith nosy: +gregory.p.smith ___ Python tracker _

[issue7978] SocketServer doesn't handle syscall interruption

Yaniv Aknin added the comment: First, let me cast my vote in favour of leaving select() itself alone, it's excessive to fix this issue with changing select(). Second, while exarkun's comments are accurate, I understand where pitrou is coming from. EINTR is not a pathological case, but it's no

[issue7978] SocketServer doesn't handle syscall interruption

Vetoshkin Nikita added the comment: > What kind of signals can be received in real-life? We use SIGUSR1 to reopen log files after rotation. sighandler works just fine, but after that Paste crashes. I suppose that implementing silent syscall restart at select.select() level is a bad idea (exp

[issue7978] SocketServer doesn't handle syscall interruption

Jean-Paul Calderone added the comment: > What kind of signals can be received in real-life? There are lots of possible answers. Here's one. You launch a child process with os.fork() (and perhaps os.exec*() as well). Eventually you'll get a SIGCHLD in the parent when the child exits. If yo

[issue7978] SocketServer doesn't handle syscall interruption

Charles-Francois Natali added the comment: In real life, you can receive for example SIGSTOP (strace, gdb, shell), but mostly SIGCHLD (any process spawning children), etc. The attached patch just restarts calls when EINTR is received, as is done in subprocess module. The mailing list is a goo

[issue7978] SocketServer doesn't handle syscall interruption

Antoine Pitrou added the comment: What kind of signals can be received in real-life? (I'm assuming that most signals are of the kind that you only receive if someone else deliberately sends it to you, in which case you are supposed to be prepared to handle it) Also, does your patch still allo

[issue7978] SocketServer doesn't handle syscall interruption

Charles-Francois Natali added the comment: A patch to make select calls EINTR-safe is attached, but: - there are many modules that suffer from the same problem, so maybe a fix at select level would be better - if not, it could be a good idea to add this EINTR-retry handler to a given module (s

[issue7978] SocketServer doesn't handle syscall interruption

Vetoshkin Nikita added the comment: Wrapping select in (taken from twisted sources) can help: def untilConcludes(f, *a, **kw): while True: try: return f(*a, **kw) except (IOError, OSError), e: if e.args[0] == errno.EINTR: continue