[ python-Bugs-1731717 ] race condition in subprocess module
Bugs item #1731717, was opened at 2007-06-06 00:19
Message generated for change (Comment added) made by astrand
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1731717&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dsagal (dsagal)
Assigned to: Peter Åstrand (astrand)
Summary: race condition in subprocess module
Initial Comment:
Python's subprocess module has a race condition: Popen() constructor has a call
to global "_cleanup()" function on whenever a Popen object gets created, and
that call causes a check for all pending Popen objects whether their subprocess
has exited - i.e. the poll() method is called for every active Popen object.
Now, if I create Popen object "foo" in one thread, and then a.wait(), and
meanwhile I create another Popen object "bar" in another thread, then a.poll()
might get called by _clean() right at the time when my first thread is running
a.wait(). But those are not synchronized - each calls os.waitpid() if
returncode is None, but the section which checks returncode and calls
os.waitpid() is not protected as a critical section should be.
The following code illustrates the problem, and is known to break reasonably
consistenly with Python2.4. Changes to subprocess in Python2.5 seems to address
a somewhat related problem, but not this one.
import sys, os, threading, subprocess, time
class X(threading.Thread):
def __init__(self, *args, **kwargs):
super(X, self).__init__(*args, **kwargs)
self.start()
def tt():
s = subprocess.Popen(("/bin/ls", "-a", "/tmp"), stdout=subprocess.PIPE,
universal_newlines=True)
# time.sleep(1)
s.communicate()[0]
for i in xrange(1000):
X(target = tt)
This typically gives a few dozen errors like these:
Exception in thread Thread-795:
Traceback (most recent call last):
File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
self.run()
File "/usr/lib/python2.4/threading.py", line 422, in run
self.__target(*self.__args, **self.__kwargs)
File "z.py", line 21, in tt
s.communicate()[0]
File "/usr/lib/python2.4/subprocess.py", line 1083, in communicate
self.wait()
File "/usr/lib/python2.4/subprocess.py", line 1007, in wait
pid, sts = os.waitpid(self.pid, 0)
OSError: [Errno 10] No child processes
Note that uncommenting time.sleep(1) fixes the problem. So does wrapping
subprocess.poll() and wait() with a lock. So does removing the call to global
_cleanup() in Popen constructor.
--
>Comment By: Peter Åstrand (astrand)
Date: 2007-08-02 11:15
Message:
Logged In: YES
user_id=344921
Originator: NO
Suddenly starting to leave Zombies is NOT an option for me. To prevent
zombies, all applications using the subprocess module would need to be
changed. This is a major breakage of backwards compatibility, IMHO.
subprocess (as well as the popen2 module) has prevented zombies
automatically from the beginning and I believe they should continue to do
so (at least by default).
A little bit of history: When I wrote the subprocess module, I stole the
idea of calling _cleanup whenever a new instance is created from the popen2
module, since I thought it was nice, lightweight and avoided having a
__del__ method (which I have some bad experience with, GC-related). This
worked great for a long time. At 2006-04-10, however, martin.v.loewis
committed patch 1467770 (revision r45234), to solve bug 1460493. It
introduces a __del__ method and some more complexity. I must admit that I
didn't review this patch in detail, but it looked like thread safeness was
being taken care of. But apparently it isn't.
Perhaps reverting to the original popen2 approach would solve the problem?
Or does the popen2 module suffer from this bug as well?
I think that we need to fix this once and for all, this time. We should
probably also consider adding the option of not having subprocess auto-wait
while we are at it, for those what would like to wait() themselves (there
should be a tracker item for this, but I can't find it right now).
Are we tight on time for fixing this bug? I'm out in the forest right
now...
--
Comment By: Guido van Rossum (gvanrossum)
Date: 2007-08-02 02:45
Message:
Logged In: YES
user_id=6380
Originator: NO
I like #2. I don't see any use for threadsafe Popen instances, and I
think that any self-respecting long-running server using Popen should
properly manage its subprocesses anyway. (And for short-running processes
it doesn't really matter if you have a few zombies.)
One could add a __del__ method that registers zombies to be reaped later,
but I don't think it's worth it, and __del
[ python-Bugs-1764761 ] Decimal comparison with None fails in Windows
Bugs item #1764761, was opened at 2007-07-31 13:34
Message generated for change (Settings changed) made by facundobatista
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1764761&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: pablohoffman.com (pablohoffman)
>Assigned to: Facundo Batista (facundobatista)
Summary: Decimal comparison with None fails in Windows
Initial Comment:
The version used to test this was:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
In Linux:
>>> from decimal import Decimal
>>> Decimal('1') < None
False
>>>
In Windows:
>>> from decimal import Decimal
>>> Decimal('1') < None
True
>>>
This is probably a Windows bug since both platforms do:
>>> 1 < None
False
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1764761&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1766421 ] poll() returns "status code", not "return code"
Bugs item #1766421, was opened at 2007-08-02 19:54 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1766421&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: sjbrown (ezide_com) Assigned to: Nobody/Anonymous (nobody) Summary: poll() returns "status code", not "return code" Initial Comment: http://docs.python.org/lib/popen3-objects.html poll() documentation should say that it returns the "status code" -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1766421&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1728488 ] -q (quiet) option for python interpreter
Feature Requests item #1728488, was opened at 2007-05-31 00:14 Message generated for change (Comment added) made by orsenthil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1728488&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Marcin Wojdyr (wojdyr) Assigned to: Nobody/Anonymous (nobody) Summary: -q (quiet) option for python interpreter Initial Comment: I'd like to suggest the new option for python: -q Do not print the version and copyright messages. These messages are also suppressed in non-interactive mode. Why: I often use python as a calculator, for a couple-lines calculations, and would prefer to avoid having printed these three lines. There is a similar option in e.g. gdb. AFAICS the implementation would require small changes in Modules/main.c, Misc/python.man and probably in other docs. If it would be accepted, I can do it. Marcin -- Comment By: O.R.Senthil Kumaran (orsenthil) Date: 2007-08-03 07:36 Message: Logged In: YES user_id=942711 Originator: NO +1 for this option. -- Comment By: Raymond Hettinger (rhettinger) Date: 2007-08-02 09:56 Message: Logged In: YES user_id=80475 Originator: NO +1 I think this would be nice. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1728488&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1764761 ] Decimal comparison with None fails in Windows
Bugs item #1764761, was opened at 2007-07-31 13:34
Message generated for change (Comment added) made by facundobatista
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1764761&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: pablohoffman.com (pablohoffman)
Assigned to: Facundo Batista (facundobatista)
Summary: Decimal comparison with None fails in Windows
Initial Comment:
The version used to test this was:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
In Linux:
>>> from decimal import Decimal
>>> Decimal('1') < None
False
>>>
In Windows:
>>> from decimal import Decimal
>>> Decimal('1') < None
True
>>>
This is probably a Windows bug since both platforms do:
>>> 1 < None
False
--
>Comment By: Facundo Batista (facundobatista)
Date: 2007-08-02 23:59
Message:
Logged In: YES
user_id=752496
Originator: NO
Solved, the problem was that __cmp__ was returning NotImplemented, which
is not allowed (is not defined, that's why the different behaviour in
different systems).
The solution was commited in revision 56682, in the decimal branch.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1764761&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
