[ python-Bugs-1754642 ] subprocess.Popen.wait fails sporadically with threads
Bugs item #1754642, was opened at 2007-07-16 09:26 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1754642&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 >Status: Closed >Resolution: Duplicate Priority: 5 Private: No Submitted By: Geoffrey Bache (gjb1002) Assigned to: Nobody/Anonymous (nobody) Summary: subprocess.Popen.wait fails sporadically with threads Initial Comment: When several threads are using the subprocess module I occasionally get stack traces that look like out, err = process.communicate() File "/usr/lib/python2.4/subprocess.py", line 1083, in communicate self.wait() File "/usr/lib/python2.4/subprocess.py", line 1007, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes See the fixed bug at https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183780&group_id=5470 The problem I am observing seems to match the one described there in the "related modules" part. After discovering that bug, I downloaded the test program attached to it and discovered that I got the errors described from time to time. I am using Python 2.4.3 on Red Hat EL4 and Python 2.4.4 on Red Hat EL3, and both have this error. I reattach my tweaked version of that test program. The only basic difference is that testing subprocess is the default option and the default number of threads increased to 50 as this seemed to mean it occurred more often. I have reproduced the original popen2 problem once on Python 2.4.4 but it's much more infrequent, and not a problem in practice for me. See the comment at the top of that script for details of expected behaviour etc. -- >Comment By: Georg Brandl (gbrandl) Date: 2007-08-04 07:24 Message: Logged In: YES user_id=849994 Originator: NO Duplicate of #1731717. -- Comment By: Donovan Baarda (abo) Date: 2007-08-03 19:37 Message: Logged In: YES user_id=10273 Originator: NO Bugs 1754642 and 1753891 both look like duplicates of bug 1731717 to me. I suggest marking them both as dups of 1731717 because that one has info on the race-condition that causes this and discussions on how to fix it. -- Comment By: Geoffrey Bache (gjb1002) Date: 2007-07-16 09:32 Message: Logged In: YES user_id=769182 Originator: YES Hmm, seems someone else reported this during the weekend. Looks rather similar to bug 1753891... I let someone else mark as duplicate if they want. Can't harm to have two different test programs for an indeterministic problem :) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1754642&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1753891 ] subprocess raising "No Child Process" OSError
Bugs item #1753891, was opened at 2007-07-14 03:06 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753891&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.4 Status: Open Resolution: Duplicate Priority: 5 Private: No Submitted By: slowfood (slowfood) Assigned to: Nobody/Anonymous (nobody) Summary: subprocess raising "No Child Process" OSError Initial Comment: The program below demostrates a "No Child Process" OSError on a multi-cpu systems. This is extracted from a large system where we are trying to manage many sub-processes, some of which end up having little/no real work to do so they return very fast, here emulated by having the sub-process be an invocation of: Executable="/bin/sleep 0" Seems like some race condition, since if you make the child process take some time (sleep 0.1) the frequency of errors decreeses. Error only shows up when there are more threads than have real CPU's by at least a factor of two, on dual core machines up NumThreads to 18 to get the failures. Same error on both Python 2.4.3 with: Linux 2.6.18-gentoo-r3 Gentoo and Python 2.4.4 with: Linux 2.6.20-gentoo-r8 Any help appreciated - = = = = Start code example === % python subprocess_noChildErr.py Exception in thread Thread-3: Traceback (most recent call last): File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap self.run() File "testCaseA.py", line 14, in run subprocess.call(Executable.split()) File "/usr/lib64/python2.4/subprocess.py", line 413, in call return Popen(*args, **kwargs).wait() File "/usr/lib64/python2.4/subprocess.py", line 1007, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes Test finished % cat subprocess_noChildErr.py import subprocess, threading # Params Executable="/bin/sleep 0" NumThreads = 18 NumIterations = 10 class TestClass(threading.Thread): def __init__(self, threadNum): self.threadNum = threadNum threading.Thread.__init__(self) def run(self): for i in range(NumIterations): subprocess.call(Executable.split()) def test(): allThreads = [] for i in range(NumThreads): allThreads.append(TestClass(i)) for i in range(NumThreads): allThreads[i].start() for i in range(NumThreads): allThreads[i].join() print "Test finished" if __name__ == '__main__': test() % python -V Python 2.4.4 % uname -a Linux 2.6.20-gentoo-r8 #2 SMP PREEMPT Sun Jul 1 13:22:56 PDT 2007 x86_64 Dual-Core AMD Opteron(tm) Processor 2212 AuthenticAMD GNU/Linux % % date Fri Jul 13 19:26:44 PDT 2007 % = = = = End code example === -- >Comment By: Georg Brandl (gbrandl) Date: 2007-08-04 07:25 Message: Logged In: YES user_id=849994 Originator: NO Closing too. Tying bugs is not supported by SF. -- Comment By: slowfood (slowfood) Date: 2007-08-03 20:07 Message: Logged In: YES user_id=1844537 Originator: YES Managed to mark it as a duplicate, but not able to tie it to 1731717. Sorry - ;;slowfood -- Comment By: slowfood (slowfood) Date: 2007-08-03 20:01 Message: Logged In: YES user_id=1844537 Originator: YES I agree with abo that this looks to be a dup of 1731717, will try to mark it as such. -- Comment By: Donovan Baarda (abo) Date: 2007-08-03 19:37 Message: Logged In: YES user_id=10273 Originator: NO Bugs 1754642 and 1753891 both look like duplicates of bug 1731717 to me. I suggest marking them both as dups of 1731717 because that one has info on the race-condition that causes this and discussions on how to fix it. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753891&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1753891 ] subprocess raising "No Child Process" OSError
Bugs item #1753891, was opened at 2007-07-14 03:06 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753891&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.4 >Status: Closed Resolution: Duplicate Priority: 5 Private: No Submitted By: slowfood (slowfood) Assigned to: Nobody/Anonymous (nobody) Summary: subprocess raising "No Child Process" OSError Initial Comment: The program below demostrates a "No Child Process" OSError on a multi-cpu systems. This is extracted from a large system where we are trying to manage many sub-processes, some of which end up having little/no real work to do so they return very fast, here emulated by having the sub-process be an invocation of: Executable="/bin/sleep 0" Seems like some race condition, since if you make the child process take some time (sleep 0.1) the frequency of errors decreeses. Error only shows up when there are more threads than have real CPU's by at least a factor of two, on dual core machines up NumThreads to 18 to get the failures. Same error on both Python 2.4.3 with: Linux 2.6.18-gentoo-r3 Gentoo and Python 2.4.4 with: Linux 2.6.20-gentoo-r8 Any help appreciated - = = = = Start code example === % python subprocess_noChildErr.py Exception in thread Thread-3: Traceback (most recent call last): File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap self.run() File "testCaseA.py", line 14, in run subprocess.call(Executable.split()) File "/usr/lib64/python2.4/subprocess.py", line 413, in call return Popen(*args, **kwargs).wait() File "/usr/lib64/python2.4/subprocess.py", line 1007, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes Test finished % cat subprocess_noChildErr.py import subprocess, threading # Params Executable="/bin/sleep 0" NumThreads = 18 NumIterations = 10 class TestClass(threading.Thread): def __init__(self, threadNum): self.threadNum = threadNum threading.Thread.__init__(self) def run(self): for i in range(NumIterations): subprocess.call(Executable.split()) def test(): allThreads = [] for i in range(NumThreads): allThreads.append(TestClass(i)) for i in range(NumThreads): allThreads[i].start() for i in range(NumThreads): allThreads[i].join() print "Test finished" if __name__ == '__main__': test() % python -V Python 2.4.4 % uname -a Linux 2.6.20-gentoo-r8 #2 SMP PREEMPT Sun Jul 1 13:22:56 PDT 2007 x86_64 Dual-Core AMD Opteron(tm) Processor 2212 AuthenticAMD GNU/Linux % % date Fri Jul 13 19:26:44 PDT 2007 % = = = = End code example === -- Comment By: Georg Brandl (gbrandl) Date: 2007-08-04 07:25 Message: Logged In: YES user_id=849994 Originator: NO Closing too. Tying bugs is not supported by SF. -- Comment By: slowfood (slowfood) Date: 2007-08-03 20:07 Message: Logged In: YES user_id=1844537 Originator: YES Managed to mark it as a duplicate, but not able to tie it to 1731717. Sorry - ;;slowfood -- Comment By: slowfood (slowfood) Date: 2007-08-03 20:01 Message: Logged In: YES user_id=1844537 Originator: YES I agree with abo that this looks to be a dup of 1731717, will try to mark it as such. -- Comment By: Donovan Baarda (abo) Date: 2007-08-03 19:37 Message: Logged In: YES user_id=10273 Originator: NO Bugs 1754642 and 1753891 both look like duplicates of bug 1731717 to me. I suggest marking them both as dups of 1731717 because that one has info on the race-condition that causes this and discussions on how to fix it. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753891&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1753395 ] struni: assertion in Windows debug build
Bugs item #1753395, was opened at 2007-07-13 13:35 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Heller (theller) Assigned to: Thomas Heller (theller) Summary: struni: assertion in Windows debug build Initial Comment: When running Lib/test/test_descr.py with a Windows debug build, I get an assertion in the MS runtime lib, in the isalpha(*p) call, file typeobject.c, line 1582. The value of '*p' at the time of the call is 4660. The assertion reported is: File: istype.c Line: 68 Expression; (unsigned)(c + 1) <= 256 -- >Comment By: Walter Dörwald (doerwalter) Date: 2007-08-04 09:42 Message: Logged In: YES user_id=89016 Originator: NO Right, this should have been *p > 255, even better would be *p > 127. Thomas, can you try that on Windows? -- Comment By: Guido van Rossum (gvanrossum) Date: 2007-08-03 22:37 Message: Logged In: YES user_id=6380 Originator: NO I wonder if the test "i > 255" wasn't meant to be "*p > 255"? Please try that, and submit if it works. Looks like Walter Doerwald introduced this bug in r55892. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753395&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1767511 ] SocketServer.DatagramRequestHandler
Bugs item #1767511, was opened at 2007-08-04 11:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1767511&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alzheimer (alzheimer) Assigned to: Nobody/Anonymous (nobody) Summary: SocketServer.DatagramRequestHandler Initial Comment: I seem to have misunderstood SocketServer completely, maybe someone can tell me if I was using the wrong thing after all. I wanted to implement an UDP protocol in Python. For sending and receiving UDP packets, I chose SocketServer.UDPSocketServer along with the DatagramRequestHandler. It is probably implied, but did not become clear to me just by reading the documentation, that apparently a server is meant to send back a packet for every packet it receives, no matter what. Which in turn means that you can't have servers talk to other servers, because they'd be sending packets back and forth endlessly then. At least, that's what the DatagramRequestHandler is doing. It issues a socket.sendto in its finish() routine. Took me a while to find out where all the empty packets where coming from... Does it make sense at all for an UDP server to do this? Sure, it works for simple protocols that do nothing but query/answer. But it's really useless for anything else. The documentation could be more verbose on this behaviour, in any case. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1767511&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1753395 ] struni: assertion in Windows debug build
Bugs item #1753395, was opened at 2007-07-13 07:35 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Heller (theller) >Assigned to: Martin v. Löwis (loewis) Summary: struni: assertion in Windows debug build Initial Comment: When running Lib/test/test_descr.py with a Windows debug build, I get an assertion in the MS runtime lib, in the isalpha(*p) call, file typeobject.c, line 1582. The value of '*p' at the time of the call is 4660. The assertion reported is: File: istype.c Line: 68 Expression; (unsigned)(c + 1) <= 256 -- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-08-04 12:44 Message: Logged In: YES user_id=6380 Originator: NO I've committed the *p > 127 patch. Committed revision 56737. However, we'll need to change this again now that we accept Unicode identifiers. Assigning to MvL. -- Comment By: Walter Dörwald (doerwalter) Date: 2007-08-04 03:42 Message: Logged In: YES user_id=89016 Originator: NO Right, this should have been *p > 255, even better would be *p > 127. Thomas, can you try that on Windows? -- Comment By: Guido van Rossum (gvanrossum) Date: 2007-08-03 16:37 Message: Logged In: YES user_id=6380 Originator: NO I wonder if the test "i > 255" wasn't meant to be "*p > 255"? Please try that, and submit if it works. Looks like Walter Doerwald introduced this bug in r55892. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753395&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1753395 ] struni: assertion in Windows debug build
Bugs item #1753395, was opened at 2007-07-13 13:35 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Heller (theller) Assigned to: Martin v. Löwis (loewis) Summary: struni: assertion in Windows debug build Initial Comment: When running Lib/test/test_descr.py with a Windows debug build, I get an assertion in the MS runtime lib, in the isalpha(*p) call, file typeobject.c, line 1582. The value of '*p' at the time of the call is 4660. The assertion reported is: File: istype.c Line: 68 Expression; (unsigned)(c + 1) <= 256 -- >Comment By: Thomas Heller (theller) Date: 2007-08-04 22:58 Message: Logged In: YES user_id=11105 Originator: YES I confirm that the assertion in the windows debug build is gone now. -- Comment By: Guido van Rossum (gvanrossum) Date: 2007-08-04 18:44 Message: Logged In: YES user_id=6380 Originator: NO I've committed the *p > 127 patch. Committed revision 56737. However, we'll need to change this again now that we accept Unicode identifiers. Assigning to MvL. -- Comment By: Walter Dörwald (doerwalter) Date: 2007-08-04 09:42 Message: Logged In: YES user_id=89016 Originator: NO Right, this should have been *p > 255, even better would be *p > 127. Thomas, can you try that on Windows? -- Comment By: Guido van Rossum (gvanrossum) Date: 2007-08-03 22:37 Message: Logged In: YES user_id=6380 Originator: NO I wonder if the test "i > 255" wasn't meant to be "*p > 255"? Please try that, and submit if it works. Looks like Walter Doerwald introduced this bug in r55892. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1753395&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
