[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
New submission from Leonardo Francalanci:
the script below (a python process is called, which calls a waitfor cmd with a
timeout of 4 seconds) is supposed to end after 4 seconds. But instead
proc.communicate stops after the 20 seconds timeout.
Everything works 100% ok if I remove the stdin/stderr/stdout parameters...
if __name__ == "__main__":
#start a python process which will wait for 4 seconds and then exit (waitfor
is set to 200):
proc_str = ["C:\\Program Files (x86)\\Anaconda3\\Python.exe",
"-c", "import
subprocess;subprocess.run('cmd /S /C waitfor g /t 200', shell=False,
timeout=4)"]
proc = subprocess.Popen(proc_str,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
shell=False,
universal_newlines=True)
#this should exit in 4 seconds (when the called process exits), but instead
exits after 20 seconds:
(proc_out, proc_err) = proc.communicate(timeout=20)
--
messages: 302051
nosy: Leonardo Francalanci
priority: normal
severity: normal
status: open
title: proc communicate not exiting on python subprocess timeout using PIPES
versions: Python 3.6
___
Python tracker
<https://bugs.python.org/issue31447>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Changes by Leonardo Francalanci : -- components: +Interpreter Core type: -> behavior ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment: The called "C:\\Program Files (x86)\\Anaconda3\\Python.exe" process exits after 4 seconds. The reason why it ends shouldn't matter, right? I expect that a call to communicate should exit as soon as the called process is not running anymore. I don't hold a pipe with "waitfor", I hold it with python.exe. -- ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment: thank you for your replies! I used waitfor because it's the simplest and shortest way to have reproducible code. The issue I'm having is obviously not with waitfor, but with another exe, but this doesn't change the fact that I need a way to exit as soon as the called process exits or the timeout ends. I don't care what the called process does. I know that the process exited, and the docs for proc.communicate say "Wait for process to terminate". Well, the process terminates, but "communicate" doesn't exit. It doesn't say "communicate will hang as long as the pipes are open". Plus: the "python.exe" process doesn't actually timeout!!! It just exits (with exception, but that's not important), and I would like to get the error from stderr! While I can't get it because "communicate" raises an exception... -- ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment: (forgot: Issue 26534 is about shell=True, I use shell=False, right?) -- ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment: I have a workaround, and I guess this means there's a bug in the current implementation of stdout/stderr=subprocess.PIPE; if I open my own pipes instead of using subprocess.PIPE everything seems to work (provided I close the pipe before reading from it): (errPipeR, errPipeW) = os.pipe(); (outPipeR, outPipeW) = os.pipe(); proc = subprocess.Popen(proc_str, stdin=subprocess.PIPE, stderr=errPipeW, stdout=outPipeW,shell=False, universal_newlines=True) proc.communicate(timeout=20) os.close(outPipeW) os.close(errPipeW) Now I can read from the "R" pipes with 0 problems (getting the error and output streams), and proc.communicate exits as soon as the called process exits (4 seconds). So, since I can use my own pipes without any problems, I don't see why the regular "subprocess.PIPE" shouldn't be working... -- ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment: I'm really sorry, you are 100% correct: it blocks on the pipe (my tests killed the process tree before reading from the pipes). Still, I think there should be a way to actually read the output also in this case... works for me when I kill the whole process stack. -- ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment:
can I at least change the call to:
subprocess.run('cmd /S /C waitfor g /t 200', shell=False, timeout=4)
in any way to avoid the problem?
I tried with
stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL;
also with close_fds=True, but nothing changes...
--
___
Python tracker
<https://bugs.python.org/issue31447>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31447] proc communicate not exiting on python subprocess timeout using PIPES
Leonardo Francalanci added the comment: That works! But when I said "also with close_fds=True", I meant that I tried WITHOUT overriding stdin, stdout, and stderr AND setting close_fds=True, but it didn't work. What worked was not overriding stdin/out/err and adding os.set_inheritable(0, False) os.set_inheritable(1, False) os.set_inheritable(2, False) before the call (no need to set close_fds) -- ___ Python tracker <https://bugs.python.org/issue31447> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
