[issue14000] Subprocess stdin.flush does not flush
New submission from Andrew Moffat :
The following code only works if the "p.stdin.close()" line is uncommented. It
seems that stdin only flushes to the process on stdin.close(), not on
stdin.flush().
---
import subprocess as subp
import threading
import time
p = subp.Popen(["tr", "[:lower:]", "[:upper:]"],
stdin=subp.PIPE, stderr=subp.PIPE, stdout=subp.PIPE, close_fds=True)
def read(stdout):
while True:
line = stdout.readline()
if not line: break
print(line.decode("utf8"))
t = threading.Thread(target=read, args=(p.stdout,))
t.daemon = True
t.start()
p.stdin.write("uppercase\n".encode())
p.stdin.flush()
#p.stdin.close()
time.sleep(1)
--
components: None
messages: 153257
nosy: amoffat
priority: normal
severity: normal
status: open
title: Subprocess stdin.flush does not flush
type: behavior
versions: Python 2.7, Python 3.1, Python 3.2
___
Python tracker
<http://bugs.python.org/issue14000>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15898] OSX TTY bug
New submission from Andrew Moffat: I'm getting some kind of race condition on OSX when trying to read the output from a pseudoterminal of a forked process. This race conditional only occurs if I run tty.tcsetattr on the master side of the pty, regardless of if I actually change the mode or not. Try running the following code on OSX multiple times. Sometimes you'll see "testing", sometimes you won't, sometimes it hangs. If you comment out "tty.tcsetattr", you will consistently see "testing". import os import pty import resource import signal import tty master, slave = pty.openpty() pid = os.fork() # BUG IS HERE # we're not making any changes to the tty mode, but # the mere act of setting a mode causes the output to only # show up sometimes. # # comment out "tty.tcsetattr" and the bug goes away mode = tty.tcgetattr(master) tty.tcsetattr(master, tty.TCSANOW, mode) # child process if pid == 0: os.setsid() os.close(master) os.dup2(slave, 0) os.dup2(slave, 1) os.dup2(slave, 2) max_fd = resource.getrlimit(resource.RLIMIT_NOFILE)[0] os.closerange(3, max_fd) # make controlling terminal. taken from pty.fork tmp_fd = os.open(os.ttyname(1), os.O_RDWR) os.close(tmp_fd) os.write(1, "testing".encode()) os._exit(255) # parent process else: os.close(slave) try: print(os.read(master, 1024)) finally: os.kill(pid, signal.SIGKILL) -- messages: 170147 nosy: amoffat priority: normal severity: normal status: open title: OSX TTY bug type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 ___ Python tracker <http://bugs.python.org/issue15898> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15898] OSX TTY bug
Andrew Moffat added the comment: If I move tcsetattr into the parent, the race condition still exists. If I move it into the child, before the write(), it works. My environment is OSX 10.7.2 inside of Virtualbox (maybe this is the problem?) I've shuffled some code around, and found case that fails consistently in python 3, but succeeds consistently on python 2.7. It doesn't use tcsetattr at all, which is making me realize the problem may be deeper than I originally thought. Are you able to confirm this, Ned? import os import pty import resource import signal import tty import time master, slave = pty.openpty() pid = os.fork() # child process if pid == 0: os.setsid() os.close(master) os.dup2(slave, 0) os.dup2(slave, 1) os.dup2(slave, 2) max_fd = resource.getrlimit(resource.RLIMIT_NOFILE)[0] os.closerange(3, max_fd) # make controlling terminal. taken from pty.fork tmp_fd = os.open(os.ttyname(1), os.O_RDWR) os.close(tmp_fd) os.write(1, "testing".encode()) os._exit(255) # parent process else: time.sleep(0.5) os.close(slave) try: print(os.read(master, 1024)) finally: os.kill(pid, signal.SIGKILL) -- ___ Python tracker <http://bugs.python.org/issue15898> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15898] OSX TTY bug
Andrew Moffat added the comment: @Ned, Ok, give me a little time to work up those suggestions. On my machine, the previous script prints "testing" consistently on 2.7, "" consistently on 3.2. I guess this is the behavior that is unexpected...it should print the same thing for both versions. But if you are unable to confirm it, that's no good. Let me work on this a little more before you guys close this. @Martin, Thanks for the tip. -- ___ Python tracker <http://bugs.python.org/issue15898> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15898] OSX TTY bug
Andrew Moffat added the comment:
Below I attached a script that reproduces the behavior of an assertion error on
a specific fork condition for a process with a pty. Here are the results
Xubuntu x64:
Python 2.7, parent_close_slave_before_child_write = False: WORKS
Python 2.7, parent_close_slave_before_child_write = True: WORKS
Python 3.2, parent_close_slave_before_child_write = False: WORKS
Python 3.2, parent_close_slave_before_child_write = True: WORKS
Mac OSX 10.7.2:
Python 2.7, parent_close_slave_before_child_write = False: WORKS
Python 2.7, parent_close_slave_before_child_write = True: WORKS
Python 3.2, parent_close_slave_before_child_write = False: >> FAILS <<
Python 3.2, parent_close_slave_before_child_write = True: WORKS
Here's dtruss -f for python 2.7 and 3.0 on Mac OSX:
2.7, parent_close_slave_before_child_write = False (WORKS)
PID/THRD SYSCALL(args) = return
356/0xc4a: open_nocancel("/dev/ptmx\0", 0x20002, 0x0) = 3 0
356/0xc4a: ioctl(0x3, 0x20007454, 0x) = 0 0
356/0xc4a: ioctl(0x3, 0x20007452, 0x0)= 0 0
356/0xc4a: ioctl(0x3, 0x40807453, 0x10041AD20)= 0 0
356/0xc4a: stat64("/dev/ttys003\0", 0x7FFF5FBFF030, 0x0) = 0 0
356/0xc4a: open_nocancel("/dev/ttys003\0", 0x20002, 0x0) = 4 0
356/0xc4a: fork() = 362 0
362/0xc61: fork() = 0 0
362/0xc61: thread_selfid(0x100420BE0, 0x3, 0x1) = 3169 0
362/0xc61: getpid(0x100420BE0, 0x3, 0x0) = 362 0
362/0xc61: select(0x0, 0x0, 0x0, 0x0, 0x7FFF5FBFF3C0) = 0 0
362/0xc61: setsid(0x0, 0x0, 0x1001A7B40) = 362 0
362/0xc61: close(0x3) = 0 0
362/0xc61: dup2(0x4, 0x0, 0x0)= 0 0
362/0xc61: dup2(0x4, 0x1, 0x0)= 1 0
362/0xc61: dup2(0x4, 0x2, 0x0)= 2 0
362/0xc61: getrlimit(0x1008, 0x7FFF5FBFF3D0, 0x0) = 0 0
... close range ...
362/0xc61: ioctl(0x1, 0x40487413, 0x7FFF5FBFF360) = 0 0
362/0xc61: fstat64(0x1, 0x7FFF5FBFF2D0, 0x0) = 0 0
362/0xc61: open_nocancel("/dev/\0", 0x14, 0x0)= 3 0
362/0xc61: fcntl_nocancel(0x3, 0x2, 0x1) = 0 0
362/0xc61: fstatfs64(0x3, 0x7FFF5FBFE7B0, 0x0)= 0 0
362/0xc61: getdirentries64(0x3, 0x1010AB600, 0x1000) = 3080 0
... lots of lstat64 ...
362/0xc61: close_nocancel(0x3)= 0 0
362/0xc61: open("/dev/ttys003\0", 0x2, 0x1FF) = 3 0
362/0xc61: close(0x3) = 0 0
362/0xc61: write(0x1, "testing\0", 0x7) = 7 0
362/0xc61: close(0x1) = 0 0
356/0xc4a: select(0x0, 0x0, 0x0, 0x0, 0x7FFF5FBFF3C0) = 0 0
356/0xc4a: close(0x4) = 0 0
356/0xc4a: read(0x3, "testing\0", 0x400) = 7 0
356/0xc4a: write_nocancel(0x1, "'testing'\n\0", 0xA) = 10 0
356/0xc4a: wait4(0x16A, 0x7FFF5FBFF3D4, 0x0) = 362 0
356/0xc4a: sigaction(0x2, 0x7FFF5FBFF800, 0x7FFF5FBFF830) = 0 0
3.0, parent_close_slave_before_child_write = False (FAILS)
PID/THRD SYSCALL(args) = return
385/0xe53: open_nocancel("/dev/ptmx\0", 0x20002, 0xD15EB8)= 3 0
385/0xe53: ioctl(0x3, 0x20007454, 0xB1BC) = 0 0
385/0xe53: ioctl(0x3, 0x20007452, 0xB1BC) = 0 0
385/0xe53: ioctl(0x3, 0x40807453, 0x64F220) = 0 0
385/0xe53: stat64("/dev/ttys003\0", 0xB164, 0x64F220) = 0 0
385/0xe53: open_nocancel("/dev/ttys003\0", 0x20002, 0x0) = 4 0
385/0xe53: fork() = 389 0
389/0xe5c: fork() = 0 0
389/0xe5c: thread_selfid(0x0, 0x0, 0x0) = 3676 0
389/0xe5c: getpid(0x0, 0x0, 0x0) = 389 0
389/0xe5c: select_nocancel(0x0, 0x0, 0x0) = 0 0
389/0xe5c: setsid(0x0, 0x0, 0x0) = 389 0
389/0xe5c: close_nocancel(0x3)= 0 0
389/0xe5c: dup2(0x4, 0x0, 0x0)= 0 0
389/0xe5c: dup2(0x4, 0x1, 0x0)= 1 0
389/0xe5c: dup2(0x4, 0x2, 0x0)= 2 0
389/0xe5c: getrlimit(0x8, 0xB46C, 0x0)= 0 0
... close range ...
389/0xe5c: ioctl(0x1, 0x402C7413, 0xB410) = 0 0
389/0xe5c: fstat64(0x1, 0xB3A4, 0xB410) = 0 0
389/0xe5c: open_nocancel("/dev/\0", 0x14, 0xBFFFE968) = 3 0
389/0xe5c: fcntl_nocancel(0x3, 0x2, 0x1) = 0 0
389/0xe5c: fstatfs64(0x3, 0xBFFFE8D8, 0x1)= 0 0
389/0xe5c: getdirentries64(0x3, 0x10DFC00, 0x1000)= 3080 0
... lots of lstat64 ...
389/0xe5c: close_nocancel(0x3)= 0 0
389/0xe5c: open_nocancel("/dev/ttys003\0", 0x20002, 0x0) = 3 0
389/0xe5c: close_nocancel(0x3)= 0 0
389/0xe5c: write_nocancel(0x1, "testing\0", 0x7) = 7 0
389/0xe5c: close_nocancel(0x1)= 0 0
385/0xe53: select_nocancel(0x0, 0x0, 0x0) = 0 0
385/0xe53: close_nocancel(0x4)= 0 0
385/0xe53: read_noca
[issue15898] OSX TTY bug
Andrew Moffat added the comment: Sorry about that, I refactored out the string at the last minute and typed "testing123" instead of the original "testing" from the trace. I'm not sure I follow the problem exactly. So you're saying that the failing version uses _cancel sys calls, and that __pthread_testcancel may result in EINTR. But that this is not happening in the trace, and data is being written successfully. I guess I'm wondering where the written bytes went and if there's any way to retrieve them? -- ___ Python tracker <http://bugs.python.org/issue15898> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
