New submission from Amrith Kumar:
Environment:
- Linux (Ubuntu 14.04 LTS, x64) running Python 2.7.
- Code uses eventlet.green.subprocess
Code establishes subprocess as:
subprocess.Popen(command,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE, ...)
It then calls communicate()
communicate() throws EAGAIN
The code calls communicate() again
This fails with "ValueError: I/O operation on closed file"
This exception comes from eventlet/greenio.py because an operation (flush) has
been attempted on a closed file.
The complete explanation of this failure is this.
When communicate() is called in this way (WITH NO INPUT but with all three
handles stdin, stdout, stderr), code in communicate() bypasses the "1 handle
optimization" and goes directly to _communicate()
_communicate() will first send any input on stdin along, and flush and close
stdin().
>From that point forward, if anything to do with stderr and stdout returns
>EAGAIN, an attempt to call communicate() again will fail because stdin has now
>been closed().
This is because the code in _communicate() preemptively closes stdin if there
is no input but does not reset stdin.
def _communicate(self, input):
if self.stdin:
# Flush stdio buffer. This might block, if the user has
# been writing to .stdin in an uncontrolled fashion.
self.stdin.flush()
if not input:
self.stdin.close()
The fix for this is relatively straightforward (conceptually).
def _communicate(self, input):
if self.stdin:
# Flush stdio buffer. This might block, if the user has
# been writing to .stdin in an uncontrolled fashion.
self.stdin.flush()
if not input:
self.stdin.close()
self.stdin = None # This should take care of it.
However, a partial workaround from the client perspective is this.
1. If you have no input, set stdin to None
2. If you do have input and you get EAGAIN, you cannot safely retry because
your input may have already been completely or partially sent to the
subprocess; you have to assume failure.
--------------------
Backtrace:
Traceback (most recent call last):
File
"/opt/stack/trove/trove/tests/unittests/guestagent/test_mongodb_manager.py",
line 58, in test_prepare_from_backup
self._prepare_dynamic(backup_id='backup_id_123abc')
File
"/opt/stack/trove/trove/tests/unittests/guestagent/test_mongodb_manager.py",
line 91, in _prepare_dynamic
backup_info=backup_info)
File "trove/guestagent/datastore/mongodb/manager.py", line 66, in
prepare
operating_system.update_owner('mongodb', 'mongodb', mount_point)
File "trove/guestagent/common/operating_system.py", line 109, in
update_owner
run_as_root=True, root_helper="sudo")
File "trove/common/utils.py", line 278, in execute_with_timeout
return execute(*args, **kwargs)
File "trove/openstack/common/processutils.py", line 186, in execute
result = obj.communicate()
File "/usr/lib/python2.7/subprocess.py", line 799, in communicate
return self._communicate(input)
File "/usr/lib/python2.7/subprocess.py", line 1396, in _communicate
self.stdin.flush()
File
"/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/eventlet/greenio.py",
line 419, in _operationOnClosedFile
raise ValueError("I/O operation on closed file")
ValueError: I/O operation on closed file
----------
components: IO
messages: 224398
nosy: amrith
priority: normal
severity: normal
status: open
title: You cannot call communicate() safely after receiving EAGAIN
versions: Python 2.7
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue22114>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
