[issue7932] print statement delayed IOError when stdout has been closed

[tholzer]

New submission from tholzer :

When printing to a closed stdout file descriptor, the print statement only 
raises an IOError at character 8192. 

The expected behaviour is that IOError gets raised immediately (i.e. on the 
first character). Compare this behaviour to writing to a closed sys.stderr.

To reproduce (using bash):


# python -V
Python 2.6.4

# python -c 'print "x" * 8191' 1>&- ; echo $?
close failed in file object destructor:
Error in sys.excepthook:

Original exception was:
0

# python -c 'print "x" * 8192' 1>&- ; echo $?
Traceback (most recent call last):
  File "", line 1, in 
IOError: [Errno 9] Bad file descriptor
1


--
components: Interpreter Core
messages: 99351
nosy: tholzer
severity: normal
status: open
title: print statement delayed IOError when stdout has been closed
type: behavior
versions: Python 2.6

___
Python tracker 
<http://bugs.python.org/issue7932>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7932] print statement delayed IOError when stdout has been closed

[tholzer]

tholzer  added the comment:

This is not quite correct:

The following 2 lines fail as expected (unbuffered):


python -u -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
python -u -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?


whereas in the following example, the first command succeeds and the second one 
fails:


python -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
python -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?


This is counter-intuitive and somewhat contradicts the documentation, as stderr 
always seems to be unbuffered (irrespective of the -u option). IMHO, they 
should either both fail or both succeed.

They seem to behave the same in 2.5 & 2.6, however in 3.1, they all die with a 
SIGABRT, which I guess could be questioned as well. IMHO, they should just 
throw an exception like in 2.6.

--

___
Python tracker 
<http://bugs.python.org/issue7932>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21058] tempfile.NamedTemporaryFile leaks file descriptor when fdopen fails

[tholzer]

New submission from tholzer:

The NamedTemporaryFile inside the standard tempfile library leaks an open file 
descriptor when fdopen fails.

Test case:

# ulimit -SHn 50
# python test1.py

from tempfile import NamedTemporaryFile

while 1:
try:
NamedTemporaryFile(mode='x')
except ValueError as ex:
pass

Output:

Traceback (most recent call last):
  File "./a2.py", line 7, in 
  File "/usr/lib/python2.7/tempfile.py", line 454, in NamedTemporaryFile
  File "/usr/lib/python2.7/tempfile.py", line 235, in _mkstemp_inner
OSError: [Errno 24] Too many open files: '/tmp/tmpI0MIEW'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 66, in 
apport_excepthook
ImportError: No module named fileutils

Original exception was:
Traceback (most recent call last):
  File "./a2.py", line 7, in 
  File "/usr/lib/python2.7/tempfile.py", line 454, in NamedTemporaryFile
  File "/usr/lib/python2.7/tempfile.py", line 235, in _mkstemp_inner
OSError: [Errno 24] Too many open files: '/tmp/tmpI0MIEW'

Patch:

@@ -452,7 +452,11 @@
 flags |= _os.O_TEMPORARY
 
 (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags)
-file = _os.fdopen(fd, mode, bufsize)
+try:
+file = _os.fdopen(fd, mode, bufsize)
+except Exception as ex:
+_os.close(fd)
+raise
 return _TemporaryFileWrapper(file, name, delete)
 
 if _os.name != 'posix' or _os.sys.platform == 'cygwin':

--
components: Library (Lib)
messages: 214778
nosy: tholzer
priority: normal
severity: normal
status: open
title: tempfile.NamedTemporaryFile leaks file descriptor when fdopen fails
type: resource usage
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue21058>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

tholzer added the comment:

We encountered the same problem, this is in the context of using PyQt 
(specifically QProcess) or twisted. They both rely on SIGCHLD for their 
notification framework.

I've attached a httplib EINTR patch for 2.6.4 & 2.7.3.

--
nosy: +tholzer
Added file: http://bugs.python.org/file35296/httplib_2.6.4_eintr_patch.py

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

Changes by tholzer :


Added file: http://bugs.python.org/file35297/httplib_2.7.3_eintr_patch.py

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

tholzer added the comment:

Here is a reproducible test case:

import threading
import signal
import os
import httplib

def killer():
while 1:
os.kill(os.getpid(), signal.SIGINT)

def go():
signal.signal(signal.SIGINT, lambda x,y: None)
thread = threading.Thread(target=killer)
thread.start()
while 1:
connection = httplib.HTTPConnection("localhost:80")
connection.connect()
connection.close()

if __name__ == '__main__':
go()

Which gives:

Traceback (most recent call last):
  File "./repro1.py", line 22, in 
go()
  File "./repro1.py", line 18, in go
connection.connect()
  File ".../lib/python2.7/httplib.py", line 757, in connect
self.timeout, self.source_address)
  File ".../lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error:

--

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

tholzer added the comment:

No problem, I've attached a patch for socket.py for Python 2.7.3.

A few notes:

getaddrinfo (and gethostbyname, etc.) are already immune to this bug, so I've 
just fixed the connect() call.

The socket does need to be closed after EINTR, otherwise a EINPROGRESS might 
get returned on subsequent connect() calls.

--
Added file: http://bugs.python.org/file35359/socket_2.7.3_eintr_patch.py

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

tholzer added the comment:

I've also attached a potential patch for the C module Modules/socketmodule.c 
inside internal_connect().

A few notes:

This seems to work both without time-out and with time-out sockets 
(non-blocking).

One concern would be a signal storm prolonging the operation beyond the 
time-out. Should we keep track of the actual time taken in this loop and check 
it against the 'timeout' parameter ?

Also, I don't think we can call PyErr_CheckSignals() in this context. Does this 
need to happen at all ?

--
Added file: http://bugs.python.org/file35360/socketmodule_2.7.6_eintr_patch.c

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

Changes by tholzer :


Added file: http://bugs.python.org/file35361/socketmodule_2.7.6_eintr_patch.c

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

Changes by tholzer :


Removed file: http://bugs.python.org/file35360/socketmodule_2.7.6_eintr_patch.c

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

tholzer added the comment:

Oops, I missed a break statement at the end of socket_2.7.3_eintr_patch.py. 
I've fixed this now in the attached patch.

@meishao

Could you please also update your socket_2_7_2_patch.py and add the missing 
break statement ?

--
Added file: http://bugs.python.org/file35406/socket_2.7.3_eintr_patch.py

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

Changes by tholzer :


Removed file: http://bugs.python.org/file35359/socket_2.7.3_eintr_patch.py

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20611] socket.create_connection() doesn't handle EINTR properly

[tholzer]

tholzer added the comment:

And a test case for smtplib:

import threading
import signal
import os
import smtplib

def go():
running = True
pid = os.getpid()

def killer():
while running: 
os.kill(pid, signal.SIGINT)

signal.signal(signal.SIGINT, lambda x,y: None)
thread = threading.Thread(target=killer)
thread.start()

while 1:
try:
smtplib.SMTP('localhost')
except Exception, ex:
running = False
raise

if __name__ == '__main__':
go()

Fails with:

socket.error: [Errno 4] Interrupted system call

--

___
Python tracker 
<http://bugs.python.org/issue20611>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7932] print statement delayed IOError when stdout has been closed

[tholzer]

tholzer added the comment:

It's still a problem in Python 2.7:

python -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr
0

But feel free to close as "won't fix", as this seems to be an edge case which 
might not be worth fixing.

--

___
Python tracker 
<http://bugs.python.org/issue7932>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com