[issue7900] posix.getgroups() failure on Mac OS X

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Michael: 
* which configure options do you use?
* which xcode version do you use? 
  (this shouldn't be relevant, I'm interested in what causes the dot 1
  suffix)
* If you use --enable-universalsdk: do you have the 10.4 SDK installed
  (should be installed in "$(xcode-select -print-path)/SDKs/")

I cannot reproduce this with r78205, OSX 10.6.2/10C540, gcc version 4.2.1 
(Apple Inc. build 5659), Xcode 3.2.2/10M2135.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7900] posix.getgroups() failure on Mac OS X

[Ronald Oussoren]

Ronald Oussoren  added the comment:

A related question: is this issue present in the 3.x trunk?

(BTW: feel free to assign all OSX related issues to me)

--
assignee:  -> ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7846] Fnmatch cache is never cleared during usage

[Andrew Clegg]

Andrew Clegg  added the comment:

Hi,

I used a global name for a couple of reasons: it is consistent with the cache 
itself and the size of the cache being defined in the same place; and because I 
wanted to allow the value to be modified by users.

I used the leading underscore to give a weak indication of internal use - for 
most users, it can be completely ignored. If someone wanted to modify the 
maximum size of the cache however, this would be a reasonable and safe thing 
for them to do.

Cheers

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7582] [patch] diff.py to use iso timestamp

[anatoly techtonik]

anatoly techtonik  added the comment:

Ok. Let's commit it at least to 2.7 - I'll create a separate issue for 
discussion of unified diff format later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1212900] Python segfaults on OpenBSD (tested 3.4 and 3.5)

[Stefan Krah]

Stefan Krah  added the comment:

It is a stack overflow, which can be prevented by setting

#define Py_DEFAULT_RECURSION_LIMIT 150

in Python/ceval.c.

Then the program behaves in the same way as on Linux (i.e. it swallows
the RuntimeError somewhere). Recommend closing.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4151] Separate build dir broken

[Matthias Klose]

Matthias Klose  added the comment:

current status with 2.7 alpha3:

FAIL: test_get_python_inc (distutils.tests.test_sysconfig.SysconfigTestCase)
--
Traceback (most recent call last):
  File 
"/home/packages/python/2.7/python2.7-2.7~a3/Lib/distutils/tests/test_sysconfig.py",
 line 47, in test_get_python_inc
self.assertTrue(os.path.isfile(python_h), python_h)
AssertionError: 
/home/packages/python/2.7/python2.7-2.7~a3/build-static/Include/Python.h

unsure about this one:
FAIL: test_finalize_options (distutils.tests.test_build.BuildTestCase)
--
Traceback (most recent call last):
  File 
"/home/packages/python/2.7/python2.7-2.7~a3/Lib/distutils/tests/test_build.py", 
line 24, in test_finalize_options
self.assertEquals(cmd.build_purelib, wanted)
AssertionError: 'build/lib.linux-i686-2.7' != 'build/lib'

--
nosy: +tarek
versions: +Python 2.7, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5217] testExtractDir (test.test_zipfile.TestWithDirectory) fails when python built with srcdir != builddir

[Matthias Klose]

Matthias Klose  added the comment:

this works with 2.7 alpha3, won't fix for 3.0

--
versions:  -Python 2.7, Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7334] ElementTree: file locking in Jython 2.5 (OSError on Windows)

[Florent Xicluna]

Florent Xicluna  added the comment:

The patch proposed on msg95349:
 * need tests
 * not reviewed (yet)

--
keywords: +patch
nosy: +flox
priority:  -> normal
stage:  -> needs patch
title: XML file locking in Jython 2.5 (OSError on Windows) -> ElementTree: file 
locking in Jython 2.5 (OSError on Windows)
Added file: http://bugs.python.org/file16231/issue7334_etree_patch.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1005895] curses for win32

[Jason Tishler]

Jason Tishler  added the comment:

Sorry, but I don't know. I haven't looked at this issue for almost five years! 
And when I did, I only looked as far to determine it wasn't Cygwin related.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7736] os.listdir hangs since opendir() and closedir() do not release GIL

[Nikolaus Rath]

Nikolaus Rath  added the comment:

Does this patch still need review? Both Martin and Antoine already commented 
that the patch is ok, so it'd be great if someone could actually apply it...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[R. David Murray]

R. David Murray  added the comment:

If by "fail" you mean "doesn't execute the command in the way I expected", then 
I believe you will be enlightened by reading the recent updates to the Popen 
documentation (see issue 6760).

--
nosy: +r.david.murray
priority:  -> normal
resolution:  -> duplicate
stage:  -> committed/rejected
status: open -> closed
superseder:  -> patch to subprocess docs to better explain Popen's 'args' 
argument
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[anatoly techtonik]

anatoly techtonik  added the comment:

I can't see changes to subprocess.call() docs in issue #6760. I reopen this 
bug, because call() definitely need a visible note about this significant 
behavior. I can not reopen #6760

http://docs.python.org/library/subprocess.html#convenience-functions

--
assignee:  -> georg.brandl
components: +Documentation
nosy: +georg.brandl
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[anatoly techtonik]

anatoly techtonik  added the comment:

I still do not agree. There should be a note in call() documentation, because 
users will waste their time experiencing the bug before reading documentation 
again.

If you can't make something just work without rereading the whole doc - that's 
not pythonic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7942] Inconsistent error types/messages for __len__ between old and new-style classes

[Paul Boddie]

New submission from Paul Boddie :

As noted here:

http://www.selenic.com/pipermail/mercurial/2010-February/030068.html

This is probably documented somewhere, and there may even be a good reason for 
the difference, but old-style classes raise TypeError when __len__ returns a 
non-int, whereas new-style classes raise OverflowError. The latter is probably 
just as valid, but the message is a bit obscure for debugging purposes.

Maybe this went away after 2.5 - if so, sorry for the noise!

Here's an illustration of the problem:

Python 2.5.4 (r254:67916, Nov  4 2009, 17:59:46)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
... def __len__(self):
... return 2**35
...
>>> c = C()
>>> len(c)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: __len__() should return an int
>>> class C(object):
... def __len__(self):
... return 2**35
...
>>> c = C()
>>> len(c)
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: long int too large to convert to int

--
components: Interpreter Core
messages: 99421
nosy: pboddie
severity: normal
status: open
title: Inconsistent error types/messages for __len__ between old and new-style 
classes
type: behavior
versions: Python 2.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[Brian Curtin]

Brian Curtin  added the comment:

"This module also defines two shortcut functions"

I think given that we say the calls are shortcuts, and that their arguments are 
the same as Popen, I take that to mean that subprocess.call experiences the 
same situations as subprocess.Popen.

If any change has to be made, I could get on board with changing "two shortcut 
functions" to "two Popen shortcut functions".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7943] Memory leak due to circular references in ssl.SSLSocket

[Péter Szabó]

New submission from Péter Szabó :

Here is how to reproduce the leak in Python 2.6.4 and Python 2.7:

  import gc
  import socket
  import ssl
  sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  gc.disable()
  gc.collect()
  count0 = gc.get_count()[0]
  for i in xrange(1000):
ssl.SSLSocket(sock)
  count1 = gc.get_count()[0]
  # This should be less than 100 without the leak, but it is >1000.
  diff = count1 - count0
  assert diff < 1000, diff

The reason why the memory usage has increased is that the SSLSocket objects 
contain a circular reference (self -> __init__ ->  recv -> lambda -> self), and 
thus Python cannot free them without garbage collection.

A side effect is that if the SSLSocket doesn't have external references, the 
underlying socket._realsocket may remain unclosed for a long time (until the 
garbage collection kicks in), because the SSLSocket still refers to it. Another 
side effect is that when there is an exception in the wrapping in 
SSLSocket.accept() (e.g. keyfile not found), then the newly accepted socket 
won't get closed immediately, and the client will experience a hanging open 
socket.

With gc.enable(), the leak goes away (the garbage collector finds and frees the 
SSLSocket objects with circular reference). Python 3.1.1 is not affected since 
it has a very different SSLSocket implementation.

Here is an easy fix: replace the self.recv = lambda ... lines in the code of 
ssl.SSLSocket.__init__ with this:

  import socket as socket_module
  for attr in socket_module._delegate_methods:
delattr(self, attr)  

See the attached patch. You can also download the patch from 
http://code.google.com/p/pts-mini-gpl/source/browse/trunk/patches/pts-python2.6.4-ssl-init-memory-leak-fix.patch

--
components: IO, Library (Lib)
files: pts-python2.6.4-ssl-init-memory-leak-fix.patch
keywords: patch
messages: 99423
nosy: Péter.Szabó
severity: normal
status: open
title: Memory leak due to circular references in ssl.SSLSocket
type: resource usage
versions: Python 2.6, Python 2.7
Added file: 
http://bugs.python.org/file16234/pts-python2.6.4-ssl-init-memory-leak-fix.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7944] Use the 'with' statement in conjunction with 'open' throughout test modules

[Dave Fugate]

New submission from Dave Fugate :

Sprinkled throughout CPython's test modules are snippets of code such as the 
following taken from 2.7A3's test_old_mailbox.py (line 141):
box = mailbox.UnixMailbox(open(self._path, 'r'))

The key thing to observe here is the file being opened yet never has it's 
'close' method explicitly called.  While this is fine for CPython's rather 
predictable garbage collections, it's quite possible that in alternate 
implementations of Python this file object won't be destroyed until well after 
line 141.  This can result in seemingly random failures of CPython's tests 
under alternate implementations of Python.

The solution to this problem would be to consistently use the 'with' statement 
(or alternatively close all open file objects) throughout all CPython test 
modules.

I've gone ahead and submitted an updated (2.7A3) version of test_old_mailbox.py 
which addresses this.  We can find other places where this is going on as well, 
but for the most part the tests already seem to be doing the right thing.

--
components: Tests
files: test_old_mailbox.py
messages: 99425
nosy: midnightdf
severity: normal
status: open
title: Use the 'with' statement in conjunction with 'open' throughout test 
modules
type: feature request
versions: Python 2.6, Python 2.7
Added file: http://bugs.python.org/file16235/test_old_mailbox.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7931] Interpreter does not terminate if daemonized while running multithreaded

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

libc.daemon() calls forks(), which duplicates only the main thread.
The other thread does not exists in the forked process, and the interpreter 
blocks while trying to wait for it...

Please use os.fork() instead, it has code to forget the threads created by the 
threading module (and clears some other locks as well, preventing deadlocks).
Or google for "daemonize python".

--
nosy: +amaury.forgeotdarc
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[R. David Murray]

R. David Murray  added the comment:

You can't even use call or check_call without referencing the Popen constructor 
documentation, so I'm not sure what you are advocating, Anatoly.  We certainly 
don't want to repeat the Popen documentation in full for each convenience 
function, and if we aren't doing that it seems to me that the current one line 
redirect to the full Popen docs is the right thing to do.  I have, however, 
turned those references into active links. so that all you have to do is click 
on Popen to get bounced up to the full docs.  (Change made in trunk in r78206, 
it will get copied to the other branches eventually.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7332] python script segment fault at PyMarshal_ReadLastObjectFromFile in import_submodule

[Thomas Smith]

Thomas Smith  added the comment:

I'm also getting segfaults in PyMarshal_ReadLastObjectFromFile in Python 2.6.2 
(on Ubuntu Jaunty).  It's very sporadic, I've been reproducing it by running a 
minimal script 100,000 times, and getting a few core dumps.  There are several 
Ubuntu bugreports in various packages that use Python:

https://bugs.launchpad.net/ubuntu/+source/apport/+bug/393022
https://bugs.launchpad.net/ubuntu/+source/gnome-python/+bug/432546
https://bugs.launchpad.net/ubuntu/+source/streamtuner/+bug/336331

I've attached a zip file with my test scripts and some gdb backtraces.  I am 
happy to spend time on this bug, although I only have a rudimentary knowledge 
of C, so I'd mainly be useful for testing.

The computer I'm having trouble on is a Dell PowerEdge T410, with a Xeon E5502, 
and it had another sporadic segfault problem in a should-be-reliable program, 
ImageMagick.  Switching to GraphicsMagick fixed that one, somehow.  If it's a 
hardware-specific bug, Python is the only program that's tickling it right 
now...

--
nosy: +Thomas.Smith
versions: +Python 2.6
Added file: http://bugs.python.org/file16236/traces.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7846] Fnmatch cache is never cleared during usage

[Éric Araujo]

Éric Araujo  added the comment:

Hi

Your rationale makes perfect sense. Perhaps add a comment suggesting 
it’s ok to change the value.

Cheers

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1212900] Python segfaults on OpenBSD (tested 3.4 and 3.5)

[Martin v . Löwis]

Changes by Martin v. Löwis :


--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7942] Inconsistent error types/messages for __len__ between old and new-style classes

[Martin v . Löwis]

Martin v. Löwis  added the comment:

I fail to see the bug in this report. What did you expect to happen instead?

--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7945] typo in operator.isCallable deprecation warning

[kenneth dombrowski]

New submission from kenneth dombrowski :

The operator documentation @ http://docs.python.org/library/operator.html 
states for operator.isCallable(obj): "Deprecated since version 2.0: Use 
isinstance(x, collections.Callable) instead.", I believe this should read since 
version 2.6

kenn...@dev2 ~ $ python
Python 2.5.4 (r254:67916, Oct  3 2009, 21:36:21)
[GCC 3.4.6 [FreeBSD] 20060305] on freebsd6
Type "help", "copyright", "credits" or "license" for more information.

>>> import collections
>>> collections.Callable
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'Callable'

--
assignee: georg.brandl
components: Documentation
messages: 99433
nosy: georg.brandl, kennethd
severity: normal
status: open
title: typo in operator.isCallable deprecation warning
versions: Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7944] Use the 'with' statement in conjunction with 'open' throughout test modules

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

Please use patches to show where the code changed.
(I generated one; it looks good to me)

I would like this kind of patches applied to CPython.
Looking at the Pypy test suite, the following test files seem easy to update:
test_bz2
test_dumbdbm
test_mailbox
test_marshal
test_mmap
test_pkgimport
test_tarfile
test_tempfile
test_traceback

--
keywords: +patch
nosy: +amaury.forgeotdarc
Added file: http://bugs.python.org/file16237/test_old_mailbox.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[anatoly techtonik]

anatoly techtonik  added the comment:

Thanks. That should be enough, although I wouldn't mind against more prominent 
notice to urge users to go and see Popen parameters. I would also add a second 
example to allow users see the difference. For instance, how to execute another 
Python script:

retcode = call("diff.py oldfile newfile", shell=True)
retcode = call([sys.executable], ["diff.py oldfile newfile"])

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[anatoly techtonik]

anatoly techtonik  added the comment:

The last one should be:
retcode = call([sys.executable, "diff.py oldfile newfile"])

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[R. David Murray]

R. David Murray  added the comment:

I think you should reread the Popen docs carefully :)

That example should be:

call([sys.executable, 'diff.py', 'oldfile', 'newfile'])

And there are examples of that call format in the Popen docs, as well as the 
explanation of exactly how to format the list correctly.  So again, I don't 
think it is a good idea to try to duplicate that in the call/check_call docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7946] Convoy effect with I/O bound threads and New GIL

[David Beazley]

New submission from David Beazley :

Background
---
In order to multitask with threads, a critical part of the Python
interpreter implementation concerns the behavior of I/O operations
such as read, write, send, and receive.  Specifically, whenever an I/O
operation is carried out, the interpreter releases the GIL so that
other threads can run while the program waits for the I/O operation to
complete.

Observed Behavior of I/O Operations

The release of the GIL for I/O is a critical aspect of making sure the
interpreter doesn't make all threads hang while waiting.  However, the
release of the GIL also assumes a worst-case scenario.  In practice,
a large number of I/O requests actually complete immediately with no
actual blocking.  For example, if a program is sending on a socket,
send() operations will typically complete immediately if buffer space
is available to accept the data.  Likewise, read() and recv()
operations may return immediately if data is already available in the
operating system.

For system calls that complete immediately, a thread quickly releases
and then almost immediately reacquires the GIL to continue running.
However, given that the I/O operation didn't block, the release of the
GIL was technically unnecessary in this case.

Behavior of the new GIL
---
A feature of the new Python GIL implementation is that the interpreter
no longer periodically signals waiting threads (e.g., the check
interval).  Instead, thread switching is based on a timed wait on a
condition variable. Should a timeout occur, a thread will indicate
that it wants the GIL and the currently running thread will be forced
to give it up.

Although this scheme solves the problem of CPU-bound threads
thrashing, it introduces a highly pronounced "convoy effect" when
CPU-bound threads and I/O bound threads get mixed together.  A major
part of the problem is caused by the bahvior of I/O as described
above.  Specifically, when an I/O bound thread executes an I/O call,
it always releases the GIL.  Since the GIL is released, a CPU bound
thread is now free to acquire the GIL and run.  However, if the I/O
call completes immediately (which is common), the I/O bound thread
immediately stalls upon return from the system call.  To get the GIL
back, it now has to go through the timeout process to force the
CPU-bound thread to release the GIL again.

It should be noted that the behavior described also occurs in Python
2, but due to the small check interval, an I/O bound thread that wants
the GIL back will tend to get it without having to wait very long.

Example
---
Here is a very simple example that illustrates the problem.  In this
example, there is one CPU-bound thread that hammers the CPU and there
is an I/O bound thread that handles some network communication (an
echo server):

# iotest.py
import time
import threading
from socket import *

# CPU-bound thread (just hammers the CPU)
def spin():
while True:
pass

# I/O-bound thread (an echo TCP server)
def echo_server():
s = socket(AF_INET, SOCK_STREAM)
s.setsockopt(SOL_SOCKET, SO_REUSEADDR,1)
s.bind(("",15000))
s.listen(1)
while True:
c,a = s.accept()
while True:
data = c.recv(8192)
if not data:
break
c.sendall(data)
c.close()
s.close()

# Launch the CPU-bound thread
t1 = threading.Thread(target=spin)
t1.daemon=True
t1.start()

# Run the I/O server
echo_server()

Here is a benchmark program that runs as a client for the echo_server()
thread defined above.  It sends a sequence of messages and reads the
response back.  It then reports some timings at the end.

# echoclient.py
from socket import *
import time

CHUNKSIZE = 16384
NUMMESSAGES = 640 # Total of 10MB

# Dummy message
msg = b"x"*CHUNKSIZE

# Connect and send messages
s = socket(AF_INET,SOCK_STREAM)
s.connect(("",15000))
start = time.time()
for n in range(NUMMESSAGES):
s.sendall(msg)
bytes_recv = len(msg)
# Get the response back
while bytes_recv > 0:
data = s.recv(bytes_recv)
bytes_recv -= len(data)
s.close()
end = time.time()
print("%0.3f seconds (%0.3f bytes/sec)" % (end-start, 
(CHUNKSIZE*NUMMESSAGES)/(end-start)))

Performance Results
---
These results are from running the above programs on a dual-core
MacBook running OS-X Snow Leopard.  I also get similar behavior on a
quad-core desktop machine.

If you run the iotest.py program using Python 2.6.4 and execute
the client, you get this result:

   bash % python echoclient.py
   1.064 seconds (9854148.739 bytes/sec)

If you switch the iotest.py to Python 3.2 and rerun, you get this
result:

   bash % python echoclient.py
   12.340 seconds (849726.150 bytes/sec)

Notice that there is a factor 12 performance difference.

Modify the iotest.py program so that there are 2 CPU-bound
threads spinning.  Just add this extra code:

t2 = threading.Thread(

[issue7945] typo in operator.isCallable deprecation warning

[Georg Brandl]

Georg Brandl  added the comment:

No, this is correct.  isCallable() was deprecated since 2.0, but the 
replacement was the builtin callable() until 2.6.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5088] optparse: inconsistent default value for append actions

[Andrew McNabb]

Andrew McNabb  added the comment:

I think that optparse is doing the right thing here.  I think that your code 
example should be changed to:

import optparse
  parser = optparse.OptionParser()
  parser.add_option("-o", "--option", action = "append")
  options, args = parser.parse_args()
  if not options.option:
options.option = ['a']
  print options

Think of the default as the initial list, and each time the -o option is 
specified, an item is appended to that initial list.

--
nosy: +amcnabb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7947] Documentation of math.pow, math.hypot, and complex.__abs__

[Mark Dickinson]

Mark Dickinson  added the comment:

Thanks for reporting this.

Rather than documenting corner cases explicitly, maybe it would be enough to 
point to the C99 standard:  the corner cases for the math functions should 
(modulo bugs) all follow Annex F of the C standard, while the corner cases for 
the cmath functions follow Annex G.  This doesn't quite tell the complete 
story, because some functions (for example the cmath.rect function) don't 
appear in the C standards, but it would avoid cluttering the function 
descriptions with special cases.

Certainly the "All functions ..." sentence is inaccurate, and should be removed 
or revised.  I believe it's true for all single-argument functions, though.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7947] Documentation of math.pow, math.hypot, and complex.__abs__

[Mark Dickinson]

Mark Dickinson  added the comment:

Hmm.  Looking at the docs, there are a number of revisions that would be 
useful.  For example, the docs mention signaling NaNs, but that doesn't make a 
lot of sense:  Python essentially treats all NaNs as quiet.  I'll add this to 
my to-do list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7942] Inconsistent error types/messages for __len__ between old and new-style classes

[Paul Boddie]

Paul Boddie  added the comment:

I would have expected a more accurate error message for the new-style class. In 
the original message which brought this to my attention, the cause was not 
particularly obvious:

http://www.selenic.com/pipermail/mercurial/2010-February/030066.html

I concede that the different mechanisms in place for new-style classes might 
make it hard to have a specific error message in such a situation, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6472] Update ElementTree with upstream changes

[Florent Xicluna]

Florent Xicluna  added the comment:

Update the 2.x patch with the last version uploaded to rietveld (patch set 5).

Improved test coverage with upstream tests and tests cases provided by Neil on 
issue #6232.

Note: the patch for 3.x is obsolete.

--
Added file: http://bugs.python.org/file16239/issue6472_etree_upstream_v5.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7930] pydoc.stripid doesn't strip ID in py25, py26, py31

[Ezio Melotti]

Ezio Melotti  added the comment:

Fixed in r78207 (trunk), r78208 (release26-maint), r78209 (py3k) and r78210 
(release31-maint).
After a short discussion on #python-dev I decided to remove the 'if', because 
it's not necessary and might creates problems with other implementations.

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5088] optparse: inconsistent default value for append actions

[R. David Murray]

R. David Murray  added the comment:

I agree with Andrew, but I think the optparse documentation should make it 
clear that this is what happens with a default value for an 'append' action.  
It is *implied* by the fact that append says it appends to a list, and default 
says the value is set before any options are parsed, but it should be made 
explicit in the docs for the 'append' action.  Doc patch welcomed.

--
assignee:  -> georg.brandl
components: +Documentation -Library (Lib)
nosy: +georg.brandl, r.david.murray
priority:  -> normal
versions: +Python 2.7, Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7942] Inconsistent error types/messages for __len__ between old and new-style classes

[R. David Murray]

R. David Murray  added the comment:

In the example you give it seems to me that the message for the new style class 
is more accurate and useful than the message for the old style class.  Looking 
at your mercurial example, it looks like the problem is the same (len returning 
a long), and again it seems to me that the new message is more accurate than 
the old, and that neither one is more confusing than the other in the given 
context(s).  Or, if anything, that the OverflowError is less confusing as it 
has more connection with the problem (committing *large* amounts of data).

I don't see this as a bug in any sense, myself.  What would consider to be a 
"specific" error message?  Perhaps this is a request for an enhancement, 
instead.

--
nosy: +r.david.murray
priority:  -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7005] ConfigParser does not handle options without values

[Fred L. Drake, Jr.]

Fred L. Drake, Jr.  added the comment:

Assigning to myself for handling.

Bumping to Python 2.7 / 3.2 since support for this syntax variation is a new 
feature.

--
assignee:  -> fdrake
versions: +Python 2.7, Python 3.2 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7948] Complete your registration to Python tracker -- key yAMyUeuHR9LS2Q86aUUs2GO4rujOhgWH

[panzi]

New submission from panzi :

On 02/17/2010 02:35 AM, Python tracker wrote:
> To complete your registration of the user "panzi" with
> Python tracker, please do one of the following:
>
> - send a reply to rep...@bugs.python.org and maintain the subject line as is 
> (the
> reply's additional "Re:" is ok),
>
> - or visit the following URL:
>
> http://bugs.python.org/?...@action=confrego&otk=yAMyUeuHR9LS2Q86aUUs2GO4rujOhgWH
>

--
messages: 99455
nosy: panzi
severity: normal
status: open
title: Complete your registration to Python tracker -- key  
yAMyUeuHR9LS2Q86aUUs2GO4rujOhgWH

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7948] Complete your registration to Python tracker -- key yAMyUeuHR9LS2Q86aUUs2GO4rujOhgWH

[Brian Curtin]

Changes by Brian Curtin :


--
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7949] idle does not handle dark gtk color schemes

[Mathias Panzenböck]

Mathias Panzenböck  added the comment:

I just noticed that the debugger also suffers from this problem. Doing a quick 
grep over the source reveals even more places that might cause problems (where 
only the bg or only the fg color is set).

I think in ScrolledList.py the background="white" should just be removed. This 
and setting the fg color to black when the bg color is set to yellow 
(Debugger.py) makes the debugger usable.

Checkboxes and radiobuttons seem to be messed up in general (forced white bg 
but using system fg color (white on white on my system); not idles but Tkinters 
or TKs fault).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7946] Convoy effect with I/O bound threads and New GIL

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Just a quick test under Linux (on a dual quad core machine):
- with iotest.py and echo_client.py both running Python 2.7: 25.562 seconds 
(410212.450 bytes/sec)
- with iotest.py and echo_client.py both running Python 3.2: 28.160 seconds 
(372362.459 bytes/sec)

As already said, the "spinning endlessly" loop is a best case for thread 
switching latency in 2.x, because the opcodes are very short. If each opcode in 
the loop has an average duration of 20 ns, and with the default check interval 
of 100, the GIL gets speculatively released every 2 us (yes, microseconds). 
That's why I suggested trying more "realistic" workloads, as in ccbench.

Also, as I told you, there might also be interactions with the various timing 
heuristics the TCP stack of the kernel applies. It would be nice to test with 
UDP.

That said, the observations are interesting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3132] implement PEP 3118 struct changes

[Meador Inge]

Meador Inge  added the comment:

Hi All,

On Sat, Feb 13, 2010 at 5:07 AM, Mark Dickinson wrote:

>
> Mark Dickinson  added the comment:
>
> Some of the proposed struct module additions look far from straightforward;
>  I find that section of the PEP significantly lacking in details and
> motivation.
>

I agree.

> "Unpacking a long-double will return a decimal object or a ctypes
> long-double."
>
> Returning a Decimal object here doesn't make a lot of sense, since Decimal
> objects aren't generally compatible with floats.  And ctypes long double
> objects don't seem to exist, as far as I can tell.  It might be better not
> to add this code.

And under what conditions would a ctype long double be used vs. a Decimal
object.

>

Another bit that's not clear to me:  how is unpacking an object pointer
> expected to work, and how would it typically be used?  What if the unpacked
> pointer no longer points to a valid Python object?  How would this work in
> other Python implementations?
>

I guess if an object associated with the packed address does not exist, then
you would unpack None (?).  This is especially a problem if the struct-sting
is being sent over the wire to another machine.

> For the 'X{}' format (pointer to a function), is this supposed to mean a
> Python function or a C function?
>

I read that as a Python function.  However, I am not completely sure how the
prototype would be enforced when unpacking.  I am also wondering, though,
how the signatures on pointers-to-functions are specified?  Are
the arguments and return type full struct strings as well?

> What's a 'specific pointer'?

I think this means a pointer to a specific type, e.g. '&d' is a pointer to a
double. If this is the case, though, the use cases are not completely clear
to me.

I also have the following questions:

* Can pointers be nested, '&&d' ?
* What nesting level can structures have? Arbitrary?
* The new array syntax claims "multi-dimensional array of whatever follows".

  Truly whatever? Arrays of structures? Arrays of pointers?
* "complex (whatever the next specifier is)".  Not really 'whatever'.  You
  can not have a 'complex bool' or 'complex int'.  What other types of
  complex are there besides complex double?
* How do array specifiers and pointer specifiers mix?  For example, would
  '(2, 2)&d' be a two-by-two array of pointers to doubles?  What about
  '&(2, 2)d'?  Is this a pointer to an two-by-two array of doubles?

The new features of the struct-string syntax are so different that I think
we
need to specify a grammar.  I think it will clarify some of the open
questions.

In addition, I was thinking that a reasonable implemenation strategy would
be to keep the current struct-string syntax mostly in place within the C
module
implementation.  The C implementation would just provide an interface to
pack\unpack sequences of primitive data elements.  Then we could write a
layer in the Python 'struct' module that took care of the higher-order
concepts like nested structures, arrays, named values, and pointers to
functions.  The higher-order concepts would be mapped to the appropriate
primitive sequence strings.

I think this will simplify the implementation and will provide a way to
phase
it.  We can implement the primitive type extensions in C first followed by
the higher-level Python stuff.  The result of each phase is immediately
usuable.

I have attached a patch against the PEP containing my current thoughts on
fleshing out the grammar and some of the current open questions.  This still
needs work, but I wanted to share to see if I am on the right track.
 Please advise on how to proceed.

--
keywords: +patch
Added file: http://bugs.python.org/file16241/unnamed
Added file: http://bugs.python.org/file16242/pep-3118.patch

___
Python tracker 

___Hi All,On Sat, Feb 13, 2010 at 5:07 AM, Mark 
Dickinson rep...@bugs.python.org> 
wrote:

Mark Dickinson dicki...@gmail.com> added the 
comment:

Some of the proposed struct module additions look far from straightforward;  I 
find that section of the PEP significantly lacking in details and 
motivation. I 
agree. 

"Unpacking a long-double will return a decimal object or a ctypes 
long-double."

Returning a Decimal object here doesn't make a lot of sense, since Decimal 
objects aren't generally compatible with floats.  And ctypes long double 
objects don't seem to exist, as far as I can tell.  It might be better not 
to add this code.
And under what conditions would a ctype long double be used 
vs. a Decimal object. 
 
Another bit that's not clear to me:  how is unpacking an object pointer 
expected to work, and how would it typically be used?  What if the unpacked 
pointer no longer points to a valid Python object?  How would this work in 
other Python implementations?
I guess if an object assoc

[issue7946] Convoy effect with I/O bound threads and New GIL

[David Beazley]

David Beazley  added the comment:

The comment on the CPU-bound workload is valid--it is definitely true that 
Python 2.6 results will degrade as the workload of each tick is increased.   
Maybe a better way to interpreter those results is as a baseline of what kind 
of I/O performance is possible if there is a quick I/O response time.

However, ignoring that and the comparison between Python 2.6 and 3.2, there is 
still a serious performance issue with I/O in 3.2.  For example, the dramatic 
decrease in I/O performance as there are more CPU bound threads competing and 
the fact that there is a huge performance gain when all but one CPU core is 
disabled. 

I tried the test using UDP packets and get virtually the exact same behavior 
described.  For instance, echoing 10MB (sent in 8k UDP packets) takes about 
0.6s in Python 2.6 and 12.0s in Python-3.2.   The time shoots up to more than 
40s if there are two CPU-bound threads. 

The problem being described really doesn't have anything to do with TCP vs. UDP 
or any part of the network stack.  It has everything to do with how the 
operating system buffers I/O requests and how I/O operations such as sends and 
receives complete immediately without blocking depending on system buffer 
characteristics (e.g., if there is space in the send buffer, a send will return 
immediately without blocking).   The fact that the GIL is released when it's 
not necessary in these cases is really the source of the problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7939] subprocess: call([arg, arg2], shell=True) vs call(arg+" "+arg2, shell=True)

[anatoly techtonik]

anatoly techtonik  added the comment:

You are right - I should read the docs, but one of the outstanding features of 
Python is that in many cases you can test the code faster than read 
documentation. =) It would help if ["ls", "-la"] example included third 
argument thus fully covering the concept.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com