[issue7060] test_multiprocessing dictionary changed size errors and hang

[STINNER Victor]

STINNER Victor added the comment:

I never seen this issue, can we close it?

#7105 has been fixed in Python 3, not in Python 2.7.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7105] weak dict iterators are fragile because of unpredictable GC runs

[STINNER Victor]

STINNER Victor added the comment:

What is the status of this issue? Does anyone still want to backport the fix to 
Python 2.7?

(I found this issue while searching for test_multiprocessing failures, and I 
found #7060 which refers this issue.)

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19633] test_wave: failures on PPC64 buildbot

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

array's constructor interprets its second memoryview argument as an iterable of 
integers.

>>> import array
>>> array.array('h', b'abcd')
array('h', [25185, 25699])
>>> array.array('h', memoryview(b'abcd'))
array('h', [97, 98, 99, 100])

array.frombytes() always interpret its argument as bytes-like object.

>>> a = array.array('h')
>>> a.frombytes(memoryview(b'abcd'))
>>> a
array('h', [25185, 25699])

First patch fixes only a half of the issue. test_unseekable_incompleted_write() 
still failed because array.fromfile() fails read incomplete data. Second patch 
also adds unittest.expectedFailure decorators for these tests.

--
Added file: http://bugs.python.org/file32680/wave_byteswap_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19633] test_wave: failures on PPC64 buildbot

[STINNER Victor]

STINNER Victor added the comment:

> array's constructor interprets its second memoryview argument as an iterable 
> of integers.

Ok so, so your fix is correct.

> First patch fixes only a half of the issue. 
> test_unseekable_incompleted_write() still failed because array.fromfile() 
> fails read incomplete data.

Why the test succeeded on little endian?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19634] test_strftime.test_y_before_1900_nonwin() fails on AIX

[STINNER Victor]

STINNER Victor added the comment:

Ok, the test passed with the second fix.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19633] test_wave: failures on PPC64 buildbot

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Why the test succeeded on little endian?

Because array.fromfile() is used only to swap 16- and 32-bit samples on 
bigendian platform.

Perhaps we need the byteswap() function in the audioop module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10158] BadInternalCall running test_multiprocessing

[STINNER Victor]

STINNER Victor added the comment:

Even with PyTrash_UNWIND_LEVEL in Include/object.h defined to 10 instead of 50, 
I cannot reproduce the issue on the 2.7 branch of Mercurial. I guess that the 
bug was already fixed in Python 2.7 and this issue has no activity since 3 
years, so I'm closing it.

--
nosy: +haypo
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16158] sporadic test_multiprocessing failure

[STINNER Victor]

STINNER Victor added the comment:

I didn't see this failure recently. The issue doesn't contain useful 
information, and has no activity since more than 1 year, so I'm closing it. 
Reopen the issue with more information if it was reproduced recently.

--
nosy: +haypo
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19633] test_wave: failures on PPC64 buildbot

[STINNER Victor]

STINNER Victor added the comment:

> Because array.fromfile() is used only to swap 16- and 32-bit samples on 
> bigendian platform.

If the file is truncated, why would the test suceed on little endian? The file 
doesn't have the same size in bytes? The test doesn't check the number of 
frames?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18887] test_multiprocessing.test_connection failure on Python 2.7

[STINNER Victor]

Changes by STINNER Victor :


--
title: test_multiprocessing.test_connection failure -> 
test_multiprocessing.test_connection failure on Python 2.7
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19565] test_multiprocessing_spawn: RuntimeError and assertion error on windows xp buildbot

[STINNER Victor]

STINNER Victor added the comment:

The initial issue (RuntimeError messages) has been fixed, I'm closing the issue.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19564] test_context() of test_multiprocessing_spawn hangs on "x86 Gentoo Non-Debug 3.x" buildbot

[STINNER Victor]

Changes by STINNER Victor :


--
title: test_multiprocessing_spawn hangs -> test_context() of 
test_multiprocessing_spawn hangs on "x86 Gentoo Non-Debug 3.x" buildbot

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13090] test_multiprocessing: memory leaks

[STINNER Victor]

STINNER Victor added the comment:

If the result of os.read() was stored in a Python daemon thread, the memory 
should be released since the following changeset. Can someone check if this 
issue still exist?

changeset:   87070:c2a13acd5e2b
user:Victor Stinner 
date:Tue Nov 12 16:37:55 2013 +0100
files:   Lib/test/test_threading.py Misc/NEWS Python/pythonrun.c
description:
Close #19466: Clear the frames of daemon threads earlier during the Python
shutdown to call objects destructors. So "unclosed file" resource warnings are
now corretly emitted for daemon threads.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19564] test_context() of test_multiprocessing_spawn hangs on "x86 Gentoo Non-Debug 3.x" buildbot

[STINNER Victor]

STINNER Victor added the comment:

See issue #12413 which may help to analyze this hang.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19634] test_strftime.test_y_before_1900_nonwin() fails on AIX

[Tim Golden]

Tim Golden added the comment:

Thanks, guys; I'm afraid I only watched the stable buildbots when I committed - 
the AIX doesn't seem to be included.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19564] test_context() of test_multiprocessing_spawn hangs on "x86 Gentoo Non-Debug 3.x" buildbot

[STINNER Victor]

STINNER Victor added the comment:

multiprocessing_faulthandler_timeout.patch: Dump the traceback of all Python 
threads in a child process if it takes more than 10 minutes to succeed. On the 
buildbot, the process hangs more than 1 hour. I guess that 10 minutes in enough 
for a single unit test.

The patch adds a private attribute "_faulthandler_timeout" to Popen classes, 
but also a "_faulthandler_timeout" parameter to spawn_main(). If you think that 
it is really too ugly, the patch may be reverted after the bug is analyzed. Or 
would it be possible to pass a command to "prepare" a child process, like 
preexec_fn of the subprocess module?

--
keywords: +patch
nosy: +neologix
Added file: 
http://bugs.python.org/file32681/multiprocessing_faulthandler_timeout.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19617] Fix usage of Py_ssize_t type in Python/compile.c

[STINNER Victor]

STINNER Victor added the comment:

> You could use the Py_SAFE_DOWNCAST macro everywhere.

I prefer to store sizes in a size type (Py_ssize_t), and only downcast where it 
is really needed, in compiler_addop_i(). So in the future, if someone wants to 
support values larger than INT_MAX, only one function need to be changed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[Christian Heimes]

New submission from Christian Heimes:

The patch silences three compiler warnings on Win64

--
components: Windows
files: dtoa.patch
keywords: patch
messages: 203264
nosy: christian.heimes, haypo, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: dtoa: conversion from '__int64' to 'int', possible loss of data
type: compile error
versions: Python 3.4
Added file: http://bugs.python.org/file32683/dtoa.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2927] expose html.parser.unescape

[Ezio Melotti]

Changes by Ezio Melotti :


--
assignee:  -> ezio.melotti
versions: +Python 3.4 -Python 3.2
Added file: http://bugs.python.org/file32682/issue2927.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

[STINNER Victor]

STINNER Victor added the comment:

> Your patch implies that the two only supported OSes are Linux and Windows :-)

It more means that Windows memory allocator is different to the one used on all 
other operating systems.

Well, if you are not convinced, we can keep the overallocation factor of 25%: 
performances are not so bad. The difference between 25% and 50% is low.

--

Benchmark on patched repr(list) (to use PyUnicodeWriter, see issue #19513) 
using different overallocation factors on Linux. The best factor is 25% (the 
current factor).


Platform: Linux-3.9.4-200.fc18.x86_64-x86_64-with-fedora-18-Spherical_Cow
Bits: int=32, long=64, long long=64, size_t=64, void*=64
Timer: time.perf_counter
SCM: hg revision=00348c0518f8+ tag=tip branch=default date="2013-11-18 10:04 
+0100"
Python unicode implementation: PEP 393
CFLAGS: -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes
CPU model: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Timer info: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)
Python version: 3.4.0a4+ (default:00348c0518f8+, Nov 18 2013, 10:11:42) [GCC 
4.7.2 20121109 (Red Hat 4.7.2-8)]
Timer precision: 40 ns

-+-+++--
Tests    |   writer-25 |    writer-12.5 |  writer-50 |  
  writer-100
-+-+++--
list("a")    |  247 ns (*) |  272 ns (+10%) |   265 ns (+7%) | 
283 ns (+14%)
list("abc")  |  435 ns (*) |   407 ns (-7%) | 430 ns |  
  427 ns
["a"]*(100)  | 8.26 us (*) | 8.4 us |  8.76 us (+6%) |  
 7.93 us
["abc"]*(100)    | 7.97 us (*) | 8.75 us (+10%) | 9.19 us (+15%) |  
 8.37 us
["a" * 100]*(100)    | 35.7 us (*) |    36.8 us |    36.6 us |  
 35.6 us
["a"]*(10**6)    |   73 ms (*) |  77.4 ms (+6%) | 84.9 ms (+16%) | 
78.3 ms (+7%)
["abc"]*(10**6)  | 76.6 ms (*) |  81.1 ms (+6%) | 90.3 ms (+18%) | 
82.4 ms (+8%)
["a" * 100]*(10**5)  | 35.1 ms (*) |    35.3 ms |    35.8 ms | 
37.6 ms (+7%)
list(range(10**6))   | 93.4 ms (*) |  102 ms (+10%) |  103 ms (+10%) | 
105 ms (+12%)
list(map(str, range(10**6))) |   97 ms (*) |    94.1 ms |    99.7 ms | 
91.2 ms (-6%)
-+-+++--
Total    |  375 ms (*) | 390 ms |  414 ms (+10%) |  
394 ms (+5%)
-+-+++--

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9566] Compilation warnings under x64 Windows

[STINNER Victor]

STINNER Victor added the comment:

I forgot to mention this issue in the following changesets.

changeset:   87128:f0a7924fac56
user:Victor Stinner 
date:Fri Nov 15 23:16:15 2013 +0100
files:   Modules/_randommodule.c
description:
Fix compiler warning on Windows 64-bit: explicit cast size_t to unsigned long


changeset:   87129:8adbd8a3a626
user:Victor Stinner 
date:Fri Nov 15 23:21:11 2013 +0100
files:   Modules/_sre.c
description:
Fix compiler warning (especially on Windows 64-bit): don't truncate Py_ssize_t
to int


changeset:   87130:baab11a466ab
user:Victor Stinner 
date:Fri Nov 15 23:26:25 2013 +0100
files:   Python/random.c
description:
Fix compiler warning in win32_urandom(): explicit cast to DWORD in
CryptGenRandom()


changeset:   87131:e086cb1c6e5a
user:Victor Stinner 
date:Sat Nov 16 00:13:29 2013 +0100
files:   Python/marshal.c
description:
Fix compiler warning (on Windows 64-bit): explicit cast Py_ssize_t to unsigned
char, n is in range [0; 255] (a tuple cannot have a negative length)


changeset:   87132:cf399d13a707
user:Victor Stinner 
date:Sat Nov 16 00:16:58 2013 +0100
files:   Include/asdl.h
description:
Fix compiler warning on Windows 64-bit: asdl_seq_SET() stores the index 
parameter
into a Py_ssize_t, instead of an int


changeset:   87133:309d855ebc3e
user:Victor Stinner 
date:Sat Nov 16 00:17:22 2013 +0100
files:   Modules/_ctypes/_ctypes.c
description:
Fix compiler warning on Windows 64 bit: _init_pos_args() result type is
Py_ssize_t, not int


changeset:   87134:7cd4c3e9e310
user:Victor Stinner 
date:Sat Nov 16 00:18:58 2013 +0100
files:   Modules/socketmodule.c
description:
Fix sock_recvfrom_guts(): recvfrom() size is limited to an int on Windows, not
on other OSes!


changeset:   87135:9e25367095c4
user:Victor Stinner 
date:Sat Nov 16 00:27:16 2013 +0100
files:   Modules/_hashopenssl.c
description:
Fix compiler warnings on Windows 64 bit: add an explicit cast from Py_ssize_t
to int, password.len was checked for being smaller than INT_MAX.


changeset:   87229:ac4272df1af6
user:Victor Stinner 
date:Mon Nov 18 01:07:38 2013 +0100
files:   Parser/grammar.c
description:
Fix compiler warnings on Windows 64-bit in grammar.c

INT_MAX states and labels should be enough for everyone


changeset:   87230:19e900e3033f
user:Victor Stinner 
date:Mon Nov 18 01:09:51 2013 +0100
files:   Parser/parsetok.c
description:
Fix a compiler warning on Windows 64-bit in parsetok.c

Python parser doesn't support lines longer than INT_MAX bytes yet


changeset:   87231:103998db4407
user:Victor Stinner 
date:Mon Nov 18 01:21:12 2013 +0100
files:   Python/getargs.c
description:
Use Py_ssize_t type for sizes in getargs.c

Fix compiler warnings on Windows 64-bit


changeset:   87232:0f7f1f2121a1
user:Victor Stinner 
date:Mon Nov 18 01:24:31 2013 +0100
files:   Modules/_sqlite/connection.c
description:
sqlite: raise an OverflowError if the result is longer than INT_MAX bytes

Fix a compiler warning on Windows 64-bit


changeset:   87233:855e172bcac4
user:Victor Stinner 
date:Mon Nov 18 01:27:30 2013 +0100
files:   Modules/_sqlite/row.c
description:
Fix a compiler warning on Windows 64-bit: _sqlite module


changeset:   87234:40d25b2b93f0
user:Victor Stinner 
date:Mon Nov 18 01:36:29 2013 +0100
files:   Modules/_sqlite/statement.c
description:
sqlite: raise an OverflowError if a string or a BLOB is longer than INT_MAX
bytes

Fix compiler warnings on Windows 64-bit


changeset:   87235:d1dc7888656f
user:Victor Stinner 
date:Mon Nov 18 02:05:31 2013 +0100
files:   Python/getargs.c
description:
PY_FORMAT_SIZE_T should not be used with PyErr_Format(), PyErr_Format("%zd") is
portable


changeset:   87236:2a01ca4b0edc
user:Victor Stinner 
date:Mon Nov 18 02:07:29 2013 +0100
files:   Modules/_sqlite/statement.c
description:
sqlite: Use Py_ssize_t to store a size instead of an int

Fix a compiler warning on Windows 64-bit

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Bernt Røskar Brenna added the comment:

I just tested on 3.4a0.

Observed the following changes: 

- subprocess_devnull now NEVER fails.
- subprocess_redirfile does not fail as often as before, but still fails.

I changed the number of tasks to 20 and increased max_workers to 5 to get 
subprocess_redirfile to fail at least one of twenty times every time I invoked 
the test script.

A typical result on 3.4 looks like this:

Platform: Windows-XP-5.1.2600-SP3

task_type #threads   result
subprocess_redirfile  5  3 errors
subprocess_redirfile  1  OK
subprocess_devnull5  OK
subprocess_devnull1  OK
subprocess_noredirect 5  OK
subprocess_noredirect 1  OK
nosubprocess  5  OK
nosubprocess  1  OK

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19639] Improve re match objects display

[Claudiu.Popa]

New submission from Claudiu.Popa:

When issue17087 was accepted, the documentation wasn't changed to reflect the 
new changes. The attached patch fixes this, replacing the old <_sre.Match 
object ...> with the deterministic repr of the match object.

--
assignee: docs@python
components: Documentation
files: re_doc.patch
keywords: patch
messages: 203268
nosy: Claudiu.Popa, docs@python
priority: normal
severity: normal
status: open
title: Improve re match objects display
versions: Python 3.4
Added file: http://bugs.python.org/file32684/re_doc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Bernt Røskar Brenna added the comment:

Increasing max_workers to 5 and running 20 tasks highlighs the 3.3-3.4 
difference (code in attached file testscript4.py):

Version: 3.4.0a4 (v3.4.0a4:e245b0d7209b, Oct 20 2013, 19:23:45) [MSC v.1600 32 b
it (Intel)]
Platform: Windows-XP-5.1.2600-SP3
Tasks: 20

task_type #threads   result
subprocess_redirfile  5  4 errors (of 20)
subprocess_redirfile  1  OK
subprocess_devnull5  OK
subprocess_devnull1  OK
subprocess_noredirect 5  OK
subprocess_noredirect 1  OK
nosubprocess  5  OK
nosubprocess  1  OK



Version: 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (
Intel)]
Platform: Windows-XP-5.1.2600-SP3
Tasks: 20

task_type #threads   result
subprocess_redirfile  5  18 errors (of 20)
subprocess_redirfile  1  OK
subprocess_devnull5  19 errors (of 20)
subprocess_devnull1  OK
subprocess_noredirect 5  OK
subprocess_noredirect 1  OK
nosubprocess  5  OK
nosubprocess  1  OK

--
Added file: http://bugs.python.org/file32685/testscript4.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19640] Drop _source attribute of namedtuple

[STINNER Victor]

New submission from STINNER Victor:

The definition of a new nametuple creates a large Python script to create the 
new type. The code stores the code in a private attribute:

namespace = dict(__name__='namedtuple_%s' % typename)
exec(class_definition, namespace)
result = namespace[typename]
result._source = class_definition

This attribute wastes memory, I don't understand the purpose of the attribute. 
It was not discussed in an issue, so I guess that there is no real use case:

changeset:   68879:bffdd7e9265c
user:Raymond Hettinger 
date:Wed Mar 23 12:52:23 2011 -0700
files:   Doc/library/collections.rst Lib/collections/__init__.py 
Lib/test/test_collections.py
description:
Expose the namedtuple source with a _source attribute.

Can we just drop this attribute to reduce the Python memory footprint?

--
messages: 203270
nosy: haypo, rhettinger
priority: normal
severity: normal
status: open
title: Drop _source attribute of namedtuple
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19640] Drop _source attribute of namedtuple

[STINNER Victor]

STINNER Victor added the comment:

I found this issue while using my tracemalloc module to analyze the memory 
consumption of Python. On the Python test suite, the _source attribute is the 
5th line allocating the memory memory:

/usr/lib/python3.4/collections/__init__.py: 676.2 kB

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19639] Improve re match objects display

[Ezio Melotti]

Changes by Ezio Melotti :


--
assignee: docs@python -> ezio.melotti
nosy: +ezio.melotti
stage:  -> patch review
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue513840] entity unescape for sgml/htmllib

[Ezio Melotti]

Changes by Ezio Melotti :


--
assignee:  -> ezio.melotti
resolution:  -> duplicate
stage: test needed -> committed/rejected
status: open -> closed
superseder:  -> expose html.parser.unescape

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Changes by Bernt Røskar Brenna :


--
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19640] Drop _source attribute of namedtuple

[Christian Heimes]

Changes by Christian Heimes :


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19633] test_wave: failures on PPC64 buildbot

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> If the file is truncated, why would the test suceed on little endian? The
> file doesn't have the same size in bytes? The test doesn't check the number
> of frames?

Because this code is not used on little endian. On little endian a data is 
read by file's read() method.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19641] Add audioop.byteswap()

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The audio modules often need convert between little endian and big endian data. 
The array module can be used to byteswap 16- and 32-bit samples, but it can't 
help with 24-bit samples. Python implemented function for swapping bytes is not 
very efficient. In any case the use of array is not so simple (see issue19276, 
issue19633).

The proposed patch adds efficient byteswap() function in the audioop module. 
byteswap(fragment, width) byteswaps every "width"-byte sample in the fragment 
and returns modified data.

--
components: Library (Lib)
files: audioop_byteswap.patch
keywords: patch
messages: 203273
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Add audioop.byteswap()
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file32686/audioop_byteswap.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19640] Drop _source attribute of namedtuple

[Eric V. Smith]

Changes by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8402] Add a function to escape metacharacters in glob/fnmatch

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Updated patch addresses Eric's comment.

--
Added file: http://bugs.python.org/file32687/glob_escape_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19564] test_context() of test_multiprocessing_spawn hangs on "x86 Gentoo Non-Debug 3.x" buildbot

[Richard Oudkerk]

Richard Oudkerk added the comment:

I don't think the patch to the _test_multiprocessing will work.  It defines 
cls._Popen but I don't see how that would be used by cls.Pool to start the 
processes.

I will have a think about a fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13633] Handling of hex character references in HTMLParser.handle_charref

[Ezio Melotti]

Changes by Ezio Melotti :


--
dependencies: +expose html.parser.unescape

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Changes by Bernt Røskar Brenna :


--
components: +IO

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[STINNER Victor]

STINNER Victor added the comment:

Ok, this issue is the corner case of the PEP 446:
http://www.python.org/dev/peps/pep-0446/#only-inherit-some-handles-on-windows

The PEP explicitly does nothing for this case. It can change in the future.

Until the point is fixed, you have to use a lock around the code spawning new 
processes to avoid that two threads spawn two processes and inherit unexpected 
files.

Example: Thread 1 creates file 1, thread 2 creates file 2, child process 
inherits files 1 and 2, instead of just file 1.

Richard proposed to use a trampoline process, the parent process would it the 
handles to inherit. Since Windows Vista, the trampoline process is no more 
needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8402] Add a function to escape metacharacters in glob/fnmatch

[Eric V. Smith]

Eric V. Smith added the comment:

Looks good to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[Stefan Krah]

Changes by Stefan Krah :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8402] Add a function to escape metacharacters in glob/fnmatch

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5fda36bff39d by Serhiy Storchaka in branch 'default':
Issue #8402: Added the escape() function to the glob module.
http://hg.python.org/cpython/rev/5fda36bff39d

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8402] Add a function to escape metacharacters in glob/fnmatch

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you Ezio and Eric for your reviews.

--
assignee:  -> serhiy.storchaka
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Bernt Røskar Brenna added the comment:

I tested, and locking around the subprocess.Popen call indeed works on Python 
3.4. 

Thanks!

Do you have any tips on how to accomplish the same thing on Python 3.3 (locking 
around Popen did not make any difference on 3.3)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13090] test_multiprocessing: memory leaks

[Stefan Krah]

Stefan Krah added the comment:

The leaks still exist here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Bernt Røskar Brenna added the comment:

Never mind, I figured it:

On Python 3.3, the combination of locking around Popen and opening the file 
that I redirect to using the code below works (code from scons):

def open_noinherit(*args, **kwargs):
fp = open(*args, **kwargs)
win32api.SetHandleInformation(msvcrt.get_osfhandle(fp.fileno()),
  win32con.HANDLE_FLAG_INHERIT,
  0)
return fp

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19642] shutil to support equivalent of: rm -f /dir/*

[Ivan Radic]

New submission from Ivan Radic:

Shutil supports deleting of files and directories but it offers no way to 
delete directory content (without deleting directory itself). Shell programming 
includes a lot of "rm -rf /dir" and "rm -f /dir/*", and there is no direct 
equivalent of these commands in Python. Educate me if I am wrong on this claim.

Sure, I can write a quick for loop, and delete each subfile and subdirectory 
manually, but adding such ability to shutil would greatly enhance readability 
and simplicity of Python file handling scripts.

Implementation could be as simply as :
import os
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
os.remove(os.path.join(root, name))
for name in dirs:
os.rmdir(os.path.join(root, name))

(example taken from http://docs.python.org/2/library/os.html#os.walk)

--
components: IO
messages: 203283
nosy: ivan.radic
priority: normal
severity: normal
status: open
title: shutil to support equivalent of: rm -f /dir/*
type: enhancement
versions: Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19642] shutil to support equivalent of: rm -f /dir/*

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +hynek, tarek
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19641] Add audioop.byteswap()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Updated patch addresses some Victor's comments. Added optimization for trivial 
case byteswap(bytes, 1).

--
Added file: http://bugs.python.org/file32688/audioop_byteswap_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[Ivan Radic]

New submission from Ivan Radic:

shutil.rmtree works nice on Windows until it hits file with read only attribute 
set. Workaround is to provide a onerror parameter as a function that checks and 
removes file attribute before attempting to delete it. Can option to delete 
read_only files be integrated in shutil.rmtree?

Example output in In Python 2.7:
shutil.rmtree("C:\\2")

Traceback (most recent call last):
  File "", line 1, in 
shutil.rmtree("C:\\2")
  File "C:\Program Files (x86)\Python.2.7.3\lib\shutil.py", line 250, in rmtree
onerror(os.remove, fullname, sys.exc_info())
  File "C:\Program Files (x86)\Python.2.7.3\lib\shutil.py", line 248, in rmtree
os.remove(fullname)
WindowsError: [Error 5] Access is denied: 'C:\\2\\read_only_file.txt'

Example output in In Python 3.3:
shutil.rmtree("C:\\2")
Traceback (most recent call last):
  File "", line 1, in 
shutil.rmtree("C:\\2")
  File "C:\Program Files (x86)\Python.3.3.0\lib\shutil.py", line 460, in rmtree
return _rmtree_unsafe(path, onerror)
  File "C:\Program Files (x86)\Python.3.3.0\lib\shutil.py", line 367, in 
_rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Program Files (x86)\Python.3.3.0\lib\shutil.py", line 365, in 
_rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\2\\read_only_file.txt'

--
components: IO
messages: 203285
nosy: ivan.radic
priority: normal
severity: normal
status: open
title: shutil rmtree fails on readonly files in Windows
type: behavior
versions: Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7105] weak dict iterators are fragile because of unpredictable GC runs

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

Attached is a version for 2.7

Btw, I think a cleaner way to deal with unpredictable GC runs is to be able to 
temporarily disable GC with a context manager.

--
nosy: +kristjan.jonsson
Added file: http://bugs.python.org/file32689/issue7105.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5673] Add timeout option to subprocess.Popen

[Thomas Guettler]

Thomas Guettler added the comment:

For Python 2.x there is a backport of the subprocess module of 3.x: 
https://pypi.python.org/pypi/subprocess32.

It has the timeout argument for call() and pipe.wait().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[Eric V. Smith]

Changes by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19575] subprocess.Popen with multiple threads: Redirected stdout/stderr files still open after process close

[Bernt Røskar Brenna]

Bernt Røskar Brenna added the comment:

And here's a function that does not require pywin32:

def open_noinherit_ctypes(*args, **kwargs):
HANDLE_FLAG_INHERIT = 1

import msvcrt
from ctypes import windll, WinError
fp = open(*args, **kwargs)
if not 
windll.kernel32.SetHandleInformation(msvcrt.get_osfhandle(fp.fileno()), 
HANDLE_FLAG_INHERIT, 0):
raise WinError()
return fp

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19644] Failing of interpreter when slicing a string in MacOS X Maverics

[Dmitry]

New submission from Dmitry:

I launch python3 on MacOS X Maverics and do the following things:
a = '12345'
a[:-0]
And the interpreter (application python3) crashes with sigfault. Everything is 
OK with python2 on this computer (the same code returnes '') and with the same 
version of Python on linux computer

--
assignee: ronaldoussoren
components: Macintosh
files: Снимок экрана 2013-11-18 в 16.39.46.png
messages: 203289
nosy: ronaldoussoren, torbad
priority: normal
severity: normal
status: open
title: Failing of interpreter when slicing a string in MacOS X Maverics
versions: Python 3.3
Added file: http://bugs.python.org/file32690/Снимок экрана 2013-11-18 в 
16.39.46.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19644] Failing of interpreter when slicing a string in MacOS X Maverics

[Ezio Melotti]

Ezio Melotti added the comment:

This is a duplicate of #18458.

--
nosy: +ezio.melotti
resolution:  -> duplicate
stage:  -> committed/rejected
status: open -> closed
superseder:  -> interactive interpreter crashes and test_readline fails on OS X 
10.9 Mavericks due to libedit update

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11344] Add os.path.splitpath(path) function

[Martin Panter]

Martin Panter added the comment:

The ntpath.splitpath() version is easy to get lost in. It would probably help 
if you spelt out all the single-letter variable names, and explained that 
tri-state root/separator = None/True/False flag. Maybe there is a less 
convoluted way to write it too, I dunno.

Also, maybe it is worth clearly documenting a couple special properties of the 
result:

* The first element is always the root component (for an absolute path), or an 
empty string (for a relative path)
* The last element is an empty string if the path name ended in a directory 
separator, except when the path is a root directory

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19617] Fix usage of Py_ssize_t type in Python/compile.c

[Benjamin Peterson]

Benjamin Peterson added the comment:

I meant where you added casts, you could use it.

2013/11/18 STINNER Victor :
>
> STINNER Victor added the comment:
>
>> You could use the Py_SAFE_DOWNCAST macro everywhere.
>
> I prefer to store sizes in a size type (Py_ssize_t), and only downcast where 
> it is really needed, in compiler_addop_i(). So in the future, if someone 
> wants to support values larger than INT_MAX, only one function need to be 
> changed.
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13090] test_multiprocessing: memory leaks

[Richard Oudkerk]

Richard Oudkerk added the comment:

> If the result of os.read() was stored in a Python daemon thread, the 
> memory should be released since the following changeset. Can someone 
> check if this issue still exist?

If a daemon thread is killed while it is blocking on os.read() then the bytes 
object used as the read buffer will never be decrefed.

--
nosy: +sbt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13090] test_multiprocessing: memory leaks

[STINNER Victor]

STINNER Victor added the comment:

> If a daemon thread is killed while it is blocking on os.read()
> then the bytes object used as the read buffer will never be decrefed.

Ah yes, so another reason to ensure that daemon threads exit normally at Python 
shutdown :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] Improving unittest assertions

[Gregory Salvan]

New submission from Gregory Salvan:

Actually unittest assertions depends on testcase class forcing us to extend it 
to add assertions and to use it to make assertions outside tests.

Seeing interests in rethinking the way assertions are done in unittest, this 
issue first intent to collect feedback in order to suggest an implementation 
that fit the most.

Some notes from private discussions:
- it was briefly discussed here #18054.
- taking care of popular solutions like py.test's rich assert
statements and the testtools matcher objects.
- avoid unnecessary complexity, staying focused on value

My opinion:
- must not change unittest api
- may be put in a seperate package (splitting "unittest" in "unittest" and 
"assertions")
- Open to hide assertions exceptions in optimized mode or providing a simple 
way to change default behaviour (log, skip... instead of throwing unhandled 
exception).

--
components: Tests
messages: 203295
nosy: Gregory.Salvan, michael.foord, ncoghlan, rbcollins
priority: normal
severity: normal
status: open
title: Improving unittest assertions
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19642] shutil to support equivalent of: rm -f /dir/*

[R. David Murray]

R. David Murray added the comment:

See also issue 13229.

You can replicate 'rm -f' like this:

   for p in glob.glob('/dir/*'): 
  os.remove(p)

That doesn't seem worth an extra function.

The annoying one to emulate is 'rm -rf /dir/*', because with the current shutil 
tools you have to make different calls depending on whether the object is a 
file or a directory.  Pathlib doesn't help with that (it has no generic 
'remove' method that applies to both directories and files), but it does make 
the iteration easier.

--
components: +Library (Lib) -IO
nosy: +pitrou, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[R. David Murray]

R. David Murray added the comment:

You are essentially asking that we have an option to make the windows behavior 
mirror the posix behavior?  (A read only file in a writable directory can be 
deleted in posix, since only the directory entry, not the file, is being 
deleted.)

That makes some sense to me.  I wonder what the windows devs think?

--
components: +Library (Lib), Windows -IO
nosy: +r.david.murray, tim.golden, zach.ware
type: behavior -> enhancement
versions: +Python 3.5 -Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[Tim Golden]

Tim Golden added the comment:

This, unfortunately, is the classic edge-case where intra-platform consistency 
and inter-platform consistency clash. 

I (on Windows) would certainly be surprised if a tree delete removed read-only 
files without my specifying some kind of override. I understand why 
cross-platform behaviour might be preferred, and I'm open to being convinced, 
but I start at -0.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[Mark Dickinson]

Mark Dickinson added the comment:

I don't much like the use of Py_SSIZE_T and Py_SAFE_DOWNCAST here:  the dtoa.c 
code knows almost nothing about Python.h (aside from right at the top), and I'd 
like to keep it that way if possible.

And in fact I'd say that we *shouldn't* be silencing these warnings; rather, we 
should take them seriously.  It looks to me as though it is possible for that 
conversion to overflow.

I'll try to take a look sometime soon.  Adding 2.7 and 3.3, since the bug is 
present there, too.

--
versions: +Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee:  -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[STINNER Victor]

STINNER Victor added the comment:

"I don't much like the use of Py_SSIZE_T and Py_SAFE_DOWNCAST here (...) 
And in fact I'd say that we *shouldn't* be silencing these warnings; rather, we 
should take them seriously."

So size_t should be used instead ("s - s1" is unsigned, s >= s1).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[R. David Murray]

R. David Murray added the comment:

Well, it would *definitely* need to be a new explicit option whose default 
value was the current behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[STINNER Victor]

STINNER Victor added the comment:

See also the "meta-issue" #9566: "Compilation warnings under x64 Windows".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[R. David Murray]

R. David Murray added the comment:

You should probably start by summarizing the assertThat proposal from issue 
18054, which suggested making a separate issue for assertThat.

--
nosy: +r.david.murray
title: Improving unittest assertions -> decouple unittest assertions from the 
TestCase class

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19640] Drop _source attribute of namedtuple

[STINNER Victor]

STINNER Victor added the comment:

>  the 5th line allocating the memory memory

oops, the 5th line allocating the *most* memory

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9878] Avoid parsing pyconfig.h and Makefile by autogenerating extension module

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I believe this has been fixed for a while.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19596] Silently skipped tests in test_importlib

[Brett Cannon]

Brett Cannon added the comment:

I've actually started doing this in the PEP 451 repo and it's working out so 
far. When that all lands I should have all the loaders ported but not the 
finders. If you want, Zachary, you can make the chances in the PEP 451 repo so 
you don't have to wonder what has or has not been cleaned up.

--
assignee:  -> zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19618] test_sysconfig_module fails on Ubuntu 12.04

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I also can't reproduce this in Python 3.3 or 3.4 on Trusty Tahr (what will be 
Ubuntu 14.04).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19638] dtoa: conversion from '__int64' to 'int', possible loss of data

[Jeremy Kloth]

Changes by Jeremy Kloth :


--
nosy: +jkloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[Ivan Radic]

Ivan Radic added the comment:

>You are essentially asking that we have an option to make the windows behavior 
>mirror the posix behavior?  (A read only file in a writable directory can be 
>deleted in posix, since only the directory entry, not the file, is being 
>deleted.)

Actually, your explanation is perfect.
I want to be able to remove some directory after I am done using it. When 
similar operation is done through file manager, dialog pops up asking for 
confirmation, I would like to have function parameter equivalent of "yes to 
all" dialog that file manager gives me.

The thing is, anyone working with files is used to think in "rm -rf" kind of 
way, and on Windows read_only files break this workflow. I discovered this 
problem few days ago when I was working on custom backup script that needs to 
work both on Linux (at home) and Windows (at work). Currently, I need to have 
some extra *windows only* code just to be able to successfully remove a 
directory.

Quick Google search discovered the workaround 
(http://stackoverflow.com/questions/1889597/deleting-directory-in-python), so I 
am set, but the original 
question: "Why oh why is this such a pain?"
and the comment: "Maybe nobody has taken the five minutes to file a bug at 
bugs.python.org" 
resonated in my head long enough to give it a try.

For me it makes sense to have this option configurable. And it make a ton of 
sense to support one line equivalent of "rm -rf".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11344] Add os.path.splitpath(path) function

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Added examples and Martin's notes to the documentation. ntpath implementation 
rewrote with regular expressions (it is now shorter and perhaps more clear).

--
Added file: http://bugs.python.org/file32691/ospath_splitpath_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16596] Skip stack unwinding when "next", "until" and "return" pdb commands executed in generator context

[Xavier de Gaye]

Xavier de Gaye added the comment:

A description of what goes wrong when stepping out of the generator
would be helpful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19437] More failures found by pyfailmalloc

[Roundup Robot]

Roundup Robot added the comment:

New changeset 8f770cdb7a19 by Victor Stinner in branch 'default':
Issue #19437: Fix error handling of CDataType_from_buffer()
http://hg.python.org/cpython/rev/8f770cdb7a19

New changeset 556bdd8d0dde by Victor Stinner in branch 'default':
Issue #19437: Fix error handling of PyCArrayType_new(), don't decreases the
http://hg.python.org/cpython/rev/556bdd8d0dde

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16596] Skip stack unwinding when "next", "until" and "return" pdb commands executed in generator context

[Guido van Rossum]

Guido van Rossum added the comment:

Basically the debugger lost control and the program ran to completion after I 
hit 'n' that returned from the coroutine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[Zachary Ware]

Zachary Ware added the comment:

I like the idea of a remove_readonly flag.  I was going to say that I'm a bit 
worried about the fact that shutil.rmtree already has a couple of keyword 
arguments, but it's nowhere near what, say, copytree has.  Call me +0.75.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[R. David Murray]

R. David Murray added the comment:

That's not a good name for the flag.  The problem is that 'read-only' means 
different things on Windows than it does on Unix.  Which is why I suggested 
that the flag control whether or not it acts "posix like" on Windows.  So 
perhaps 'posix_compat'?  That feels a bit odd, though, since on posix it does 
nothing...so it's really about behavioral consistency across platforms

Hmm.  It's really hard to think of a name that conveys succinctly what we are 
talking about here.

A more radical notion would be something like 'delete_control' and have it be 
tri-valued: 'unixlike', 'windowslike', and 'native', with the default being 
native.  Bad names, most likely, but you get the idea.  The disadvantage is 
that it would be even more code to implement ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[Tim Golden]

Tim Golden added the comment:

TBH I'm still fairly -0 and edging towards -0.5. If we didn't already 
have two keyword args I might be convinced towards a jfdi=True flag. 
But, as the OP described, the current params already allow for a 
workaround of sorts and another param of the semantics we're discussing 
would surely just be confusing?

(Or you could just "preprocess" on Windows, eg attrib -r /s)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19643] shutil rmtree fails on readonly files in Windows

[Zachary Ware]

Zachary Ware added the comment:

I make no claims of being good at naming things :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17397] ttk::themes missing from ttk.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Since splitlist() works with Tcl_Obj-s, proposed test should pass.

klappnase, could you please sign a Contributor Licensing Agreement?

http://www.python.org/psf/contrib/contrib-form/

What is your real name? What should I add in the Misc/ACKS file?

--
assignee:  -> serhiy.storchaka
stage: test needed -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7105] weak dict iterators are fragile because of unpredictable GC runs

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Btw, I think a cleaner way to deal with unpredictable GC runs is to be able 
> to temporarily disable GC with a context manager

Disabling the GC in a library function sounds very ugly to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

[Roundup Robot]

Roundup Robot added the comment:

New changeset 093b9838a41c by Victor Stinner in branch 'default':
Issue #19581: Change the overallocation factor of _PyUnicodeWriter on Windows
http://hg.python.org/cpython/rev/093b9838a41c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19513] Use PyUnicodeWriter instead of PyAccu in repr(tuple) and repr(list)

[Roundup Robot]

Roundup Robot added the comment:

New changeset fc7ceb001eec by Victor Stinner in branch 'default':
Issue #19513: repr(list) now uses the PyUnicodeWriter API, it is faster than
http://hg.python.org/cpython/rev/fc7ceb001eec

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19581] PyUnicodeWriter: change the overallocation factor for Windows

[STINNER Victor]

Changes by STINNER Victor :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19513] Use PyUnicodeWriter instead of PyAccu in repr(tuple) and repr(list)

[STINNER Victor]

STINNER Victor added the comment:

I checked in list_repr_writer-2.patch, thanks Serhiy for your review.

And now the patch for repr(tuple).

Result of bench_tuple_repr.py:


Common platform:
CFLAGS: -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes
Bits: int=32, long=64, long long=64, size_t=64, void*=64
Platform: Linux-3.9.4-200.fc18.x86_64-x86_64-with-fedora-18-Spherical_Cow
CPU model: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Timer info: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)
Timer: time.perf_counter
Python unicode implementation: PEP 393

Platform of campaign pyaccu:
SCM: hg revision=fc7ceb001eec tag=tip branch=default date="2013-11-18 21:11 
+0100"
Timer precision: 41 ns
Date: 2013-11-18 21:29:45
Python version: 3.4.0a4+ (default:fc7ceb001eec, Nov 18 2013, 21:29:41) [GCC 
4.7.2 20121109 (Red Hat 4.7.2-8)]

Platform of campaign writer:
SCM: hg revision=fc7ceb001eec+ tag=tip branch=default date="2013-11-18 21:11 
+0100"
Timer precision: 39 ns
Date: 2013-11-18 21:28:53
Python version: 3.4.0a4+ (default:fc7ceb001eec+, Nov 18 2013, 21:28:24) [GCC 
4.7.2 20121109 (Red Hat 4.7.2-8)]

--+-+---
Tests |  pyaccu | writer
--+-+---
tuple("a")|  355 ns (*) |  318 ns (-11%)
tuple("abc")  |  492 ns (*) | 481 ns
("a",)*(100)  |  8.1 us (*) |  7.39 us (-9%)
("abc",)*(100)| 8.16 us (*) |  7.51 us (-8%)
("a" * 100,)*(100)| 35.2 us (*) |36.3 us
("a",)*(10**6)|   88 ms (*) | 75.9 ms (-14%)
("abc",)*(10**6)  | 88.5 ms (*) | 75.3 ms (-15%)
("a" * 100,)*(10**5)  | 44.8 ms (*) | 34.9 ms (-22%)
tuple(range(10**6))   |  104 ms (*) | 92.5 ms (-11%)
tuple(map(str, range(10**6))) |  105 ms (*) | 90.5 ms (-14%)
--+-+---
Total |  431 ms (*) |  369 ms (-14%)
--+-+---

--
Added file: http://bugs.python.org/file32692/tuple_repr_writer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19513] Use PyUnicodeWriter instead of PyAccu in repr(tuple) and repr(list)

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file32693/bench_tuple_repr.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19646] Use PyUnicodeWriter in repr(dict)

[STINNER Victor]

New submission from STINNER Victor:

Attached patch modify dict_repr() function to use the _PyUnicodeWriter API 
instead of building a list of short strings with PyUnicode_AppendAndDel() and 
calling PyUnicode_Join() at the end to join the list. PyUnicode_Append() is 
inefficient because it has to allocate a new string instead of reusing the same 
buffer.

_PyUnicodeWriter API has a different design. It overallocates a buffer to write 
Unicode characters and shrink the buffer at the end. It is faster according to 
my micro benchmark.


$ ./python ~/prog/HG/misc/python/benchmark.py compare_to pyaccu writer
Common platform:
CPU model: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Python unicode implementation: PEP 393
CFLAGS: -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g 
-fwrapv -O3 -Wall -Wstrict-prototypes
Timer precision: 40 ns
Timer info: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)
Platform: Linux-3.9.4-200.fc18.x86_64-x86_64-with-fedora-18-Spherical_Cow
Bits: int=32, long=64, long long=64, size_t=64, void*=64
Timer: time.perf_counter

Platform of campaign pyaccu:
Date: 2013-11-18 21:37:44
Python version: 3.4.0a4+ (default:fc7ceb001eec, Nov 18 2013, 21:29:41) [GCC 
4.7.2 20121109 (Red Hat 4.7.2-8)]
SCM: hg revision=fc7ceb001eec tag=tip branch=default date="2013-11-18 21:11 
+0100"

Platform of campaign writer:
Date: 2013-11-18 22:10:40
Python version: 3.4.0a4+ (default:fc7ceb001eec+, Nov 18 2013, 22:10:12) [GCC 
4.7.2 20121109 (Red Hat 4.7.2-8)]
SCM: hg revision=fc7ceb001eec+ tag=tip branch=default date="2013-11-18 21:11 
+0100"

--+-+--
Tests |  pyaccu |    writer
--+-+--
{"a": 1}  |  603 ns (*) | 496 ns (-18%)
dict(zip("abc", range(3)))    | 1.05 us (*) | 904 ns (-14%)
{"%03d":"abc" for k in range(10)} |  631 ns (*) | 501 ns (-21%)
{"%100d":"abc" for k in range(10)}    |  660 ns (*) | 484 ns (-27%)
{k:"a" for k in range(10**3)} |  235 us (*) | 166 us (-30%)
{k:"abc" for k in range(10**3)}   |  245 us (*) | 177 us (-28%)
{"%100d":"abc" for k in range(10**3)} |  668 ns (*) | 478 ns (-28%)
{k:"a" for k in range(10**6)} |  258 ms (*) | 186 ms (-28%)
{k:"abc" for k in range(10**6)}   |  265 ms (*) | 184 ms (-31%)
{"%100d":"abc" for k in range(10**6)} |  652 ns (*) | 489 ns (-25%)
--+-+--
Total |  523 ms (*) | 369 ms (-29%)
--+-+--

--
components: Unicode
files: dict_repr_writer.patch
keywords: patch
messages: 203322
nosy: ezio.melotti, haypo, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Use PyUnicodeWriter in repr(dict)
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file32694/dict_repr_writer.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19646] Use PyUnicodeWriter in repr(dict)

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file32695/bench_dict_repr.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19513] Use PyUnicodeWriter instead of PyAccu in repr(tuple) and repr(list)

[Roundup Robot]

Roundup Robot added the comment:

New changeset ead9043f69df by Victor Stinner in branch 'default':
Issue #19513: Simplify list_repr()
http://hg.python.org/cpython/rev/ead9043f69df

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19513] Use PyUnicodeWriter instead of PyAccu in repr(tuple) and repr(list)

[STINNER Victor]

STINNER Victor added the comment:

Ok, here is a new version hyper optimized for the final ",)" string :-)

--
Added file: http://bugs.python.org/file32696/tuple_repr_writer-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19647] unittest.TestSuite consumes tests

[Stefan Holek]

New submission from Stefan Holek:

This test passed in Python <= 3.3 but fails in 3.4:

def testTestSuiteConsumesTest(self):
class MyTestCase(unittest.TestCase):
def testMethod(self):
pass
test = MyTestCase('testMethod')
suite = unittest.TestSuite((test,))
result = unittest.TestResult()
self.assertEqual(next(iter(suite)), test)
suite.run(result)
self.assertEqual(next(iter(suite)), test)

The suite contains None after it has been run in Python 3.4.

--
components: Library (Lib)
messages: 203325
nosy: stefanholek
priority: normal
severity: normal
status: open
title: unittest.TestSuite consumes tests
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19647] unittest.TestSuite consumes tests

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +michael.foord

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19513] Use PyUnicodeWriter instead of PyAccu in repr(tuple) and repr(list)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Old version looks better to me (it is simpler and performance of writing final 
",)" is not worth additional complication).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19596] Silently skipped tests in test_importlib

[Zachary Ware]

Zachary Ware added the comment:

Alright, I'll try to do that later this evening.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17457] Unittest discover fails with namespace packages and builtin modules

[Eric Snow]

Eric Snow added the comment:

I left a review relative to the use of the _path attribute (which shouldn't be 
used).

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18864] Implementation for PEP 451 (importlib.machinery.ModuleSpec)

[Nick Coghlan]

Nick Coghlan added the comment:

Just a heads up - since the two of you appear to have this well in hand,
don't wait for a review from me before committing it. I have plenty of
other things to look at before the beta deadline :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com