[issue18472] Update PEP 8 to encourage modern conventions

[Florent Xicluna]

Changes by Florent Xicluna :


--
nosy: +flox

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[STINNER Victor]

STINNER Victor added the comment:

It looks like Windows supports also lazy initialization of memory pages 
initialized to zero.

According to my microbenchmark on Linux and Windows, only bytes(n) and 
bytearray(n) are really faster with use_calloc.patch. Most changes of 
use_calloc.patch are maybe useless since all bytes are initilized to zero, but 
just after that they are replaced with new bytes.

Results of bench_alloc2.py on Windows 7: original vs 
calloc-4.patch+use_calloc.patch:

Common platform:
Timer: time.perf_counter
Python unicode implementation: PEP 393
Bits: int=32, long=32, long long=64, size_t=32, void*=32
Platform: Windows-7-6.1.7601-SP1
CFLAGS: None
Timer info: namespace(adjustable=False, implementation='QueryPerformanceCounter(
)', monotonic=True, resolution=1e-08)

Platform of campaign orig:
SCM: hg revision=4b97092aa4bd branch=default date="2014-04-27 18:02 +0100"
Date: 2014-04-28 09:35:30
Python version: 3.5.0a0 (default, Apr 28 2014, 09:33:30) [MSC v.1600 32 bit (Int
el)]
Timer precision: 4.47 us

Platform of campaign calloc:
SCM: hg revision=4f0aaa8804c6 tag=tip branch=default date="2014-04-28 09:27 +020
0"
Date: 2014-04-28 09:37:37
Python version: 3.5.0a0 (default:4f0aaa8804c6, Apr 28 2014, 09:37:03) [MSC v.160
0 32 bit (Intel)]
Timer precision: 4.44 us

---+-+
Tests  |    orig |  calloc
---+-+
object()   |  121 ns (*) |   109 ns (-10%)
b'A' * 10  |   77 ns (*) |   79 ns
b'A' * 10**3   |  159 ns (*) |    168 ns (+5%)
b'A' * 10**6   |  428 us (*) |  415 us
'A' * 10   |   87 ns (*) |   89 ns
'A' * 10**3    |  175 ns (*) |  177 ns
'A' * 10**6    |  429 us (*) |    454 us (+6%)
'A' * 10**8    | 48.4 ms (*) |   49 ms
(None,) * 10**0    |   49 ns (*) |   51 ns
(None,) * 10**1    |  115 ns (*) |    99 ns (-14%)
(None,) * 10**2    |  433 ns (*) |  422 ns
(None,) * 10**3    | 3.58 us (*) | 3.57 us
(None,) * 10**4    | 34.9 us (*) | 34.9 us
(None,) * 10**5    |  347 us (*) |  351 us
(None,) * 10**6    | 5.14 ms (*) |   4.85 ms (-6%)
(None,) * 10**7    | 53.2 ms (*) |   50.2 ms (-6%)
(None,) * 10**8    |  563 ms (*) |    515 ms (-9%)
([None] * 10)[1:-1]    |  217 ns (*) |  217 ns
([None] * 10**3)[1:-1] | 3.89 us (*) | 3.92 us
([None] * 10**6)[1:-1] | 5.13 ms (*) | 5.17 ms
([None] * 10**8)[1:-1] |  634 ms (*) |   533 ms (-16%)
bytes(10)  |  193 ns (*) |    206 ns (+7%)
bytes(10**3)   |  266 ns (*) |   296 ns (+12%)
bytes(10**6)   |  414 us (*) |  3.89 us (-99%)
bytes(10**8)   | 44.2 ms (*) | 4.56 us (-100%)
bytearray(10)  |  229 ns (*) |    243 ns (+6%)
bytearray(10**3)   |  301 ns (*) |   330 ns (+10%)
bytearray(10**6)   |  421 us (*) |  3.89 us (-99%)
bytearray(10**8)   | 44.4 ms (*) | 4.56 us (-100%)
---+-+
Total  | 1.4 sec (*) | 1.16 sec (-17%)
---+-+

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21372] multiprocessing.util.register_after_fork inconsistency

[Marc Schlaich]

New submission from Marc Schlaich:

multiprocessing.util.register_after_fork does not behave consistently on 
Windows because the `_afterfork_registry` is not transferred to the subprocess. 
The following example fails on Windows while it works perfectly on Linux:


import multiprocessing.util


def hook(*args):
print (args)


def func():
print ('func')


if __name__ == '__main__':
multiprocessing.util.register_after_fork(hook, hook)
p = multiprocessing.Process(target=func)
p.start()

--
components: Windows
messages: 217347
nosy: schlamar
priority: normal
severity: normal
status: open
title: multiprocessing.util.register_after_fork inconsistency
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[STINNER Victor]

STINNER Victor added the comment:

Changes on the pickle module don't look like an interesting optimization. It 
even looks slower.

$ python perf.py -b 
fastpickle,fastunpickle,pickle,pickle_dict,pickle_list,slowpickle,slowunpickle,unpickle
 ../default/python.orig ../default/python.calloc
...

Report on Linux selma 3.13.9-200.fc20.x86_64 #1 SMP Fri Apr 4 12:13:05 UTC 2014 
x86_64 x86_64
Total CPU cores: 4

### fastpickle ###
Min: 0.364510 -> 0.374144: 1.03x slower
Avg: 0.367882 -> 0.377714: 1.03x slower
Significant (t=-11.54)
Stddev: 0.00493 -> 0.00347: 1.4209x smaller

The following not significant results are hidden, use -v to show them:
fastunpickle, pickle_dict, pickle_list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[STINNER Victor]

STINNER Victor added the comment:

Patch version 5. This patch is ready for a review.

Summary of calloc-5.patch:

- add the following functions:

  * void* PyMem_RawCalloc(size_t nelem, size_t elsize)
  * void* PyMem_Calloc(size_t nelem, size_t elsize)
  * void* PyObject_Calloc(size_t nelem, size_t elsize)
  * PyObject* _PyObject_GC_Calloc(size_t basicsize)

- add "void* calloc(void *ctx, size_t nelem, size_t elsize)" field to the 
PyMemAllocator structure
- optimize bytes(n) and bytearray(n) to allocate objects using calloc() instead 
of malloc()
- update tracemalloc to trace also calloc()
- document new functions and add unit tests for the calloc "hook" (in _testcapi)


Changes between versions 4 and 5:

- revert all changes except bytes(n) and bytearray(n) of use_calloc.patch: they 
were useless according to benchmarks
- _PyObject_GC_Calloc() now takes a single parameter
- add versionadded and versionchanged fields in the documentation


According to benchmarks, calloc() is only useful for large allocation (1 MB?) 
if only a part of the memory block is modified (to non-zero bytes) just after 
the allocation. Untouched memory pages don't use physical memory and don't use 
RSS memory pages, but it is possible to read their content (null bytes). Using 
calloc() instead of malloc()+memset(0) doens't look to be faster (it may be a 
little bit slower) if all bytes are set just after the allocation.

I chose to only use one parameter for _PyObject_GC_Calloc() because this 
function is used to allocate Python objects. A structure of a Python object 
must start with PyObject_HEAD or PyObject_VAR_HEAD and so the total size of an 
object cannot be expressed as NELEM * ELEMSIZE.

I have no use case for _PyObject_GC_Calloc(), but it makes sense to use it to 
allocate a large Python object tracked by the GC and using a single memory 
block for the Python header + data.

PyObject_Calloc() simply use memset(0) for small objects (<= 512 bytes). It 
delegates the allocation to PyMem_RawCalloc(), and so indirectly to calloc(), 
for larger objects.

Note: use_calloc.patch is no more needed, I merged the two patches since only 
bytes(n) and bytearray(n) now use calloc().

--
Added file: http://bugs.python.org/file35072/calloc-5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20305] Android's incomplete locale.h implementation prevents cross-compilation

[Stefan Krah]

Changes by Stefan Krah :


--
nosy: +lizhenhua

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21371] struct lconv does not have decimal_point on android platform

[Stefan Krah]

Stefan Krah added the comment:

This looks like a duplicate.

--
nosy: +skrah
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Android's incomplete locale.h implementation prevents 
cross-compilation

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[STINNER Victor]

STINNER Victor added the comment:

Demo of calloc-5.patch on Linux. Thanks to calloc(), bytes(50 * 1024 * 1024) 
doesn't allocate memory for null bytes and so the RSS memory is unchanged (+148 
kB, not +50 MB), but tracemalloc says that 50 MB were allocated.

$ ./python -X tracemalloc
Python 3.5.0a0 (default:4b97092aa4bd+, Apr 28 2014, 10:40:53) 
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, tracemalloc
>>> os.system("grep RSS /proc/%s/status" % os.getpid())
VmRSS: 10736 kB
0
>>> before = tracemalloc.get_traced_memory()[0]
>>> large = bytes(50 * 1024 * 1024)
>>> import sys
>>> sys.getsizeof(large) / 1024.
51200.0478515625
>>> (tracemalloc.get_traced_memory()[0] - before) / 1024.
51198.1962890625
>>> os.system("grep RSS /proc/%s/status" % os.getpid())
VmRSS: 10884 kB
0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5404] Cross-compiling Python

[Stefan Krah]

Stefan Krah added the comment:

Ping. Is this still an issue for anyone in 3.4?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21220] Enhance obmalloc allocation strategy

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

>> This significantly helps fragmentation in programs with dynamic memory 
>> usage, e.g. long running programs.

> On which programs? The fragmentation of the memory depends a lot on how the 
> program allocates memory. For example, if a program has no "temporary memory 
> peak", it should not be a victim of the memory fragmentation.

"Long running programs."  E.g. web servers and so on.  Where there is a churn 
in memory usage.  As objects are allocated and released with some sort of 
churn.  New objects will be allocated from lower-address pages, where higher 
address pages are increasingly likely to be freed as no-longer used objects are 
released.  This is the second best thing to a sliding-block allocator and is 
motivated by the same requirements that makes such an sliding-block allocator 
(such as pypy uses) desirable in the first place. 


> To measure the improvment of such memory allocator, more benchmarks (speed 
> and fragmentation) should be run than a single test (memcruch.py included in 
> the test) written to benchmark the allocator.

Yes.  Memcrunch was specifically written to simulate a case where objects are 
continuously created and released, such as might be expected in a server, with 
a peak in memory usage followed by lower memory usage, and to demonstrate that 
the pages allocated during the peak will be released later as memory churn 
causes memory usage to migrate toward lower addresses.

However, following Antoine's advice I ran the Benchmarks testsuite and found an 
adverse effect in the n-body benchmark.  That can have two causes:
a) the insertion cost of the block when a block moves from 'full' to 'used'.  
This is a rare event and should be unimportant.  I will instrument this for 
this test and see if it is really the reason
b) Cache effects because a newly 'used' block is not immediately allocated 
from.  Again, it happens rarely that a block is linked at the head so this 
shouldn't be significant.

Because of this, this change isn't yet ready to be applied.
If however I manage to change the policy so that memory usage becomes better 
while still preserving performance of the benchmark tests, I will report back :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20188] ALPN support for TLS

[Cory Benfield]

Changes by Cory Benfield :


--
nosy: +Lukasa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21369] Extended modes for tarfile.TarFile()

[Sworddragon]

Sworddragon added the comment:

The TarFile class provides more options. Alternatively a file object could be 
used but this means additional code (and maybe IO overhead).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale

[Antoine Pitrou]

Antoine Pitrou added the comment:

> > We should not overcomplicate this. I suggest that we simply use utf-8 under 
> > the C locale.
> 
> Do you mean utf8/strict or utf8/surrogateescape?
> 
> utf8/strict doesn't work (os.listdir raises an unicode error) if your
> system is configured to use latin1 (ex: filenames are stored in this
> encoding), but unfortunately your program is running in an empty
> environment (so will use the POSIX locale).

The issue is about stdin and stdout, I'm not sure why os.listdir would
be affected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13559] Use sendfile where possible in httplib

[Giampaolo Rodola']

Changes by Giampaolo Rodola' :


--
dependencies: +Add a new socket.sendfile() method
versions: +Python 3.5 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20306] Lack of pw_gecos field in Android's struct passwd causes cross-compilation for the pwd module to fail

[Antoine Pitrou]

Antoine Pitrou added the comment:

Interesting. It seems pw_gecos isn't mandated by POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/pwd.h.html

I wonder if there's a better way to do this (autoconf check?) than an 
Android-specific #ifdef, though.

--
nosy: +pitrou
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[Stefan Krah]

Stefan Krah added the comment:

With the latest patch the decimal benchmark with a lot of small
allocations is consistently 2% slower. Large factorials (where
the operands are initialized to zero for the number-theoretic
transform) have the same performance with and without the patch.

It would be interesting to see some NumPy benchmarks (Nathaniel?).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21369] Extended modes for tarfile.TarFile()

[Lars Gustäbel]

Lars Gustäbel added the comment:

You can pass keyword arguments to tarfile.open(), which will be passed to the 
TarFile constructor. You can also use pass fileobj arguments to tarfile.open().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13204] sys.flags.__new__ crashes

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thanks for the patch, Jessica. It seems to work under Windows here.

--
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13204] sys.flags.__new__ crashes

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7052fdd90a11 by Antoine Pitrou in branch '3.4':
Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it 
raises a TypeError.
http://hg.python.org/cpython/rev/7052fdd90a11

New changeset a14012352f65 by Antoine Pitrou in branch 'default':
Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it 
raises a TypeError.
http://hg.python.org/cpython/rev/a14012352f65

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21369] Extended modes for tarfile.TarFile()

[Sworddragon]

Sworddragon added the comment:

Interesting, after reading the documentation again I would now assume that is 
what **kwargs is for.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13204] sys.flags.__new__ crashes

[Antoine Pitrou]

Antoine Pitrou added the comment:

I've committed the patch to 3.4 and 3.5. I'm closing the issue, I don't think 
fixing 2.7 is important at this point.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21369] Extended modes for tarfile.TarFile()

[Lars Gustäbel]

Lars Gustäbel added the comment:

Jup. That's it.

--
priority: normal -> low
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Donald Stufft]

Donald Stufft added the comment:

"Depleting" /dev/urandom isn't actually a thing. /dev/urandom on all modern 
*nix OSs uses a fast PRNG which is secure as long as it has received enough 
bytes of initial entropy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Charles-François Natali]

Charles-François Natali added the comment:

> "Depleting" /dev/urandom isn't actually a thing. /dev/urandom on all modern 
> *nix OSs uses a fast PRNG which is secure as long as it has received enough 
> bytes of initial entropy.

I didn't say "deplete /dev/urandom", I said that when reading from
/dev/urandom "you're depleting your entropy pool". So reading from
/dev/urandom won't block, but it can starve processes that read from
/dev/random, and that's a problem.

See 
https://groups.google.com/forum/#!msg/fa.linux.kernel/Ocl01d8TzT0/KDCon2ZUm1AJ

I think since 2.6 Linux uses two different entropy pools for
/dev/random and /dev/urandom, but that might not be true for every OS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1820] Enhance Object/structseq.c to match namedtuple and tuple api

[Stefan Krah]

Stefan Krah added the comment:

Is accessing _fields a common operation?  Personally I'd use a
PyGetSetDef and generate the tuple on access (perhaps cache the
result).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13096] ctypes: segfault with large POINTER type names

[Meador Inge]

Meador Inge added the comment:

Thanks for the review and reminder about this issue, jesstess.  I will apply 
the patch later today.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20306] Lack of pw_gecos field in Android's struct passwd causes cross-compilation for the pwd module to fail

[Shiz]

Shiz added the comment:

Ah, yes, if it's not actually mandated by POSIX, something like 
HAVE_PASSWD_PW_GECOS would be more appropriate. I'll rework the patch into 
something more generic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Donald Stufft]

Donald Stufft added the comment:

I don't think what you're worrying about here is something that has a high 
chance of happening, if it even occurs in the wild at all. To be clear in order 
for that to matter at all in the context of this ticket, some software would 
need to be reading from /dev/random (which almost zero software should be 
doing) on a system where you have a high number of threads or async handlers 
all reading from /dev/urandom at the same time and reading enough data off of 
it to drop the entropy estimation in the primary pool down below whatever 
amount of data that the other process is attempting to read from /dev/random. 
In that case no "trouble" will occur and the process reading from /dev/random 
will just block waiting on additional entropy to be collected so that the 
entropy estimation is high enough to fulfill the request.

AFAIK there are zero practical concerns from reading as much as you want off of 
/dev/urandom as often as you want.

What this change does is make it "safe" to just simply use os.urandom in order 
to generate random bytes in a Python application. The current situation is such 
that any use of os.urandom is potentially a place where your application will 
crash in highly concurrent environments. This will drive people to either:

A) Poorly reimplement the persistent FD option, especially troublesome on 
Windows because the simple approach to doing so will flat out not work on 
Windows
B) Use a userspace CSPRNG, this is considered ill advised by most reputable 
cryptographer's as most of them have had issues at one point in time or 
another, and they all generally depend on the security of /dev/urandom anyways 
so if /dev/urandom fall they fall as well.

Using os.urandom is the *right* thing to do for getting random in an 
application, but the current implementation effectively punishes people who use 
it if their application is highly concurrent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21373] robotparser: Automatically call modified function in read()

[Maxime Lorant]

New submission from Maxime Lorant:

For the moment, RobotFileParser (on both Python 2.x and 3.x) has a method 
modified, but it is never called in the class itself, hence the last_checked 
attribute is always at 0 if the user doesn't call modified() explicitly. 

I would suggest to add a call to modified() at the end of the read() method. It 
makes more sense to have a last_checked value (returns in mtime()) updated by 
the class itself. Especially when the doc says: "Returns the time the 
``robots.txt`` file was last fetched.". Currently this sentence isn't true, 
since the value has to be updated by the user.

--
components: Library (Lib)
files: robotparser.diff
keywords: patch
messages: 217370
nosy: mlorant, orsenthil
priority: normal
severity: normal
status: open
title: robotparser: Automatically call modified function in read()
type: enhancement
versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file35073/robotparser.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[William Tisäter]

William Tisäter added the comment:

That makes sense.

I proceeded and updated `Lib/gzip.py` to use `io.DEFAULT_BUFFER_SIZE` instead. 
This will change the existing behaviour in two ways:

* Start using 1024 * 8 as buffer size instead of 1024.

* Add one more kwarg (`buffer_size`) to `GzipFile.__init__()`.

Ps. This is my first patch, tell me if I'm missing something.

--
Added file: http://bugs.python.org/file35074/20962_default-buffer-size.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Charles-François Natali]

Charles-François Natali added the comment:

> Using os.urandom is the *right* thing to do for getting random in an 
> application, but the current implementation effectively punishes people who 
> use it if their application is highly concurrent.

And I argue that this scenario is almost as likely as the one you
depict above: we never had a bug report before, and if you have a look
at the the bug report which led to the change in question, it's not
clear at all that all threads were indeed reading from /dev/urandom
when EMFILE was raised. Since reading from /dev/urandom shouldn't
block, it's not clear at all how a realistic workload would actually
hit the file descriptor limit because RLIMIT_NOFILE threads are
reading from /dev/urandom.

But don't get me wrong, I'm not saying this change is useless, it
actually makes sense to use a persistent FD. But backporting always
has a risk, which has to be balanced.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC

[akira]

akira added the comment:

I've updated the patch:

- fixed the code example in the documentation to use int instead of
  float result
- removed assertion on the int returned type (float won't lose precision
  for the practical dates but guaranteeing an integer would be nice)
- reworded the scary comment
- removed tests that test the tests

Ready for review.

--
Added file: http://bugs.python.org/file35075/ssl_cert_time_to_seconds-ps6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Donald Stufft]

Donald Stufft added the comment:

> But backporting always has a risk, which has to be balanced.

Sure, which is why a PEP was written, discussed and accepted to find that 
balance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[STINNER Victor]

STINNER Victor added the comment:

> With the latest patch the decimal benchmark with a lot of small
> allocations is consistently 2% slower.

Does your benchmark use bytes(int) or bytearray(int)? If not, I guess that your 
benchmark is not reliable because only these two functions are changed by 
calloc-5.patch, except if there is a bug in my patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21362] concurrent.futures does not validate that max_workers is proper

[Jim Jewett]

Jim Jewett added the comment:

I confirm the bug.
The patch looks good.

--
nosy: +Jim.Jewett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Nick Coghlan]

Nick Coghlan added the comment:

I'd like to revisit this after PEP 432 is in place, since having to do this 
dance for arg processing when running on Linux in the POSIX locale is somewhat 
lame:

argv = sys.argv
encoding = locale.getpreferredencoding() # Hope nobody changed the locale!
fixed_encoding = read_encoding_from("/etc/locale.conf") # For example
argvb = [arg.encode(encoding, "surrogateescape") for arg in argv]
fixed_argv = [arg.decode(fixed_encoding, "surrogateescape") for arg in 
argvb]

(For stricter parsing, leave out the second "surrogateescape")

Now, if PEP 432 resolves the system encoding issue such that we are able to use 
the right encoding even when locale.getpreferredencoding() returns the wrong 
answer, then it may not be worthwhile to also provide sys.argvb (especially 
since it won't help hybrid 2/3 code). On the other hand, like os.environb, it 
does make it easier for POSIX-only code paths that wants to handle boundary 
encoding issues directly to stick with consuming the binary data directly and 
avoid the interpreter's automatic conversion to the text domain.

Note also that os.environb is only available when os.supports_bytes_environ is 
True, so it would make sense to only provide sys.argvb in the circumstances 
where we provide os.environb.

--
assignee:  -> ncoghlan
nosy: +ncoghlan
resolution: wont fix -> later
status: closed -> open
versions: +Python 3.5 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1599254] mailbox: other programs' messages can vanish without trace

[Jim Jewett]

Jim Jewett added the comment:

pinging David Watson:  What is the status?  If I understand correctly, (and I 
may well not), you have already opened other issues for parts of this, and 
(only) the final patch is ready for patch (and hopefully) commit review.  Is 
this correct?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20974] email module docs say not compatible with current python version

[Jim Jewett]

Jim Jewett added the comment:

I don't know for sure if the compatibility claims are correct, but the patch 
looks good.

--
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[Stefan Krah]

Stefan Krah added the comment:

Hmm, obmalloc.c changed as well, so already the gcc optimizer can take
different paths and produce different results.

Also I did set mpd_callocfunc to PyMem_Calloc(). 2% slowdown is far
from being a tragic result, so I guess we can ignore that.

The bytes() speedup is very nice. Allocations that took one second
are practically instant now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21374] DecimalTuple.__module__ is set to _frozen_importlib

[Stefan Krah]

New submission from Stefan Krah:

>>> x = Decimal(9).as_tuple()
>>> import pickle
>>> pickle.dumps(x)
Traceback (most recent call last):
  File "", line 1, in 
_pickle.PicklingError: Can't pickle : 
attribute lookup DecimalTuple on _frozen_importlib failed

--
components: Extension Modules
messages: 217381
nosy: skrah
priority: normal
severity: normal
stage: needs patch
status: open
title: DecimalTuple.__module__ is set to _frozen_importlib
type: behavior
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[Charles-François Natali]

Charles-François Natali added the comment:

> Also I did set mpd_callocfunc to PyMem_Calloc(). 2% slowdown is far
> from being a tragic result, so I guess we can ignore that.

Agreed.

> The bytes() speedup is very nice. Allocations that took one second
> are practically instant now.

Indeed.
Victor, thanks for the great work!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19354] test_format fails on RHEL-6

[Stefan Krah]

Changes by Stefan Krah :


--
resolution:  -> out of date
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21370] segfault from simple traceback.format_exc call

[Stefan Krah]

Stefan Krah added the comment:

I cannot reproduce this. Which platform? Does it happen with
Python 3.4?

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21354] PyCFunction_New no longer exposed by python DLL breaking bdist_wininst installers

[Andrew Svetlov]

Andrew Svetlov added the comment:

Ok, I'll take a look. Sorry, probably I've missed python3.def file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale

[Nick Coghlan]

Nick Coghlan added the comment:

Victor was referring to code like "print(os.listdir())". Those are the
motivating cases for ensuring round trips from system APIs to the standard
streams work correctly.

There's also the problem that sys.argv currently relies on the locale
encoding directly, because the filesystem encoding hasn't been worked out
at that point (see issue 8776). So this current change will also make
"print(sys.argv)" work more reliably in the POSIX locale.

The conclusion I have come to is that any further decoupling of Python 3
from the locale encoding will actually depend on getting the PEP 432
bootstrapping changes implemented, reviewed and the PEP approved, so we
have more interpreter infrastructure in place by the time the interpreter
starts trying to figure out all these boundary encoding issues.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale

[Antoine Pitrou]

Antoine Pitrou added the comment:

> The conclusion I have come to is that any further decoupling of Python 3
> from the locale encoding will actually depend on getting the PEP 432
> bootstrapping changes implemented, reviewed and the PEP approved, so we
> have more interpreter infrastructure in place by the time the interpreter
> starts trying to figure out all these boundary encoding issues.

Yeah. My proposal had more to do with the fact that we should some day
switch to utf-8 by default on all POSIX systems, regardless of what the
system advertises as "best encoding".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale

[Nick Coghlan]

Nick Coghlan added the comment:

Antoine Pitrou added the comment:
> Yeah. My proposal had more to do with the fact that we should some day
> switch to utf-8 by default on all POSIX systems, regardless of what the
> system advertises as "best encoding".

Yeah, that seems like a plausible future to me as well, and knowing it's a
step along that path actually gives me more motivation to get back to
working on the startup issues :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1533105] NetBSD build with --with-pydebug causes SIGSEGV

[Stefan Krah]

Stefan Krah added the comment:

Given msg84518 and msg118909 I think this can be closed.

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21375] Fix and test overflow behavior in the C version of heapq

[Raymond Hettinger]

New submission from Raymond Hettinger:

The Py_ssizet indexes can overflow the "childpos" variable:

childpos = 2*pos + 1;/* leftmost child position  */
while (childpos < endpos) ...


http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html

--
assignee: rhettinger
components: Extension Modules
messages: 217389
nosy: rhettinger
priority: normal
severity: normal
stage: needs patch
status: open
title: Fix and test overflow behavior in the C version of heapq
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21376] asyncio docs refer to wrong TimeoutError

[Philip Sequeira]

New submission from Philip Sequeira:

Example: https://docs.python.org/3.4/library/asyncio-task.html
TimeoutError is mentioned several times, and links to the OSError subclass. 
However, the actual TimeoutError raised by asyncio stuff is the one from 
concurrent.futures, which is not compatible. The docs as they are seem to 
suggest that something like "except TimeoutError" would be appropriate, when in 
fact that would not produce the expected behavior; "except 
asyncio.TimeoutError" is what you'd want.

--
assignee: docs@python
components: Documentation
messages: 217390
nosy: docs@python, qmega
priority: normal
severity: normal
status: open
title: asyncio docs refer to wrong TimeoutError
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Antoine Pitrou]

Antoine Pitrou added the comment:

> So I'd suggest, instead of using an hardcoded value, to simply reuse 
> io.DEFAULT_BUFFER_SIZE.
> That way, if some day we decide to change it, all user code wil benefit from 
> the change.

I don't think io.DEFAULT_BUFFER_SIZE makes much sense as a heuristic for the 
gzip module (or compressed files in general). Perhaps gzip should get its own 
DEFAULT_BUFFER_SIZE?

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1284316] Win32: Security problem with default installation directory

[Steve Dower]

Changes by Steve Dower :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thanks for the updated patch, Akira! I'm gonna take a look right now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Charles-François Natali]

Charles-François Natali added the comment:

> I don't think io.DEFAULT_BUFFER_SIZE makes much sense as a heuristic for the 
> gzip module (or compressed files in general). Perhaps gzip should get its own 
> DEFAULT_BUFFER_SIZE?

Do you mean from a namespace point of vue, or from a performance point of view?
Because this size is used to read/write from underlying the file
object, so using the io default would make sense, no?

Sure, it might not be optimal for compressed files, but I gues that
the optimal value is function of the compression-level block size and
many other factors which are just too varied to come up with a
reasonable heuristic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC

[Antoine Pitrou]

Antoine Pitrou added the comment:

I've committed the patch. Thank you very much for contributing!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Sure, it might not be optimal for compressed files, but I gues that
> the optimal value is function of the compression-level block size and
> many other factors which are just too varied to come up with a
> reasonable heuristic.

Well, I think that compressed files in general would benefit from a
larger buffer size than plain binary I/O, but that's just a hunch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19940] ssl.cert_time_to_seconds() returns wrong results if local timezone is not UTC

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7191c37238d5 by Antoine Pitrou in branch 'default':
Issue #19940: ssl.cert_time_to_seconds() now interprets the given time string 
in the UTC timezone (as specified in RFC 5280), not the local timezone.
http://hg.python.org/cpython/rev/7191c37238d5

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21312] Update thread_foobar.h to include timed locking and TLS support

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ok, noone chimed in, so I committed the patch :-)
Thank you very much for your contribution!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21312] Update thread_foobar.h to include timed locking and TLS support

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7bb1bda5dcef by Antoine Pitrou in branch 'default':
Issue #21312: Update the thread_foobar.h template file to include newer 
threading APIs.  Patch by Jack McCracken.
http://hg.python.org/cpython/rev/7bb1bda5dcef

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16104] Compileall script: add option to use multiple cores

[Jim Jewett]

Jim Jewett added the comment:

Trying to put bounds on the disagreements.  Does anyone disagree with any of 
the following:

(1)  compileall currently runs single-threaded in a single process.


(2)  This enhancement intends to allow parallelization by process.


(3)  Users MAY need to express whether they (require/forbid/are expressly 
apathetic concerning) paralellization.

(3A)  There is some doubt that this even needs to be user-controlled.

(3B)  If it is user-controlled, the patch proposes adding a "processes" 
parameter to do this.

(3C)  There have been suggestions of other names (notably "workers"), but *if* 
it is user-controlled, the idea of a new parameter is not controversial.


(4)  Users MAY need to control the degree of parallelization.

(4A)  If so, setting the value of the new parameter to a positive integer > 1 
is an acceptable solution.

(4B)  There is not yet consensus on how to represent "Use multi-processing, 
with the default degree for this system.", "Do NOT use multiprocessing.", or "I 
don't care."

(4C)  Suggested values have included 1, 0, -1, any negative number, None, and 
specific strings.  The precise mapping between some of these and the three 
cases of 4B is not agreed.


(5)  If multiprocessing is explicitly requested, what should happen when it is 
not available?

(5A)  Fall back to the current way, without multi-processing.

(5B)  Fall back to the current way, without multi-processing, but issue a 
Warning.

(5C)  Raise an Exception.  (ValueError, ImportError, NotImplemented?)


(6)  Portions of the documentation unrelated to this should be fixed.  But 
ideally, that would be done separately, and it will NOT be a pre-requisite to 
this patch.


---

Another potential value set

None (the default) ==> let the system parallelize as best it can -- as it does 
in multiprocessing.  If the system picks "not in parallel at all", that is also 
OK, and no warning is raised.

0 ==> Do not parallelize.

positive integers ==> Use that many processes.

negative ==> ValueError

Would these uses of 0 and negative be too surprising for someone?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21376] asyncio docs refer to wrong TimeoutError

[Antoine Pitrou]

Antoine Pitrou added the comment:

Gasp. Perhaps concurrent.futures.TimeoutError can inherit from the standard 
TimeoutError? The following patch doesn't seem to disrupt the test suite:

diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py
--- a/Lib/concurrent/futures/_base.py
+++ b/Lib/concurrent/futures/_base.py
@@ -49,7 +49,7 @@ class CancelledError(Error):
 """The Future was cancelled."""
 pass
 
-class TimeoutError(Error):
+class TimeoutError(Error, TimeoutError):
 """The operation exceeded the given deadline."""
 pass

--
assignee: docs@python -> 
components: +Library (Lib)
nosy: +giampaolo.rodola, gvanrossum, haypo, pitrou, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16104] Compileall script: add option to use multiple cores

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Skip Montanaro]

Skip Montanaro added the comment:

On Mon, Apr 28, 2014 at 1:59 PM, Antoine Pitrou  wrote:
> Well, I think that compressed files in general would benefit from a
> larger buffer size than plain binary I/O, but that's just a hunch.

I agree. When writing my patch, my (perhaps specious) thinking went like this.

* We have a big-ass file, so we compress it.
* On average, when seeking to another point in that file, we probably
want to go a long way.
* It's possible that operating system read-ahead semantics will make
read performance relatively high.
* That would put more burden on the Python code to be efficient.
* Larger buffer sizes will reduce the amount of Python bytecode which
must be executed.

So, if I have a filesystem block size of 8192 bytes, while that would
represent some sort of "optimal" chunk size, in practice, I think
operating system read-ahead and post-read processing of the bytes read
will tend to suggest larger chunk sizes. Hence my naive choice of 16k
bytes for _CHUNK_SIZE in my patch.

Skip

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21376] asyncio docs refer to wrong TimeoutError

[Guido van Rossum]

Guido van Rossum added the comment:

I considered this, and decided against unifying the two TimeoutErrors.

First the builtin TimeoutError is specifically a subclass of OSError 
representing the case where errno is ETIMEDOUT.  But asyncio.TimeoutError means 
nothing of the sort.

Second, the precedent is concurrent.futures.TimeoutError. The asyncio one is 
used under the same conditions as that one.

I think we should just update the links in the docs to be correct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5404] Cross-compiling Python

[Gregory P. Smith]

Gregory P. Smith added the comment:

I haven't tried a cross compile in ages.  If nothing else I don't think this 
issue should be closed until we have at least one buildbot setup to cross 
compile it and run it on the target platform.

That's on my long "todo for python" wish list but I haven't had time to figure 
out how to set that up yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18314] Have os.unlink remove junction points

[Tim Golden]

Tim Golden added the comment:

I'm just pinging #python-dev to see if there's a way to request a buildbot 
build from a specific server-side clone. 

Meanwhile, though, I definitely introduced a change into your code which I 
thought I had reverted, but clearly hadn't! The code, as committed, used 
PyMem_RawAlloc in place of the calloc() call you had, but didn't replace the 
later free() by its PyMem counterpart.

If I don't get any joy with the clone-specific buildbot question, I'll just 
rebuild from your original patch, re-commit, and watch the buildbots.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21372] multiprocessing.util.register_after_fork inconsistency

[Ned Deily]

Changes by Ned Deily :


--
nosy: +sbt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21057] TextIOWrapper does not support reading bytearrays or memoryviews

[Antoine Pitrou]

Antoine Pitrou added the comment:

> My usecase is that I have a binary stream class that internally uses 
> memoryviews.

Ok, I think it is a reasonable use case. I'm gonna look at your patch and give 
it a review.

--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21377] PyBytes_Concat could try to concat in-place

[Antoine Pitrou]

New submission from Antoine Pitrou:

Currently, PyBytes_Concat always creates a new bytes object for the result. 
However, when Py_REFCNT(*pv) == 1, it could instead call _PyBytes_Resize() and 
then concat the second argument in place.

(like e.g. _PyUnicode_Append does)

--
components: Interpreter Core
messages: 217406
nosy: haypo, nikratio, pitrou
priority: normal
severity: normal
status: open
title: PyBytes_Concat could try to concat in-place
type: performance
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21321] itertools.islice() doesn't release reference to the source iterator when the slice is exhausted

[Anton Afanasyev]

Anton Afanasyev added the comment:

Hi Antoine,
I have no found a way to check resource usage in test infrastructure and I 
don't think it could be done carefully. The only method I found to test issue 
is straightforward: just to check source iterator is not referenced from 
itertools.islice() after the latter has been exhausted:


a = [random.random() for i in range(10)]
before = sys.getrefcount(a)
b = islice(a, 5)
for i in b: pass
after = sys.getrefcount(a)
self.assertEqual(before, after)


Attaching "issue21321_2.7_e3217efa6edd_3.diff" and 
"issue21321_3.4_8c8315bac6a8_3.diff" patches with this test included in 
"Lib/test/test_itertools.py".

--
Added file: http://bugs.python.org/file35076/issue21321_2.7_e3217efa6edd_3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21321] itertools.islice() doesn't release reference to the source iterator when the slice is exhausted

[Anton Afanasyev]

Changes by Anton Afanasyev :


Added file: http://bugs.python.org/file35077/issue21321_3.4_8c8315bac6a8_3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Raymond Hettinger]

Raymond Hettinger added the comment:

Without commenting on this specific proposal, I would like to make an overall 
observation that Python is impairing its usability by adding 
too-many-ways-to-it in a number of categories (file descriptor variants of file 
methods, multiple versions of time.time, byte variants of everything that is 
done with strings).  Python 3 was intended to be a cleaner, more learnable 
version of Python.  Instead, it is growing enums, multiple dispatch, and 
multiple variants of every function.   Professional programmers can be well 
served by some of the these tools, but the Python universe is much larger than 
that and the other users are not being well served by these additions (too many 
choices impairs usability and learnability).

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17861] put opcode information in one place

[David Bolen]

David Bolen added the comment:

This generator script breaks the daily OSX DMG builds on my bolen-dmg 
buildslave for the 3.x branch.

The issue is with the use of "with" as the slave uses Python 2.5 to build the 
installer.  Now, that's old, and I'm not even sure how necessary the daily 
builds are to keep running, but I believe the Mac build-installer script still 
tries to target 2.5 as a minimum - though I'm not sure if that's true for all 
build tools.  But the fix is fairly simple (a __future__ import).

See attached patch.

Alternatively, if the minimum Python to build the installer should be bumped, I 
can work on that for the slave.

--
nosy: +db3l
Added file: http://bugs.python.org/file35078/generate_opcode_h.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21321] itertools.islice() doesn't release reference to the source iterator when the slice is exhausted

[Antoine Pitrou]

Antoine Pitrou added the comment:

Anton, the test is wrong: it is taking a reference to the iterable object (the 
list), not the iterator.

To check the reference to the original iterator is released, something like 
this would work:

>>> import itertools, weakref
>>> it = (x for x in (1, 2))
>>> wr = weakref.ref(it)
>>> it = itertools.islice(it, 1)
>>> wr() is None
False
>>> list(it)
[1]
>>> wr() is None  # returns True with the patch, False without
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21321] itertools.islice() doesn't release reference to the source iterator when the slice is exhausted

[Antoine Pitrou]

Antoine Pitrou added the comment:

(note I haven't looked at the C part of the patch)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Charles-François Natali]

Charles-François Natali added the comment:

That could make sense, dunno.

Note that the bz2 module uses a harcoded 8K value.

Note that the buffer size should probably be passed to the open() call.

Also, the allocation is quite peculiar: it uses an exponential buffer
size, starting at a tiny value:

202 # Starts small, scales exponentially
203 self.min_readsize = 100

In short, I think the overall buffering should be rewritten :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Skip Montanaro]

Skip Montanaro added the comment:

On Mon, Apr 28, 2014 at 3:08 PM, Charles-François Natali
 wrote:
> In short, I think the overall buffering should be rewritten :-)

Perhaps so, but I think we should open a separate ticket for that
instead of instituting some feature creep here (no matter how
reasonable the concept or its changes would be).

S

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20962] Rather modest chunk size in gzip.GzipFile

[Charles-François Natali]

Charles-François Natali added the comment:

> Perhaps so, but I think we should open a separate ticket for that
> instead of instituting some feature creep here (no matter how
> reasonable the concept or its changes would be).

Agreed.

The patch looks good to me, so feel free to commit!
(FWIW, gzip apparently uses a fixed-32K buffer read).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor added the comment:

Today I regret os.environb (I added it). If I remember correctly, os.environb 
was added before the PEP 383 (surrogateescape). This PEP makes os.environb 
almost useless. In Python 3, Unicode is the natural choice, and thanks to the 
PEP 383, it's still possible to use any "raw bytes".

argvb can be computed in one line: list(map(os.fsencode, sys.argv)).

I now suggest to close this issue as wontfix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17861] put opcode information in one place

[Ned Deily]

Ned Deily added the comment:

Thanks, David. Ideally, the generator script shouldn't run during an installer 
build since presumably the generated file should be up-to-date in the repo.  
"make touch" could handle that but the installer build does use a separate 
build/object directory and doesn't currently use "make touch".  For Python 3.5, 
the installer build now does need at least a Python 2.6 to build the 
documentation but there's no reason to unnecessarily break the main build with 
such a simple fix available.

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17861] put opcode information in one place

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1fd9c3f6cf68 by Ned Deily in branch 'default':
Issue #17861: Allow generate_opcode_h to run with a system Python 2.5.
http://hg.python.org/cpython/rev/1fd9c3f6cf68

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21378] ImportError: No module named 'concurrent.futures'

[Bill Bergmann]

New submission from Bill Bergmann:

python 3.4 attempting to run example at 
https://docs.python.org/3/library/concurrent.futures.html
17.4.2.1

$ python3 17_4_2.py 
Traceback (most recent call last):
  File "", line 2195, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "17_4_2.py", line 1, in 
import concurrent.futures
ImportError: No module named 'concurrent.futures'; 'concurrent' is not a package

os: OS X 10.6

from $env:
PATH=/Library/Frameworks/Python.framework/Versions/3.4/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:
 ...

PYTHONPATH=/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/:/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/:
 ...

--
components: Library (Lib)
messages: 217418
nosy: Bill.Bergmann
priority: normal
severity: normal
status: open
title: ImportError: No module named 'concurrent.futures'
type: crash
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17386] Bring Doc/make.bat as close to Doc/Makefile as possible

[Zachary Ware]

Zachary Ware added the comment:

New patch, fixes a typo bug in a useless statement (by removing the statement) 
and a few pesky tabs that made their way into make.bat.  Also, a little better 
organization in the vars at the top.

--
Added file: http://bugs.python.org/file35079/issue17386.v2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21378] ImportError: No module named 'concurrent.futures'

[Claudiu.Popa]

Claudiu.Popa added the comment:

Do you have a module/file named concurrent.py in your PATH?

--
nosy: +Claudiu.Popa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21379] StopIteration is silenced when raised by generator within context manager

[Hannan Aharonov]

New submission from Hannan Aharonov:

If the code that defines a context manager performs some operations on a 
generator that cause it to raise StopIteration, this exception is propagated to 
the context manager's __exit__() method and there swallowed by a try..except 
block. 

This can cause unexpected/unintuitive behaviour.
The following simplified example shows an infinite loop caused by this:

---
from contextlib import contextmanager

@contextmanager
def manager(gen):
yield
gen.next()

emptygen = (_ for _ in ())

while True:
with manager(emptygen):
print "Infinite Loop"
---

In this case, even though the generator is empty, the loop will never stop, 
because the generator's StopIteration exception is swallowed by the context 
manager. (If `gen.next()` was wrapped by try..except that reraises exception 
as, say, RuntimeError, the program would stop.)

I've looked at the source (here: 
http://hg.python.org/cpython/file/tip/Lib/contextlib.py line 67)
and I understand why this is happening: the context manager can't tell the 
difference between the two kinds of StopIteration - one that is raised by its 
own completion, and one that is raised by a call in its body. 

I don't know if this is a bug or expected behavior of nested generators. In any 
case, I haven't seen this issue mentioned in the documentation, and the 
behavior is unintuitive.

I've searched the bug tracker and found a similar issue that handled swallowing 
of StopIteration in the code wrapped by the context manager 
(http://bugs.python.org/issue1705170), and another similar issue 
(http://bugs.python.org/issue16610) that was closed without resolution.

--
components: Interpreter Core, Library (Lib)
messages: 217421
nosy: hannan.aha, ncoghlan
priority: normal
severity: normal
status: open
title: StopIteration is silenced when raised by generator within context manager
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21379] StopIteration is silenced when raised by generator within context manager

[Hannan Aharonov]

Changes by Hannan Aharonov :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21378] ImportError: No module named 'concurrent.futures'

[Ned Deily]

Ned Deily added the comment:

FWIW, works for me using the python.org 3.4 64-bin installer.  What is the 
output of:

python3 -c 'import sys;print(sys.version)'

As Claudiu suggests, check for a concurrent.py shadowing the standard library 
version.

Also, why are you setting PYTHONPATH to include 
/Library/Frameworks/Python.framework/Versions/{3.4,2.7}/lib/python3.4/site-packages/?
  Those are the standard locations for site-packages are included automatically 
in sys.path by the respective interpreters.  Setting PYTHONPATH should not 
normally be needed.

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21233] Add *Calloc functions to CPython memory allocation API

[STINNER Victor]

STINNER Victor added the comment:

> Hmm, obmalloc.c changed as well, so already the gcc optimizer can take
> different paths and produce different results.

If decimal depends on allocator performances, you should maybe try to
implement a freelist.

> Also I did set mpd_callocfunc to PyMem_Calloc().

I don't understand. 2% slowdown is when you use calloc? Do you have the
same speed if you don't use calloc? According to my benchmarks, calloc is
slower if some bytes are modified later.

> The bytes() speedup is very nice. Allocations that took one second
> are practically instant now.

Is it really useful? Who need bytes(10**8) object?

Faster creation of bytearray(int) may be useful in real applications. I
really like bytearray and memoryview to avoid memory copies.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21380] timezone support in strftime methods broken

[Michael P. Soulier]

New submission from Michael P. Soulier:

msoulier@cappuccino:~$ python
Python 2.7.3 (default, Mar 13 2014, 11:03:55)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> now = datetime.now()
>>> now.strftime("%z")
''
>>> import time
>>> time.strftime("%z", time.localtime(time.time()))
'+'
>>>

I think those calls should be printing the UTC offset, no?

--
components: Extension Modules
messages: 217424
nosy: msoulier
priority: normal
severity: normal
status: open
title: timezone support in strftime methods broken
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[STINNER Victor]

STINNER Victor added the comment:

Please don't backport this "feature". We had to wait 20 years before someone 
requested the feature, but only a few months before the first user reported an 
issue ("regression"?).

IMO it would be much better to use explicitly a random.SystemRandom instance 
which would keep the fd open, and only use it if you hit the bug (*many* 
threads calling os.urandom in parallel.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7511] msvc9compiler.py: ValueError when trying to compile with VC Express

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +brian.curtin, tim.golden

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19217] Calling assertEquals for moderately long list takes too long

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ezio, do you intend to add a test to your patch?

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21225] io.py: Improve docstrings for classes

[Roundup Robot]

Roundup Robot added the comment:

New changeset 6e23afdee4e4 by Andrew Kuchling in branch '2.7':
#21225: copy docstrings from base classes
http://hg.python.org/cpython/rev/6e23afdee4e4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21225] io.py: Improve docstrings for classes

[Antoine Pitrou]

Antoine Pitrou added the comment:

I've backported Andrew's change.

--
nosy: +pitrou
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Donald Stufft]

Donald Stufft added the comment:

Well except random.SystemRandom doesn't keep the file open (At least in 2.7) 
and actually it just calls os.urandom under the covers, also it doesn't make it 
very nice to get a glob of random bytes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5404] Cross-compiling Python

[Stefan Krah]

Stefan Krah added the comment:

Here's a cross-compile script for arm-linux-gnueabi. Building 3.4 works
on Ubuntu 14.04.  I cannot run the tests, since I only have an old
Debian-ARM qemu image with the wrong glibc version.

For 3.5 we have a regression due to the new matrix operator.

--
Added file: http://bugs.python.org/file35080/arm-linux-gnueabi.sh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21353] document Popen.args attribute

[Antoine Pitrou]

Antoine Pitrou added the comment:

Gregory, do you think this is ok to document?

--
nosy: +gregory.p.smith, pitrou
stage:  -> patch review
versions:  -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17227] devguide: buggy heading numbers

[Antoine Pitrou]

Antoine Pitrou added the comment:

I agree with Chris, changing the tocdepth doesn't sound like the right 
solution. Is this a Sphinx bug? Georg, could you please take a look?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21305] PEP 466: update os.urandom

[Donald Stufft]

Donald Stufft added the comment:

Just verified that 3.x also does not exhibit this behavior with 
random.SystemRandom (except implicitly through os.urandom doing it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21273] don't defined socket constants which are not implemented for GNU/Hurd

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ok, rejecting.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com