[issue39350] Remove deprecated fractions.gcd()

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

bpo-22486 added math.gcd() and deprecated fractions.gcd() in Python 3.5: commit 
48e47aaa28d6dfdae128142ffcbc4b0642422e90.

The function was deprecated during 4 cycles (3.5, 3.6, 3.7, 3.8): I propose 
attached PR to remove it.

--
components: Library (Lib)
messages: 360095
nosy: vstinner
priority: normal
severity: normal
status: open
title: Remove deprecated fractions.gcd()
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22486] Add math.gcd()

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +17417
pull_request: https://github.com/python/cpython/pull/18021

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39350] Remove deprecated fractions.gcd()

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17416
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18021

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39351] Remove base64.encodestring() and base64.decodestring() aliases, deprecated since Python 3.1

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

base64.encodestring() and base64.decodestring() are aliases deprecated since 
Python 3.1: encodebytes() and decodebytes() should be used instead.

In Python 3, "string" means Unicode, whereas these functions really work at the 
bytes level:

>>> base64.encodestring("text")
TypeError: expected bytes-like object, not str

>>> base64.decodestring("text")
TypeError: expected bytes-like object, not str

encodebytes() and decodebytes() names are explicit on the expected types (bytes 
or bytes-like).

This issue is similar to bpo-38916: "Remove array.fromstring() and 
array.tostring() aliases, deprecated since Python 3.2".

Attached PR removes the deprecated aliases base64.encodestring() and 
base64.decodestring().

--
components: Library (Lib)
messages: 360096
nosy: vstinner
priority: normal
severity: normal
status: open
title: Remove base64.encodestring() and base64.decodestring() aliases, 
deprecated since Python 3.1
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39351] Remove base64.encodestring() and base64.decodestring() aliases, deprecated since Python 3.1

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17418
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18022

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39351] Remove base64.encodestring() and base64.decodestring() aliases, deprecated since Python 3.1

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

The deprecation was introduced 11 years ago in bpo-3613 by this change:

commit b54d801280e3f510782e2855504710947d10f053
Author: Georg Brandl 
Date:   Thu Jun 4 09:11:51 2009 +

#3613: add base64.encodebytes and decodebytes as the new spelling of 
encodestring and decodestring; deprecate the latter.

The aliases were deprecated for 7 cycles (Python 3.1 to Python 3.8).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39352] Remove the formatter module, deprecated since Python 3.4

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

The formatter module has been deprecated in Python 3.4 by bpo-18716: commit 
1448ecf470013cee63c0682f615c5256928dc6b0.

In 2014, its removal was scheduled in Python 3.6: commit 
29636aeaccaf6a1412e0dc7c230db29cccf68381.

But bpo-25407 cancelled the removal from Python 3.6: commit 
5ad5a7d31f5328c73df523b6ade330d88573717e

"The new PEP 4 policy of any module existing in both 2.7 and 3.5 applies here, 
hence the module will be with us for a bit longer."

In the meanwhile, I'm not aware of anyone opposed to the removal.

Python 2.7 reached it's end of life, so I propose to remove the module:
https://docs.python.org/3.8/library/formatter.html

If someone needs this module, it's a single formatter.py file: it can easily be 
copied from Python 3.8. The intent here is to reduce the size of the standard 
library to remove the maintenance burden on Python core developers.

Note: I'm surprised, but it seems like the formatter module has no test!?

Attached PR removes the module.

--
components: Library (Lib)
messages: 360098
nosy: brett.cannon, vstinner
priority: normal
severity: normal
status: open
title: Remove the formatter module, deprecated since Python 3.4
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39352] Remove the formatter module, deprecated since Python 3.4

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17419
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18023

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39352] Remove the formatter module, deprecated since Python 3.4

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

I did a quick & dirty code search in GitHub to check for "import formatter". 
There is no "import formatter" in the 5 first pages of code search in the 
Python programming language. IMHO it seems that this module is basically 
unused. To be honest, I even didn't know that it existed!?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39353] Deprecate the binhex module

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

The binhex module encodes and decodes Apple Macintosh binhex4 data. It was 
originally developed for TRS-80. In the 1980s and early 1990s it was used on 
classic Mac OS 9 to encode binary email attachments.

Mac OS 9 is now heavily outdated, replaced by "macOS" (previously known as "Mac 
OS X").

I propose to emit a DeprecationWarning in the binhex module and document that 
it's deprecated.

I don't think that we have to schedule its removal yet, it can be decided 
later. A first deprecation warning emitted at runtime might help to warn last 
users, if there is any.

There are two binhex open issues:

* bpo-29566: no activity for almost 2 years (February 2017)
* bpo-34063: no activity for 1 year 1/2 (July 2018)

If we ignore global cleanup (done on the whole Python code base at once, not 
specific to binhex), the binhex was basically untouched since it has been 
ported to Python 3 (10 years ago). Maybe it means that it is very stable, which 
can be seen as a quality ;-)

I looked for "import binhex" in the first 10 pages of GitHub code search 
(restricted to Python programming language). I mostly found copies of Python 
test_binhex.py, no real usage of binhex.

On Stackoverflow, the latest questions about binhex has been asked in 2012:
https://stackoverflow.com/questions/12467973/binhex-decoding-using-java-code

I also found an answer suggesting to use binascii.a2b_hex() to decode a string 
the hexadecimal string "2020202020202020202020205635514d385a5856":
https://stackoverflow.com/questions/9683278/how-to-get-hard-disk-drivers-serial-number-in-python/9683837#9683837

But binascii.unhexlify() does the same than binascii.a2b_hex().

Attached PR deprecates binhex.

--
components: Library (Lib)
messages: 360100
nosy: serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Deprecate the binhex module
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39349] Add "cancel" parameter to concurrent.futures.Executor.shutdown()

2020-01-16 Thread Kyle Stanley


Change by Kyle Stanley :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39354] collections.UserString format and format_map return a string

2020-01-16 Thread bizywizy


New submission from bizywizy :

collections.UserString.format and collections.UserString.format_map return a 
string instaed of UserString.

This is quite weird because I expect that the %-syntax and `format` method have 
to produce the same result.

```
>>> isinstance(UserString('Hello %s') % 'World', UserString)
True
>>> isinstance(UserString('Hello {}').format('World'), UserString)
False
```

--
components: Library (Lib)
messages: 360101
nosy: bizywizy
priority: normal
severity: normal
status: open
title: collections.UserString format and format_map return a string
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39353] Deprecate the binhex module

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17420
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18025

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39351] Remove base64.encodestring() and base64.decodestring() aliases, deprecated since Python 3.1

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset 210c19e3c5b86535a73487fa737752de8eb1d866 by Victor Stinner in 
branch 'master':
bpo-39351: Remove base64.encodestring() (GH-18022)
https://github.com/python/cpython/commit/210c19e3c5b86535a73487fa737752de8eb1d866


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39351] Remove base64.encodestring() and base64.decodestring() aliases, deprecated since Python 3.1

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword

2020-01-16 Thread Keith

New submission from Keith :

The Python library will not compile with a C++2020 compiler because the code 
uses the reserved “module” keyword
 
For example, in warnings.h, we have the following code:
 
#ifndef Py_LIMITED_API
PyAPI_FUNC(int) PyErr_WarnExplicitObject(
PyObject *category,
PyObject *message,
PyObject *filename,
int lineno,
PyObject *module,
PyObject *registry);
 
 
In modsupport.h we have the following code:
PyAPI_FUNC(int) PyModule_ExecDef(PyObject *module, PyModuleDef *def);
 
We can fix this by using a different identifier, for example “pyModule” instead 
of “module”

--
components: C API
messages: 360103
nosy: aCuria
priority: normal
severity: normal
status: open
title: The Python library will not compile with a C++2020 compiler because the 
code uses the reserved “module” keyword
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39354] collections.UserString format and format_map return a string

2020-01-16 Thread bizywizy


Change by bizywizy :


--
keywords: +patch
pull_requests: +17421
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18026

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39318] NamedTemporaryFile could cause double-close on an fd if _TemporaryFileWrapper throws

2020-01-16 Thread Paul Ollis


Paul Ollis  added the comment:

> I thought that if this raises a (normal) exception, it always means that it
> did not have overtaken the `fd`, i.e. never results in an unreferenced file
> object which has taken ownership of `fd`.

The current CPython implementation does not guard against this happening. Some
incorrect combinations of arguments will definitely cause the problem. It is
also possible that it could occur under other circumstances.

> It this is not true right now, you are right that this is problematic. But
> this should be simple to fix on the CPython side, right? I.e. to make sure
> whenever some exception is raised here, even if some temporary file object
> already was constructed, it will not close `fd`. It should be consistent in
> this behavior, otherwise indeed, the semantics are broken.

I agree. I think it should be fairly simple to fix for CPython.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39353] Deprecate the binhex module

2020-01-16 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

binhex provides an encoding different from binascii.a2b_hex() and 
binascii.unhexlify().

If deprecate the binhex module, we perhaps should deprecate also 
binascii.b2a_hqx(), binascii.a2b_hqx(), binascii.rlecode_hqx(), 
binascii.rledecode_hqx() and binascii.crc_hqx().

But it would be nice to create a separate package on PyPI that provides such 
functionality before removing it from the stdlib.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword

2020-01-16 Thread Karthikeyan Singaravelan


Change by Karthikeyan Singaravelan :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword

2020-01-16 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

Names of arguments can be just removed from function declarations in header 
files.

--
keywords: +easy (C)
nosy: +serhiy.storchaka
stage:  -> needs patch
type:  -> enhancement
versions:  -Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39356] zipfile suprising "except DeprecationWarning:" block

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

Lib/zipfile.py contains the following code:

try:
filename, flag_bits = zinfo._encodeFilenameFlags()
centdir = struct.pack(structCentralDir,
  stringCentralDir, create_version,
  zinfo.create_system, extract_version, zinfo.reserved,
  flag_bits, zinfo.compress_type, dostime, dosdate,
  zinfo.CRC, compress_size, file_size,
  len(filename), len(extra_data), len(zinfo.comment),
  0, zinfo.internal_attr, zinfo.external_attr,
  header_offset)
except DeprecationWarning:
print((structCentralDir, stringCentralDir, create_version,
   zinfo.create_system, extract_version, zinfo.reserved,
   zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,
   zinfo.CRC, compress_size, file_size,
   len(zinfo.filename), len(extra_data), len(zinfo.comment),
   0, zinfo.internal_attr, zinfo.external_attr,
   header_offset), file=sys.stderr)
raise

It is not considered as good programmating method to put print() statement in 
production code: usually, it's only used for debugging :-)

The "except DeprecationWarning:" with its print has been added 12 years ago by:

commit bf02e3bb21b2d75cba4ce409a14ae64dbc2dd6d2
Author: Gregory P. Smith 
Date:   Wed Mar 19 03:14:41 2008 +

Fix the struct module DeprecationWarnings that zipfile was triggering by
removing all use of signed struct values.

test_zipfile and test_zipfile64 pass.  no more warnings.

But I don't recall any complain about a DeprecationWarning on struct.pack() in 
zipfile.

I propose attached PR to remove it.

--
components: Library (Lib)
messages: 360107
nosy: vstinner
priority: normal
severity: normal
status: open
title: zipfile suprising "except DeprecationWarning:" block
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36051] Drop the GIL during large bytes.join operations?

2020-01-16 Thread Bruce Merry


Bruce Merry  added the comment:

I think I've addressed the concerns that were raised in this bug, but let me 
know if I've missed any.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39356] zipfile suprising "except DeprecationWarning:" block

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17423
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18027

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39353] Deprecate the binhex module

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

> But it would be nice to create a separate package on PyPI that provides such 
> functionality before removing it from the stdlib.

This issue is *not* about removing the module :-) Only to mark it as deprecated.

> If deprecate the binhex module, we perhaps should deprecate also 
> binascii.b2a_hqx(), binascii.a2b_hqx(), binascii.rlecode_hqx(), 
> binascii.rledecode_hqx() and binascii.crc_hqx().

I have no opinion on that. I mean: in case of doubt, I prefer to not deprecate 
them. I never used any of these functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

Qt has a similar issue with "slots": bpo-1086854 and bpo-38007.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36051] Drop the GIL during large bytes.join operations?

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39350] Remove deprecated fractions.gcd()

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset 4691a2f2a2b8174a6c958ce6976ed5f3354c9504 by Victor Stinner in 
branch 'master':
bpo-39350: Remove deprecated fractions.gcd() (GH-18021)
https://github.com/python/cpython/commit/4691a2f2a2b8174a6c958ce6976ed5f3354c9504


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22486] Add math.gcd()

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset 4691a2f2a2b8174a6c958ce6976ed5f3354c9504 by Victor Stinner in 
branch 'master':
bpo-39350: Remove deprecated fractions.gcd() (GH-18021)
https://github.com/python/cpython/commit/4691a2f2a2b8174a6c958ce6976ed5f3354c9504


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39350] Remove deprecated fractions.gcd()

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39354] collections.UserString format and format_map return a string

2020-01-16 Thread Eric V. Smith


Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39353] Deprecate the binhex module

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

> I also found an answer suggesting to use binascii.a2b_hex() to decode a 
> string the hexadecimal string "2020202020202020202020205635514d385a5856":
https://stackoverflow.com/questions/9683278/how-to-get-hard-disk-drivers-serial-number-in-python/9683837#9683837

Oh wait, it is just unrelated :-) It's a typo in the StackOverflow answer: 
"import binhex; binascii.a2b_hex(...)". It should be "import binascii" 
obviously ;-) I didn't notice at the first read. binhex has no a2b_hex() 
function :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39357] bz2: Remove deprecated buffering parameter of bz2.BZ2File

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

The "buffering" parameter of bz2.BZ2File is deprecated for 12 years.

Using it was emitting a DeprecationWarning since Python 3.0.

Attached PR removes it.

--
components: Library (Lib)
messages: 360114
nosy: vstinner
priority: normal
severity: normal
status: open
title: bz2: Remove deprecated buffering parameter of bz2.BZ2File
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword

2020-01-16 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

Qt has different issue. "slots" is not a keyword, and the issue can be avoided 
by including Python.h before Qt.h or undefining the "slots" macro.

It could be a harder issue if "module" would be a field name of a public 
structure. But names of arguments are not part of the API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39357] bz2: Remove deprecated buffering parameter of bz2.BZ2File

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +17424
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18028

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35247] test.test_socket.RDSTest.testPeek hangs indefinitely

2020-01-16 Thread Kresimir Kumericki


Kresimir Kumericki  added the comment:

Sorry if this necrobumping is inappropriate, but I just experienced the same 
issue of test.test_socket.RDSTest.testPeek() hanging indefinitely, while trying 
to recompile Python 3.8.1 on Arch Linux.

--
nosy: +kkumer
versions: +Python 3.8 -Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

2020-01-16 Thread hai shi


Change by hai shi :


--
pull_requests: +17425
pull_request: https://github.com/python/cpython/pull/18030

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39358] test_code.CoExtra leads to double-free when ce_size >1

2020-01-16 Thread Sergei Lebedev


New submission from Sergei Lebedev :

tl;dr Passing a Python function as a freefunc to _PyEval_RequestCodeExtraIndex 
leads to double-free. In general, I think that freefunc should not be allowed 
to call other Python functions.

---

test_code.CoExtra registers a new co_extra slot with a ctypes-wrapped freefunc

# All defined in globals().
LAST_FREED = None

def myfree(ptr):
global LAST_FREED
LAST_FREED = ptr

FREE_FUNC = freefunc(myfree)
FREE_INDEX = RequestCodeExtraIndex(FREE_FUNC)

Imagine that we have registered another co_extra slot FOO_INDEX of type Foo and 
a freefunc FreeFoo. Furthermore, assume that 

* FOO_INDEX < FREE_INDEX
* FOO_INDEX is set on any executed code object including myfree. 

Consider what happens when we collect the globals() of the test_code module. 
myfree is referenced by globals() and FREE_FUNC. If FREE_FUNC is DECREF'd 
first, then by the time we get to myfree it has a refcount of 1 and DECREF'ing 
it leads to a code_dealloc call. Recall that the code object corresponding to 
myfree has two co_extra slots: 

* FOO_INDEX pointing to some Foo*, and
* FREE_INDEX with a value of NULL.

So, code_dealloc will first call FreeFoo (because FOO_INDEX < FREE_INDEX) and 
then the ctypes wrapper of myfree. The following sequence of calls looks 
roughly like this

_CallPythonObject
...
PyEval_EvalCodeEx
_PyEval_EvalCodeWithName
frame_dealloc
code_dealloc # !

The argument of the last code_dealloc call is *the same* myfree code object 
(!). This means that code_dealloc will attempt to call FreeFoo on an already 
free'd pointer leading to a crash.

--
components: Tests
messages: 360117
nosy: slebedev
priority: normal
severity: normal
status: open
title: test_code.CoExtra leads to double-free when ce_size >1
type: crash
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39358] test_code.CoExtra leads to double-free when ce_size >1

2020-01-16 Thread Sergei Lebedev


Change by Sergei Lebedev :


--
nosy: +dino.viehland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34775] pathlib.PurePath division raises TypeError instead of returning NotImplemented

2020-01-16 Thread Maor Kleinberger


Change by Maor Kleinberger :


--
pull_requests: +17426
pull_request: https://github.com/python/cpython/pull/12361

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message

2020-01-16 Thread Daniel Hillier


New submission from Daniel Hillier :

Setting the ZipFile.pwd attribute directly skips the check to ensure the 
password is a bytes object and, if not, return a user friendly TypeError("pwd: 
expected bytes, got ") informing them of that.

--
components: Library (Lib)
messages: 360118
nosy: dhillier
priority: normal
severity: normal
status: open
title: zipfile: add missing "pwd: expected bytes, got str" exception message
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread gaborbernat


New submission from gaborbernat :

Assume the following code:

```python
from multiprocessing.pool import ThreadPool
class A(object):
def __init__(self):
self.pool = ThreadPool()
def __del__(self):
self.pool.close()
self.pool.join()
a = A()
print(a)
```

The code snippet above hangs forever on Python 3.8+ (works ok on Python 3.7 and 
earlier). An example output where I've added some extra prints on to the thread 
joins:

```
<__main__.A object at 0x1104d6070>
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
done
join thread  None
```

I've tested on MacOs, but could replicate on Linux too within the CI.

--
components: Library (Lib)
messages: 360119
nosy: gaborbernat, vstinner
priority: normal
severity: normal
status: open
title: python3.8 regression - ThreadPool join via __del__ hangs forever
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread gaborbernat


Change by gaborbernat :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message

2020-01-16 Thread Daniel Hillier


Change by Daniel Hillier :


--
keywords: +patch
pull_requests: +17427
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18031

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39361] [C API] Document PyTypeObject.tp_print removal in What's New In Python 3.9

2020-01-16 Thread STINNER Victor


New submission from STINNER Victor :

commit aacc77fbd77640a8f03638216fa09372cc21673d
Author: Jeroen Demeyer 
Date:   Wed May 29 20:31:52 2019 +0200

bpo-36974: implement PEP 590 (GH-13185)


Co-authored-by: Jeroen Demeyer 
Co-authored-by: Mark Shannon 

removed PyTypeObject.tp_print:

diff --git a/Include/cpython/object.h b/Include/cpython/object.h
index ba52a48358..a65aaf6482 100644
--- a/Include/cpython/object.h
+++ b/Include/cpython/object.h
@@ -182,7 +182,7 @@ typedef struct _typeobject {
 /* Methods to implement standard operations */
 
 destructor tp_dealloc;
-printfunc tp_print;
+Py_ssize_t tp_vectorcall_offset;
 getattrfunc tp_getattr;
 setattrfunc tp_setattr;
 PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2)


Would it be possible to just document it in What's New in Python 3.9? Near:
https://docs.python.org/dev/whatsnew/3.9.html#build-and-c-api-changes


For example, this incompatible change broke zbar project:
https://bugzilla.redhat.com/show_bug.cgi?id=1791745

--
components: C API
messages: 360120
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Document PyTypeObject.tp_print removal in What's New In Python 
3.9
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36974] Implement PEP 590

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

I created bpo-39361: [C API] Document PyTypeObject.tp_print removal in What's 
New In Python 3.9.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31031] Unify duplicate bits_in_digit and bit_length

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset c5b79003f5fe6aa28a2a028680367839ba8677db by Victor Stinner 
(Niklas Fiekas) in branch 'master':
bpo-31031: Unify duplicate bits_in_digit and bit_length (GH-2866)
https://github.com/python/cpython/commit/c5b79003f5fe6aa28a2a028680367839ba8677db


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31031] Unify duplicate bits_in_digit and bit_length

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

Thanks Niklas Fiekas for merging duplicated code ;-) It should make 
factorial_partial_product() a little bit more efficient!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31031] Unify duplicate bits_in_digit and bit_length

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

"On BSD, there's a library function called fls() that we could use, and GCC 
provides __builtin_clz()."

See also rejected bpo-29782 and GH-594.

If someone wants to experiment that, I suggest to add 
Include/internal/pycore_pymath.h, move _Py_bit_length() there and convert it to 
a static inline function.

Since the code is now unified, it makes a little bit more sense to add 
specialized code, *if* the code is not too complex.

I'm still not sure that GH-594 is worth it, since it was not possible to see 
the speedup on a microbenchmark :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31031] Unify duplicate bits_in_digit and bit_length

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39357] bz2: Remove deprecated buffering parameter of bz2.BZ2File

2020-01-16 Thread STINNER Victor


Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39357] bz2: Remove deprecated buffering parameter of bz2.BZ2File

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset 9baf242fc733ab8a52a0b6201d95c6fdb8251745 by Victor Stinner in 
branch 'master':
bpo-39357: Remove buffering parameter of bz2.BZ2File (GH-18028)
https://github.com/python/cpython/commit/9baf242fc733ab8a52a0b6201d95c6fdb8251745


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39362] add option to make chunksize adaptive for multiprocessing.pool methods

2020-01-16 Thread Forest


Change by Forest :


--
components: +Library (Lib) -macOS
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39362] add option to make chunksize adaptive for multiprocessing.pool methods

2020-01-16 Thread Forest


New submission from Forest :

In the multiprocessing Pool methods like map, chunksize determines the 
trade-off between computation per task and inter-process communication. Setting 
chunksize appropriately has a large effect on efficiency.


However, for users directly interacting with the map methods, the way to find 
the appropriate chunksize is by manually checking different sizes and observing 
the program behavior.

For library developers, you have to hope that you set an reasonable value that 
will work okay across different hardware, operating systems, and task 
characteristics.

Generally, users of these methods want maximum throughput. It would be great if 
the map-like methods could adapt their chunksize towards that goal.

Something along the lines of this:

n_items = 0
queue = Queue(N)
while True:
chunk = tuple(itertools.islice(iterable, chunk_size))
if chunk:
queue.put(chunk)

n_items += chunk_size
i += 1

if i % 10:
time_delta = max(time.perf_counter() - t0, 0.0001)

current_rate = n_items / time_delta

# chunk_size is always either growing or shrinking, if
# the shrinking led to a faster rate, keep
# shrinking. Same with growing. If the rate decreased,
# reverse directions
if current_rate < last_rate:
multiplier = 1 / multiplier

chunk_size = int(min(max(chunk_size * multiplier, 1), 
upper_bound))

last_rate = current_rate
n_items = 0
t0 = time.perf_counter()


Would such a feature be desirable?

--
components: macOS
messages: 360126
nosy: fgregg, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: add option to make chunksize adaptive for multiprocessing.pool methods

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38076] Make struct module PEP-384 compatible

2020-01-16 Thread Pablo Galindo Salgado


Pablo Galindo Salgado  added the comment:

Executing this simple code after this commit segfaults:

from multiprocessing.pool import Pool
class A(object):
def __init__(self):
self.pool = Pool()
def __del__(self):
self.pool.close()
self.pool.join()
a = A()

[1]28019 segmentation fault  ./python.exe ../lel.py

The reason is that there is a call to PyModule_GetState with the module being 
NULL:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS 
(code=1, address=0x8)
  * frame #0: 0x0001000b9737 
python.exe`PyModule_GetState(m=0x) at moduleobject.c:565:10
frame #1: 0x0001013d5972 
_struct.cpython-39d-darwin.so`s_pack(self=0x0001032fae50, 
args=0x0001033051f8, nargs=1) at _struct.c:1858:5
frame #2: 0x0001013d4a8a 
_struct.cpython-39d-darwin.so`pack(self=0x0001013be890, 
args=0x0001033051f0, nargs=2) at _struct.c:2165:14
frame #3: 0x0001000b6e5d 
python.exe`cfunction_vectorcall_FASTCALL(func=0x0001013beb30, 
args=0x0001033051f0, nargsf=9223372036854775810, 
kwnames=0x) at methodobject.c:380:24
frame #4: 0x0001001efa89 
python.exe`_PyObject_Vectorcall(callable=0x0001013beb30, 
args=0x0001033051f0, nargsf=9223372036854775810, 
kwnames=0x) at abstract.h:106:21
frame #5: 0x0001001efbfa 
python.exe`call_function(tstate=0x000101003ec0, 
pp_stack=0x7ffeefbf6e60, oparg=2, kwnames=0x) at 
ceval.c:4984:13
frame #6: 0x0001001ea00c 
python.exe`_PyEval_EvalFrameDefault(f=0x000103305050, throwflag=0) at 
ceval.c:3465:23
frame #7: 0x0001001d8c87 
python.exe`PyEval_EvalFrameEx(f=0x000103305050, throwflag=0) at 
ceval.c:737:12
frame #8: 0x00010004374e 
python.exe`function_code_fastcall(co=0x000103165380, 
args=0x000103301400, nargs=2, globals=0x000103131d10) at call.c:293:14
frame #9: 0x00010004317a 
python.exe`_PyFunction_Vectorcall(func=0x00010324d910, 
stack=0x0001033013f0, nargsf=9223372036854775810, 
kwnames=0x) at call.c:331:20
frame #10: 0x0001001efa89 
python.exe`_PyObject_Vectorcall(callable=0x00010324d910, 
args=0x0001033013f0, nargsf=9223372036854775810, 
kwnames=0x) at abstract.h:106:21
frame #11: 0x0001001efbfa 
python.exe`call_function(tstate=0x000101003ec0, 
pp_stack=0x7ffeefbf8950, oparg=2, kwnames=0x) at 
ceval.c:4984:13
frame #12: 0x0001001ea09c 
python.exe`_PyEval_EvalFrameDefault(f=0x000103301250, throwflag=0) at 
ceval.c:3482:23
frame #13: 0x0001001d8c87 
python.exe`PyEval_EvalFrameEx(f=0x000103301250, throwflag=0) at 
ceval.c:737:12
frame #14: 0x0001001f126c 
python.exe`_PyEval_EvalCodeWithName(_co=0x000103159930, 
globals=0x000103131d10, locals=0x, args=0x0001032b2f68, 
argcount=2, kwnames=0x, kwargs=0x0001032b2f78, kwcount=0, 
kwstep=1, defs=0x000103173f18, defcount=2, kwdefs=0x, 
closure=0x, name=0x00010315c2e0, 
qualname=0x000103158430) at ceval.c:4296:14
frame #15: 0x0001000435d0 
python.exe`_PyFunction_Vectorcall(func=0x00010324d230, 
stack=0x0001032b2f68, nargsf=9223372036854775810, 
kwnames=0x) at call.c:356:12
frame #16: 0x0001001efa89 
python.exe`_PyObject_Vectorcall(callable=0x00010324d230, 
args=0x0001032b2f68, nargsf=9223372036854775810, 
kwnames=0x) at abstract.h:106:21
frame #17: 0x0001001efbfa 
python.exe`call_function(tstate=0x000101003ec0, 
pp_stack=0x7ffeefbfa610, oparg=2, kwnames=0x) at 
ceval.c:4984:13
frame #18: 0x0001001ea09c 
python.exe`_PyEval_EvalFrameDefault(f=0x0001032b2de0, throwflag=0) at 
ceval.c:3482:23
frame #19: 0x0001001d8c87 
python.exe`PyEval_EvalFrameEx(f=0x0001032b2de0, throwflag=0) at 
ceval.c:737:12
frame #20: 0x00010004374e 
python.exe`function_code_fastcall(co=0x0001031d5860, 
args=0x000103302fd8, nargs=2, globals=0x0001031d1b90) at call.c:293:14
frame #21: 0x00010004317a 
python.exe`_PyFunction_Vectorcall(func=0x000103256b90, 
stack=0x000103302fc8, nargsf=9223372036854775810, 
kwnames=0x) at call.c:331:20
frame #22: 0x0001001efa89 
python.exe`_PyObject_Vectorcall(callable=0x000103256b90, 
args=0x000103302fc8, nargsf=9223372036854775810, 
kwnames=0x) at abstract.h:106:21
frame #23: 0x0001001efbfa 
python.exe`call_function(tstate=0x000101003ec0, 
pp_stack=0x7ffeefbfc100, oparg=2, kwnames=0x) at 
ceval.c:4984:13
frame #24: 0x0001001ea09c 
python.exe`_PyEval_EvalFrameDefault(f=0x000103302e50, throwflag=0) at 
ceval.c:3482:23
frame #25: 0x0001001d8c87 
python.exe`PyEval_EvalFrameEx(f=0x000103302e50, throwflag=0) at 
ceval.c:737:12
frame #26: 0

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

2020-01-16 Thread hai shi


Change by hai shi :


--
pull_requests: +17428
pull_request: https://github.com/python/cpython/pull/18032

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread gaborbernat


gaborbernat  added the comment:

An FYI update, the code is used by swagger-codegen to generate HTTP clients 
(https://github.com/swagger-api/swagger-codegen/blob/master/modules/swagger-codegen/src/main/resources/python/api_client.mustache#L73).
 The example below basically replicates creating such client as a global.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31862] Port the standard library to PEP 489 multiphase initialization

2020-01-16 Thread Mariatta


Change by Mariatta :


--
pull_requests:  -13420

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38076] Make struct module PEP-384 compatible

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

I can reproduce the crash on Linux. I interrupt the problem with CTRL+c (why is 
it blocked? I don't know). Then it does crash.

First, _PyImport_Cleanup() does clear all modules including _struct.

Then, _PyImport_Cleanup() calls gc.collect() which finalize_garbage() which 
calls A.__del__().

Problem: at this point, the _struct became unusable.

--

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00473f30 in PyModule_GetState (m=0x0) at Objects/moduleobject.c:565
565 if (!PyModule_Check(m)) {
(gdb) py-bt
Traceback (most recent call first):
  
  File "/home/vstinner/python/master/Lib/multiprocessing/connection.py", line 
400, in _send_bytes
header = struct.pack("!i", n)
  File "/home/vstinner/python/master/Lib/multiprocessing/connection.py", line 
200, in send_bytes
self._send_bytes(m[offset:offset + size])
  File "/home/vstinner/python/master/Lib/multiprocessing/queues.py", line 368, 
in put
self._writer.send_bytes(obj)
  File "/home/vstinner/python/master/Lib/multiprocessing/pool.py", line 649, in 
close
self._change_notifier.put(None)
  File "/home/vstinner/python/master/x.py", line 7, in __del__
self.pool.close()
  Garbage-collecting

In debug mode, the crash occurs in s_pack() at:

assert(PyStruct_Check(self));

--

#define _structmodulestate(o) ((_structmodulestate *)PyModule_GetState(o))

static struct PyModuleDef _structmodule;

#define _structmodulestate_global 
_structmodulestate(PyState_FindModule(&_structmodule))

#define PyStruct_Check(op) PyObject_TypeCheck(op, (PyTypeObject 
*)_structmodulestate_global->PyStructType)

The problem is "_structmodulestate_global->PyStructType": 
PyState_FindModule(&_structmodule) returns NULL, _structmodulestate() calls 
PyModule_GetState(NULL) which does crash.

--

The question is why the _struct module was cleared whereas there was still a 
reference to it. Is it part of a reference cycle?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

This code snippet seems to trigger a crash in the _struct module:
https://bugs.python.org/issue38076#msg360127

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38076] Make struct module PEP-384 compatible

2020-01-16 Thread Dino Viehland


Change by Dino Viehland :


--
nosy: +eelizondo -dino.viehland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread gaborbernat


gaborbernat  added the comment:

Reported the issue downstream too under 
https://github.com/swagger-api/swagger-codegen/issues/9991

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38076] Make struct module PEP-384 compatible

2020-01-16 Thread Dino Viehland


Dino Viehland  added the comment:

It seems problematic that_PyInterpreterState_ClearModules runs before all 
instances from a module have been cleared.  If PyState_FindModule is no longer 
able to return module state then there's no way for a module to reliably work 
at shutdown other than having all instances hold onto the module (or module 
state) directly from all of their insatances.  Certainly that would mimic more 
closely what happens w/ pure Python instances and modules - the type will hold 
onto the functions which will hold onto the module global state.

--
nosy: +dino.viehland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39360] python3.8 regression - ThreadPool join via __del__ hangs forever

2020-01-16 Thread Pablo Galindo Salgado


Pablo Galindo Salgado  added the comment:

This is not a regression: the code only worked on 3.7 because there is a sleep 
in the pool maintainance loop that makes the pool more likely being more 
consistent, but is the same problem as there is a race condition. For example, 
reducing the sleep in the loop (notice 3.8 does not use a sleep but instead 
uses a more resilient system of sentinels) for the 3.7 version reproduces the 
problem:

diff --git a/Lib/multiprocessing/pool.py b/Lib/multiprocessing/pool.py
index 3e9a0d6b48..f8d438d87c 100644
--- a/Lib/multiprocessing/pool.py
+++ b/Lib/multiprocessing/pool.py
@@ -410,7 +410,7 @@ class Pool(object):
 # is terminated.
 while thread._state == RUN or (pool._cache and thread._state != 
TERMINATE):
 pool._maintain_pool()
-time.sleep(0.1)
+time.sleep(0.0001)
 # send sentinel to stop workers
 pool._taskqueue.put(None)
 util.debug('worker handler exiting')

With that patch, 3.7 hangs as well. The problem here is that something 
regarding the locks inside the SimpleQueue of inbound tasks is not in a 
consistent state when Python finalization calls __del__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39362] add option to make chunksize adaptive for multiprocessing.pool methods

2020-01-16 Thread Ned Deily


Change by Ned Deily :


--
nosy: +davin, pitrou -ned.deily, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39362] add option to make chunksize adaptive for multiprocessing.pool methods

2020-01-16 Thread Pablo Galindo Salgado


Change by Pablo Galindo Salgado :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39363] zipfile with multiprocessing: zipfile.BadZipFile

2020-01-16 Thread maxime-lemonnier


Change by maxime-lemonnier :


Added file: https://bugs.python.org/file48848/foo_bar_small.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39363] zipfile with multiprocessing: zipfile.BadZipFile

2020-01-16 Thread maxime-lemonnier


New submission from maxime-lemonnier :

zipfile sometimes throws zipfile.BadZipFile when opening the same zip file from 
multiple processes

see attached file to reproduce the error. You'll need a zipfile with multiple 
files in it to reproduce.

--
components: Library (Lib)
files: test_filesource.py
messages: 360134
nosy: maxime-lemonnier
priority: normal
severity: normal
status: open
title: zipfile with multiprocessing: zipfile.BadZipFile
type: crash
versions: Python 3.6
Added file: https://bugs.python.org/file48847/test_filesource.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39363] zipfile with multiprocessing: zipfile.BadZipFile

2020-01-16 Thread maxime-lemonnier


maxime-lemonnier  added the comment:

Here's my console output:

python3 test_filesource.py
lock
file
access_mode = file, nb processes = 1, res = 110289, 0.08039402961730957 ms/frame
file
access_mode = file, nb processes = 4, res = 110289, 0.32297492027282715 ms/frame
lock
access_mode = lock, nb processes = 4, res = 110289, 0.2950408458709717 ms/frame
lock
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
  File "/path/to/script/test_filesource.py", line 64, in read_small
return fs_small[i%len(fs_small)][42]
  File "/path/to/script/test_filesource.py", line 55, in __getitem__
data_bytes = self.access( lambda archive: archive.read(member))
  File "/path/to/script/test_filesource.py", line 27, in access_lock
return f(self.archive)
  File "/path/to/script/test_filesource.py", line 55, in 
data_bytes = self.access( lambda archive: archive.read(member))
  File "/usr/lib/python3.6/zipfile.py", line 1337, in read
with self.open(name, "r", pwd) as fp:
  File "/usr/lib/python3.6/zipfile.py", line 1419, in open
% (zinfo.orig_filename, fname))
zipfile.BadZipFile: File name in directory '0005.pkl' and header 
b'0004.pkl' differ.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/path/to/script/test_filesource.py", line 90, in 
f(4, "lock") #crash
  File "/path/to/script/test_filesource.py", line 81, in f
for i in pool.imap_unordered(read_small, frames):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 735, in next
raise value
zipfile.BadZipFile: File name in directory '0005.pkl' and header 
b'0004.pkl' differ.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38342] ImportError: cannot import name 'MetadataPathFinder' from 'importlib.metadata'

2020-01-16 Thread Mariatta


Mariatta  added the comment:

Just wanted to add that I got the same error in Python 3.8.1. 

I ran the script that @Jaraco wrote above:

python -c "import importlib.metadata, shutil, pathlib; file = 
pathlib.Path(importlib.metadata.__file__); str(file).endswith('__init__.py') 
and shutil.rmtree(file.parent) and print('removed', file.parent)"

I wonder if the fix (e.g. running the script above) should be more visible, 
perhaps in some documentation somewhere, instead of having to dig this up from 
the bugtracker.

Is it possible for the Python installer itself to execute that script?

Thanks.

--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38076] Make struct module PEP-384 compatible

2020-01-16 Thread Dino Viehland


Dino Viehland  added the comment:

This is a relatively simple repro of the underlying problem:

import _struct

s = _struct.Struct('i')

class C:
def __del__(self):
s.pack(42, 100)

_struct.x = C()

It's a little bit different in that it is actually causing the module to 
attempt to throw an exception instead of do a type check.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38076] Make struct module PEP-384 compatible

2020-01-16 Thread Dino Viehland


Dino Viehland  added the comment:

And here's a variation which doesn't involve any instances from the module:

import _struct

class C:
def __init__(self):
self.pack = _struct.pack
def __del__(self):
self.pack('I', -42)

_struct.x = C()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> Later, I closed my pysandbox beause it was "broken by design":
https://lwn.net/Articles/574215/

Thanks for the link, insightful. Still unclear, by design of what it's broken 
;-).


> Paul Sokolovsky wrote in bpo-36220 than his idea is also to implement a 
> sandbox

Depends on the definition of "sandbox". If it's "But I was contacted by
different companies interested to use pysandbox in production on their
web application.  So I think that there is a real need to execute
arbitrary untrusted code." (re: https://lwn.net/Articles/574323/). Then no, I 
definitely not developing such a "sandbox".

If by "sandbox" you mean "ability to override some namespacing rules used by 
Python", then yep, I, and other folks (according to these >1 issue reports) 
interested in that. It's unclear why demise of your particular project should 
affect all other possible projects interested in dealing with namespacing 
matters. I for example interested in study of some properties of corpus of 
Python code. I can understand your claim that you "own" the partial features 
you introduced, but claiming them to be useful for only our own usecase is 
somewhat ..., well, short-sighted, just as claiming that all such uses should 
cease and desist is as insightful as claiming that, for example, list 
comprehensions are useless because you faced a case when one needs to be 
rewritten as explicit loop.

> IMHO we should reject dict subclasses to make Python (especially ceval.c) 
> more efficient.

I'm afraid efficiency story for CPython is irrecoverably lost, unless someone 
secretly works on adding JIT to it, but as we know, JIT work happily happens 
outside of it, in multiples. CPython still can either serve as a common 
platform for prototyping and experimentation, or complicate that by a committee 
fiat. Because the current situation is that CPython is too dynamic to be 
"slow", but not enough dynamic to perform useful instrumentation easily. For 
example, to comfortably do study on the proportions of actual usage of various 
"overdynamic" features.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

s/only our own usecase/only your own usecase/ (missing "y" typo)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Guido van Rossum


Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread ppperry


ppperry  added the comment:

I was not attempting to run untrusted Python code when I filed this bug report.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36220] LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses for globals() differently

2020-01-16 Thread Paul Sokolovsky


Change by Paul Sokolovsky :


--
pull_requests: +17429
pull_request: https://github.com/python/cpython/pull/18033

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31485] Tkinter widget.unbind(sequence, funcid) unbind all bindings

2020-01-16 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

Serhiy, if I understand your last message (msg302299), the docstring should 
read:

   Unbind for this widget the event SEQUENCE.
   If FUNCID is given, only unbind the function identified with FUNCID
   and also delete that command.

The main doc change is to specify what happens when funcid is not given while 
the code change in the patch is to no longer unbind everything when funcid *is* 
given.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

Ok, so the patch for my usecase (STORE_GLOBAL) is vividly trivial, so to go 
thru the full circle, I posted it: https://github.com/python/cpython/pull/18033 
.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Pablo Galindo Salgado


Pablo Galindo Salgado  added the comment:

> IMHO we should reject dict subclasses to make Python (especially ceval.c) 
> more efficient. The global type must be checked at every place where it's 
> possible to specify or override globals. Like the exec() function.

I agree with Victor in that we should reject dict subclasses everywhere.

--
keywords: +3.6regression

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31485] Tkinter widget.unbind(sequence, funcid) unbind all bindings

2020-01-16 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

Serhiy, given that the code did not match the docstring and that we are 
changing the code to match the clarified docstring, should we consider this a 
bugfix, and, should we backport it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

> Depends on the definition of "sandbox".

In bpo-36220, it clearly means "execute untrusted code": hide access to the 
filesystem for example.


> I for example interested in study of some properties of corpus of Python code.

This issue is a feature request: you ask to modify Python so you can pass dict 
subclasses as namespaces and expect CPython to respect the mapping protocol 
(__getitem__/__setitem__, rather than accessing directly dict internals (well, 
don't call overriden __getitem__/__setitem__ methods).

Currently, we are in a gray area: it's supported in some places, but not 
everywhere.

Implementing such features which require to introduce new code, which means 
increasing the maintenance burden, and it will slow down Python, since every 
access to a namespace would require to check if the namespace is a dict or a 
dict subclass.

Paul: you're are in front of 3 core developers who are rejecting your feature 
request. It doesn't mean that it is not valid. It means that we don't consider 
your use case as important enough to justify to slow down Python for everything 
and to increase the maintenance burden.

Namespace performances are really critical for overall Python performances.

My idea is only to reject the feature in CPython implementation, not the in 
Python language specification. It means that other Python implementation would 
be free to decide to implement it.

You're free to fork Python and modify the C code to implement your use case, to 
study Python internals.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread STINNER Victor


STINNER Victor  added the comment:

More detail about the implementation. Currently, namespaces are accessed 
directly with C functions which expect to get an exact dict type: these 
functions are designed for performance, not to handle dict subclasses, on 
purpose.

Supporting dict subclasses first require to check if the namespace type is 
dict: that's the first performance overhead. Then new code should be added to 
handle subclasses: that's the additional maintenance burden. This code path 
should be tested, maintained, reviewed, etc.

The risk is also to forget to support subclasses in one function which would 
break the feature. Same problem is new code is added which doesn't support dict 
subclasses.

This feature is far from being free to implement. It's very expensive on 
multiple aspects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39364] Automatically tabulate module contents in the docs

2020-01-16 Thread alex c


New submission from alex c :

By default, the docs.python.org page for a module does not list or tabulate the 
contents of that module. This makes it difficult to browse a module's functions 
or get a bird's-eye view.

For example, the logging module 
(https://docs.python.org/3/library/logging.html) has almost 70 functions, 
methods, and attributes. But it's impossible to scan them without scrolling the 
entire length of the entry (~18 pages of US letter). Compare to the 
browsability of itertools (https://docs.python.org/3/library/itertools.html), 
which manually tabulates its functions in the first section.

docs.python.org should automatically generate a TOC of the module's contents 
(classes, functions, etc) in the navigation sidebar, below the existing sidebar 
sections (perhaps in a collapsible section). Rust's documentation does this 
(example: https://doc.rust-lang.org/std/time/struct.Duration.html), and 
doc.rust-lang.org also effectively allows the entire page to function as a TOC 
by providing a "collapse page" button.

--
assignee: docs@python
components: Documentation
messages: 360148
nosy: alexchandel, docs@python
priority: normal
severity: normal
status: open
title: Automatically tabulate module contents in the docs
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread STINNER Victor

STINNER Victor  added the comment:

exec() function is currently quite clear on which types are accepted or not: 
exact dict for globals, mapping for locals.

*  "If only globals is provided, it must be a dictionary (and not a subclass of 
dictionary), which will be used for both the global and the local variables."
* "If provided, locals can be any mapping object."

Moreover, the requirement on the exact dict type allowed to implement an 
optimization on globals in Python 3.8:

"LOAD_GLOBAL instruction now uses new “per opcode cache” mechanism. It is about 
40% faster now. (Contributed by Yury Selivanov and Inada Naoki in bpo-26219.)"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38342] ImportError: cannot import name 'MetadataPathFinder' from 'importlib.metadata'

2020-01-16 Thread Jason R. Coombs


Jason R. Coombs  added the comment:

Given that this issue only affects those who upgraded from beta versions, I'm 
inclined to say it shouldn't be part of the installer, and that the long tail 
of users affected probably can track it down here.

I don't feel strongly about it though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39308] Literal[True] prints as Literal[1] in some cases

2020-01-16 Thread Guido van Rossum


Guido van Rossum  added the comment:

In the PR (https://github.com/python/cpython/pull/17974) we found that there's 
no great solution for this. This issue will probably linger for a while, but 
maybe we should just set an example and close it, since we're unlikely to take 
any action?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39298] add BLAKE3 to hashlib

2020-01-16 Thread Jack O'Connor


Jack O'Connor  added the comment:

Ok, I've added Rust bindings to the BLAKE3 C implementation, so that I can 
benchmark it in a vaguely consistent way. My laptop is an i5-8250U, which 
should be very similar to yours. (Both are "Kaby Lake Refresh".) My end result 
do look similar to yours with TurboBoost on, but pretty different with 
TurboBoost off:

with TurboBoost on
--
K12 GCC| 2159 MB/s
BLAKE3 Rust| 1787 MB/s
BLAKE3 C Clang | 1588 MB/s
BLAKE3 C GCC   | 1453 MB/s

with TurboBoost off
---
BLAKE3 Rust| 1288 MB/s
K12 GCC| 1060 MB/s
BLAKE3 C Clang | 1094 MB/s
BLAKE3 C GCC   |  943 MB/s

The difference seems to be that with TurboBoost on, the BLAKE3 benchmarks have 
my CPU sitting around 2.4 GHz, while for the K12 benchmarks it's more like 2.9 
GHz. With TurboBoost off, both benchmarks run at 1.6 GHz, and BLAKE3 does 
better. I'm not sure what causes that frequency difference. Perhaps some 
high-power instruction that the BLAKE3 implementation is emitting?

To reproduce these numbers you can clone these two repos (the latter is where I 
happen to have a K12 benchmark):

https://github.com/BLAKE3-team/BLAKE3
https://github.com/oconnor663/blake2_simd

Then in both cases checkout the "bench_406668786" branch, where I've put some 
benchmarks with the same input length you used.

For Rust BLAKE3, at the root of the BLAKE3 repo, run: cargo +nightly bench 
406668786

For C BLAKE3, the command is the same, but run it in the 
"./c/blake3_c_rust_bindings" directory. The build defaults to GCC, and you can 
"export CC=clang" to switch it.

For my K12 benchmark, at the root of the blake2_simd repo, run: cargo +nightly 
bench --features=kangarootwelve 406668786

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> you ask to modify Python so you can pass dict subclasses as namespaces and 
> expect CPython to respect the mapping protocol

But no, per your own account, you made noticeable, though not complete, code 
changes in that direction. The only thing I'm saying "now that it's all there, 
it's only logical to fix the missing parts, about which people report issues". 
While you suddenly come up with "alternative" solution - throw it all away.

And I always just took your word for it, but now did some git logging/blaming. 
So, LOAD_GLOBAL handling of dict subclasses was introduced by you in 
https://github.com/python/cpython/commit/b0b224233e481d979430a54d257d871424ff19fb
 , in 2012, almost 8 years ago. 

And what about STORE_NAME handling? I traced that to 
https://github.com/python/cpython/commit/214b1c3aaea3e83302df9ea37a37b3c7548b92b1
 of 2004.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> Paul: you're are in front of 3 core developers who are rejecting your feature 
> request.

But no, it's not my feature request. There were 2 tickets by at least 2 people. 
I just saw my case to be similar to cases of those people, so instead of 
creating duplicate bug reports, I chimed in, to show the general issue: various 
name-related opcodes don't treat namespace objects consistently.

And if I'm in front on 3 core developers here, then only because someone's 
Rubber Ducky (https://en.wikipedia.org/wiki/Rubber_duck_debugging) took a good 
vacation. Because I may imagine following "debugging session":

- Hey Ducky, some time ago I hacked on one project. As you know, I'm a core 
developer, so I kinda can add things on my whim, so I added some stuff to 
CPython core for that project, not consistent, not complete. Since then, I lost 
interest in my project, and as you can imagine, couldn't care less about the 
code in the core. The problem? It was almost 8 years ago. People discovered 
those features, and came to use on them. Not only that, they raise heads and 
demand it to be implemented more thoroughly and consistently. So, don't you 
think now would be good time to punish them and just rip that code out?

- You say how could I possible to do that on my own? No worries, I have 2 more 
buddies vouching for me, we core developers never let each other down.

- You're saying that removing features after 8 years is problematic? No 
worries, we can always say it was a regression from just 3 years ago.

- Hey Ducky, there's a more problem still, there's a particularly obnoxious 
dude, who initially tried to argue need for adding a feature based on my 8-year 
old code. If we support stuff in LOAD_GLOBAL, he says, then it should be piece 
of cake to support it in STORE_GLOBAL, which is called much less frequently. 
But we got him into the cold with a news of removal that 8-year code. Still he 
can't calm down, and changing tactics arguing that due to the way that globals 
are used at the module level (as locals too), then STORE_GLOBAL should behave 
consistently with STORE_NAME, tossing some pretty simple exec() code showing 
inconsistency. Should we just ignore him, or go for total punishment and make 
it "consistent" just the same as above, by removing support in STORE_NAME, 
instead of adding to STORE_GLOBAL? Now, that code is 16 years old. Do you think 
we can go for a twist like that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> Namespace performances are really critical for overall Python performances.

Yeah, that's why I'd like for myself and other people to make it easy to 
explore the behavior of namespace lookups, to see how to optimize them.


> You're free to fork Python and modify the C code to implement your use case, 
> to study Python internals.

Thanks, already done. I'm now looking for a way to share those studies with 
other people, who won't patch and compile CPython. So, no worries, I've already 
lost, I need that fix yesterday, in actual versions of CPython deployed around 
the world. The rest if just idea that bugs which aren't fixed, persist; and the 
very interesting case of decision making by CPython core developers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Paul Sokolovsky


Paul Sokolovsky  added the comment:

> exec() function is currently quite clear

A recent case: https://bugs.python.org/issue38316, co_stacksize was quite clear 
what it is. Turned out, a bug in the documentation (likely, just someone forgot 
to update it to the actual code). That's just one case, there're tons of bugs 
in the docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17005] Add a topological sort algorithm

2020-01-16 Thread Pablo Galindo Salgado


Pablo Galindo Salgado  added the comment:

I have been playing with the API for a while and I have to say that I have 
fallen in love with it: I tried to reimplement many snippets of parallel and 
non-parallel topological-order processing of a graph with this and it always 
fits very nicely.

I have shown this API also to some library maintainers that have to deal with 
topological sorting (like the tox maintainer) and they are very enthusiastic 
about the API as well.

Tim, I have updated PR 11583 to use the proposed API with test and docs. Could 
you make a review when you have some time so we can polish it for the 3.9 
release?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39365] Support (SEEK_END/SEEK_CUR) relative seeking in StringIO

2020-01-16 Thread random832


New submission from random832 :

Currently this fails with a (misleading in the case of SEEK_END) "Can't do 
nonzero cur-relative seeks" error, but there's no conceptual reason it 
shouldn't be possible.

The offset should simply be taken as a character offset, without any pretense 
that the "file" represents bytes in some Unicode encoding. This is already done 
for SEEK_START and tell, and has not caused any problems.

--
components: IO
messages: 360158
nosy: random832
priority: normal
severity: normal
status: open
title: Support (SEEK_END/SEEK_CUR) relative seeking in StringIO
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39354] collections.UserString format and format_map return a string

2020-01-16 Thread Raymond Hettinger


Raymond Hettinger  added the comment:

It is too late is the life of UserString for a design change, so I'm going to 
have to close this as more likely to harm existing code than to benefit future 
code.

Also, despite you expectations, this isn't atypical for Python classes.  For 
example, the __add__ method on integer subclasses returns actual ints rather 
than instances of the subclass.

In the case of UserString, the current design can be useful because it make 
UserString instances more substitutable for regular strings.

--
nosy: +rhettinger
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39349] Add "cancel_futures" parameter to concurrent.futures.Executor.shutdown()

2020-01-16 Thread Kyle Stanley


Kyle Stanley  added the comment:

As of now, I have the implementation for ThreadPoolExecutor working correctly, 
and a unit test added to ensure its behavior. It was a bit more involved than I 
originally anticipated, as I had to make a minor change in the _worker() 
function to allow the new parameter to be compatible with wait (which is 
important, as it prevents dangling threads).

With my initial implementation, using "wait=True" and "cancel_futures=True" was 
resulting in any threads that called work_queue.get(block=True) to hang 
indefinitely. In order to remedy this, I changed it to use 
work_queue.get_nowait(). If a queue.Empty exception occurs, it checks for a 
global constant _cancelling_futures (set to True just in shutdown before it 
starts draining the work queue). If it's true, the while True loop is broken, 
otherwise it continues to the next iteration. This effectively has the same 
behavior as before. 

I experimented with a few different possible solutions, and the above was the 
only one that worked while still maintaining the current behavior as much as 
possible. From what I can tell, this was the only viable means of implementing 
the new parameter without making it entirely incompatible with "wait=True".

At this point, I believe the only remaining step for the ThreadPoolExecutor 
implementation is to update the documentation and decide on the name. After 
working with it, I'm leaning more towards *cancel_futures*, as I think this 
more accurately describes what it does compared to just *cancel* (which is a 
bit vague IMO).

--
title: Add "cancel" parameter to concurrent.futures.Executor.shutdown() -> Add 
"cancel_futures" parameter to concurrent.futures.Executor.shutdown()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32615] Inconsistent behavior if globals is a dict subclass

2020-01-16 Thread Inada Naoki


Change by Inada Naoki :


--
superseder:  -> Allow mappings as globals (was: Fix dictionary subclass ...)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38840] incorrect __all__ list in multiprocessing.managers module

2020-01-16 Thread Zackery Spytz


Change by Zackery Spytz :


--
pull_requests: +17430
pull_request: https://github.com/python/cpython/pull/18034

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38840] incorrect __all__ list in multiprocessing.managers module

2020-01-16 Thread Zackery Spytz


Change by Zackery Spytz :


--
nosy: +ZackerySpytz
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   >