[issue35564] [DOC] Sphinx 2.0 will require master_doc variable set in conf.py

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +12413

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35564] [DOC] Sphinx 2.0 will require master_doc variable set in conf.py

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +12414

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35564] [DOC] Sphinx 2.0 will require master_doc variable set in conf.py

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +12415

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36374] A possible null pointer dereference in compile.c's merge_consts_recursive()

[SilentGhost]

Change by SilentGhost :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +12416

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36378] Add support to load from paths to json.load

[Dani Fojo]

New submission from Dani Fojo :

Add support to json.load to read from a string or Path object containing the 
path to a json file. Many libraries (like Numpy) support this behavior.

--
components: Library (Lib)
messages: 338442
nosy: Dani Fojo
priority: normal
severity: normal
status: open
title: Add support to load from paths to json.load
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36378] Add support to load from paths to json.load

[SilentGhost]

Change by SilentGhost :


--
nosy: +ezio.melotti, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36378] Add support to load from paths to json.load

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +12417
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36378] Add support to load from paths to json.load

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Inada Naoki  added the comment:

> Also PC/winreg.c. In this case winreg.SetValue() needs a length of size DWORD 
> instead of ssize_t.

As MSDN, cbData is ignored.
https://docs.microsoft.com/en-us/windows/desktop/api/winreg/nf-winreg-regsetvaluew

Can I skip overflow check?

If I can not, what is DWORD_MAX? (INT_MAX?)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36374] A possible null pointer dereference in compile.c's merge_consts_recursive()

[miss-islington]

miss-islington  added the comment:


New changeset 9b4a1b1e23d4a7cb18ad26f405bdc741af69f342 by Miss Islington (bot) 
(Zackery Spytz) in branch 'master':
bpo-36374: Fix a possible null pointer dereference (GH-12449)
https://github.com/python/cpython/commit/9b4a1b1e23d4a7cb18ad26f405bdc741af69f342


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36374] A possible null pointer dereference in compile.c's merge_consts_recursive()

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +12418

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +12419

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35564] [DOC] Sphinx 2.0 will require master_doc variable set in conf.py

[Julien Palard]

Julien Palard  added the comment:


New changeset 07b8018d75f3d4495708cf1d4175f33b40e13d30 by Julien Palard in 
branch '2.7':
[2.7] bpo-35564: add master_doc='contents' to conf.py (GH-12462)
https://github.com/python/cpython/commit/07b8018d75f3d4495708cf1d4175f33b40e13d30


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35564] [DOC] Sphinx 2.0 will require master_doc variable set in conf.py

[Julien Palard]

Julien Palard  added the comment:


New changeset 756cfd88920c2349d4546024856c406409b0ab7b by Julien Palard in 
branch '3.7':
[3.7] bpo-35564: add master_doc='contents' to conf.py (GH-12460)
https://github.com/python/cpython/commit/756cfd88920c2349d4546024856c406409b0ab7b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Inada Naoki  added the comment:


New changeset c5a216e0b97712bf19b4a6b7655c6bf22a367edd by Inada Naoki in branch 
'master':
bpo-8677: use PY_SSIZE_T_CLEAN in Modules/_gdbmodule.c (GH-12464)
https://github.com/python/cpython/commit/c5a216e0b97712bf19b4a6b7655c6bf22a367edd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36378] Add support to load from paths to json.load

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

This expands the API where json.load now only accepts open file objects to 
support str and pathlike objects. This was discussed in 
https://mail.python.org/pipermail/python-ideas/2017-March/045303.html where 
there were alternatives proposed as below : 

with path.open() as f: obj = json.load(f)

some_path.write_text(json.dumps(obj), encoding='utf8')
json.loads(some_path.read_text(encoding='utf8'))

Also supporting str and pathlib passes responsibility to json.load to open them 
and also close them properly? I think this needs a python-ideas discussion if 
the proposal has something to add other than the previous thread where many 
alternatives were mentioned for terse code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Inada Naoki  added the comment:


New changeset e9a1dcb4237cb2be71ab05883d472038ea9caf62 by Inada Naoki in branch 
'master':
bpo-8677: use PY_SSIZE_T_CLEAN in socketmodule.c (GH-12467)
https://github.com/python/cpython/commit/e9a1dcb4237cb2be71ab05883d472038ea9caf62


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36324] Inverse cumulative normal distribution function

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Tue, Mar 19, 2019 at 01:06:45AM +, Steven D'Aprano wrote:
> Later I will do some spot checks against the results returned by the Nspire 
> calculator

Looks good to me, they agree to 6 decimal places in my tests. Following 
Mark's earlier investigation, I expect that where they differ, it is the 
Nspire which is wrong.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Inada Naoki  added the comment:


New changeset d5f18a63cc2dfe8d0adec8bce384a8c569b0f3dc by Inada Naoki in branch 
'master':
bpo-8677: use PY_SSIZE_T_CLEAN in PC/winreg.c (GH-12466)
https://github.com/python/cpython/commit/d5f18a63cc2dfe8d0adec8bce384a8c569b0f3dc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36373] asyncio.gather: no docs for deprecated loop parameter

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi,

Reading the asyncio.gather code not seem to be deprecrated.
loop is used on a lot of line of code. 

Maybe, the deprecate idea is on other place where I don't know.

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36347] Renaming the constants for the .flags of PyMemberDef

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi @Josh,

In the PR, I don't remove READONLY. READONLY becomes an alias of
PY_READONLY.

I think I should split my PR in two PRs

1. Improve the current documentation and the code
2. Change the code in the Modules/ with the new macros.

@Serhyi what is your suggestion?

Thank you

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30668] DOC: missing word in license.rst

[Cheryl Sabella]

Cheryl Sabella  added the comment:

The first note was fixed, but the second still remains.  Assigning to @Mariatta 
for the sprints.

--
assignee: docs@python -> Mariatta
nosy: +Mariatta -christian.heimes, docs@python
stage:  -> needs patch
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36377] Python 'datastructures.html' docs page needs improvement because of ambiguity

[Emmanuel Arias]

Change by Emmanuel Arias :


--
keywords: +patch
pull_requests: +12420
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33406] [ctypes] increase the refcount of a callback function

[Cheryl Sabella]

Change by Cheryl Sabella :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36355] Remove documentation and internal use of the *RESTRICTED constants for PyMemberDef's flags field

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Josh, I don't know for the uselessness or not of these macros, but these macros 
are used.

RESTRICTED is used in Objects/classobject.c and Objects/funcobject.c

PY_WRITE_RESTRICTED is used in Objects/funcobject.c and Objects/methodobject.c

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[STINNER Victor]

STINNER Victor  added the comment:

PC/winreg.c:

if (value_length >= INT_MAX) {
PyErr_SetString(PyExc_OverflowError,
"the value is too long");
return NULL;
}

PY_DWORD_MAX should be used here. It's twice larger than INT_MAX ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +12421

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36367] tokenizer.c memory leak in case of realloc failure

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36367] tokenizer.c memory leak in case of realloc failure

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12423

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Inada Naoki  added the comment:


New changeset cc60cdd9c44dd15e441603ee5f78e09ea3e76929 by Inada Naoki in branch 
'master':
bpo-8677: use PY_DWORD_MAX instead of INT_MAX (GH-12469)
https://github.com/python/cpython/commit/cc60cdd9c44dd15e441603ee5f78e09ea3e76929


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36367] tokenizer.c memory leak in case of realloc failure

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 65b9849f0f07a000d751c96d9d711aeb24c95224 by Victor Stinner in 
branch '3.7':
bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442) (GH-12471)
https://github.com/python/cpython/commit/65b9849f0f07a000d751c96d9d711aeb24c95224


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36367] tokenizer.c memory leak in case of realloc failure

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 469b0a50d990bcb441910b23194c131e403c2833 by Victor Stinner in 
branch '2.7':
bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442) (GH-12470)
https://github.com/python/cpython/commit/469b0a50d990bcb441910b23194c131e403c2833


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Zuzu_Typ]

New submission from Zuzu_Typ :

Using the C-API, the inplace_pow numbermethod is always called with the third 
argument pointing to an invalid address.

The reason is likely that self.__ipow__ only takes one argument, resulting in a 
binaryfunc (self, arg), though inplace_pow is a ternaryfunc.
When trying to use the third argument in any way, Python crashes.

The third arg should be nonexistent, NULL or Py_None.

--
components: Build, Extension Modules
messages: 338458
nosy: Zuzu_Typ
priority: normal
severity: normal
status: open
title: nb_inplace_pow is always called with an invalid argument
type: crash
versions: Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36367] tokenizer.c memory leak in case of realloc failure

[STINNER Victor]

Change by STINNER Victor :


--
versions: +Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Никита Сметанин]

New submission from Никита Сметанин :

All of collections.Counter in-place operators: +=, -=, |= and &= are obviously 
expected to have time complexity O(b) as in the following example:

a = Counter(...)  # e.g 1M elements
b = Counter(...)  # e.g. 10 elements
a += b

But in fact, all of them are having O(a + b) time complexity due to inefficient 
implementation of _keep_positive method, which checks ALL of the elements of 
"a" counter after modification while it's required only to check CHANGED 
elements (no more than a size of "b") having time complexity O(b).

See 
https://github.com/python/cpython/blob/master/Lib/collections/__init__.py#L819 

It also unclear if there's even a need to check for non-positives with 
_keep_positive in ops like __iadd__, __iand__ and __ior__ (except __isub__) as 
it expects Counters which are always positive.

This unobvious inefficiency leads to unnecessary large execution times while, 
for example, iteratively accumulating some small Counters into a large one 
(frequent case). In this case .update method works much faster, but it doesn't 
check for zeros or negatives for some reason (is this a bug too?).

--
components: Library (Lib)
messages: 338461
nosy: Никита Сметанин
priority: normal
severity: normal
status: open
title: collections.Counter in-place operators are unexpectedly slow
type: performance
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[Stefan Krah]

Stefan Krah  added the comment:

Thanks, but it is still not going to happen. Look at the increased code size
in e.g. blake2s_impl.c.h.


I want to know what is going on in the code.  Also, the performance
improvements are in argument parsing, but not when you have numerical
code like a * b, where a and b are already decimals.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36366] Patcher stop method should be idempotent

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
keywords: +patch
pull_requests: +12424
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[SilentGhost]

Change by SilentGhost :


--
versions:  -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

> It also unclear if there's even a need to check for non-positives with 
> _keep_positive in ops like __iadd__, __iand__ and __ior__ (except __isub__) 
> as it expects Counters which are always positive.

Counter accepts dictionaries and keyword arguments where keys can have negative 
or zero as values where _keep_positive needs to be checked for add operation 
too.

>>> Counter(a=3) + Counter(a=-1)
Counter({'a': 2})

Counter can have keys where the value is already 0 during construction that are 
not deleted and thus during __iadd__ all the keys have to be checked for zero. 
Below is the behavior in Python 3.7 and checking only for changed "b" key for 
non-zero in the example would leave "a" key also in the result with the 
optimization proposed?

$ python3.7
Python 3.7.1rc2 (v3.7.1rc2:6c06ef7dc3, Oct 13 2018, 05:10:29)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import Counter
>>> a = Counter(a=0, b=1)
>>> a
Counter({'b': 1, 'a': 0}) # Not removed during construction
>>> b = Counter(b=1)
>>> a += b
>>> a
Counter({'b': 2})

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +12425

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36342] test_venv failure when the platform lacks a functional sem_open()

[Xavier de Gaye]

Xavier de Gaye  added the comment:

It is a test_venv issue related to bpo-32126 and also related to the other 
issues listed in bpo-32126.

Changing the title as it is ambiguous indeed.

check_output() in test_venv.py does not print the full stack trace as it is 
done by test_asyncio in bpo-32126, is this another issue ?

--
title: test_venv fails on Android with clang -> test_venv failure when the 
platform lacks a functional  sem_open()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36381] Preapre for mandatory PY_SSIZE_T_CLEAN

[Inada Naoki]

New submission from Inada Naoki :

Raise warning for # use without PY_SSIZE_T_CLEAN.

* 3.8: PendingDeprecationWarning
* 3.9: DeprecationWarning
* 3.10 (or 4.0): Remove PY_SSIZE_T_CLEAN and use Py_ssize_t always

--
components: Extension Modules
messages: 338466
nosy: inada.naoki, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Preapre for mandatory PY_SSIZE_T_CLEAN
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

@xtreak __init__ delegates work to update() which has the same behavior:

Python 3.7.2 (default, Feb 12 2019, 08:15:36)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import Counter
>>> d = Counter()
>>> d.update(a=0)
>>> d
Counter({'a': 0})


If we want to keep this behavior could we keep an internal list of keys whose 
value is 0 that we append to at 
https://github.com/python/cpython/blob/master/Lib/collections/__init__.py#L652 
so we doing operations we just check values of keys which are updated and the 
content of this list?

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[Inada Naoki]

Inada Naoki  added the comment:

> Would it be possible to emit a deprecation warning, maybe at runtime, when 
> PY_SSIZE_T_CLEAN is not defined?

I created bpo-36381 for it.  Let's close this long living issue.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36381] Preapre for mandatory PY_SSIZE_T_CLEAN

[Inada Naoki]

Change by Inada Naoki :


--
keywords: +patch
pull_requests: +12426
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21960] Better path handling in Idle find in files

[Cheryl Sabella]

Change by Cheryl Sabella :


--
resolution:  -> duplicate
stage: test needed -> resolved
status: open -> closed
superseder:  -> IDLE: always display full grep path

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36323] IDLE: always display full grep path

[Cheryl Sabella]

Cheryl Sabella  added the comment:

See also #21960.  I'm going to close that one in favor of this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] "Fatal Python error: Cannot recover from stack overflow" from SymPy tests

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

FWIW I just tried both f8e46e9e741f253803e9b8be03287e5dd16abd4d and 
f8e46e9e741f253803e9b8be03287e5dd16abd4d with the reproducer given and none of 
them segfault.

I don't think there is a bug in the Python interpreter, there is some ways to 
trigger this error when a RecursionError is not handled quickly enough. I 
suppose you just have a problem in Sympy. I also find suspicious that you 
reproducer change many lines in Sympy and would look at that first.

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Nikita Smetanin]

Nikita Smetanin  added the comment:

@xtreak I agree, also, this behavior is stated in documentation, but it's quite 
inconsistent in many ways, like in the following examples:
Counter(a=-1) + Counter(a=-2) produces empty Counter() instead of Counter(a=-2) 
which is unexpected, but 
Counter(a=-1) + Counter(a=2) produces correct result Counter(a=1). The same is 
with in-place operators.

That makes no sense.

It's clear that in-place addition, for example, isn't a good place to remove 
negative elements which wasn't involved in this operation at all. The only 
possible optimization here (if we don't want to change the behavior) is to keep 
track of positive / non-positive elements in a separate set.

The better solution could be in providing two classes — Counter for any 
(signed) values with consistent operations and MultisetCounter (e.g.) or 
specific methods (like .multiset_add) for positive-only values and 
positive-checks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I would defer to Raymond at this point and would prefer keeping current 
__iadd__ behavior and . See also issue23509 that discussed about C  
implementation of _keep_positive and some other optimizations.

> Counter(a=-1) + Counter(a=-2) produces empty Counter() instead of 
> Counter(a=-2) which is unexpected, but 

I think you meant Counter(a=-3) here

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36382] socket.getfqdn() returns domain "mshome.net"

[Markus]

New submission from Markus :

In a corporate network, `wmic computersystem get domain` returns the correct 
domain.

On some clients, the Python query "socket.getfqdn()" returns the wrong domain, 
namely "mshome.net"

>>> import socket
>>> socket.getfqdn()
'*.mshome.net'

I have only found only very old nominations of that domain.

Problems persists after reboot.

Tried versions 
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit 
(AMD64)] on win32
Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 32 bit 
(Intel)] on win32

Currently, I suspect this is a DHCP configuration problem.

--
components: Windows
messages: 338472
nosy: markuskramerIgitt, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: socket.getfqdn() returns domain "mshome.net"
type: behavior
versions: Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20309] Not all method descriptors are callable

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

See also PEP 579 (issue 11) and the thread 
https://mail.python.org/pipermail/python-ideas/2018-June/051572.html

--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36381] Preapre for mandatory PY_SSIZE_T_CLEAN

[STINNER Victor]

STINNER Victor  added the comment:

> * 3.8: PendingDeprecationWarning

... Or maybe use directly DeprecationWarning? :-)
https://discuss.python.org/t/pendingdeprecationwarning-is-really-useful/1038

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] "Fatal Python error: Cannot recover from stack overflow" from SymPy tests

[Stefan Krah]

Stefan Krah  added the comment:

This occurs when handling a recursion error uses more than 50 extra nested 
function calls:

if (tstate->overflowed) {
if (tstate->recursion_depth > recursion_limit + 50) {
/* Overflowing while handling an overflow. Give up. */
Py_FatalError("Cannot recover from stack overflow.");
}
return 0;
}



You can set the recursion limit with sys.setrecursionlimit(), but it is the 
extra stack depth that matters here.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] "Fatal Python error: Cannot recover from stack overflow" from SymPy tests

[Stefan Krah]

Stefan Krah  added the comment:

It can still be an issue in CPython, like in #14537.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Josh Rosenberg]

Josh Rosenberg  added the comment:

@Nikita: Your examples aren't unexpected at all, per the documented behavior:

>Several mathematical operations are provided for combining Counter objects to 
>produce multisets (counters that have counts greater than zero). Addition and 
>subtraction combine counters by adding or subtracting the counts of 
>corresponding elements. Intersection and union return the minimum and maximum 
>of corresponding counts. Each operation can accept inputs with signed counts, 
>but the output will exclude results with counts of zero or less.

That's pretty explicit; "Each operation can accept inputs with signed counts" 
means the inputs can have arbitrary counts (negative, zero or positive), but 
"the output will exclude results with counts of zero or less" means the output 
will *only* contain positive counts.

This is repeated again in the design "Note" at the bottom:

* The multiset methods are designed only for use cases with positive values. 
The inputs may be negative or zero, but only outputs with positive values are 
created.

while also noting (reiterating in the case of subtract, whose method docs 
already specify this) that the update and subtract methods are appropriate for 
use when both inputs and outputs must preserve negative or zero values.

So both of your examples are "expected" per the documentation. 
"Counter(a=-1) + Counter(a=-2) produces empty Counter()" is 100% required by 
the documentation.

And, again per the documentation, _keep_positive is necessary, even for the 
in-place ops, because there is no expectation that the inputs are positive.

So your original motivating case could be spelled:

a = Counter(...)  # e.g 1M elements
b = Counter(...)  # e.g. 10 elements
# or just b = {...} or b = some_iterable_of_values_to_count since update works 
just
# fine with plain dicts and iterables to count, and making a second Counter is
# unnecessary overhead

a.update(b)

and it would work (and since _keep_positive isn't involved, it would run 
faster). Yes, if you want to eliminate non-positive counts you'll eventually 
need to do:

a += Counter()

(or abusing implementation details, a._keep_positive()), but that's largely 
unavoidable; the only way to adhere to the documented guarantees without a per 
mathematical operation _keep_positive would be to slow down many operations 
(and dramatically increase memory requirements) by separately tracking the 
non-positive values as they're created, just in case a math operation would 
need to remove them.

And when I say "slow down", I don't mean by a little. Right now, Counter 
doesn't overload __setitem__ (it just inherits dict's implementation). So when 
you do:

c = Counter()
c['a'] += 1
c['a'] += 1

that calls dict.__getitem__ twice (the first of which ends up calling 
Counter.__missing__ because the key 'a' doesn't exist), and dict.__setitem__ 
twice (to store back 1, then 2).

To make Counter track non-positive values live, you'd need to reimplement 
__setitem__ as:

def __setitem__(self, elem, count):
if count <= 0:
self._nonpositive.add(elem)
else:
self._nonpositive.discard(elem)
return super().__setitem__(elem, count)

which would significant overhead, because:

1. Any movement from the C layer back to the Python layer is a massive 
performance hit in general
2. It wouldn't just slow __setitem__, it would also slow down the 
_count_elements accelerator for counting iterables (the thing that finally made 
Counter faster than defaultdict(int) for its intended use); the accelerator 
only uses the fast path when the mapping in question is a dict subclass and 
inherited dict's get and __setitem__ methods unmodified. If either one is 
overridden, it falls back to the slow path, which misses out on several major 
improvements available only with the concrete dict API.

On top of the performance hit, it wouldn't be remotely thread-safe: Counter 
(and most Python level types) aren't fully thread-safe to start with, but they 
generally break in predictable ways (e.g. doing c[key] += 1 in two threads 
simultaneously might only increment once, not twice, but at least the logical 
outcome of at least *one* of the operations occurred, not something wholly 
unrelated).

The race condition here would break code a long time after the actual race 
occurred; two threads could try to set the same value, one positive, one 
negative, and interleaved improperly, the negative value could be stored in the 
dict, but discarded from _nonpositive (or vice versa, but that's less 
problematic for correctness). Now when something should be eliminating 
non-positive values, it doesn't know that the negative value is even there, so 
it survives indefinitely (unless something else sets the count for it again 
anyway).

For the record, if you want to make a Counter without this behavior, it's 
actually pretty easy:

class SignedCounter(Counter):
def _keep_positive(self):
pass

but the interlocking documente

[issue36382] socket.getfqdn() returns domain "mshome.net"

[Steve Dower]

Steve Dower  added the comment:

We just return the result of GetComputerNameEx [1] here - we don't use WMI at 
all (and given the complexities, we are not going to start).

Are you able to provide any more information? In particular, if the 
documentation for GetComputerNameEx (below) helps you understand why these 
machines are different, that would be helpful to know.

Obviously (I hope) we can't diagnose a DHCP issue on your network, so claiming 
this is a bug in Python is likely premature.

[1]: 
https://docs.microsoft.com/en-us/windows/desktop/api/sysinfoapi/nf-sysinfoapi-getcomputernameexw

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] "Fatal Python error: Cannot recover from stack overflow" from SymPy tests

[Stefan Krah]

Stefan Krah  added the comment:

The tests pass here on Linux with 3.8 (cc60cdd9c4) and a very low 
sys.setrecursionlimit(150).

The fail properly with RecursionError at sys.setrecursionlimit(125).


So I guess we'd need a gdb stack trace from OS X in case there's a CPython 
issue that is OS X specific.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36381] Deprecate "#" argument format without PY_SSIZE_T_CLEAN

[STINNER Victor]

Change by STINNER Victor :


--
title: Preapre for mandatory PY_SSIZE_T_CLEAN -> Deprecate "#" argument format 
without PY_SSIZE_T_CLEAN

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8677] Modules needing PY_SSIZE_T_CLEAN

[STINNER Victor]

STINNER Victor  added the comment:

> Let's close this long living issue.

Thanks INADA-san for fixing last issues and for creating the deprecation issue!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Josh Rosenberg]

Josh Rosenberg  added the comment:

object.__ipow__ is documented to take an optional third argument (though there 
is no way to pass it aside from explicitly calling __ipow__ directly since 
there is no syntax support for three-arg pow, in place or otherwise), so it's 
not some incompatibility with object.__ipow__'s signature.

How are you seeing garbage passed? In the CPython C code base, I only see 
PyNumber_InPlacePower called in two places; ceval.c (to handle **=, which only 
handles two operands) and _operator.c (to implement operator.__ipow__, which 
unlike object.__ipow__, only takes two arguments, not three). In both cases, 
the third argument is explicitly passed in as Py_None.

PyNumber_InPlacePower itself then passes along that third argument to 
ternary_op as its third argument, and every code path that calls the retrieved 
slot consistently passes that argument along as the third argument to the 
slotted ternaryfunc.

I suppose an extension module might incorrectly call PyNumber_InPlacePower 
without passing the third argument, but that's a problem on their end (and 
should be caught by the compiler unless all diagnostics are suppressed).

But I'm not seeing the problem here. The code path is probably untested (given 
all numeric types in the CPython core are immutable, so none of them set 
nb_inplace_pow), but it looks correct at first glance. Do you have code that 
reproduces the error?

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36382] socket.getfqdn() returns domain "mshome.net"

[Markus]

Markus  added the comment:

Dear Steve,
in fact not a Python bug at all.

I used 2 commands: 
  ipconfig /release
  ipconfig /renew
and rebootet.

This fixed the issue, for now.

Also, I found this domain in c:\Windows\System32\drivers\etc\hosts.ics

Unclear who created that entry.


Documenting this here could help others with the same problem.

Closing this issue.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] "Fatal Python error: Cannot recover from stack overflow" from SymPy tests

[Stefan Krah]

Stefan Krah  added the comment:

Whoops, I tested the wrong branch, getting a proper abort() now. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36301] Add _Py_PreInitialize() function

[STINNER Victor]

STINNER Victor  added the comment:

In term of API, we get something like:

_PyInitError err;

_PyPreConfig preconfig = _PyPreConfig_INIT;
preconfig.utf8_mode = 1;
preconfig.allocator = "malloc";

_PyInitError err = _Py_PreInitializeFromPreConfig(&preconfig);
if (_Py_INIT_FAILED(err)) {
_Py_ExitInitError(err);
}

/* PyMem_RawMalloc and Py_DecodeLocale can now be used */

_PyCoreConfig config = _PyCoreConfig_INIT;
config.user_site_directory = 0;

err = _Py_InitializeFromConfig(&config);
if (_Py_INIT_FAILED(err)) {
_Py_ExitInitError(err);
}

/* ... use Python ... */

Py_Finalize();

/* Note: no need to "free" preconfig nor config memory, they use constants 
*/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35564] [DOC] Sphinx 2.0 will require master_doc variable set in conf.py

[Ned Deily]

Ned Deily  added the comment:


New changeset 4508bc37dd80c71adfaa0925a67c438389817076 by Ned Deily (Julien 
Palard) in branch '3.6':
[3.6] bpo-35564: add master_doc='contents' to conf.py (GH-11290). (GH-12461)
https://github.com/python/cpython/commit/4508bc37dd80c71adfaa0925a67c438389817076


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Raymond Hettinger]

Raymond Hettinger  added the comment:

As Josh pointed out, this is how counters are documented and tested to work, so 
it is an unchangeable decision (i.e. it would break code that relying on the 
saturating arithmetic feature).  Accordingly, I'm marking this as closed, not a 
bug.

On the plus side (no pun intended), a counter is just a kind of dictionary so 
you can easily implement your own behaviors with just a simple for-loop:

   for x in a:
   b[a] += a

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21960] Better path handling in Idle find in files

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36382] socket.getfqdn() returns domain "mshome.net"

[Markus]

Markus  added the comment:

I found the IP of mshome.net in an Ethernet adapter "vEthernet"
It seems that this adapter stems from Hyper-V.


It therefore seems that socket.getfqdn() reported the wrong network adapter 
once.

Because I cannot reproduce this, I leave this issue closed.

--
resolution: not a bug -> works for me

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36368] server process of shared_memory shuts down if KeyboardInterrupt

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Your patch sounds good on the principle, but can you make a patch out of it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36368] server process of shared_memory shuts down if KeyboardInterrupt

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Sorry - I meant make a *PR* out of it :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] Check for PyErr_Occurred() after PyImport_GetModule().

[Stefan Krah]

Stefan Krah  added the comment:

The issue is that PyImport_GetModule() can legitimately NULL (not found)
but also NULL after an error occurred in PyDict_GetItemWithError().

So one (quick and dirty) approach that fixes this abort() is:

diff --git a/Python/import.c b/Python/import.c
index bf3a99414f..22eecd7cd6 100644
--- a/Python/import.c
+++ b/Python/import.c
@@ -1735,7 +1735,10 @@ PyImport_ImportModuleLevelObject(PyObject *name, 
PyObject *globals,
 }
 
 mod = PyImport_GetModule(abs_name);
-if (mod != NULL && mod != Py_None) {
+if (mod == NULL && PyErr_Occurred()) {
+goto error;
+}
+else if (mod != NULL && mod != Py_None) {
 _Py_IDENTIFIER(__spec__);
 _Py_IDENTIFIER(_lock_unlock_module);
 PyObject *spec;


cc Brett as the import expert, perhaps he would like another approach.

--
nosy: +brett.cannon
title: "Fatal Python error: Cannot recover from stack overflow" from SymPy 
tests -> Check for PyErr_Occurred() after PyImport_GetModule().

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] Check for PyErr_Occurred() after PyImport_GetModule().

[Stefan Krah]

Change by Stefan Krah :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Hi what do you think of a patch to this effect that would speed up operations 
without changing the current semantics?


diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py
index cff75a48d6..fe5d5b2dca 100644
--- a/Lib/collections/__init__.py
+++ b/Lib/collections/__init__.py
@@ -561,6 +561,7 @@ class Counter(dict):
 if len(args) > 1:
 raise TypeError('expected at most 1 arguments, got %d' % len(args))
 super(Counter, self).__init__()
+self._nonpositives = []
 self.update(*args, **kwds)
 
 def __missing__(self, key):
@@ -652,6 +653,12 @@ class Counter(dict):
 self[elem] = count + self_get(elem, 0)
 else:
 super(Counter, self).update(iterable) # fast path when 
counter is empty
+for elem, count in self.items():
+try:
+if not count > 0:
+self._nonpositives.append(elem)
+except TypeError:
+pass
 else:
 _count_elements(self, iterable)
 if kwds:
@@ -818,9 +825,10 @@ class Counter(dict):
 
 def _keep_positive(self):
 '''Internal method to strip elements with a negative or zero count'''
-nonpositive = [elem for elem, count in self.items() if not count > 0]
-for elem in nonpositive:
-del self[elem]
+for elem in self._nonpositives:
+if not self[elem] > 0:
+del self[elem]
+self._nonpositives.clear()
 return self
 
 def __iadd__(self, other):
@@ -833,7 +841,10 @@ class Counter(dict):
 
 '''
 for elem, count in other.items():
-self[elem] += count
+count = self[elem] + count
+self[elem] = count
+if not count > 0:
+self._nonpositives.append(elem)
 return self._keep_positive()
 
 def __isub__(self, other):
@@ -846,7 +857,10 @@ class Counter(dict):
 
 '''
 for elem, count in other.items():
-self[elem] -= count
+count = self[elem] - count
+self[elem] = count
+if not count > 0:
+del self[elem]
 return self._keep_positive()
 
 def __ior__(self, other):
@@ -858,10 +872,11 @@ class Counter(dict):
 Counter({'b': 3, 'c': 2, 'a': 1})
 
 '''
-for elem, other_count in other.items():
-count = self[elem]
-if other_count > count:
-self[elem] = other_count
+for elem, count in other.items():
+count = max(count, self[elem])
+self[elem] = count
+if not count > 0:
+self._nonpositives.append(elem)
 return self._keep_positive()
 
 def __iand__(self, other):
@@ -874,9 +889,10 @@ class Counter(dict):
 
 '''
 for elem, count in self.items():
-other_count = other[elem]
-if other_count < count:
-self[elem] = other_count
+count = min(other[elem], count)
+self[elem] = count
+if not count > 0:
+self._nonpositives.append(elem)
 return self._keep_positive()
 
 



It seems to give great improvements for the case Nikita Smetanin gave:

➜  cpython git:(speed-up-Counter) ✗ ./python.exe tests.py
Setup took:  9.10102138099
Computation took:  1.63429864197e-05
➜  cpython git:(speed-up-Counter) ✗ python3 tests.py
Setup took:  11.615437155
Computation took:  0.3418315480004


Is there a performance test suite for Counter I could use to check this does 
not add regressions in some cases?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I don't want to complexify the normal case.  Please don't go down the path of 
turning something simple into a mess.

Because counters are just a dict subclass, users are free to make updates in 
any way they want.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21625] Make help() beginner helpful when no PAGER or LESS variable

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I recently opened Python in a Mac Terminal (bash) window, tried help(ob), and 
being a beginner in this situation, had the same terrible frustrating 
experience.  I am leaving this open to implement David Murray's suggestion and 
changed the title accordingly.

On Windows, in Command Prompt and Power Shell, a prompt appears when help 
output is done and the text remains.  This is the same as when one enters any 
other Python statement that produces output.  (The only difference is the 
intermediate paging.)  I think that this standard Python behavior, or something 
close, should be the default help behavior on all systems.  (It is on IDLE.)

--
nosy: +terry.reedy
stage:  -> needs patch
title: help()'s more-mode is frustrating -> Make help() beginner helpful when 
no PAGER or LESS variable
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35771] IDLE: Fix tooltip Hovertiptest failure

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35675] IDLE: Refactor config_key module.

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35467] IDLE: unrequested pasting into Shell after restart

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33964] IDLE maxosc.overrideRootMenu: remove unused menudict

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21696] Idle: test configuration files

[Terry J. Reedy]

Change by Terry J. Reedy :


--
components: +IDLE
versions: +Python 3.7, Python 3.8 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31329] Add idlelib module entry to doc

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21588] Idle: make editor title bar user configurable

[Terry J. Reedy]

Change by Terry J. Reedy :


--
components: +IDLE
versions: +Python 3.7, Python 3.8 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21647] Idle unittests: make gui, mock switching easier.

[Terry J. Reedy]

Change by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE
versions: +Python 3.7, Python 3.8 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36383] In Windows 10 virtual environments distutils.sysconfig.get_python_inc() reports base Python include directory

[Mark Campanelli]

New submission from Mark Campanelli :

On Windows 10 64bit, using virtualenv in Python 2.7.15 or venv in Python 3.7.2, 
calling distutils.sysconfig.get_python_inc() returns the path to the include 
directory for the (respective) Python base installation instead of the include 
directory installed with the virtual environment. This happens regardless of 
the True/False value of the  plat_specific argument. I would expect the virtual 
environment's include directory to be returned.

--
components: Distutils
messages: 338494
nosy: Mark Campanelli, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: In Windows 10 virtual environments distutils.sysconfig.get_python_inc() 
reports base Python include directory
type: behavior
versions: Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36342] test_venv failure when executed by test_multiprocessing and the platform lacks a functional sem_open()

[Brett Cannon]

Brett Cannon  added the comment:

I guess my confusion comes from the fact that test_venv in isolation is a 
totally fine test suite, it just fails when test_multiprocessing runs it due to 
something multiprocessing-related.

I've tried tweaking the title to reflect the fact that it's a test_venv failure 
triggered by test_multiprocessing.

--
title: test_venv failure when the platform lacks a functional  sem_open() -> 
test_venv failure when executed by test_multiprocessing and the platform lacks 
a functional sem_open()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] Check for PyErr_Occurred() after PyImport_GetModule().

[Brett Cannon]

Brett Cannon  added the comment:

Pulling in Eric and Nick as they have played w/ the C API more recently.

--
nosy: +eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31329] Add idlelib module entry to doc

[Terry J. Reedy]

Terry J. Reedy  added the comment:

See also #14944, about the Using doc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36370] Check for PyErr_Occurred() after PyImport_GetModule().

[Stefan Krah]

Stefan Krah  added the comment:

Actually I just see that this behavior of PyImport_GetModule() is documented:

"Return the already imported module with the given name. If the module has not 
been imported yet then returns NULL but does not set an error. Returns NULL and 
sets an error if the lookup failed."

New in version 3.7.


So it should indeed be just a matter of always checking for PyErr_Occurred().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14944] Setup & Usage documentation for pydoc, IDLE, & 2to3

[Terry J. Reedy]

Terry J. Reedy  added the comment:

idle.html, derived from idle.rst, is now the IDLE help file.

pip also has pip3 command; perhaps ensurepip should be mentioned.

--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Zuzu_Typ]

Zuzu_Typ  added the comment:

Even though __ipow__ might be documented to take a third argument, if you build 
an inplace_pow function using the C-API, you can only pass one argument to it.

You can see that in the attached screenshot.

The example class shown in the screenshot can be found here: 
https://github.com/Zuzu-Typ/Python-C-API-extension-template

With the little template I wasn't able to reproduce the crash, but I did 
reassure myself that the third object is neither Py_None nor NULL, by adding 
"if (obj2 == Py_None || obj2 == NULL) return NULL;" before line 469 in 
"template.c", because calling __ipow__ still returned an example_class 
instance, instead of an error message, as it should if it returned NULL.

--
Added file: https://bugs.python.org/file48225/Doesn't Work.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35866] concurrent.futures deadlock

[Hugh Redelmeier]

Hugh Redelmeier  added the comment:

@jwilk: thanks for creating cf-deadlock.py

I can replicate the test program hang on Fedora 29 with 
python3-3.7.2-4.fc29.x86_64

The test program hasn't yet hung on Fedora 29 with older packages, in 
particular 
python3-3.7.1-4.fc29.x86_64

My interest is due to the fact that the libreswan.org test suite has started to 
hang and we don't know why.  It might well be this bug.

--
nosy: +hugh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Stefan Krah]

Stefan Krah  added the comment:

Like Josh I don't quite understand the problem description. This
for example works:

>>> class C(int):
... def __ipow__(self, other, mod=None):
... return pow(self, other, mod)
... 
>>> 
>>> x = C(10)
>>> x
10
>>> x **= 3
>>> x
1000

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Zuzu_Typ]

Zuzu_Typ  added the comment:

This isn't about the CPython Interpreter, it's about the C-API, the APIT for 
writing c-extensions for Python.

I know it works in CPython.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Stefan Krah]

Stefan Krah  added the comment:

Ok, got it. I think __ipow__ should be a ternaryfunc, like so:

diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index 403f3caaee..914d076b5c 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -7032,7 +7032,7 @@ static slotdef slotdefs[] = {
 IBSLOT("__imod__", nb_inplace_remainder, slot_nb_inplace_remainder,
wrap_binaryfunc, "%="),
 IBSLOT("__ipow__", nb_inplace_power, slot_nb_inplace_power,
-   wrap_binaryfunc, "**="),
+   wrap_ternaryfunc, "**="),
 IBSLOT("__ilshift__", nb_inplace_lshift, slot_nb_inplace_lshift,
wrap_binaryfunc, "<<="),
 IBSLOT("__irshift__", nb_inplace_rshift, slot_nb_inplace_rshift,



On the other hand it is odd if "**=" can never use the third argument.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36379] nb_inplace_pow is always called with an invalid argument

[Stefan Krah]

Change by Stefan Krah :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36380] collections.Counter in-place operators are unexpectedly slow

[Raymond Hettinger]

Raymond Hettinger  added the comment:

FWIW, the promised semantics of saturating arithmetic require that 
_keep_positive be run on entire the entire counter:

>>> c1 = Counter(a=-3, b=4)
>>> +c1
Counter({'b': 4})

>>> from collections import Counter
>>> c1 = Counter(a=-3, b=4)
>>> c2 = Counter(b=5, c=5)
>>> c1 += c2 # The "a" entry gets removed
>>> c1
Counter({'b': 9, 'c': 5})

When this behavior isn't wanted, use the update() method which is documented 
not to perform removal of non-positive values.  That method is fast in pure 
python and has an even faster C helper function:

>>> c1 = Counter(a=-3, b=4)
>>> c2 = Counter(b=5, c=5)
>>> c1.update(c2)
>>> c1
Counter({'b': 9, 'c': 5, 'a': -3})

And as mentioned before, the Counter API is not encapsulated -- the underlying 
structure is fully exposed, so a user is free to do anything with it that they 
can do with a regular dictionary.

A final thought is to be careful when proposing to rewrite the internals of the 
existing methods.  These methods are careful to retain ordering of inputs, to 
not use auxiliary data fields (it is just a dict variant with no other 
structure), and to make only minimal assumptions about the datatype for the 
count.  They are designed to not be magical.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com