[issue34919] Crash caused by certain characters in a string

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Just noticed that you have mentioned that it doesn't crash in Linux in the 
GitHub repo. I tried the same in Ubuntu and if it's due to the mentioned issue 
I think it should crash in Linux too. Anyway it's worth giving 3.7RC1 a try to 
validate this. You can also attach the core dump stack trace file if possible. 
Unfortunately, I don't have a Windows machine to test this.

Thanks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34969] Add --fast, --best to the gzip CLI

[pmpp]

pmpp  added the comment:

Hi, on platform without gzip ( there are some , including some widely used OS  
eg: https://github.com/bazelbuild/rules_docker/issues/507 )
ability to use python gzip cli is extremely usefull as a fallback.

Though as discussed on irc default compression to 6 is a good tradeoff for the 
basic options fast (-1) / best(-9) and require less memory for decompression on 
target (could be embedded).

--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect all_tasks manipulation in asyncio.all_tasks()

[Andrew Svetlov]

New submission from Andrew Svetlov :

Traceback (most recent call last):
  File "C:\Users\max\AppData\Local\Programs\Python\Python37\lib\threading.py", 
line 917, in _bootstrap_inner
self.run()
  File 
"D:\Proj\antwork\code\antwork_backend\server\data_handlers\order\playback_hd.py",
 line 147, in run
send_to_ws(channel=self.room_name, data=json.loads(msg))
  File "D:\Proj\antwork\code\antwork_backend\server\data_exchange.py", line 26, 
in send_to_ws
asyncio.run(channel_layer.group_send(channel, send_data))
  File 
"C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\runners.py", 
line 46, in run
_cancel_all_tasks(loop)
  File 
"C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\runners.py", 
line 54, in _cancel_all_tasks
to_cancel = tasks.all_tasks(loop)
  File 
"C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", 
line 38, in all_tasks
return {t for t in _all_tasks
  File 
"C:\Users\max\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", 
line 38, in 
return {t for t in _all_tasks
  File 
"C:\Users\max\AppData\Local\Programs\Python\Python37\lib\_weakrefset.py", line 
61, in __iter__
for itemref in self.data:
RuntimeError: Set changed size during iteration


https://github.com/python/cpython/commit/416c1ebd9896b394790dcb4f9f035b1a44ebe9ff#commitcomment-30887909

--
components: asyncio
messages: 327632
nosy: asvetlov, ned.deily, yselivanov
priority: release blocker
severity: normal
status: open
title: Protect all_tasks manipulation in asyncio.all_tasks()
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Change by Andrew Svetlov :


--
title: Protect all_tasks manipulation in asyncio.all_tasks() -> Protect tasks 
weak set manipulation in asyncio.all_tasks()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The issue is hard to reproduce, the problem happens on an implicit garbage 
collector run during iteration over `_all_tasks` weakset.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34919] Crash caused by certain characters in a string

[cwickens]

cwickens  added the comment:

Actually I was mistaken: it does reproduce with 3.7.0 on MacOS. I just tried it 
again; I guess the first time I must have copied the problematic string 
incorrectly (as even changing by 1 char seems to affect whether it crashes).

Below is the crash report. Anyway, my next step will be to try with the 3.7.1 
RC. Though I guess I need to uninstall my existing 3.7 and I wonder how this 
will affect my existing projects and virtualenvs.

Process:   Python [558]
Path:  
/Library/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:Python
Version:   3.7.0 (3.7.0)
Code Type: X86-64 (Native)
Parent Process:bash [476]
Responsible:   Terminal [467]
User ID:   502

Date/Time: 2018-10-13 16:59:09.845 +0900
OS Version:Mac OS X 10.11.2 (15C50)
Report Version:11
Anonymous UUID:091D73EC-486B-5A63-9222-A9291F26FB6E


Time Awake Since Boot: 330 seconds

System Integrity Protection: enabled

Crashed Thread:0  Dispatch queue: com.apple.main-thread

Exception Type:EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:   EXC_I386_GPFLT

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   org.python.python   0x0001017f75aa pymalloc_alloc + 74
1   org.python.python   0x0001017f5493 _PyObject_Calloc + 35
2   org.python.python   0x0001017d1cdb PyList_New + 123
3   org.python.python   0x0001017d4758 list_subscript + 280
4   org.python.python   0x0001017a1068 PyObject_GetItem + 72
5   org.python.python   0x000101871514 
_PyEval_EvalFrameDefault + 4596
6   org.python.python   0x0001017ba1f0 
function_code_fastcall + 128
7   org.python.python   0x0001017b97e4 
_PyFunction_FastCallDict + 148
8   org.python.python   0x0001017bab4f 
_PyObject_FastCall_Prepend + 127
9   org.python.python   0x00010180ea00 slot_mp_subscript + 
160
10  org.python.python   0x0001017a1068 PyObject_GetItem + 72
11  org.python.python   0x000101871514 
_PyEval_EvalFrameDefault + 4596
12  org.python.python   0x00010187a096 
_PyEval_EvalCodeWithName + 2422
13  org.python.python   0x0001017b9d81 
_PyFunction_FastCallKeywords + 257
14  org.python.python   0x0001018795bb call_function + 795
15  org.python.python   0x000101876601 
_PyEval_EvalFrameDefault + 25313
16  org.python.python   0x0001017ba1f0 
function_code_fastcall + 128
17  org.python.python   0x0001018795bb call_function + 795
18  org.python.python   0x000101876601 
_PyEval_EvalFrameDefault + 25313
19  org.python.python   0x00010187a096 
_PyEval_EvalCodeWithName + 2422
20  org.python.python   0x0001017b9d81 
_PyFunction_FastCallKeywords + 257
21  org.python.python   0x0001018795bb call_function + 795
22  org.python.python   0x000101876601 
_PyEval_EvalFrameDefault + 25313
23  org.python.python   0x0001017ba1f0 
function_code_fastcall + 128
24  org.python.python   0x0001018795bb call_function + 795
25  org.python.python   0x000101876601 
_PyEval_EvalFrameDefault + 25313
26  org.python.python   0x00010187a096 
_PyEval_EvalCodeWithName + 2422
27  org.python.python   0x0001017b9d81 
_PyFunction_FastCallKeywords + 257
28  org.python.python   0x0001018795bb call_function + 795
29  org.python.python   0x000101876560 
_PyEval_EvalFrameDefault + 25152
30  org.python.python   0x00010187a096 
_PyEval_EvalCodeWithName + 2422
31  org.python.python   0x0001017b9d81 
_PyFunction_FastCallKeywords + 257
32  org.python.python   0x0001018795bb call_function + 795
33  org.python.python   0x000101876560 
_PyEval_EvalFrameDefault + 25152
34  org.python.python   0x0001017ba1f0 
function_code_fastcall + 128
35  org.python.python   0x0001018795bb call_function + 795
36  org.python.python   0x000101876601 
_PyEval_EvalFrameDefault + 25313
37  org.python.python   0x00010187a096 
_PyEval_EvalCodeWithName + 2422
38  org.python.python   0x0001017b9d81 
_PyFunction_FastCallKeywords + 257
39  org.python.python   0x0001018795bb call_function + 795
40  org.python.python   0x000101876560 
_PyEval_EvalFrameDefault + 25152
41  org.python.python   0x00010187a096 
_PyEval_EvalCodeWithName + 2422
42  org

[issue34919] Crash caused by certain characters in a string

[cwickens]

Change by cwickens :


--
components:  -Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Change by Andrew Svetlov :


--
keywords: +patch
pull_requests: +9211
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Ned, would be nice to have this trivial bugfix in 3.7.1.

The problem happens if wakes up on weakset iteration.
The chance is very low, it leads to a flaky scary bug.

--
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Ned Deily]

Ned Deily  added the comment:

3.7.1rc2 is about to be released so to add this would require retagging and 
remanufacturing release bits.  If the chances of it happening are "very low", I 
would prefer to not further delay rc2.  If you and Yury agree that the risk is 
low and the benefit is high, we could consider risking cherry-picking it to 
3.7.1 final.  Sound OK?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34969] Add --fast, --best to the gzip CLI

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Julien,

Currently, the default compresslevel for gzip.open, GzipFile is 9, for the best 
method compression. Maybe we could define 6 as the tradeoff and specify 1 (for 
fast), 9 (for best), 6 (tradeoff).

Maybe in an other issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34971] add support for tls/ssl sessions in asyncio

[Rémi Cardona]

New submission from Rémi Cardona :

cpython has had TLS session support since 3.6, using the SSLContext.wrap_* 
methods. Unfortunately, this support is not available when using asyncio's 
create_connection.

While I've managed to monkeypatch asyncio.sslproto._SSLPipe from my own code 
(it's a filthy hack but it's short and it gets the job done) running on 3.6.6, 
I feel this should be properly supported out of the box.

A patch is ready (tests work), a github PR will be created shortly.

Notes in no particular order:
- argument and attribute naming is all over the place, but I could not decide 
between "sslsession" (matching "sslcontext") and "ssl_session" (matching 
"ssl_handshake_timeout") so I just picked one
- tested on jessie (with openssl 1.0.2 from jessie-backports) and on gentoo
- the new asyncio tests added in the patch are adapted from test_ssl.py's 
test_session, with the server-side stats left out. I felt they were not useful 
if one assumes that the hard work is done by SSLContext.wrap_*.
- I did not reuse test_asyncio.utils.run_test_server which AIUI creates a new 
server-side context for each incoming connection, thus breaking sessions 
completely

TIA for considering this bug and patch

--
components: asyncio
messages: 327638
nosy: RemiCardona, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: add support for tls/ssl sessions in asyncio
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34919] Crash caused by certain characters in a string

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks @cwickens for the confirmation. It's still weird that I used one of the 
problematic strings in the repo and it didn't crash with 3.7.0 on my Ubuntu 
machine. I tried testing with Mac OS again since you mentioned it was 
reproducible now. I cloned the repo and even with the normal repo it crashes 
with 3.7.0 on Mac. I installed 3.7.1 RC1 and it fixes the issue. 

Please find my testing as below with the default clone from git : 

Python 3.7.0 for Mac OS installed from python.org segfaults : 

➜  py37bug git:(master) python3.7 -m ../venv py37-venv
➜  py37bug git:(master) source ../venv py37-venv/bin/activate
(py37-venv) ➜  py37bug git:(master) python3.7
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D
(py37-venv) ➜  py37bug git:(master) pip install otree
(py37-venv) ➜  py37bug git:(master) otree resetdb --noinput
[1]2590 segmentation fault  otree resetdb --noinput


Python 3.7.1 RC 1 for Mac OS installed from python.org works fine : 

➜  py37bug git:(master) python3.7 -m ../venv py37-rc-1-venv
➜  py37bug git:(master) source ../venv py37-rc-1-venv/bin/activate
(py37-rc-1-venv) ➜  py37bug git:(master) python3.7
Python 3.7.1rc1 (v3.7.1rc1:2064bcf6ce, Sep 26 2018, 03:09:51)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D
(py37-rc-1-venv) ➜  py37bug git:(master) pip install otree
(py37-rc-1-venv) ➜  py37bug git:(master) otree resetdb --noinput
INFO Database engine: SQLite
INFO Retrieving Existing Tables...
INFO Dropping Tables...
INFO Creating Database 'default'...
Operations to perform:
  Synchronize unmigrated apps: auth, channels, contenttypes, demoapp, djhuey, 
forms, idmap, messages, otree, sessions, staticfiles
  Apply all migrations: (none)
Synchronizing apps without migrations:
  Creating tables...
Creating table otree_pagecompletion
Creating table otree_pagetimeout
Creating table otree_completedgroupwaitpage
Creating table otree_completedsubsessionwaitpage
Creating table otree_participanttoplayerlookup
Creating table otree_participantlockmodel
Creating table otree_undefinedformmodel
Creating table otree_roomtosession
Creating table otree_failedsessioncreation
Creating table otree_participantroomvisit
Creating table otree_browserbotslaunchersessioncode
Creating table otree_chatmessage
Creating table otree_session
Creating table otree_participant
Creating table auth_permission
Creating table auth_group
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table demoapp_subsession
Creating table demoapp_group
Creating table demoapp_player
Running deferred SQL...
Running migrations:
  No migrations to apply.
INFO Created new tables and columns.

For some odd reason even with Python 3.7.0 installed the segfault still shows 
3.7.1RC1 in my Mac OS report. I had RC1 installed first and then did an install 
of 3.7.0 for testing this so I don't know why it's not updated to 3.7.0 in the 
segfault but my tests show with 3.7.1RC1 it works fine. If you can try with 
3.7.1 RC1 as you are available then it will be a helpful confirmation. 
Attaching the core dump similar to yours for reference.

Hope it helps. 

Thanks

--
Added file: 
https://bugs.python.org/file47865/Python_2018-10-13-141913_Karthikeyans-MacBook-Air.crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34968] loop.call_soon_threadsafe should be documented to be re-entrant-safe too

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Agree.
Docs update sounds good.
Nathaniel, would you create a docs patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34971] add support for tls/ssl sessions in asyncio

[Andrew Svetlov]

Andrew Svetlov  added the comment:

TLS session support is awesome.

IFAIK ssl_proto.py is under heavy reconstruction now.
Please coordinate your work with Yuri.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Let's wait for Yuri opinion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22851] 2.7 core crashes with generator.gi_frame.f_restricted

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 68ddb59417ee0b0dedf5c8b66a304138c9ce0a63 by Serhiy Storchaka 
(Zackery Spytz) in branch '2.7':
[2.7] bpo-22851: Fix a segfault when accessing generator.gi_frame.f_restricted. 
(GH-9348)
https://github.com/python/cpython/commit/68ddb59417ee0b0dedf5c8b66a304138c9ce0a63


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22872] multiprocessing.Queue raises AssertionError

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 0461704060474cb358d3495322950c4fd00616a0 by Serhiy Storchaka 
(Zackery Spytz) in branch 'master':
bpo-22872: multiprocessing.Queue's put() and get() now raise ValueError if the 
queue is closed. (GH-9010)
https://github.com/python/cpython/commit/0461704060474cb358d3495322950c4fd00616a0


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16965] 2to3 should rewrite execfile() to open in 'rb' mode

[miss-islington]

Change by miss-islington :


--
pull_requests: +9212

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16965] 2to3 should rewrite execfile() to open in 'rb' mode

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset d4d60134b29290049e28df54f23493de4f1824b6 by Serhiy Storchaka 
(Zackery Spytz) in branch 'master':
bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569)
https://github.com/python/cpython/commit/d4d60134b29290049e28df54f23493de4f1824b6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16965] 2to3 should rewrite execfile() to open in 'rb' mode

[miss-islington]

Change by miss-islington :


--
pull_requests: +9213

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22851] 2.7 core crashes with generator.gi_frame.f_restricted

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16965] 2to3 should rewrite execfile() to open in 'rb' mode

[miss-islington]

miss-islington  added the comment:


New changeset 22381394ad1541db0b652eed802601f62510d22f by Miss Islington (bot) 
in branch '3.7':
bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569)
https://github.com/python/cpython/commit/22381394ad1541db0b652eed802601f62510d22f


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34971] add support for tls/ssl sessions in asyncio

[Rémi Cardona]

Change by Rémi Cardona :


--
keywords: +patch
pull_requests: +9214
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16965] 2to3 should rewrite execfile() to open in 'rb' mode

[miss-islington]

miss-islington  added the comment:


New changeset 950fa40eee483f7507cd825b574a018b957dd253 by Miss Islington (bot) 
in branch '3.6':
bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569)
https://github.com/python/cpython/commit/950fa40eee483f7507cd825b574a018b957dd253


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34972] json dump silently converts int keys to string

[My-Tien Nguyen]

Change by My-Tien Nguyen :


--
nosy: My-Tien Nguyen
priority: normal
severity: normal
status: open
title: json dump silently converts int keys to string

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34972] json dump silently converts int keys to string

[My-Tien Nguyen]

New submission from My-Tien Nguyen :

When int keys are silently converted to string on json serialization, the user 
needs to remember to convert it back to int on loading.
I think that a warning should be shown at least.

In my case I serialize a dict to json with int keys, later load it back into a 
dict (resulting in a dict with string keys) and test for existence of an int 
key in the dict which will then return False incorrectly.

I am aware that json does not support int keys, but this can be easily 
forgotten.

--
components: +Library (Lib)
type:  -> behavior
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Exposing internal buffer means committing on a new API contract forever.

I feel a need for reacher read*() API but pretty sure that making internal 
buffer public is a bad idea. With BufferedProtocol it could be even worse: SLAB 
allocators can spit a buffer into several separate chunks.

`str.startswith()` supports a tuple of separators, maybe we can do the same for 
streaming API

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34972] json dump silently converts int keys to string

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the report. There was a related issue few days back issue32816. I 
think this is a documented behavior at 
https://docs.python.org/3.8/library/json.html#json.dumps . Having a warning in 
place might break code and I don't know if there is a safe way to introduce 
this as a code level warning given that this is a documented behavior in Python 
2 and 3. I think this is the case with other languages too like JavaScript 
itself converting int to string without warning adhering to JSON standard. 
Correct me if I am wrong or other languages have a warning related to this

> Note: Keys in key/value pairs of JSON are always of the type str. When a 
> dictionary is converted into JSON, all the keys of the dictionary are coerced 
> to strings. As a result of this, if a dictionary is converted into JSON and 
> then back into a dictionary, the dictionary may not equal the original one. 
> That is, loads(dumps(x)) != x if x has non-string keys

You can try doing json.loads(data, parse_int=int) but it will try converting 
the values.

>>> json.loads(json.dumps({1:'1'}), parse_int=int)
{'1': '1'}
>>> json.loads(json.dumps({1:1}), parse_int=int) 
{'1': 1}


Thanks

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30250] StringIO module truncate behavior of current position

[Martin Panter]

Change by Martin Panter :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
title: StreamIO truncate behavior of current position -> StringIO module 
truncate behavior of current position

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34953] Implement `mmap.mmap.__repr__`

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34882] f(a=1, *args) should be a SyntaxError

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Yury Selivanov]

Yury Selivanov  added the comment:

>> we could consider risking cherry-picking it to 3.7.1 final.  Sound OK?
> Let's wait for Yuri opinion.

I agree it's a pretty serious bug; basically a time bomb that can crash a 
perfectly fine asyncio application.  The PR itself seems to be safe to 
cherry-pick to 3.7.1 even after rc2.  I suggest to merge it asap though to have 
some time for buildbots to test it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34968] loop.call_soon_threadsafe should be documented to be re-entrant-safe too

[Yury Selivanov]

Yury Selivanov  added the comment:

> AFAICT the stdlib's implementation of call_soon_threadsafe is already 
> reentrant-safe

What would make it not reentrant-safe?  We'll need to document that for the 
benefit of asyncio and third-party maintainers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34973] Crash in bytes constructor with mutating list

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Constructing bytes from mutating list can cause a crash.

class X:
def __index__(self):
if len(a) < 1000:
a.append(self)
return 0

a = [X()]
bytes(a)

This is not an issue about weird integer-like objects. It is about . The size 
of the list can be changed in other thread while iterate it in the bytes 
constructor.

The optimization for the case of the list argument was added in issue6688. The 
code was refactored several times since, but this flaw was always.

--
assignee: serhiy.storchaka
components: Interpreter Core
messages: 327653
nosy: alexandre.vassalotti, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Crash in bytes constructor with mutating list
type: crash
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34972] json dump silently converts int keys to string

[My-Tien Nguyen]

My-Tien Nguyen  added the comment:

I don’t think, “other languages do that too” is a good argument here. This 
would apply if behaving differently would break user expectation. But here we 
would do nothing more than explicitly inform the user of a relevant operation. 
If they already expected that behaviour, they can disregard the warning.

I don’t see how `parse_int`would help me here, I would need a `parse_str=int`, 
but then it would try to parse every string, and I don’t see the use case for 
that.

I would suggest a warning similar to this:

--- json/encoder.py
+++ json/encoder.py
@@ -1,6 +1,7 @@
 """Implementation of JSONEncoder
 """
 import re
+import warnings
 
 try:
 from _json import encode_basestring_ascii as c_encode_basestring_ascii
@@ -353,7 +354,9 @@
 items = sorted(dct.items(), key=lambda kv: kv[0])
 else:
 items = dct.items()
+non_str_key = False
 for key, value in items:
+non_str_key = non_str_key or not isinstance(key, str)
 if isinstance(key, str):
 pass
 # JavaScript is weakly typed for these, so it makes sense to
@@ -403,6 +406,8 @@
 else:
 chunks = _iterencode(value, _current_indent_level)
 yield from chunks
+if non_str_key:
+warnings.warn("Encountered non-string key(s), converted to 
string.", RuntimeWarning)
 if newline_indent is not None:
 _current_indent_level -= 1
 yield '\n' + _indent * _current_indent_level

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34973] Crash in bytes constructor with mutating list

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +9215
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34973] Crash in bytes constructor with mutating list

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +9216

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34973] Crash in bytes constructor with mutating list

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I have two options for solving this result.

1. Add necessary checks for the list case. This will add a bunch of code and 
will slow down handling all lists because of additional checks.

2. Use the optimization for lists only when there are no other references to 
it. If no other references, the list can not be changed. This will not add much 
code, and may even slightly speed up cases for tuples and lists. But the code 
will be more fragile.

Of course there is an option of removing this optimization at all. But 
bytes([x]) and bytes((x,)) are common cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34932] Add macOS TCP_KEEPALIVE to available socket options

[Steve Dower]

Steve Dower  added the comment:

Yes, looks like TCP_KEEPIDLE is only available from 1709, so it should get the 
same handling as the TCP_KEEPCNT. If TCP_KEEPALIVE is just a synonym, then it 
applies for both.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Bruce Merry]

Bruce Merry  added the comment:

A sequence of possible terminators would cover my immediate use case and 
certainly be an improvement.

To facilitate more general use cases without exposing implementation details, 
would it be practical and maintainable to have a "putback" method that prepends 
data to the buffer? It might not be fast in all cases (e.g. it might have to 
make a copy of what's still in the buffer), but possibly BufferedReader could 
detect the common case (putting back a suffix of what's just been read) and 
adjust its offsets into its internal buffer (although I'm not at all familiar 
with BufferedReader, so feel free to tell me I'm talking nonsense).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34945] regression with ./python -m test and pdb

[Steve Dower]

Steve Dower  added the comment:

Thanks for spotting this!

There's no "rationale" for hiding the output - what happened is I streamlined 
the test suite to always capture output, instead of some tests deciding whether 
they ought to capture it or not (and some doing incredibly weird things when 
capturing, and others capturing the capture).

While I was testing I went immediately to `print(file=sys.__stderr__)`, since 
that's what I normally do, but it shouldn't be too hard to make the test suite 
only capture output when collecting it for the test results.

I'll try and get to it in the next two weeks, depending on how my conference 
schedules work out. I'm also happy to review someone else's change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Steve Dower]

Steve Dower  added the comment:

I meant why are you using an embedded application with a virtual environment? 
What sort of application do you have that requires users to configure a virtual 
environment, rather than providing its own set of libraries?

The embedding scenarios I'm aware of almost always want privacy/isolation from 
whatever a user has installed/configured, so that they can work reliably even 
when users modify other parts of their own system. I'm trying to understand 
what scenario (other than "I am an interactive Python shell") would want to 
automatically pick up the configuration rather than having its own 
configuration files/settings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34939] Possibly spurious SyntaxError: annotated name can't be global

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +9217
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33234] Improve list() pre-sizing for inputs with known lengths

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9218

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[miss-islington]

miss-islington  added the comment:


New changeset 97cf0828727ac2a269c89c5aa09570a69a22c83c by Miss Islington (bot) 
(Andrew Svetlov) in branch 'master':
bpo-34970: Protect tasks weak set manipulation in asyncio.all_tasks() (GH-9837)
https://github.com/python/cpython/commit/97cf0828727ac2a269c89c5aa09570a69a22c83c


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[miss-islington]

Change by miss-islington :


--
pull_requests: +9219
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] Multiple tests (test_socket, test_multiprocessing_*) fail due to incorrect recvmsg(2) buffer lengths, causing failures on FreeBSD CURRENT

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9220

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25246] Alternative algorithm for deque_remove()

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9221

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34974] bytes and bytearray constructors replace unexpected exceptions

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

bytes() and bytearray() replace some unexpected exceptions (including 
MemoryError and KeyboardInterrupt) with TypeError.

1) Exception in __index__.

class X:
def __index__(self):
raise MemoryError

`bytes(X())` results in:

Traceback (most recent call last):
  File "", line 1, in 
TypeError: cannot convert 'X' object to bytes

`bytearray(X())` results in:

Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'X' object is not iterable

This is related to issue29159.

2) Exception in __iter__.

class X:
def __iter__(self):
raise MemoryError

`bytes(X())` results in:

Traceback (most recent call last):
  File "", line 1, in 
TypeError: cannot convert 'X' object to bytes

`bytearray(X())` works correctly (raises a MemoryError).

The proper solution is to replace just a TypeError and allow other exceptions 
to emerge.

--
assignee: serhiy.storchaka
components: Interpreter Core
messages: 327661
nosy: belopolsky, inada.naoki, serhiy.storchaka
priority: normal
severity: normal
status: open
title: bytes and bytearray constructors replace unexpected exceptions
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 5dbb1b7df1d9ecaa6b5344028cd1777502cf5c73 by Andrew Svetlov (Miss 
Islington (bot)) in branch '3.7':
bpo-34970: Protect tasks weak set manipulation in asyncio.all_tasks() (GH-9837) 
(GH-9849)
https://github.com/python/cpython/commit/5dbb1b7df1d9ecaa6b5344028cd1777502cf5c73


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34974] bytes and bytearray constructors replace unexpected exceptions

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +9222
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Ned, the fix has landed on both master and 3.7

Your turn, please.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34919] Crash caused by certain characters in a string

[Terry J. Reedy]

Terry J. Reedy  added the comment:

An upgrade of the same version and bitness binary by the PSF Windows and Mac 
installers is done, by default, in place, so your Python scripts should not 
notice the difference, except for the bug fixes.  At least on Windows, 
downgrading requires uninstall, but one can choose to leave the directory and 
added files, including packages in site-packages, alone, so that installing the 
older maintenance version should again leave everything the same except for 
adding bugs back.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26301] ceval.c: reintroduce fast-path for list[index] in BINARY_SUBSCR

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9223

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34975] start_tls() difficult when using asyncio.start_server()

[Ian Good]

New submission from Ian Good :

There does not seem to be a public API for replacing the transport of the 
StreamReader / StreamWriter provided to the callback of a call to 
asyncio.start_server().

The only way I have found to use the new SSL transport is to update protected 
members of the StreamReaderProtocol object, e.g.:

async def callback(reader, writer):
loop = asyncio.get_event_loop()
transport = writer.transport
protocol = transport.get_protocol()
new_transport = await loop.start_tls(
transport, protocol, ssl_context, server_side=True)
protocol._stream_reader = StreamReader(loop=loop)
protocol._client_connected_cb = do_stuff_after_start_tls
protocol.connection_made(new_transport)

async def do_stuff_after_start_tls(ssl_reader, ssl_writer): ...

--
components: asyncio
messages: 327665
nosy: asvetlov, icgood, yselivanov
priority: normal
severity: normal
status: open
title: start_tls() difficult when using asyncio.start_server()
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Ned Deily]

Ned Deily  added the comment:

Thanks, Andrew and Yury!  Just leave the issue open as a "release blocker" for 
now and I will take care of it for 3.7.1 final.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34970] Protect tasks weak set manipulation in asyncio.all_tasks()

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Cool!
Thanks, Ned

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34927] Tkinter-related segfault on macOS (regression between 3.7.0 and 3.7.1rc1)

[Ned Deily]

Ned Deily  added the comment:


New changeset f55c3ae657595d12ce78aca76c9c6b998d632424 by Ned Deily in branch 
'3.6':
bpo-34370: Revert to using released Tk 8.6.8 with macOS installers
https://github.com/python/cpython/commit/f55c3ae657595d12ce78aca76c9c6b998d632424


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34370] Tkinter scroll issues on macOS

[Ned Deily]

Ned Deily  added the comment:


New changeset f55c3ae657595d12ce78aca76c9c6b998d632424 by Ned Deily in branch 
'3.6':
bpo-34370: Revert to using released Tk 8.6.8 with macOS installers
https://github.com/python/cpython/commit/f55c3ae657595d12ce78aca76c9c6b998d632424


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34927] Tkinter-related segfault on macOS (regression between 3.7.0 and 3.7.1rc1)

[Ned Deily]

Ned Deily  added the comment:


New changeset d8b6425e58a1fccdf8ddbbcde63066c13c1bcfaf by Ned Deily in branch 
'3.7':
bpo-34370: Revert to using released Tk 8.6.8 with macOS installers
https://github.com/python/cpython/commit/d8b6425e58a1fccdf8ddbbcde63066c13c1bcfaf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34370] Tkinter scroll issues on macOS

[Ned Deily]

Ned Deily  added the comment:


New changeset d8b6425e58a1fccdf8ddbbcde63066c13c1bcfaf by Ned Deily in branch 
'3.7':
bpo-34370: Revert to using released Tk 8.6.8 with macOS installers
https://github.com/python/cpython/commit/d8b6425e58a1fccdf8ddbbcde63066c13c1bcfaf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34976] IDLE: Replace the search dialog with a search bar

[Tal Einat]

New submission from Tal Einat :

Search dialogs are clunky and arguably outdated. Most modern editors, IDEs and 
browsers use a search bar instead.

I created a search bar for IDLE as an extension years ago, it can be found on 
PyPI as "SearchBar". I hadn't updated it in recent years and it doesn't work 
with modern IDLE. I doubt that it has seen much use.

I suggest that having this built into IDLE would be a great benefit to its 
users.

--
assignee: terry.reedy
components: IDLE
messages: 327672
nosy: cheryl.sabella, taleinat, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: Replace the search dialog with a search bar
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34976] IDLE: Replace the search dialog with a search bar

[Tal Einat]

Change by Tal Einat :


--
keywords: +patch
pull_requests: +9224
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34976] IDLE: Replace the search dialog with a search bar

[Tal Einat]

Tal Einat  added the comment:

See PR GH-9855 for a working implementation based on my old "SearchBar" 
extension.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34975] start_tls() difficult when using asyncio.start_server()

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks for raising the problem.
I'm in the middle of streams API refactoring.

https://github.com/asvetlov/cpython/blob/async-streams/Lib/asyncio/streams.py#L801-L812
 is a draft.

A help is very appreciated.
Would you pick up this snippet and make a pull request with tests and 
documentation update?

--
assignee:  -> asvetlov
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33729] Hashlib/blake2* missing 'data' keyword argument

[Ned Deily]

Ned Deily  added the comment:

Serhiy's fixes (thanks!) are now released in 3.7.0rc2 and 3.6.7rc2 so I'm 
removing the "release blocker" status.  If there is nothing more to be done for 
this issue, can we close it?

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[Tal Einat]

Tal Einat  added the comment:

Ping? Do we want to get this in for 3.7.1 and 3.6.7?

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34922] hashlib segmentation fault

[Ned Deily]

Ned Deily  added the comment:

Serhiy's fixes (thanks!) are now released in 3.7.0rc2 and 3.6.7rc2 so I'm 
removing the "release blocker" status.  If there is nothing more to be done for 
this issue, can we close it?

shuoz:
> oh brother, maybe this worth open a cve.

Note that Serhiy believes that this is not a security issue since it is 
unlikely that the crash can be triggered by user data.  Anyone can cause 
segfaults or do damage if they have unrestricted access to a Python 
interpreter; that's a threat model for any language that allows sometime like 
Python's os.system or subprocess.  A better question is can a user of an 
application written in Python likely cause a DOS or create a privilege 
escalation.  Is that the case here?

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34975] start_tls() difficult when using asyncio.start_server()

[Yury Selivanov]

Yury Selivanov  added the comment:

One thing: I'm -1 on adding starttls to current stream api; let's add it only 
to the new one (same for sendfile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Steve Dower]

New submission from Steve Dower :

The Windows Store can now install unrestricted apps, which means we can provide 
the Python interpreter in there.

Advantages:
* far more reliable installation
* can have specific executables on PATH
* automatic updates
* other apps can act as extensions (with permission), so things like Mu could 
also become store apps in the future

Disadvantages:
* only --user installs for pip will work
* harder to find/modify installed files

I think for a (very significant) subset of our users, this will be a much 
better experience than downloading our current installer. It also has the 
advantage of making "Python" appear in searches in the start menu, which will 
link directly to a one-click "Install" button.

The biggest issue is likely to be pip not installing with --user by default, as 
well as subsequent issues with apps that require users to navigate to their 
%AppData% directories manually (since these will be redirected to different 
locations).

But until we get a package available for testing, it will be hard to figure out 
what needs fixing. Given this is just distribution and not a new platform, I 
plan to enable it for Python 3.7 (probably 3.7.2, but it may be possible to 
bundle 3.7.1 as-is for testing).

--
assignee: steve.dower
components: Windows
messages: 327679
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Release Windows Store app containing Python
type: enhancement
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31522] _mboxMMDF.get_string() fails to pass param to get_bytes()

[Cheryl Sabella]

Change by Cheryl Sabella :


--
keywords: +patch
pull_requests: +9225
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31522] _mboxMMDF.get_string() fails to pass param to get_bytes()

[Cheryl Sabella]

Change by Cheryl Sabella :


--
versions: +Python 3.8 -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests: +9226
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23554] EchoServerClientProtocol class should be called EchoServerProtocol

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +9227
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[Terry J. Reedy]

Terry J. Reedy  added the comment:

There is still no rush as guarding the ctypes import on Windows builds that can 
run IDLE may never be needed.  But I grepped the stdlib .py code and uuid is 
the only .py module that imports ctypes, and all are directly or indirectly 
guarded in the code or test_uuid.  So I moved the import inside the existing 
try block and added ImportError to those caught.

--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +9229
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +9230

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Steve Dower]

Change by Steve Dower :


--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[miss-islington]

Change by miss-islington :


--
pull_requests: +9231
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset d274afb5e579a5d9d990f68f9af856cf4c918779 by Terry Jan Reedy in 
branch 'master':
bpo-33656: Move pyshell ctypes import inside try block. (GH-9858)
https://github.com/python/cpython/commit/d274afb5e579a5d9d990f68f9af856cf4c918779


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[miss-islington]

Change by miss-islington :


--
pull_requests: +9232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34978] check type of object in fix_dict.py in 2to3

[Pranav Devarakonda]

New submission from Pranav Devarakonda :

fix_dict.py applies fixes to every instance of keys(), items() or values() 
irrespective of the type of object. Since 2to3 cannot check the type of the 
object, we can at least add the check to the generated code like...

d.keys() -> list(d.keys) if type(d) == dict else d.keys()
and similarly 

d.viewkeys() -> d.keys() if type(d) == dict else d.viewkeys()

PFA the tweaked fixer.

--
components: 2to3 (2.x to 3.x conversion tool)
files: fix_dict.py
messages: 327682
nosy: benjamin.peterson, devarakondapranav
priority: normal
severity: normal
status: open
title: check type of object in fix_dict.py in 2to3
type: enhancement
Added file: https://bugs.python.org/file47866/fix_dict.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[miss-islington]

miss-islington  added the comment:


New changeset 77e0abe228564a5be23284bd8e963c11952eb55b by Miss Islington (bot) 
in branch '3.7':
bpo-33656: Move pyshell ctypes import inside try block. (GH-9858)
https://github.com/python/cpython/commit/77e0abe228564a5be23284bd8e963c11952eb55b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34972] json dump silently converts int keys to string

[Eric V. Smith]

Eric V. Smith  added the comment:

I can't think of another place where we issue a warning for anything similar. 
I'm opposed to any changes here: it's clearly documented behavior.

It's like being surprised .ini files convert to strings: it's just how that 
format works.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[miss-islington]

miss-islington  added the comment:


New changeset 68299305c7898eb8eb0cea6cad19c299d0d1ba2d by Miss Islington (bot) 
in branch '3.6':
bpo-33656: Move pyshell ctypes import inside try block. (GH-9858)
https://github.com/python/cpython/commit/68299305c7898eb8eb0cea6cad19c299d0d1ba2d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33656] IDLE: Turn on DPI awareness on Windows

[Terry J. Reedy]

Change by Terry J. Reedy :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26441] email.charset: to_splittable and from_splittable are not there anymore!

[Braden Groom]

Change by Braden Groom :


--
keywords: +patch
pull_requests: +9234
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34979] Python throws “SyntaxError: Non-UTF-8 code start with \xe8...” when parse source file

[Lu jaymin]

New submission from Lu jaymin :

```
# demo.py
s = 
'测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试'
```
The file on above is for testing, it's encoding is utf-8, the length of `s` is 
1020 bytes(3 * 340).

When execute `python3 demo.py` on terminal, Python will throws the following 
error:

```
$ python3 -V
Python 3.6.4

$ python3 demo.py
  File "demo.py", line 2
SyntaxError: Non-UTF-8 code starting with '\xe8' in file demo.py on line 2, but 
no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
```

I've found this error occurred on about line 630(the bottom of the function 
`decoding_fgets`) of the file `cpython/Parser/tokenizer.c` after I read 
Python-3.6.6's source code.

When Python execute xxx.py, Python will call the function `decoding_fgets` to 
read one line of raw bytes from file and save the raw bytes to a buffer, the 
initial length of the buffer is 1024 bytes, `decoding_fgets` will use the 
function `valid_utf8` to check raw bytes's encoding.

If the lenght of raw bytes is too long(like greater than 1023 bytes), then 
Python will call `decoding_fgets` multiple times and increase buffer's size by 
1024 bytes every time.so raw bytes read by `decoding_fgets` is maybe 
incomplete, for example, raw bytes contains a part of bytes of a character, 
that will cause `valide_utf8` failed.

I suggest that we should always use `fp_readl` to read source coe from file.

--
components: Interpreter Core
messages: 327686
nosy: Lu jaymin
priority: normal
severity: normal
status: open
title: Python throws “SyntaxError: Non-UTF-8 code start with \xe8...” when 
parse source file
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34927] Tkinter-related segfault on macOS (regression between 3.7.0 and 3.7.1rc1)

[Ned Deily]

Ned Deily  added the comment:

It appears that the segfault behavior *was* introduced by some change in the Tk 
8.6 development snapshot I chose to use for the python.org macOS installer 
builds in 3.7.1rc1 and 3.6.7rc2.  For 3.7.1rc2 and 3.6.7rc2, I reverted back to 
using the standard Tk 8.6.8 release and the segfault seems to have gone away on 
the systems where I was able to reproduce it with rc1 - whether it segfaulted 
seems to be system dependent.  We'll probably wait for the official release of 
Tk 8.6.9 before trying to upgrade Tk again.  Let us know if the problem 
reappears.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34972] json dump silently converts int keys to string

[Steve Dower]

Steve Dower  added the comment:

Agreed with Eric. json.dump needs to produce valid JSON, which requires keys to 
be strings.

Try using pickle if you need to preserve full Python semantics.

--
nosy: +steve.dower
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34979] Python throws “SyntaxError: Non-UTF-8 code start with \xe8...” when parse source file

[Xiang Zhang]

Change by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34979] Python throws “SyntaxError: Non-UTF-8 code start with \xe8...” when parse source file

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the report. Is this a case of encoding not being declared at the top 
of the file or am I missing something?

➜  cpython git:(master) cat ../backups/bpo34979.py
s = 
'测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试'

print("str len : ", len(s))
print("bytes len : ", len(s.encode('utf-8')))
➜  cpython git:(master) ./python.exe ../backups/bpo34979.py
  File "../backups/bpo34979.py", line 1
SyntaxError: Non-UTF-8 code starting with '\xe8' in file ../backups/bpo34979.py 
on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ 
for details

# With encoding declared

➜  cpython git:(master) cat ../backups/bpo34979.py
# -*- coding: utf-8 -*-

s = 
'测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试'

print("str len : ", len(s))
print("bytes len : ", len(s.encode('utf-8')))
➜  cpython git:(master) ./python.exe ../backups/bpo34979.py
str len :  340
bytes len :  1020

# Double the original string

➜  cpython git:(master) cat ../backups/bpo34979.py
# -*- coding: utf-8 -*-

s = 
'测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试'

print("str len : ", len(s))
print("bytes len : ", len(s.encode('utf-8')))
➜  cpython git:(master) ./python.exe ../backups/bpo34979.py
str len :  680
bytes len :  2040


Thanks

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34978] check type of object in fix_dict.py in 2to3

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the patch but I feel adding type checks will add more complexity to 
the code being converted and there are a lot of cases to handle. I have some 
code below where running 2to3 will just wrap the keys() call to a list but with 
the patch the if call might be have an inner if call if I am understanding the 
patch correctly as you have handled the for loop case.


$ cat ../backups/bpo34978.py
a = {1: 1}

True if 1 in a.keys() else False
$ 2to3 -w ../backups/bpo34978.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored ../backups/bpo34978.py
--- ../backups/bpo34978.py  (original)
+++ ../backups/bpo34978.py  (refactored)
@@ -1,3 +1,3 @@
 a = {1: 1}

-True if 1 in a.keys() else False
+True if 1 in list(a.keys()) else False
RefactoringTool: Files that need to be modified:
RefactoringTool: ../backups/bpo34978.py
➜  cpython git:(master) cat ../backups/bpo34978.py
a = {1: 1}

True if 1 in a.keys() else False


With the patch the above might be as below which throws syntax error.

True if 1 in list(a.keys()) if type(a) == dict else a.keys() else False

I think this is one case which can be handled like for but there might be other 
places where this might return invalid results that I couldn't think of like 
nested if clauses in list comprehensions and so on. I think this fixer was 
written with the notion that .keys() and .values() is a very common method for 
dict and these methods are not something many people define themselves as far 
as I have seen from other's code since they are more attached with dict so that 
the fixer affects those cases. I think the gain is minimal here with cases to 
handle. 

`list(d.keys())` seems to be more Pythonic than `list(d.keys) if type(d) == 
dict else d.keys()` though it provides some additional type checks.

# With current 2to3

keys = list(a.keys())

if 1 in list(a.keys()):
True
else:
False



# With patch this also increases line length and more to understand when 
looking at the code

keys = list(a.keys()) if type(a) == dict else a.keys()

if 1 in list(a.keys()) if type(a) == dict else a.keys():
True
else:
False


I couldn't find any discussions at the moment to see if this was discussed 
earlier when 2to3 was written. This is just my suggestion and I will leave it 
to Benjamin and others for thoughts on this. Feel free to correct me if I am 
misunderstanding the patch where it was handled.


Thanks

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34977] Release Windows Store app containing Python

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

+1 for reliable installation. There were some cases (issue33220, issue30944) 
where the installer was detected as a Virus by some vendors which would be 
reduced since this is installed from the App store as a verified app. I hope 
this will also reduce the scenarios where users need to resort to bug tracker 
with log files when there are installation failures [1] providing a more smooth 
one click install.

There is a long standing open issue [2] to make `pip install --user` as a 
default which I hope gets resolved and there is an issue with using `pip 
install --user` inside a virtualenv which seems to block the earlier issue [3]

Thanks a lot for this!

[1] 
https://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid&ignore=file%3Acontent&%40search_text=windows+msi+install&submit=search&status=-1%2C1%2C2%2C3

[2] https://github.com/pypa/pip/issues/1668

[3] https://github.com/pypa/pip/issues/5702

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34927] Tkinter-related segfault on macOS (regression between 3.7.0 and 3.7.1rc1)

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the release Ned. I can confirm that there is no segfault with Python 
3.7.1rc2 (v3.7.1rc2:6c06ef7dc3).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com