[issue28353] os.fwalk() unhandled exception when error occurs accessing symbolic link target

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch with tests. Needed to test it on Windows.

--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28353] os.fwalk() unhandled exception when error occurs accessing symbolic link target

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +patch
Added file: http://bugs.python.org/file44982/fwalk_oserror.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24870] Optimize ascii and latin1 decoder with surrogateescape and surrogatepass error handlers

[INADA Naoki]

Changes by INADA Naoki :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24454] Improve the usability of the match object named group API

[irdb]

Changes by irdb :


--
nosy: +irdb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28373] input() prints to original stdout even if sys.stdout is wrapped

[Arnon Yaari]

New submission from Arnon Yaari:

When I wrap sys.stdout with a custom object that overrides the 'write' method, 
regular prints use the custom write method, but the input() function prints the 
prompt to the original stdout.
This is broken on Python 3.5. Earlier versions work correctly.
In the following example 'write' does nothing, so I expect no output, but the 
input() function outputs to stdout anyway:

import sys

class StreamWrapper(object):
def __init__(self, wrapped):
self.__wrapped = wrapped

def __getattr__(self, name):
# 'write' is overridden but for every other function, like 'flush', use 
the original wrapped stream
return getattr(self.__wrapped, name)

def write(self, text):
pass

orig_stdout = sys.stdout
sys.stdout = StreamWrapper(orig_stdout)
print('a')   # this prints nothing
input('b')   # this should print nothing, but prints 'b' (in Python 3.5 and up 
only)

Looks like this was broken in http://bugs.python.org/issue24402 . Adding the 
'fileno' function from this issue fixes the problem, but it's just a 
workaround. This affects the colorama package: 
https://github.com/tartley/colorama/issues/103

--
messages: 278179
nosy: wiggin15
priority: normal
severity: normal
status: open
title: input() prints to original stdout even if sys.stdout is wrapped
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28199] Compact dict resizing is doing too much work

[INADA Naoki]

Changes by INADA Naoki :


Added file: http://bugs.python.org/file44983/dictresize3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28374] SyntaxError: invalid token in python2.7/test/test_grammar.py

[Jan Welker]

New submission from Jan Welker:

I compiled Python 2.7.12 from source and ran the tests unsuccessful.

Compiling /usr/lib/python2.7/test/test_grammar.py ...
  File "/usr/lib/python2.7/test/test_grammar.py", line 80
self.assertEqual(1 if 1else 0, 1)
   ^
SyntaxError: invalid token

It looks like a space (" ") is missing before the else. The same bug can be 
found in the next line 81.

--
components: Tests
messages: 278180
nosy: welker
priority: normal
severity: normal
status: open
title: SyntaxError: invalid token in python2.7/test/test_grammar.py
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28373] input() prints to original stdout even if sys.stdout is wrapped

[Adam Bartoš]

Adam Bartoš added the comment:

A related issue is that the REPL doesn't use sys.stdin for input, see #17620. 
Another related issue is #28333. I think that the situation around stdio in 
Python is complicated an inflexible (by stdio I mean all the interactions 
between REPL, input(), print(), sys.std* streams, sys.displayhook, 
sys.excepthook, C-level readline hooks). It would be nice to tidy up these 
interactions and document them at one place.

Currently, input() tries to detect whether sys.stdin and sys.stdout are 
interactive and have the right filenos, and handles the cases different way. I 
propose input() to be a thin wrapper (stripping a newline, generating EOFError) 
around proposed sys.readlinehook(). By default, sys.readlinehook would be GNU 
readline on Unix and stdio_readline (which just uses sys.stdout and sys.stdin) 
on Windows. I think that would fix all the problems like this one and 
changing/wrapping sys.std* streams would just work.

My proposal is at 
https://mail.python.org/pipermail/python-dev/2015-November/142246.html and 
there is discission at #17620. Recently, the related issue #1602 was fixed and 
there is hope there will be progress with #17620.

--
nosy: +Drekin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18597] On Windows sys.stdin.readline() doesn't handle Ctrl-C properly

[Adam Bartoš]

Adam Bartoš added the comment:

Maybe this was fixed with the recent fix of #1602.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28373] input() prints to original stdout even if sys.stdout is wrapped

[Adam Bartoš]

Adam Bartoš added the comment:

Other related issues are #1927 and #24829.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26964] Incorrect documentation for `-u`-flag

[Berker Peksag]

Changes by Berker Peksag :


--
nosy: +berker.peksag
stage:  -> patch review
type:  -> behavior
versions: +Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27232] os.fspath() should not use repr() on error

[Berker Peksag]

Berker Peksag added the comment:

This has already been fixed in ea7b6a7827a4:

>>> import os
>>> os.fspath(None)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: expected str, bytes or os.PathLike object, not NoneType

--
nosy: +berker.peksag
resolution:  -> out of date
status: open -> closed
superseder: Show the address in the repr for class objects -> 
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27231] Support the fspath protocol in the posixpath module

[Berker Peksag]

Berker Peksag added the comment:

It looks like the fspath protocol support has already been implemented in 
b64f83d6ff24.

--
nosy: +berker.peksag
resolution:  -> out of date
stage:  -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] assertion failure in rangeobject.c

[Oren Milman]

New submission from Oren Milman:

 current state 
An assertion in Objects/rangeobject.c might fail:
>>> type(iter(range(0)))

>>> type(iter(range(0)))(1, 1, 0)
Assertion failed: step != 0, file ..\Objects\rangeobject.c, line 895

This is caused by the lack of a check whether 'step' is zero, during the 
creation of a range_iterator object, in rangeiter_new.

Note that during the creation of a range object, the function range_new does 
check that, by calling validate_step, which leads to the following behavior:
>>> range(1, 1, 0)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: range() arg 3 must not be zero
>>>


 proposed changes 
1. In Objects/rangeobject.c in rangeiter_new:
- in case 'step' is zero, raise a ValueError
- in error messages, replace 'rangeiter' with 'range_iterator', as the 
latter is the name of the type in Python code

2. In Lib/test/test_range.py, add tests for calling the range_iterator type 
(i.e. creating a range_iterator object)


 diff 
The proposed patches diff file is attached.


 tests 
I ran 'python_d.exe -m test -j3' (on my 64-bit Windows 10) with and without the 
patches, and got quite the same output. (That also means my new tests in 
test_range passed.)
The outputs of both runs are attached.

--
components: Interpreter Core
files: CPythonTestOutput.txt
messages: 278187
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: assertion failure in rangeobject.c
type: crash
versions: Python 3.7
Added file: http://bugs.python.org/file44984/CPythonTestOutput.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28375] cgi.py spam in Apache server logs

[Sebastian Rittau]

New submission from Sebastian Rittau:

I am using cgi.py in WSGI applications, using Apache and mod_wsgi. 
Unfortunately cgi.py keeps spamming the error log with messages like the 
following:

Exception ignored in: 
Traceback (most recent call last):
  File "/usr/lib/python3.5/cgi.py", line 566, in __del__
NameError: name 'AttributeError' is not defined

This is mostly likely due to the warning about __del__ in 
, i.e. AttributeError will 
already have been cleaned at the time FieldStorage is collected. One workaround 
that seems to work for me is to cache AttributeError in the constructor of 
FieldStorage in self and use that attribute during __del__.

--
messages: 278186
nosy: srittau
priority: normal
severity: normal
status: open
title: cgi.py spam in Apache server logs
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] assertion failure in rangeobject.c

[Oren Milman]

Changes by Oren Milman :


Added file: http://bugs.python.org/file44985/patchedCPythonTestOutput_ver1.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] assertion failure in rangeobject.c

[Oren Milman]

Changes by Oren Milman :


--
keywords: +patch
Added file: http://bugs.python.org/file44986/issue28376_ver1.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] rangeiter_new fails when creating a range of step 0

[Emanuel Barry]

Emanuel Barry added the comment:

I'm not able to trigger an assertion error when running the latest trunk in 
debug mode. I get a "valid" range_iterator object though, and using __reduce__ 
gives me direct access to `range(0, 0, 0)` (which is completely worthless).

Error or not, this should be fixed, and your patch LGTM. Thanks!

(I changed the type to 'behaviour' since I can't reproduce the assertion, but 
it doesn't make much of a difference)

--
nosy: +ebarry
stage:  -> patch review
title: assertion failure in rangeobject.c -> rangeiter_new fails when creating 
a range of step 0
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] rangeiter_new fails when creating a range of step 0

[INADA Naoki]

INADA Naoki added the comment:

patch is LGTM.

But there is one hidden inconsistency:

>>> r = range(2**100)
>>> type(iter(r))

>>> type(iter(r))(1, 1, 0)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: cannot create 'longrange_iterator' instances

Should we have same tp_new method for longrange_iterator?

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28374] SyntaxError: invalid token in python2.7/test/test_grammar.py

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the report! How did you run the test? I think there is something 
wrong with your installation (or you probably ran the test with the system 
Python.)

$ ./python -m test.regrtest test_grammar
[1/1] test_grammar
1 test OK.
[48645 refs]

$ hg summary
parent: 104329:59260b38f7cd 
 make 'where' Py_ssize_t

--
nosy: +berker.peksag
resolution:  -> not a bug
stage:  -> resolved
status: open -> pending
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10716] Modernize pydoc to use better HTML and separate CSS

[R. David Murray]

R. David Murray added the comment:

My memory of the thread (I haven't gone back and checked) was that the thought 
was to drop html output altogether because the sphinx docs were more useful for 
anyone wanting to use html, and network connectivity was so universal in this 
day and age that having a local html server for offline uses was of minimal 
utility.

So your vote that it be kept because it is useful to idle and can be made more 
useful by improving the html seems to me to decide the issue in favor of making 
the improvements (given someone willing to do the patch review and any 
additional needed coding).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28377] struct.unpack of Bool

[Weihong Guan]

New submission from Weihong Guan:

According to the doc:

For the '?' format character, the return value is either True or False. When 
packing, the truth value of the argument object is used. Either 0 or 1 in the 
native or standard bool representation will be packed, and any non-zero value 
will be True when unpacking.

But it behaves more like & 1 operation. so False for even value, and True for 
odd value.

unpack fmt '?H' should reqiure buff of length 3, same as 'H?', not 4.

--
components: ctypes
files: Screen Shot 2016-10-06 at 20.59.43.png
messages: 278192
nosy: Weihong Guan
priority: normal
severity: normal
status: open
title: struct.unpack of Bool
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file44987/Screen Shot 2016-10-06 at 
20.59.43.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18597] On Windows sys.stdin.readline() doesn't handle Ctrl-C properly

[Eryk Sun]

Eryk Sun added the comment:

Switching to ReadConsoleW in 3.6+ solves the problem with not seeing 
ERROR_OPERATION_ABORTED in Windows 8+, and with proper handling this 
potentially solves issues with Ctrl+C handling (when I last checked there were 
still bugs with this in the 3.6 beta). However, the problem still exists in 2.7 
and 3.5, where the only possible solution is to switch to ReadConsoleA. Maybe 
once the new PyOS_StdioReadline code in 3.6 is stable, it can be backported to 
3.5 using ReadConsoleA instead of ReadConsoleW. 2.7 will probably remain broken.

--
versions:  -Python 3.3, Python 3.4, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18597] On Windows sys.stdin.readline() doesn't handle Ctrl-C properly

[Adam Bartoš]

Adam Bartoš added the comment:

The main reason I have extended the support of win_unicode_console to Python 
2.7 was that the related issues won't be fixed there, so using 
win_unicode_console may fix this as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] rangeiter_new fails when creating a range of step 0

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka
versions: +Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28377] struct.unpack of Bool

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Maybe this is a duplicate of issue22012.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28378] urllib2 does not handle cookies with `,`

[Grzegorz Sikorski]

New submission from Grzegorz Sikorski:

I have a usecase when the server sends two cookies in separate `Set-Cookie` 
headers. One of the cookie includes a `,` (comma). It seems this is not handled 
properly, as the library always try to fold multiple headers with the same name 
into a single comma-separated string. While this is valid for other header 
fields, `Set-Cookie` should never be folded, as RFC 6265 says:
```
   Origin servers SHOULD NOT fold multiple Set-Cookie header fields into
   a single header field.  The usual mechanism for folding HTTP headers
   fields (i.e., as defined in [RFC2616]) might change the semantics of
   the Set-Cookie header field because the %x2C (",") character is used
   by Set-Cookie in a way that conflicts with such folding.
```

--
components: Library (Lib)
messages: 278196
nosy: Grzegorz Sikorski
priority: normal
severity: normal
status: open
title: urllib2 does not handle cookies with `,`
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28355] wsgiref simple_server PATH_INFO treats slashes and %2F the same

[Ned Deily]

Changes by Ned Deily :


--
nosy: +pje

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27859] argparse - subparsers does not retain namespace

[Ned Deily]

Changes by Ned Deily :


--
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> argparse set_defaults on subcommands should override top level 
set_defaults

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28376] rangeiter_new fails when creating a range of step 0

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Good point Naoki. I think we can remove tp_new method from range_iterator in 
3.7. Seems it is never used.

The patch LGTM for 3.5-3.6, but the test should be marked as CPython 
implementation detail (@support.cpython_only).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28365] idle forgets that saved console session is not a python file after restart

[A.J.]

A.J. added the comment:

There is no other way to explain it without fail every time no matter what code 
I write there is always syntax error highlighted on the five in the version 
banner.

--
Added file: http://bugs.python.org/file44988/2016-10-06 (4).png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27386] Asyncio server hang when clients connect and immediately disconnect

[Yury Selivanov]

Yury Selivanov added the comment:

Alright, I've backported the fix to 3.4. Closing this.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27759] selectors incorrectly retain invalid file descriptors

[Roundup Robot]

Roundup Robot added the comment:

New changeset 8cc1fca83fb8 by Yury Selivanov in branch '3.4':
Issue #27759: Fix selectors incorrectly retain invalid file descriptors.
https://hg.python.org/cpython/rev/8cc1fca83fb8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27392] Add a server_side keyword parameter to create_connection

[Yury Selivanov]

Yury Selivanov added the comment:

AFAICT this issue was resolved in https://github.com/python/asyncio/pull/378. 
Closing this one. Thanks, Jim!

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28379] PyUnicode_CopyCharacters could lead to undefined behaviour

[Xiang Zhang]

New submission from Xiang Zhang:

Currently PyUnicode_CopyCharacters doesn't check arguments thoroughly. This 
could lead to undefined behaviour or crash in debug mode. For example, 
from_start > len(from), how_many < 0. Another case is that when how_many > 
len(from), it will choose len(from) but this can still fail since from_start 
can > 0. The doc of it is also not perfect, it does not necessarily return 0 on 
success.

--
components: Interpreter Core
files: PyUnicode_CopyCharacters.patch
keywords: patch
messages: 278202
nosy: haypo, serhiy.storchaka, xiang.zhang
priority: normal
severity: normal
stage: patch review
status: open
title: PyUnicode_CopyCharacters could lead to undefined behaviour
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file44989/PyUnicode_CopyCharacters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28379] PyUnicode_CopyCharacters could lead to undefined behaviour

[Xiang Zhang]

Changes by Xiang Zhang :


Removed file: http://bugs.python.org/file44989/PyUnicode_CopyCharacters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28379] PyUnicode_CopyCharacters could lead to undefined behaviour

[Xiang Zhang]

Changes by Xiang Zhang :


Added file: http://bugs.python.org/file44990/PyUnicode_CopyCharacters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27972] Confusing error during cyclic yield

[Yury Selivanov]

Yury Selivanov added the comment:

This is an interesting mind twister.  The key problem is that 
`self.runner_task` is blocked on *itself*: so Task._fut_waiter is set to the 
Task.

Therefore when the task is being cancelled, `Task.cancel` simply recurses.

One way to solve this is to prohibit tasks to await on themselves.  Right now 
the following code "kind of" works:

  async def f():
loop.call_later(1, lambda: t.set_result(42))
return await t

  loop = asyncio.get_event_loop()
  t = loop.create_task(f())
  print(loop.run_until_complete(t))

albeit it logs errors about invalid Future state.

My proposal is to raise a ValueError if Task is asked to await on itself.

Guido, what do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[Yury Selivanov]

Yury Selivanov added the comment:

The most recent patch segfaults... Will try to debug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28379] PyUnicode_CopyCharacters could lead to undefined behaviour

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Added comments on Rietveld. I don't know whether tests for this function are 
needed. It is public, but is not a part of stable API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[Yury Selivanov]

Yury Selivanov added the comment:

INADA, would you be able to take a look?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18789] XML Vunerability Table Unclear

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18789] XML Vunerability Table Unclear

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

I'll work on this :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28380] Mock functions with autospec don't support assert_called_once, assert_called, assert_not_called

[Yannick Brehon]

New submission from Yannick Brehon:

If one defines a mock for a function, using autospec=True, then the mock will 
correctly support assert_called_once_with(), among others, but not 
assert_called_once, assert_called, and assert_not_called.
The attached file contains a fix for the issue.

--
components: Library (Lib)
files: fix_autospecced_mock_functions.patch
keywords: patch
messages: 278208
nosy: Yannick Brehon
priority: normal
severity: normal
status: open
title: Mock functions with autospec don't support assert_called_once, 
assert_called, assert_not_called
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: 
http://bugs.python.org/file44991/fix_autospecced_mock_functions.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21140] Idle: saving Shell or an OutputWindow should default to .txt

[Terry J. Reedy]

Terry J. Reedy added the comment:

A more drastic change would be to refuse to save OutputWindow and Shell as 
.py(w,o) files.  #28365 is about a beginner who apparently save a short Shell 
session as .py, quit IDLE, loaded the saved session in an editor, and tried to 
run it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28365] IDLE: don't offer to save text files as .py

[Terry J. Reedy]

Terry J. Reedy added the comment:

I am closing this as a duplicate of existing #21140, to make .txt instead of 
.py the extension for files known not to be code files.  I There is an existing 
simple patch.  I will modify it to not even offer .py as a secondary choice.

The reason Paul had to close IDLE to open the file in an(other) editor window 
is that Python allows only one editor window per file, and Shell is a subclass 
of OutputWindow, which is a subclass of EditorWindow. (The latter inheritance 
is backwards; OutputWindow should be the base TextEditor and 'CodeEditor' a 
subclass thereof.) When Shell is saved, the file name appears on the recent 
files list, and attempts to open it make the exiting editor the active window. 

Angie, as Paul explained, your screenshot shows a text file, mislabeled as a 
.py python file, in the editor.  When you run non-code text as code, you get a 
SyntaxError.  The text file appears to be a saved interactive shell session.  
"Python 3" by itself is a SyntaxError and will continue to be an error no 
matter what follows.

It is possible that your Python installation, or the IDLE part of it, is badly 
broken, and needs to be repaired or re-installed.  But I think it much much  
more likely that you have made a mistake.  If you want to discuss this further, 
please post to python-list (or its news.gmane.org mirror).  Include a link to 
this issue, give a detailed description of what you did (keystrokes and mouse 
clicks), and ask any questions.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Idle: saving Shell or an OutputWindow should default to .txt
title: idle forgets that saved console session is not a python file after 
restart -> IDLE: don't offer to save text files as .py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28365] IDLE: don't offer to save text files as .py

[Terry J. Reedy]

Terry J. Reedy added the comment:

PS. A.J., you will likely be a happier user if you make a subdirectory under 
c:/Users/angie and put your files there instead of in the installation 
directory under the hidden AppDate

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28380] Mock functions with autospec don't support assert_called_once, assert_called, assert_not_called

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28381] Add a "starcaller" function

[Josh Rosenberg]

New submission from Josh Rosenberg:

Not sure if this is the right venue to propose this, but I'd like to propose 
adding a starcaller method to the standard library, either functools or 
operator (not sure if the proposal is more like partial or more like 
methodcaller).

Basically, right now, when you want to take an existing function and call it 
repeatedly with preconstructed tuples using functional tools (e.g. sorted or 
groupby's key argument, multiprocessing.Pool's various map-like methods), you 
need to either have a special starmethod (e.g. Pool.map vs. Pool.starmap), or 
if Python doesn't provide one, you need use Python-defined functions (and for 
stuff like Pool, you can't even use lambda due to pickling issues, so you need 
to define the utility somewhere else). This means that you can't reuse builtins 
for stuff like the old "find runs of consecutive numbers" example from the 
itertools docs ( https://docs.python.org/2.6/library/itertools.html#examples ).

In terms of behavior:

starfunc = starcaller(func)

would be behave roughly the same as the following (less efficient) Python 2 
code:

starfunc = functools.partial(apply, func)

(and perhaps starcaller(func, kwonly=True) would behave like 
functools.partial(apply, func, ()) to extend it to dicts).

This would make it possible to write the old consecutive numbers example:

for k, g in groupby(enumerate(data), lambda (i,x):i-x):

(which is no longer legal since Py3 banned sequence unpacking in arguments like 
that) as:

for k, g in groupby(enumerate(data), starcaller(operator.sub)):

It would also mean that instead of constantly needing to write new "star 
methods", you can just reuse existing functions with new batteries. For 
example, multiprocessing.Pool has a map and starmap function, but imap and 
imap_unordered have no starimap or starimap_unordered equivalents, which means 
using them at all requires you to def a function at trivial wrapper at global 
scope (possibly a long way away from the point of use) just to use an existing 
function ( def whydidineedthis(args): return existingfunction(*args) ).

With starcaller, there is no need to add starimap and friends, because the same 
result is easily achieved with:

with multiprocessing.Pool() as pool:
for res in pool.imap(starcaller(existingfunction), ...):

Is this a reasonable proposal? I could get an implementation ready fairly 
quickly, I'd just need to feedback on the name, which module it should be put 
in (functools or operator seem equally plausible) and whether the idea of using 
a keyword argument to the constructor makes more sense (kwonly=True) vs. 
expecting users to do functools.partial(starcaller(existingfunc), ()), vs. a 
separate class like doublestarcaller.

--
components: Library (Lib)
messages: 278212
nosy: josh.r
priority: normal
severity: normal
status: open
title: Add a "starcaller" function
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28380] Mock functions with autospec don't support assert_called_once, assert_called, assert_not_called

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4e39b4e57673 by Gregory P. Smith in branch '3.6':
Fixes issue28380: unittest.mock Mock autospec functions now properly support
https://hg.python.org/cpython/rev/4e39b4e57673

New changeset fca5c4a63251 by Gregory P. Smith in branch 'default':
Issue #28380: unittest.mock Mock autospec functions now properly support
https://hg.python.org/cpython/rev/fca5c4a63251

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28380] Mock functions with autospec don't support assert_called_once, assert_called, assert_not_called

[Gregory P. Smith]

Gregory P. Smith added the comment:

thanks!  I didn't apply the fix to 3.5 (or earlier - those are closed) as it 
could arguably be seen as adding a new API and there are valid workarounds by 
asserting on the list of calls directly.

--
resolution:  -> fixed
stage:  -> commit review
status: open -> closed
versions:  -Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27972] Confusing error during cyclic yield

[Guido van Rossum]

Guido van Rossum added the comment:

It's pretty perverse. But how would you detect this case? Does it
require changes to CPython or only to asyncio? Does it require a spec
change anywhere?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27972] Confusing error during cyclic yield

[Guido van Rossum]

Guido van Rossum added the comment:

Is that enough? What if the recursion involves several tasks waiting
for each other in a cycle?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27972] Confusing error during cyclic yield

[Yury Selivanov]

Yury Selivanov added the comment:

> It's pretty perverse. But how would you detect this case?

In Task._step, we can check if the future the task is about to await on is 
"self".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27972] Confusing error during cyclic yield

[Yury Selivanov]

Yury Selivanov added the comment:

> Is that enough? What if the recursion involves several tasks waiting
for each other in a cycle?

I'm not sure... Maybe it's OK when two tasks await on each other, I think the 
current Task implementation should be able to handle that.  The problem with 
the Task awaiting itself is that the current implementation just crashes with a 
RecursionError.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27972] Confusing error during cyclic yield

[Guido van Rossum]

Guido van Rossum added the comment:

Maybe it could be fixed rather than making this a checked failure?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28381] Add a "starcaller" function

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Hi Josh,

I think python ideas mailing list might have been a better venue for this.
https://mail.python.org/mailman/listinfo/python-ideas

--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28381] Add a "starcaller" function

[Ned Deily]

Changes by Ned Deily :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28206] signal.Signals not documented

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28382] Possible deadlock after many multiprocessing.Process are launch

[Alexis]

New submission from Alexis:

I am launching a process inside a pool worker, using the multiprocessing module.
After a while, a deadlock append when I am trying to join the process.

Here is a simple version of the code:

import sys, time, multiprocessing
from multiprocessing.pool import ThreadPool

def main():
# Launch 8 workers
pool = ThreadPool(8)
it = pool.imap(run, range(500))
while True:
try:
it.next()
except StopIteration:
break

def run(value):
# Each worker launch its own Process
process = multiprocessing.Process(target=run_and_might_segfault, 
args=(value,))
process.start()

while process.is_alive():
sys.stdout.write('.')
sys.stdout.flush()
time.sleep(0.1)

# Will never join after a while, because of a mystery deadlock
process.join()

def run_and_might_segfault(value):
print(value)

if __name__ == '__main__':
main()

And here is a possible output:

~ python m.py
..0
1
8
.9
...10
..11
12
13
14
16

As you can see, process.is_alive() is alway true after few iterations, the 
process will never join.

If I CTRL-C the script a get this stacktrace:

Traceback (most recent call last):
  File 
"/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py",
 line 680, in next
item = self._items.popleft()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "m.py", line 30, in 
main()
  File "m.py", line 9, in main
it.next()
  File 
"/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5
/lib/python3.5/multiprocessing/pool.py", line 684, in next
self._cond.wait(timeout)
  File 
"/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5
/lib/python3.5/threading.py", line 293, in wait
waiter.acquire()
KeyboardInterrupt

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File 
"/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5
/lib/python3.5/multiprocessing/popen_fork.py", line 29, in poll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

Using python 3.5.1 on macos, also tried with 3.5.2 with same issue.
Same result on Debian.
I tried using python 2.7, and it is working well. May be a python 3.5 issue 
only?

Here is the link of the stackoverflow question:
http://stackoverflow.com/questions/39884898/large-amount-of-multiprocessing-process-causing-deadlock

--
components: Library (Lib)
messages: 278221
nosy: Hadhoke
priority: normal
severity: normal
status: open
title: Possible deadlock after many multiprocessing.Process are launch
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28381] Add a "starcaller" function

[Steven D'Aprano]

Steven D'Aprano added the comment:

This was discussed on Python-Ideas back in July:

https://mail.python.org/pipermail/python-ideas/2016-July/041153.html

I don't recall any opposition, although Nick suggested that possibly a better 
idea was to resurrect the `apply` built-in into functools:

https://mail.python.org/pipermail/python-ideas/2016-July/041159.html

--
nosy: +ncoghlan, steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28199] Compact dict resizing is doing too much work

[Raymond Hettinger]

Raymond Hettinger added the comment:

For the simple case with no dummy entries, I was expecting a fast path that 
just realloced the keys/values/hashes arrays and then updated the index table 
with reinsertion logic that only touches the indices.   Use realloc() is nice 
because it makes it possible that the keys/values/hashes don't have to be 
recopied and if they did, it would use a fast memcpy to move them to the newly 
resized array.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[INADA Naoki]

INADA Naoki added the comment:

FutureIter_throw is wrong, maybe.
Removing FutureIter_send and FutureIter_throw from FutureIter_methods solves 
the segv and test passed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21443] asyncio logging documentation clarifications

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Hi, I added the paragraph explaining how to change the log level for asyncio.
Please check it out.
Thanks :)

--
keywords: +patch
nosy: +Mariatta
Added file: http://bugs.python.org/file44992/issue21443.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28199] Compact dict resizing is doing too much work

[INADA Naoki]

INADA Naoki added the comment:

Since entries array is embedded in PyDictKeysObject, we can't realloc entries.
And while values are split array, dictresize() convert split table into combine 
table.

Split table may have enough size of ma_values at first in typical case.
And in not typical case, split table may not be used.
So I think realloc ma_values is premature optimization, unless profiler says 
make_keys_shared()
is slow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26081] Implement asyncio Future in C to improve performance

[INADA Naoki]

INADA Naoki added the comment:

fixed

--
Added file: http://bugs.python.org/file44993/fastfuture2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26171] heap overflow in zipimporter module

[Parvesh jain]

Parvesh jain added the comment:

I think patches put up in http://bugs.python.org/msg258736 is at least not 
sufficient enough for Python 2.7. 
POC script(crash.py) provided with the issue calls get_data with data_size = -1.
I am using Python 2.7.8 . I patched the same with the solution provided in 
https://hg.python.org/cpython/rev/985fc64c60d6 . I was still able to reproduce 
the issue and it failed with 

Traceback (most recent call last):
  File "crash.py", line 25, in 
print(importer.get_data(FILE))
IOError: zipimport: can't read data
Segmentation fault (core dumped)


but I couldn't reproduce the same with latest 2.7.12:-

jchang@qasus-ubun12x64-001:~/Downloads/Python-2.7.12$ python2.7 -V
Python 2.7.12
jchang@qasus-ubun12x64-001:~/Downloads/Python-2.7.12$ python2.7 crash.py
Traceback (most recent call last):
  File "crash.py", line 25, in 
print(importer.get_data(FILE))
zipimport.ZipImportError: negative data size

As we can see issue does happen in 2.7.12 because of following extra check :-

if (data_size < 0) {
PyErr_Format(ZipImportError, "negative data size");
return NULL;
}

which was merged in https://hg.python.org/cpython/rev/2edbdb79cd6d. 

I was thinking of backporting the same to Python 2.7.8 as well to completely 
address this issue. Could you guys confirm if my understanding is correct on 
this ? Thanks

--
nosy: +Parvesh jain

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28383] __hash__ documentation recommends naive XOR to combine but this is suboptimal

[Kevin Norris]

New submission from Kevin Norris:

The documentation for __hash__ contains this text:

"The only required property is that objects which compare equal have the same 
hash value; it is advised to somehow mix together (e.g. using exclusive or) the 
hash values for the components of the object that also play a part in 
comparison of objects."

The recommendation of "using exclusive or" is likely to result in programmers 
naively doing this:

def __hash__(self):
return hash(self.thing1) ^ hash(self.thing2) ^ hash(self.thing3)

In the event that (say) self.thing1 and self.thing2 have almost or exactly the 
same hash (with "almost" referring to bitwise differences rather than integral 
distance), this wipes out most or all of the entropy from both values and 
greatly increases the likelihood of hash collisions.  Indeed, Python's own 
tuple type does not do this (while it does use XOR, it also does some other 
math to ensure the bits are as mixed up as is practical).[1]

Because the correct algorithm is both nontrivial to implement and already 
exists in the tuple type's __hash__, I propose that the documentation be 
updated to recommend something like the following:

def __hash__(self):
return hash((self.thing1, self.thing2, self.thing3))

One possible wording:

"The only required property is that objects which compare equal have the same 
hash value; it is advised to mix together the hash values of the components of 
the object that also play a part in comparison of objects by packing them into 
a tuple and hashing the tuple: [code example]"

[1]: https://hg.python.org/cpython/file/fca5c4a63251/Objects/tupleobject.c#l348

--
assignee: docs@python
components: Documentation
messages: 278229
nosy: Kevin.Norris, docs@python
priority: normal
severity: normal
status: open
title: __hash__ documentation recommends naive XOR to combine but this is 
suboptimal
type: performance
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18789] XML Vunerability Table Unclear

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Hi, here is the patch. I followed Raymond's suggestion to use 'vulnerable' or 
'safe' instead of the original 'True' or 'False'.

Please check it out.

Thanks :)

--
keywords: +patch
Added file: http://bugs.python.org/file44994/issue18789.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18967] Find a less conflict prone approach to Misc/NEWS

[Nick Coghlan]

Nick Coghlan added the comment:

I came across OpenStack's tool for this problem today: 
http://docs.openstack.org/developer/reno/design.html

I think it's significantly more complex than we need for CPython, but also 
still interesting as a point of reference.

It's already mentioned in PEP 512, but I'll also add a reference here to 
https://pypi.python.org/pypi/towncrier, Amber Brown's release note manager that 
allows Twisted style release notes management to be used with other projects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com