[issue15745] Numerous utime ns tests fail on FreeBSD w/ ZFS (update: and NetBSD w/ FFS, Solaris w/ UFS)

[Martin Panter]

Martin Panter added the comment:

The commit causes test_os to emit DeprecationWarning warnings, which it didn’t 
before:

[vadmium@localhost cpython]$ hg update 4335d898be59
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
[vadmium@localhost cpython]$ ./python -bWdefault -m test test_os
[1/1] test_os
/media/disk/home/proj/python/cpython/Lib/unittest/case.py:638: 
DeprecationWarning: stat_float_times() is deprecated
  function(*args, **kwargs)
[× 11 . . .]

--
nosy: +vadmium

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24462] bytearray.find Buffer Over-read

[DmitryJ]

DmitryJ added the comment:

Quick analysis tells this can be attributed to the following code (in 2.7):

https://hg.python.org/cpython/file/a8e24d776e99/Objects/stringlib/fastsearch.h#l110
https://hg.python.org/cpython/file/a8e24d776e99/Objects/stringlib/fastsearch.h#l116

Suppose i = 0, then s[i+m] causes OOB access when m=n. Note only one iteration 
is possible in case of m=n due to loop condition of i <= (w = n-m = 0). 
Theoretically, one can try disclosing one adjacent byte, but more likely 
results are nothing (or potentially invalid match result) or a potential crash 
in an unlucky case of s[m] hitting an unmapped page.

The same code lives in 3.2 (and likely any prior 3.x release), and 3.3 seems to 
be affected as well. 3.4 code has a modified version, but has the same problem 
(ss = s + m - 1; if (!STRINGLIB_BLOOM(mask, ss[i+1])) ...).

--
nosy: +dev_zzo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24462] bytearray.find Buffer Over-read

[DmitryJ]

DmitryJ added the comment:

>From the author's page at http://effbot.org/zone/stringlib.htm

"Note that the above Python code may access s[n], which would result in an 
IndexError exception. For the CPython implementation, this is not really a 
problem, since CPython adds trailing NULL entries to both 8-bit and Unicode 
strings."

Apparently, this flaw was known to the author, but was not documented in C code.

A possible quick-and-dirty solution is to treat m=n as a special case and 
resort to memcmp() or somesuch as there is no actual need to perform multiple 
match tries. This should fix things for bytearray and str in case str's 
implementation changes from appending a trailing NUL.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24339] iso6937 encoding missing

[John Helour]

Changes by John Helour :


Removed file: http://bugs.python.org/file39575/iso6937.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23255] SimpleHTTPRequestHandler refactor for more extensible usage.

[Berker Peksag]

Berker Peksag added the comment:

Ned is correct. I started to review the patch, but couldn't find time to do a 
complete review. I'll take a look at it in a week or two. Thanks!

--
assignee:  -> berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24421] Race condition compiling Modules/_math.c

[Tal Einat]

Tal Einat added the comment:

Adding Thomas Wouters to the nosy list, since he's listed on the Experts Index 
under "autoconf/makefiles" as an interest area. Hopefully he can help move this 
forward.

--
nosy: +twouters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23020] New matmul operator crashes modules compiled with CPython3.4

[Tal Einat]

Changes by Tal Einat :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24463] Python 3.4 bugs

[vasya yugov]

New submission from vasya yugov:

This code:

w = [[0] * 2] * 2
w[1][1] = 1
print(w)

prints
[[0, 1], [0, 1]]
Is it a bug?

--
components: Macintosh
messages: 245460
nosy: ned.deily, ronaldoussoren, vasya yugov
priority: normal
severity: normal
status: open
title: Python 3.4 bugs
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24450] Add gi_yieldfrom calculated property to generator object

[Benno Leslie]

Benno Leslie added the comment:

I've tried to address all the issues raised in the review of the first patch.

--
Added file: http://bugs.python.org/file39725/gi_yieldfrom.v1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24463] Python 3.4 bugs

[Skip Montanaro]

Skip Montanaro added the comment:

Not a bug. The two elements of w are references to the same list:

>>> w = [[0] * 2] * 2
>>> w
[[0, 0], [0, 0]]
>>> [id(elt) for elt in w]
[21743952, 21743952]

--
nosy: +skip.montanaro
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24464] Got warning when compiling sqlite3 module on Mac OSX

[Vajrasky Kok]

New submission from Vajrasky Kok:

I got this warning when compiling sqlite3 module.

gcc -Wno-unused-result -Wsign-compare -g -O0 -Wall -Wstrict-prototypes 
-Werror=declaration-after-statement -DMODULE_NAME="sqlite3" 
-DSQLITE_OMIT_LOAD_EXTENSION=1 -IModules/_sqlite -I/usr/include -I./Include -I. 
-IInclude -I/usr/local/include -I/Users/sky/Code/python/cpython/Include 
-I/Users/sky/Code/python/cpython -c 
/Users/sky/Code/python/cpython/Modules/_sqlite/module.c -o 
build/temp.macosx-10.10-x86_64-3.6-pydebug/Users/sky/Code/python/cpython/Modules/_sqlite/module.o
/Users/sky/Code/python/cpython/Modules/_sqlite/module.c:136:10: warning: 
'sqlite3_enable_shared_cache' is
  deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
rc = sqlite3_enable_shared_cache(do_enable);
 ^
/usr/include/sqlite3.h:5006:16: note: 'sqlite3_enable_shared_cache' has been 
explicitly marked deprecated here
SQLITE_API int sqlite3_enable_shared_cache(int) 
__OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0, __MAC_10_7, __I...
   ^
1 warning generated.

The enable_shared_cache method will fail if it is used afterwards.
"Changing the shared_cache flag failed"

Here is the patch to remove the warning.

--
components: Macintosh
files: remove_warning_compile__sqlite.patch
keywords: patch
messages: 245463
nosy: ned.deily, ronaldoussoren, vajrasky
priority: normal
severity: normal
status: open
title: Got warning when compiling sqlite3 module on Mac OSX
versions: Python 3.4, Python 3.5
Added file: 
http://bugs.python.org/file39726/remove_warning_compile__sqlite.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make tar files created by shutil.make_archive() have deterministic sorting

[Sam Thursfield]

New submission from Sam Thursfield:

I want shutil.make_archive() to produce deterministic output when given 
identical data as inputs.

Right now there are two holes in this. One is that mtimes might not match. This 
can be fixed by the caller. The second is that the order that files in a 
subdirectory get added to the tarfile is not deterministic. This can't be fixed 
by the caller.

Attached is a trivial patch to sort the results of os.listdir() to ensure the 
output tarfile is stable.

This only applies to the 'tar' format.

I've attached my testcase for this, which creates 3 tarfiles in /tmp. When this 
patch is applied, the 3 tarfiles it creates are identical according to 
`sha1sum`. Without this patch, they are all different.

--
components: Library (Lib)
files: tar-reproducible-testcase.py
messages: 245464
nosy: samthursfield
priority: normal
severity: normal
status: open
title: Make tar files created by shutil.make_archive() have deterministic 
sorting
type: enhancement
Added file: http://bugs.python.org/file39727/tar-reproducible-testcase.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make tar files created by shutil.make_archive() have deterministic sorting

[Sam Thursfield]

Changes by Sam Thursfield :


--
keywords: +patch
Added file: http://bugs.python.org/file39728/tarfile-stable-ordering.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make tarfile have deterministic sorting

[R. David Murray]

R. David Murray added the comment:

This would go beyond what the tar command itself does.  I'm not sure we want to 
do that, as we are pretty much modeling our behavior on tar.  However, that 
doesn't automatically mean we can't do it.   We'll see what other people think. 
 Personally I'm -0.

I've changed the issue title since your proposed patch is to tarfile, not 
shutil.

--
nosy: +r.david.murray
title: Make tar files created by shutil.make_archive() have deterministic 
sorting -> Make tarfile have deterministic sorting
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make tarfile have deterministic sorting

[Lars Gustäbel]

Lars Gustäbel added the comment:

You don't need to patch the tarfile module. You could use os.walk() in 
shutil._make_tarball() and add each file with TarFile.add(recursive=False).

--
nosy: +lars.gustaebel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make tarfile have deterministic sorting

[Sam Thursfield]

Sam Thursfield added the comment:

Thanks for the comments! Would you be happy for the patch to be merged if it 
was implemented by modifying shutil.make_archive() instead? I will rework it if 
so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24466] extend_path explanation in documentation is ambiguous

[alanf]

New submission from alanf:

The description of pkgutil.extend_path in the doc (e.g., 
https://docs.python.org/2/library/pkgutil.html , 
https://docs.python.org/3/library/pkgutil.html ) is so ambiguous that I had to 
run a test to understand its behavior.

The doc says:

This will add to the package’s __path__ all subdirectories of directories on 
sys.path named after the package.

It wasn't clear to me how this should be parsed. Using parentheses to group the 
clauses, there's this possibility :

(1) This will add to the package’s __path__ all (subdirectories of directories 
on sys.path) named after the package.

That is, given all subdirectories of directories on sys.path, find the ones 
that are named after the package, and add them to the package's __path__.

or:

(2) This will add to the package’s __path__ all subdirectories of (directories 
on sys.path named after the package).

That is, given all directories on sys.path that are named after the package, 
add all their subdirectories to the package's __path__.

or: 

(3) This will add to the package’s __path__ all subdirectories of ((directories 
on sys.path) named after the package).

That is, given all directories on sys.path that are named after the package, 
add all their subdirectories to the package's __path__.

or:

(4) This will add to the package’s __path__ all (subdirectories of (directories 
on sys.path)) named after the package.

That is, given all directories on sys.path, add any of their subdirectories 
that are named after the package to the package's __path__.

It was also unclear to me whether the subdirectories were meant to be traversed 
recursively.

My testing indicates that (4) is the correct parse, and that the subdirectories 
are not meant to be traversed recursively.

I suggest this wording: For each directory on sys.path that has a subdirectory 
that matches the package name, add the subdirectory to the package's __path__.

--
assignee: docs@python
components: Documentation
messages: 245468
nosy: alanf, docs@python
priority: normal
severity: normal
status: open
title: extend_path explanation in documentation is ambiguous
type: enhancement
versions: Python 2.7, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make tarfile have deterministic sorting

[Raymond Hettinger]

Raymond Hettinger added the comment:

I don't see any downside for this simple patch and think there is some merit 
for wanting a reproducible archive.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

python-bugs-list@python.org

[Tin Tvrtković]

Tin Tvrtković added the comment:

We're actually getting bitten by this in production through the Riak Python 
client, so this isn't a strictly theoretical problem.

--
nosy: +tinchester

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

python-bugs-list@python.org

[Antoine Pitrou]

Antoine Pitrou added the comment:

Yes, we need to fix this. Sorry for being a bit slow.

--
versions: +Python 3.5, Python 3.6 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23255] SimpleHTTPRequestHandler refactor for more extensible usage.

[Ent]

Ent added the comment:

Thanks Ned & Berker,

I can only imagine the amount of work the core devs have to deal with.
Hope my patch makes it through in next version.

Regards,
Ent

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24419] In argparse action append_const doesn't work for positional arguments

[paul j3]

paul j3 added the comment:

You can give the positional any custom name (the first parameter).  You just 
can't reuse it (by giving 2 positionals the same name).  And if you don't like 
what the 'help' shows, you can set the 'metavar'.  That way only you see the 
positional's name.

The name of a positional can be the 'dest' of an optional.  But wouldn't that 
be confusing?  Setting the same attribute with a required postional and one or 
more optional optionals?

'nargs' is another way of assigning more than one value to a Namespace 
attribute.  You just can't put an optional between two such values.

`argparse` is a parser, a way of identifying what the user gives you.  It is 
better to err on the side of preserving information.  Different argument dests 
does that.   You can always combine values after parsing.

args.foo.append(args.bar)   # or .extend()
args.x = [args.foo, args.bar]

Don't try to force argparse to do something special when you can just as easily 
do that later with normal Python expressions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19235] Add a dedicated subclass for recursion errors

[Elazar Gershuni]

Elazar Gershuni added the comment:

So what holds it back now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24400] Awaitable ABC incompatible with functools.singledispatch

[Yury Selivanov]

Yury Selivanov added the comment:

Another iteration of the patch is attached.  Nick, I think it's ready for your 
review.

--
stage: needs patch -> patch review
type:  -> enhancement
Added file: http://bugs.python.org/file39729/corotype.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24429] msvcrt error when embedded

[eryksun]

eryksun added the comment:

> shapely's installation instructions from windows are to use 
> chris gohlke's prebuilt binaries from here: 
> http://www.lfd.uci.edu/~gohlke/pythonlibs/

Christoph Gohlke's Shapely‑1.5.9‑cp27‑none‑win_amd64.whl includes a version of 
geos_c.dll that has the VC90 manifest embedded as resource 2, just like 
python27.dll. The DLL also exports a GEOSFree function, which is what shapely 
actually uses. That said, the geos.py module still defines a global free() 
using cdll.msvcrt.free. As far as I can see, it never actually calls it. 
Otherwise it would surely crash the process due to a heap mismatch.

Steve, since you haven't closed this issue, have you considered my suggestion 
to export _Py_ActivateActCtx and _Py_DeactivateActCtx for use by C extensions 
such as _ctypes.pyd? These functions are better than manually creating a 
context from the manifest that's embedded in python27.dll because they use the 
context that was active when python27.dll was initially loaded.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24419] In argparse action append_const doesn't work for positional arguments

[py.user]

py.user added the comment:

paul j3 wrote:
> You can give the positional any custom name (the first parameter).

The dest argument is not required for giving name for an optional.
You can either make it automatically or set by dest, it's handy and clear.

>>> import argparse
>>> 
>>> parser = argparse.ArgumentParser()
>>> _ = parser.add_argument('-a', '--aa')
>>> _ = parser.add_argument('-b', '--bb', dest='x')
>>> args = parser.parse_args([])
>>> print(args)
Namespace(aa=None, x=None)
>>>

But if you do the same thing with a positional, it throws an exception. Why?
(I'm a UNIX user and waiting predictable behaviour.)

And the situation with another action (not only append_const, but future 
extensions) shows that dest may be required.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19235] Add a dedicated subclass for recursion errors

[Yury Selivanov]

Yury Selivanov added the comment:

+1.

This, unfortunately, can't go in 3.5 (too late), but I can commit this in 3.6.

--
assignee:  -> yselivanov
nosy: +yselivanov
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19235] Add a dedicated subclass for recursion errors

[Elazar Gershuni]

Elazar Gershuni added the comment:

Well that's a déjà vu.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6839] zipfile can't extract file

[Sean Goodwin]

Changes by Sean Goodwin :


--
nosy: +Sean Goodwin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19235] Add a dedicated subclass for recursion errors

[Yury Selivanov]

Yury Selivanov added the comment:

Larry, is there any chance this can be committed in 3.5 (the change is fully 
backwards compatible)?

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24429] msvcrt error when embedded

[Steve Dower]

Steve Dower added the comment:

> Steve, since you haven't closed this issue, have you considered my suggestion 
> to export _Py_ActivateActCtx and _Py_DeactivateActCtx for use by C extensions 
> such as _ctypes.pyd? These functions are better than manually creating a 
> context from the manifest that's embedded in python27.dll because they use 
> the context that was active when python27.dll was initially loaded.

I'm always fairly slow to close issues - don't read too much into that :)

I don't see any value in exporting them for other C extensions, since they can 
also capture the initial context when they are loaded. They really need exports 
for Python. windll.python27._Py_ActivateActCtx would suffice and I wouldn't 
want to go any further than that - this is *very* advanced functionality that I 
would expect most people to get wrong.

Someone prepare a patch. I'm not -1 yet (and of course, I'm not the sole 
gatekeeper here, so one of the core devs who's still working on 2.7 can fix it 
too).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24419] In argparse action append_const doesn't work for positional arguments

[paul j3]

paul j3 added the comment:

(Important correction at the end of this post)

The test that you are complaining about occurs at the start of the 
'add_argument' method:

def add_argument(self, *args, **kwargs):
"""
add_argument(dest, ..., name=value, ...)
add_argument(option_string, option_string, ..., name=value, ...)
"""

# if no positional args are supplied or only one is supplied and
# it doesn't look like an option string, parse a positional
# argument
chars = self.prefix_chars
if not args or len(args) == 1 and args[0][0] not in chars:
if args and 'dest' in kwargs:
raise ValueError('dest supplied twice for positional argument')
kwargs = self._get_positional_kwargs(*args, **kwargs)

# otherwise, we're adding an optional argument
else:
kwargs = self._get_optional_kwargs(*args, **kwargs)
...

and the 2 methods it calls:

def _get_positional_kwargs(self, dest, **kwargs):
# code to deduce the 'required' parameter ...
# return the keyword arguments with no option strings
return dict(kwargs, dest=dest, option_strings=[])

def _get_optional_kwargs(self, *args, **kwargs):
# determine short and long option strings

# infer destination, '--foo-bar' -> 'foo_bar' and '-x' -> 'x'
dest = kwargs.pop('dest', None)
if dest is None:
if long_option_strings:
dest_option_string = long_option_strings[0]
else:
dest_option_string = option_strings[0]
dest = dest_option_string.lstrip(self.prefix_chars)
if not dest:
msg = _('dest= is required for options like %r')
raise ValueError(msg % option_string)
dest = dest.replace('-', '_')

# return the updated keyword arguments
return dict(kwargs, dest=dest, option_strings=option_strings)

At the 'add_argument' stage, a big difference between positionals and optionals 
is in how 'dest' is deduced.  Note the doc string.

During parsing, positionals are distinguished from optionals by the 
'option_strings' attribute (empty or not).  'dest' is not used during parsing, 
except by the Action '__call__'.

-

I just thought of another way around this constraint - set 'dest' after the 
action is created:

p=argparse.ArgumentParser()
a1=p.add_argument('foo',action='append')
a2=p.add_argument('bar',action='append')
a1.dest='x'
a2.dest='x'
args=p.parse_args(['one','two'])

produces

Namespace(x=['one', 'two'])

This works because after the action has been created, no one checks whether the 
'dest' value is duplicated or even looks pretty (except when trying to format 
it for the help.

You could also write a custom Action class, one that mangles the 'dest' to your 
heart's delight.  The primary use of 'self.dest' is in the expression:

setattr(namespace, self.dest, items)

you could replace this line in the Action '__call__' with

setattr(namespace, 'secret#dest', items)

-

I was mistaken on one thing - you can reuse positional 'names':

 a1=p.add_argument('foo',action='append')
 a2=p.add_argument('foo',action='append',type=int)
 p.parse_args(['a','3'])

produces:

 Namespace(foo=['a', 3])

There is a 'name' conflict handler, but it only pays attention to the option 
strings (flags for optionals).  You can't have two arguments using '-f' or 
'--foo'.  But you can have 2 or more positionals with the same 'dest'.  You 
just have to set the dest the right way.

This last point renders the whole issue moot.  But I'll leave it at the end to 
reflect my train of thought.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24400] Awaitable ABC incompatible with functools.singledispatch

[Yury Selivanov]

Changes by Yury Selivanov :


Added file: http://bugs.python.org/file39730/corotype.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24467] bytearray pop and remove Buffer Over-read

[JohnLeitch]

New submission from JohnLeitch:

The bytearray pop and remove methods suffer from buffer over-reads caused by 
memmove use under the assumption that PyByteArrayObject ob_size is less than 
ob_alloc, leading to a single byte over-read. This condition can be triggered 
by creating a bytearray from a range of length 0x10, then calling pop with a 
valid index:

bytearray(range(0x10)).pop(0)

The result is a memmove that reads off the end of src:

0:000> r
eax=071aeff0 ebx= ecx=071aeff1 edx=0010 esi=06ff80c8 edi=0010
eip=6234b315 esp=0027fc98 ebp=0027fca0 iopl=0 nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b efl=0202
MSVCR90!memmove+0x5:
6234b315 8b750c  mov esi,dword ptr [ebp+0Ch] 
ss:002b:0027fcac=071aeff1
0:000> dV
dst = 0x071aeff0 ""
src = 0x071aeff1 "???"
  count = 0x10
0:000> db poi(dst)
071aeff0  00 01 02 03 04 05 06 07-08 09 0a 0b 0c 0d 0e 0f  
071af000  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af010  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af020  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af030  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af040  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af050  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af060  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
0:000> db poi(src)
071aeff1  01 02 03 04 05 06 07 08-09 0a 0b 0c 0d 0e 0f ??  ...?
071af001  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af011  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af021  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af031  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af041  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af051  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
071af061  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  
0:000> g
(1968.1a88): Access violation - code c005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=0c0b0a09 ebx= ecx=0004 edx= esi=071aeff1 edi=071aeff0
eip=6234b468 esp=0027fc98 ebp=0027fca0 iopl=0 nv up ei ng nz ac pe cy
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b efl=00010297
MSVCR90!UnwindUpVec+0x50:
6234b468 8b448efcmov eax,dword ptr [esi+ecx*4-4] 
ds:002b:071aeffd=
0:000> k
ChildEBP RetAddr  
0027fca0 1e0856aa MSVCR90!UnwindUpVec+0x50 
[f:\dd\vctools\crt_bld\SELF_X86\crt\src\Intel\MEMCPY.ASM @ 322]
0027fcc0 1e0aafd7 python27!bytearray_pop+0x8a 
[c:\build27\cpython\objects\bytearrayobject.c @ 2378]
0027fcd8 1e0edd10 python27!PyCFunction_Call+0x47 
[c:\build27\cpython\objects\methodobject.c @ 81]
0027fd04 1e0f017a python27!call_function+0x2b0 
[c:\build27\cpython\python\ceval.c @ 4033]
0027fd74 1e0f1150 python27!PyEval_EvalFrameEx+0x239a 
[c:\build27\cpython\python\ceval.c @ 2682]
0027fda8 1e0f11b2 python27!PyEval_EvalCodeEx+0x690 
[c:\build27\cpython\python\ceval.c @ 3265]
0027fdd4 1e11707a python27!PyEval_EvalCode+0x22 
[c:\build27\cpython\python\ceval.c @ 672]
0027fdec 1e1181c5 python27!run_mod+0x2a [c:\build27\cpython\python\pythonrun.c 
@ 1371]
0027fe0c 1e118760 python27!PyRun_FileExFlags+0x75 
[c:\build27\cpython\python\pythonrun.c @ 1358]
0027fe4c 1e1190d9 python27!PyRun_SimpleFileExFlags+0x190 
[c:\build27\cpython\python\pythonrun.c @ 950]
0027fe68 1e038d35 python27!PyRun_AnyFileExFlags+0x59 
[c:\build27\cpython\python\pythonrun.c @ 753]
0027fee4 1d001017 python27!Py_Main+0x965 [c:\build27\cpython\modules\main.c @ 
643]
0027fef0 1d0011b6 pythonw!WinMain+0x17 [c:\build27\cpython\pc\winmain.c @ 15]
0027ff80 76477c04 pythonw!__tmainCRTStartup+0x140 
[f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 578]
0027ff94 7799ad1f KERNEL32!BaseThreadInitThunk+0x24
0027ffdc 7799acea ntdll!__RtlUserThreadStart+0x2f
0027ffec  ntdll!_RtlUserThreadStart+0x1b

If the over-read is allowed to succeed, a byte adjacent to the buffer is copied:

0:000> r
eax=01d8e978 ebx= ecx= edx=003a esi=01dc80c8 edi=0010
eip=1e08569a esp=0027fd0c ebp=01d5aa10 iopl=0 nv up ei pl nz na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b efl=0206
python27!bytearray_pop+0x7a:
1e08569a 8bd7mov edx,edi
0:000> dt self
Local var @ 0x27fd20 Type PyByteArrayObject*
0x01dc80c8 
   +0x000 ob_refcnt: 0n2
   +0x004 ob_type  : 0x1e21a6d0 _typeobject
   +0x008 ob_size  : 0n16
   +0x00c ob_exports   : 0n0
   +0x010 ob_alloc : 0n16
   +0x014 ob_bytes : 0x01d8e978  ""
0:000> db 0x01d8e978
01d8e978  00 01 02 03 04 05 06 07-08 09 0a 0b 0c 0d 0e 0f  
01d8

[issue21297] csv.skipinitialspace only skips spaces, not "whitespace" in general

[Brandon Milam]

Brandon Milam added the comment:

This is my first attempt at working with the test suite but I believe this is 
what you were asking for. Due to this being my first attempt at writing tests I 
have included it as a separate patch file. Any further changes just let me know.

--
Added file: http://bugs.python.org/file39732/skipinitialspace_test.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Jacek Kołodziej]

Changes by Jacek Kołodziej :


Added file: 
http://bugs.python.org/file39733/Issue23883_support_check__all__.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Jacek Kołodziej]

Changes by Jacek Kołodziej :


Added file: http://bugs.python.org/file39734/Issue23883_test_gettext.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Jacek Kołodziej]

Jacek Kołodziej added the comment:

Thank you for feedback, Martin. I've amended the the patch.

Next, I've prepared some initial test.support.check__all__ helper, based on 
generalization of all previous patches. Its name/params' descriptions may be a 
bit rough - amendments/suggestions for such will be strongly appreciated: 
Issue23883_support_check__all__.patch

I've added missing test.test_gettext.MiscTestCase, based on aforementioned 
check__all__ helper: Issue23883_test_gettext.patch 

I've also took the liberty of working on some more modules. These are: csv 
(using new helper), enum, ftplib, logging, optparse, pickletools, threading and 
wave: Issue23883_all.patch

ftplib and threading have more functions (missing in their __all__ variables) 
that appear to be documented than mentioned in msg240217 - namely:
* ftplib.error_temp 
https://docs.python.org/3/library/ftplib.html#ftplib.error_temp
* ftplib.error_proto 
https://docs.python.org/3/library/ftplib.html#ftplib.error_proto
* threading.main_thread 
https://docs.python.org/3/library/threading.html#threading.main_thread

so I've added them as well.

--
Added file: http://bugs.python.org/file39735/Issue23883_all.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23883] __all__ lists are incomplete

[Jacek Kołodziej]

Jacek Kołodziej added the comment:

> ftplib and threading have more functions

I've meant function and exceptions, of course. Sorry for the noise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24468] Expose compiler flag constants as code object attributes

[Nick Coghlan]

Changes by Nick Coghlan :


--
dependencies: +Awaitable ABC incompatible with functools.singledispatch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24468] Expose compiler flag constants as code object attributes

[Nick Coghlan]

New submission from Nick Coghlan:

As part of the PEP 492 implementation, Yury has needed to hardcode compile flag 
contants in various places, with adjacent comments explaining what the magic 
numbers mean.

It occurred to me that there's a way we could make those constants readily 
available to any code manipulating code objects: expose them as read-only 
attributes via the code object type.

Does this seem like a reasonable idea?

If yes, would it be reasonable to classify it as part of the PEP 492 
implementation process and include it during the 3.5 beta cycle?

--
messages: 245487
nosy: gvanrossum, larry, ncoghlan, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Expose compiler flag constants as code object attributes
type: enhancement
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24468] Expose compiler flag constants as code object attributes

[Larry Hastings]

Larry Hastings added the comment:

Probably, though I want to see a sample implementation before I agree to 
anything.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24468] Expose compiler flag constants as code object attributes

[Nick Coghlan]

Nick Coghlan added the comment:

In my last set of review comments on issue 24400 I suggested changing the 
Python level attributes for coroutine objects to cr_frame, cr_code, and 
cr_running.

It's possible that may provide a different way to eliminate some of the current 
compiler flag checks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24400] Awaitable ABC incompatible with functools.singledispatch

[Nick Coghlan]

Nick Coghlan added the comment:

In my last set of review comments, I suggested changing the Python level 
attributes for coroutine objects to cr_frame, cr_code, and cr_running.

That reminded me that now that coroutines are their own type, we should also 
give them their own state introspection API, matching the API for generators: 
https://docs.python.org/3/library/inspect.html#current-state-of-a-generator

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24450] Add gi_yieldfrom calculated property to generator object

[Nick Coghlan]

Nick Coghlan added the comment:

Marking this as dependent on issue 24400, as that refactors the PEP 492 
implementation to make coroutines their own type (albeit one that shares a 
memory layout and some attribute names with generators at the C layer).

I'd suggest cr_await as the calculated property for coroutines that corresponds 
to gi_yieldfrom for generators.

--
dependencies: +Awaitable ABC incompatible with functools.singledispatch
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24469] Py2.x int free list can grow without bounds

[Stefan Behnel]

New submission from Stefan Behnel:

A Cython user noticed a memory leak when C-inheriting from "int".

http://thread.gmane.org/gmane.comp.python.cython.devel/15689

The Cython code to reproduce this is simply this:

cdef class ExtendedInt(int): pass
 
for j in xrange(1000):
ExtendedInt(j)

The problem is due to the free-list of the int type. It uses this code for 
deallocation:

"""
static void
int_dealloc(PyIntObject *v)
{
if (PyInt_CheckExact(v)) {
Py_TYPE(v) = (struct _typeobject *)free_list;
free_list = v;
}
else
Py_TYPE(v)->tp_free((PyObject *)v);
}

static void
int_free(PyIntObject *v)
{
Py_TYPE(v) = (struct _typeobject *)free_list;
free_list = v;
}
"""

Now, when C-inheriting from PyInt_Type without providing an own tp_free 
implementation, PyType_Ready() will inherit the supertype's tp_free slot, which 
means that int_dealloc() above will end up calling int_free() in all cases, not 
only for the exact-int case. Thus, whether or not it's exactly "int" or a 
subtype, the object will always be added to the free-list on deallocation.

However, in the subtype case, the free-list is actually ignored by int_new() 
and int_subtype_new(), so that as long as the user code only creates tons of 
int subtype instances and not plain int instances, the freelist will keep 
growing without bounds by pushing dead subtype objects onto it that are never 
reused.

There are two problems here:

1) int subtypes should not be added to the freelist, or at least not unless 
they have exactly the same struct size as PyIntObject (which the ExtendedInt 
type above does but other subtypes may not)

2) if a free-list is used, it should be used in all instantiation cases, not 
just in PyInt_FromLong().

Fixing 1) by adding a type check to int_free() would be enough to fix the 
overall problem. Here's a quickly hacked up change that seems to work for me:

"""
static void
int_free(PyIntObject *v)
{
if (PyInt_CheckExact(v)) {
Py_TYPE(v) = (struct _typeobject *)free_list;
free_list = v;
}
else if (Py_TYPE(v)->tp_flags & Py_TPFLAGS_HAVE_GC)
PyObject_GC_Del(v);  // untested by probably necessary
else
PyObject_Del(v);
}
"""

--
components: Interpreter Core
messages: 245492
nosy: scoder
priority: normal
severity: normal
status: open
title: Py2.x int free list can grow without bounds
type: crash
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com