[issue25764] PyObject_Call() is called with an exception set in subprocess

[STINNER Victor]

STINNER Victor added the comment:

> Assertion failed: !PyErr_Occurred(), file Objects/abstract.c, line 2158

Quick reminder: it's not a major bug if PyObject_Call() is called with an 
exception set. The exception will be kept in some cases. But I added the 
assertion because the exception is lost in some other cases. Example: calling 
hasattr(obj, name) replaces the current exception with an AttributeError and 
then clears the AttributeError exception if the object doesn't have the 
attribute.

> My guess is that fork() has failed,

Oh good job, it's exactly that: I'm able to reproduce the bug with your unit 
test (from your attached patch).

If I recall correctly, there is no overallocation on Solaris (OpenIndiana), and 
this buildbot is running low memory (MemoryError is very common). fork() can 
likely fail under low memory.

> and because preexec_fn is in use, the code tries to call gc.enable() with the 
> fork() exception already set. My patch adds a test and fixes that.

Hum, I think that your patch can be simplified. Replace:

if (pid == -1) {
PyErr_SetFromErrno(PyExc_OSError);
...
if (need_to_reenable_gc && _enable_gc(gc_module)) ...

with:

if (need_to_reenable_gc && _enable_gc(gc_module)) ...
...
if (pid == -1) {
PyErr_SetFromErrno(PyExc_OSError);

What do you think?

> Victor already fixed a similar error for the cleanup branch of 
> subprocess_fork_exec() in Issue 22290.

I reproduced the issue on Linux with pyfailmalloc.

You can play with pyfailmalloc to try to find similar bugs. The Bitbucket 
repository contains the latest debug_cpython.patch which adds some more checks 
to make the analysis faster and more effective, and debug_cpython.gdb contains 
commands to run it.

https://bitbucket.org/haypo/pyfailmalloc

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25764] PyObject_Call() is called with an exception set in subprocess

[Martin Panter]

Martin Panter added the comment:

Couldn’t re-enabling garbage collection cause errno to be overwritten? Even if 
I saved errno until later, there is still the RuntimeError from releasing the 
import lock to worry about.

Now that I look closer, I see the lock release code path sets this exception, 
but does not return NULL. Another bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25759] Python 2.7.11rc1 not building with Visual Studio 2015

[Kovid Goyal]

Kovid Goyal added the comment:

I take it back, my methodology in reproducing the function calls used by the 
gzip module was flawed. 

It does look like a bug in the CRT, but I have not been able to isolate a 
simple way of reproducing it. I have however, found a workaround for it, that 
has an acceptable performance impact. 

https://github.com/kovidgoyal/cpython/commit/72ae720ab057b1ac0402d67a7195d575d34afbbd

Now all tests pass (except for tcl/tk and distutils, neither of which I care 
about -- well I will probably need to fix up distutils at some point, but not 
now :). Running testsuite as

./PCbuild/amd64/python_d.exe Lib/test/regrtest.py -u 
network,cpu,subprocess,urlfetch

@steve: Thank you for all the work you did porting python 3.x to VS 2015, that 
certainly made by life a lot easier.

I would of course, be ecstatic if you were to consider merging my work into the 
python 2.7 branch, but if not, I understand -- no one likes to maintain a 
legacy codebase.

In any case, for interested third parties, my work is available here:

https://github.com/kovidgoyal/cpython (2.7 branch)

and instructions on building python on windows using a nice cygwin environment 
are here: 

https://github.com/kovidgoyal/calibre/blob/master/setup/installer/windows/notes2.rst

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25627] distutils : file "bdist_rpm.py" does not quote filenames when executing the rpm command

[Martin Panter]

Martin Panter added the comment:

I left some comments on the code review. I think this definitely needs someone 
able to test it; I don’t think I am up to it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25769] Crash due to using weakref referent without acquiring a strong reference

[Armin Rigo]

Armin Rigo added the comment:

PyWeakref_GET_OBJECT() is inherently dangerous: the weakref might go away not 
only the next time the GIL is released, but also the next time the code does 
any seemingly unrelated Py_DECREF() (which might decref the object into 
inexistence) or memory allocation (which might trigger the GC).

I'd suggest adding the comment above in the docs and reviewing all usages 
inside CPython's code base.

--
nosy: +arigo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25627] distutils : file "bdist_rpm.py" does not quote filenames when executing the rpm command

[SilentGhost]

SilentGhost added the comment:

Updated version of the patch including Martin's suggestions.

--
Added file: http://bugs.python.org/file41197/issue25627_5.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25769] Crash due to using weakref referent without acquiring a strong reference

[Armin Rigo]

Armin Rigo added the comment:

Actually, Py_DECREF() may also release the GIL by calling a Python __del__() 
method; it doesn't have to directly decref the exact object.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25759] Python 2.7.11rc1 not building with Visual Studio 2015

[R. David Murray]

R. David Murray added the comment:

Unfortunately it is unlikely we'll merge it, since because of the compiled 
extension issue we have a negative motivation for supporting anything other 
than the release compiler currently used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15500] Python should support exporting thread names to the OS

[R. David Murray]

Changes by R. David Murray :


--
stage:  -> needs patch
title: Python should support naming threads -> Python should support exporting 
thread names to the OS
versions: +Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25759] Python 2.7.11rc1 not building with Visual Studio 2015

[Kovid Goyal]

Kovid Goyal added the comment:

No worries, as I said, I understand, I would probably do the same, were I in 
your shoes. I have found that being a maintainer of a complex software project 
tends to naturally increase conservatism :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25718] itertools.accumulate __reduce__/__setstate__ bug

[Roundup Robot]

Roundup Robot added the comment:

New changeset 095d21df9374 by Serhiy Storchaka in branch '3.4':
Issue #25718: Fixed copying object with state with boolean value is false.
https://hg.python.org/cpython/rev/095d21df9374

New changeset 3d39e1e2dcb3 by Serhiy Storchaka in branch '2.7':
Issue #25718: Fixed copying object with state with boolean value is false.
https://hg.python.org/cpython/rev/3d39e1e2dcb3

New changeset fecb07050aae by Serhiy Storchaka in branch '3.5':
Issue #25718: Fixed copying object with state with boolean value is false.
https://hg.python.org/cpython/rev/fecb07050aae

New changeset 962086677306 by Serhiy Storchaka in branch 'default':
Issue #25718: Fixed copying object with state with boolean value is false.
https://hg.python.org/cpython/rev/962086677306

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25765] Installation error

[Steve Dower]

Steve Dower added the comment:

Hard to tell, but you may just need to reboot. I'd also make sure you've 
installed all Windows Updates that you need, as the one we're trying to install 
should be in there (which means our installer doesn't have to do it).

The error code is from Windows Update (which we sometimes need to run as part 
of the install), and there's so much spam out there these days that it is 
impossible to find out what it means.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25765] Installation error

[Eryk Sun]

Eryk Sun added the comment:

> The error code is from Windows Update (which we sometimes need to 
> run as part of the install), and there's so much spam out there 
> these days that it is impossible to find out what it means.

The error is WU_E_NOT_INITIALIZED [1], "the object could not be initialized" -- 
whatever object it's talking about. 

[1]: https://support.microsoft.com/en-us/kb/938205

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25718] itertools.accumulate __reduce__/__setstate__ bug

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The bug in the copy module is fixed. As for accumulate's __reduce__, I don't 
think we can do something.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

This seems entirely reproducible in a local Xenial sbuild using the source 
packages in https://launchpad.net/~doko/+archive/ubuntu/toolchain/+packages

Interestingly, inspection of the built artifacts in the schroot after all is 
done yields:

# find . -name copy_reg.py? | xargs ls -l
-rw-rw-r-- 1 barry barry 5597 Nov 30 16:52 ./Lib/copy_reg.pyc
-rw-rw-r-- 1 barry barry 5553 Nov 30 16:53 ./Lib/copy_reg.pyo
-rw-rw-r-- 1 barry barry 5091 Nov 30 17:03 
./debian/tmp-dbg/usr/lib/python2.7/copy_reg.pyc
-rw-rw-r-- 1 barry barry 5047 Nov 30 17:03 
./debian/tmp-dbg/usr/lib/python2.7/copy_reg.pyo
-rw-rw-r-- 1 barry barry 5091 Nov 30 17:02 
./debian/tmp-shdbg/usr/lib/python2.7/copy_reg.pyc
-rw-rw-r-- 1 barry barry 5047 Nov 30 17:02 
./debian/tmp-shdbg/usr/lib/python2.7/copy_reg.pyo

Those are debug builds, so it seems reasonable they are different sizes than 
the non-debug artifacts in Lib, right?  And since they are self-consistent in 
size, it seems like it wouldn't be caused by a pyc/pyo write bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

First failure in test run afaict:

==
ERROR: test_recursive_dict_subclass_and_inst 
(test.test_cpickle.cStringIOCPicklerFastTests)
--
Traceback (most recent call last):
  File "/build/python2.7-96oGYh/python2.7-2.7.11~rc1/Lib/test/test_cpickle.py", 
line 175, in wrapper
func(self)
  File "/build/python2.7-96oGYh/python2.7-2.7.11~rc1/Lib/test/pickletester.py", 
line 819, in test_recursive_dict_subclass_and_inst
self.check_recursive_collection_and_inst(MyDict.fromkeys)
  File "/build/python2.7-96oGYh/python2.7-2.7.11~rc1/Lib/test/pickletester.py", 
line 790, in check_recursive_collection_and_inst
s = self.dumps(y, proto)
  File "/build/python2.7-96oGYh/python2.7-2.7.11~rc1/Lib/test/test_cpickle.py", 
line 141, in dumps
p.dump(arg)
AttributeError: 'module' object has no attribute '_reduce_ex'

==

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

But if I just run test_cpickle.py, it succeeds!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you please add debugging output to check that the copy_reg module is 
empty if tests failed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I set a breakpoint where the error occurs.  I'm in test_cpickle.py dumps().  
Doing cPickle.dumps(arg)  in the bp, I get the error.  arg is:

(Pdb) arg
{<__main__.H object at 0x7ff8e2a870d0>: None}
(Pdb) type(arg)


Look again at the AttributeError:

(Pdb) cPickle.dumps(arg)
*** AttributeError: 'module' object has no attribute '_reduce_ex'

First, why is that trying to find the attribute of a module object instead of 
the MyDict instance?  I even tried this just to discount the effects of 
__main__.H:

(Pdb) cPickle.dumps(type(arg)())
*** AttributeError: 'module' object has no attribute '_reduce_ex'

(as you'd expect, type(arg)() is just an instance of a MyDict.

Two questions: why is it trying to get the attribute on a module object, and 
why is the attribute it's trying to get '_reduce_ex' instead of '__reduce_ex__'?

The attribute to get is taken from the __reduce_ex___str in cPickle.c, which 
gets defined as:

#define INIT_STR(S) if (!( S ## _str=PyString_InternFromString(#S)))  return -1;
...
INIT_STR(__reduce_ex__);

This is a compile-time definition, so it makes no sense that it would be trying 
to find the wrong attribute in some cases (e.g. when running the full 
regrtest.py) but not in others (e.g. running just test_cpickle.py).  But I can 
find no other references to a string containing "reduce_ex" so the cPickle 
version has to be it.  And that gets initialized in init_stuff(), called from 
initcPickle() so I don't see any other way for it to get corrupted.

cPickle.dumps({}) works just fine, so it's an artifact of the MyDict.  That's 
all I've figured out for now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Default __reduce_ex__ and __reduce__ implementations call copy_reg._reduce_ex. 
cPickle.dumps({}) works because __reduce_ex__ is not called, dict is supported 
by pickle directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25698] test regressions introduced with the fix for #22995

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

... for protocols < 2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25751] ctypes.util , Shell Injection in find_library()

[Dimitri John Ledkov]

Dimitri John Ledkov added the comment:

Whilst fixed in 3.5, it is not fixed in 2.7. And the reporter believes this is 
a security vulnerability.

Can this be cherry-picked into the 2.7 branch?

--
nosy: +xnox

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25751] ctypes.util , Shell Injection in find_library()

[Martin Panter]

Martin Panter added the comment:

I do not believe 3.5 is fixed either:

$ python3.5
Python 3.5.0 (default, Sep 20 2015, 11:28:25) 
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes.util
>>> ctypes.util.find_library("; echo Hello shell >&2")
Hello shell
>>> 

Issue 22636 has a patch with some review comments, but is is still open. I 
think it needs someone to take another look, perhaps update the patch, and get 
it committed.

I will mark the other issue as a bug fix for 2.7 and 3.4+.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22636] avoid using a shell in ctypes.util: replace os.popen with subprocess

[Martin Panter]

Martin Panter added the comment:

Marking for bug fix in 2.7, requested in Issue 25751.

--
type: enhancement -> behavior
versions: +Python 2.7, Python 3.4, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25770] expose name, args, and kwargs from methodcaller

[Joe Jevnik]

New submission from Joe Jevnik:

This patch adds 3 properties to methodcaller objects for inspecting the object 
at runtime:

1. 'name': the name of the method to call
2. 'args': the position arguments to pass to the method
3. 'keywords': the keyword arguments to pass to the method

args and keywords act like functools.partial (that is why I did not name it 
kwargs).

I noticed that recently the repr changed to expose this information which helps 
in the debugging use case; however, this allows us to use the methodcaller to 
maybe construct a new methodcaller with different args, or call a different 
method with the same args.

--
components: Library (Lib)
files: methodcaller-attrs.patch
keywords: patch
messages: 255636
nosy: ll
priority: normal
severity: normal
status: open
title: expose name, args, and kwargs from methodcaller
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file41198/methodcaller-attrs.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25770] expose name, args, and kwargs from methodcaller

[Emanuel Barry]

Changes by Emanuel Barry :


--
nosy: +ebarry
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25764] PyObject_Call() is called with an exception set in subprocess

[Martin Panter]

Martin Panter added the comment:

This patch fixes the exception when releasing the import lock. I also moved the 
Py_XDECREF(gc_module) back out to where it was to keep the patch to a minimum, 
but I don’t really see a way to simplify it much more.

Apparently Solaris supports posix_spawn(), which doesn’t require double the 
memory like fork(). But implementing that would be a significant project, and 
was rejected in Issue 20104.

--
Added file: http://bugs.python.org/file41199/preexec-fork.2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25708] runpy hides traceback for some exceptions

[Martin Panter]

Martin Panter added the comment:

I think there are a couple of related bugs here, with open reports:

1. Issue 14285: runpy catches various exceptions including AttributeError, and 
incorrectly assumes this means you gave it a bad module or package name.

2. Issue 19771: runpy incorrectly interprets exceptions such as in (1) as 
meaning the package doesn’t have a __main__ submodule. But these exceptions can 
happen even if hello.__main__ exists. My patch there changes the error message 
to:

/home/proj/python/cpython/python: Error while finding spec for 'hello.__main__' 
(AttributeError: module 'random' has no attribute 'dsgjdgj')

3. Issue 16217: runpy should handle exceptions from user code by printing out 
the relevant traceback, rather than letting the Python interpreter print out 
all the internal runpy frames.

Marking this as a dupe of Issue 14285, but I think a proper solution to that 
would probably affect the other two as well.

--
nosy: +martin.panter
resolution:  -> duplicate
status: open -> closed
superseder:  -> Traceback wrong on ImportError while executing a package

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14285] Traceback wrong on ImportError while executing a package

[Martin Panter]

Martin Panter added the comment:

I suspect a proper solution of this bug would also help with Issue 16217 
(provide only the relevant traceback for exceptions from user code).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25767] asyncio documentation section 18.5.2.3.1. (Windows) links to French Wikipedia in English docs

[Zachary Ware]

Zachary Ware added the comment:

Thanks for the report!

--
nosy: +zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25767] asyncio documentation section 18.5.2.3.1. (Windows) links to French Wikipedia in English docs

[Roundup Robot]

Roundup Robot added the comment:

New changeset f475379bf22c by Zachary Ware in branch '3.4':
Issue #25767: Link to English Wikipedia instead of French.
https://hg.python.org/cpython/rev/f475379bf22c

New changeset fee19d2d7713 by Zachary Ware in branch '3.5':
Issue #25767: Merge with 3.4
https://hg.python.org/cpython/rev/fee19d2d7713

New changeset 734247d5d0f9 by Zachary Ware in branch 'default':
Closes #25767: Merge with 3.5
https://hg.python.org/cpython/rev/734247d5d0f9

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25771] importlib: '.submodule' is not a relative name (no leading dot)

[Martin Panter]

New submission from Martin Panter:

>>> import importlib.util
>>> importlib.util.resolve_name(".submodule", None)
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/proj/python/cpython/Lib/importlib/util.py", line 26, in 
resolve_name
'(no leading dot)'.format(name))
ValueError: '.submodule' is not a relative name (no leading dot)

This message sounds like nonsense. Perhaps the it should say something like:

'.submodule' should be an absolute name (no leading dot)

or:

relative import of '.submodule' not allowed outside of a package

--
keywords: easy
messages: 255642
nosy: martin.panter
priority: normal
severity: normal
status: open
title: importlib: '.submodule' is not a relative name (no leading dot)
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com