INADA Naoki added the comment:
This is use after free, not overflow.
This patch is based on Python 3.6, but I think 3.5 has same issue.
I'll check it.
--
keywords: +patch
Added file: http://bugs.python.org/file46514/29438-sharedkey-useafterfree.
INADA Naoki added the comment:
(PYTHONMALLOC=malloc valgrind find it soon. :)
--
___
Python tracker
<http://bugs.python.org/issue29438>
___
___
Python-bugs-list m
INADA Naoki added the comment:
At least, ordering of namespace dict and kwargs dict are language spec for 3.6.
This option breaks it. When this option is set, CPython 3.6 is not Python 3.6.
--
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
I can reproduce it on Python 3.5 with attached script.
I think this bug is from Python 3.3, since key-sharing dict is implemented.
"Trigger key sharing dict resize while callbacks (weakref or __del__) called
from setitem" is step to reproduce.
It
INADA Naoki added the comment:
It's similar to issue27945, but different.
I confirmed this issue is in 3.4 too.
https://github.com/python/cpython/blob/3.4/Objects/dictobject.c#L3798
// _PyObjectDict_SetItem()
if ((tp->tp_flags & Py_TPFLAGS_HEAPTYPE) && (cac
INADA Naoki added the comment:
see https://mail.python.org/pipermail/python-dev/2016-September/146348.html
kwargs, __duct__, and namespace passed to metaclass are ordered by language
design.
order of other dicts are implementation detail
Changes by INADA Naoki :
--
keywords: +patch
Added file:
http://bugs.python.org/file46521/29438-sharedkey-useafterfree-py35.patch
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
LGTM
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue29453>
___
___
Python-bugs-list mailing list
Unsub
INADA Naoki added the comment:
I've tried to update ast_opt.c[t] without changing AST.
But I can't find clear way to solve "foo" + "bar" docstring problem.
This patch adds only docstring to AST.
--
Added file: http://bugs.python.org
New submission from INADA Naoki:
spin off of #11549.
http://bugs.python.org/issue11549#msg130955
> b) Docstring is now an attribute of Module, FunctionDef and ClassDef, >
> rather than a first statement. Docstring is a special syntactic
> construction, it's not an execu
New submission from INADA Naoki:
spin off of #11549.
This patch uses code generator to traverse AST.
--
files: ast-constant-folding.patch
keywords: patch
messages: 287186
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: AST-level Constant folding
versions: Python
Changes by INADA Naoki :
--
dependencies: +Change docstring to attribute from first statement.
___
Python tracker
<http://bugs.python.org/issue29469>
___
___
Pytho
INADA Naoki added the comment:
I submit new issues:
* #29463 for AST change (change docstring from first statement to attribute).
* #29469 for constant folding
Note that this issue contains more peephole -> AST optimization changes.
But I want to start these two patch to ease review
Changes by INADA Naoki :
Added file:
http://bugs.python.org/file46556/29438-sharedkey-useafterfree-py36.patch
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
Do you mean
https://github.com/pypa/pip/blob/403e398330c8e841e4633aceda859430f5f7b913/pip/_vendor/distlib/markers.py
?
This doesn't affect, maybe.
Constant folding is executed right before producing bytecode.
It doesn't affect to PyCF_ONLY_AST.
BTW,
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46567/ast-docstring-2.patch
___
Python tracker
<http://bugs.python.org/issue29463>
___
___
Python-bug
INADA Naoki added the comment:
lnotab is changed too.
-0,0,0,115,12,0,0,0,8,4,4,2,8,8,8,12,
-8,25,8,13,114,18,0,0,0,99,0,0,0,0,0,0,
+0,0,0,115,10,0,0,0,12,6,8,8,8,12,8,25,
+8,13,114,18,0,0,0,99,0,0,0,0,0,0,0,0,
115 is header for bytes type.
next 4 bytes is it's length (l
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46569/ast-docstring-3.patch
___
Python tracker
<http://bugs.python.org/issue29463>
___
___
Python-bug
Changes by INADA Naoki :
--
title: Change docstring to attribute from first statement. -> Add `docstring`
attribute to AST nodes
___
Python tracker
<http://bugs.python.org/issu
INADA Naoki added the comment:
I like this idea.
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue29476>
___
___
Python-bugs-list mailin
Changes by INADA Naoki :
Added file:
http://bugs.python.org/file46572/29438-sharedkey-useafterfree-py36-2.patch
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
to: Serhiy
I can reproduce the issue by 29438-minimum.py, on Python 3.7 with pydebug.
But since this issue is "use after free", it may and may not crash.
It's up to how freed memory block is used from another part of Python.
Deterministic test whi
INADA Naoki added the comment:
> Why res == 0 is added? If PyDict_SetItem() triggers recursive calling of
> _PyObjectDict_SetItem() which calls PyDict_SetItem() it may be possible that
> the first PyDict_SetItem() is failed while the dict is changed by the second
> PyDict_S
INADA Naoki added the comment:
Oh, I misunderstood something.
patched Python 3.7 and system's Python 3.5 shows same output for code below.
I'll check what is actually changed.
inada-n@x250 ~/w/p/ast-docstring> cat -n x.py
1 """module docstring""
INADA Naoki added the comment:
I think it violates O(1).
"s.add(x); s.discard(x);" loop creates dummy chain.
Average chain length is determined by size of the set. So it is O(len(s))
rather than amortized O(1).
--
___
Python trac
INADA Naoki added the comment:
This patch affects firstlineno and lnotab of module and class, but not
functions.
module:
- at 0x7f053a8f8b70, file "Lib/importlib/_bootstrap.py",
line 8>
+ at 0x7fdefbf10340, file "Lib/importlib/_bootstrap.py",
line 25>
filename L
INADA Naoki added the comment:
So what's new entry may be:
+* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and
+ ``ClassDef`` AST nodes now have a new ``docstring`` attribute.
+ The first statement in their body is not considered as a docstring anymore.
+ This affects ``co_firstl
INADA Naoki added the comment:
Now I doubt about this patch is really good.
docstring of Module and Class generates two bytecode.
So it's a real, executed statement.
LOAD_CONST 0 ("docstring")
STORE_NAME 0 ("__doc__")
--
_
New submission from INADA Naoki:
PyObject_GetAttrString creates temporary unicode object and intern it.
It totally redundant.
--
files: getattrstring-nointern.patch
keywords: patch
messages: 287386
nosy: inada.naoki
priority: normal
severity: normal
stage: patch review
status: open
INADA Naoki added the comment:
Maybe, we can skip Method object entirely using _PyObject_GetMethod().
Currently it is used only in LOAD_METHOD.
But PyObject_CallMethod(), _PyObject_CallMethodId(),
PyObject_CallMethodObjArgs(), _PyObject_CallMethodIdObjArgs() can use it too
INADA Naoki added the comment:
We'll move to github soon. So no need for caring hgignore.
While there is `.vscode` in gitignore already, I'm not fan of
adding hundreds of IDE specific rule to gitignore.
Why don't you use gitignore_global?
--
nos
INADA Naoki added the comment:
I agree with Berker.
But some git/github tips are very useful for CPython core developers while they
aren't CPython specific.
And some of them are not in beginner's guide.
I think it's worth enough to add link to such tips.
* Use gitignore_globa
INADA Naoki added the comment:
> But PyObject_CallMethod(), _PyObject_CallMethodId(),
> PyObject_CallMethodObjArgs(), _PyObject_CallMethodIdObjArgs() can use it too.
CallMethod[Id]ObjArgs() can use it easily.
But format support is not so easy.
--
Added file: http://bugs.pyth
INADA Naoki added the comment:
callmethod.patch:
+ ../python.default -m perf compare_to default.json patched2.json -G
--min-speed=1
Slower (5):
- logging_silent: 717 ns +- 9 ns -> 737 ns +- 8 ns: 1.03x slower (+3%)
- fannkuch: 1.04 sec +- 0.01 sec -> 1.06 sec +- 0.02 sec: 1.02x slow
INADA Naoki added the comment:
I'm sorry, callmethod.patch is tuned other place, and causing SEGV.
method_fastcall2.patch is tuning same function (call_method() in typeobject.c),
and uses trick to bypass temporary method object (same to
_PyObject_GetMethod()).
$ ./python -m perf t
INADA Naoki added the comment:
method_fastcall3.patch implement the trick in more general way.
(I haven't ran test yet since it's midnight. I'll post result later.)
--
Added file: http://bugs.python.org/file46610/method_fastcall3.patch
___
INADA Naoki added the comment:
method_fastcall4.patch looks clean enough, and performance benefit seems nice.
I don't know current test suite covers unusual special methods.
Maybe, we can extend test_class to cover !unbound (e.g. @classmethod)
INADA Naoki added the comment:
Thanks for finishing my draft patch, Victor.
callmetohd2.patch is same trick for PyObject_CallMethod* APIs in abstract.c.
It fixes segv in callmethod.patch.
And APIs receiving format string can do same trick when format is empty too.
As I grepping
INADA Naoki added the comment:
> I'm not sure about the change on PyObject_CallMethod*() only for empty format
> string.
There are many place using _PyObject_CallMethodId() to call method without args.
Maybe, we should recommend to use _PyObject_CallMethodIdObjArgs() when no
arg
INADA Naoki added the comment:
performance benefit is small.
https://gist.github.com/methane/32fe57cd4aaac1c5c37f75cbbfbe7562
--
Added file: http://bugs.python.org/file46619/callmethod3.patch
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
Yes, I used --enable-optimization this time.
But my patch is not good for branch prediction of my CPU in this time.
I'm willing Object/call.c solves such placement issue.
BTW, since benefit of GetMethod is small, how about this?
* Add _PyMethod_FastCallKey
INADA Naoki added the comment:
Since Python 3.5's key sharing dict allows deletion, py35-2.patch is slightly
different from py36-2.patch.
Since dictresize won't happen in normal (no weakref/__del__ callback) deletion,
I removed `CACHED_KEYS(tp) = NULL` entirely.
--
Added f
INADA Naoki added the comment:
I'll send PR to github. Please continue there.
--
___
Python tracker
<http://bugs.python.org/issue29438>
___
___
Python-bugs-l
Changes by INADA Naoki :
--
pull_requests: +30
___
Python tracker
<http://bugs.python.org/issue29438>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Thanks. I don't familiar with language frontend.
I'll check NEWS entry.
--
title: Add `docstring` attribute to AST nodes -> Add `docstring` field to AST
nodes
___
Python tracker
<http://bugs.pytho
Changes by INADA Naoki :
--
pull_requests: +46
___
Python tracker
<http://bugs.python.org/issue29463>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
patch LGTM.
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue29532>
___
___
Python-bugs-list mailing list
Unsub
INADA Naoki added the comment:
Christian Heimes: Could you create pull request?
https://github.com/tiran/cpython/tree/siphash13 is 404 now.
--
___
Python tracker
<http://bugs.python.org/issue29
New submission from INADA Naoki:
As reading call.c, PyEval_Call* APIs are mostly duplicate of PyObject_Call*
APIs.
While they are not documented, they seems part of limited APIs.
So we can't remove them easily. Let's deprecate them for now.
--
components: Interpreter Cor
Changes by INADA Naoki :
--
pull_requests: +54
___
Python tracker
<http://bugs.python.org/issue29548>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Thanks, Lemburg to pointing it out.
Here is detail of the difference.
## PyEval_CallFunction(), PyEval_CallMethod()
They are very similar to PyObject_CallFunction() and PyObject_CallMethod().
difference are:
* PyEval_Call...() doesn't respect Py_SSIZE_T_
Changes by INADA Naoki :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue29548>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
All pull requests are merged.
https://github.com/python/cpython/pull/17 (master)
https://github.com/python/cpython/pull/39 (3.6)
https://github.com/python/cpython/pull/40 (3.5)
--
resolution: -> fixed
stage: patch review -> resolved
status: open -&g
INADA Naoki added the comment:
PR 87 fixes the regression in 3.6 branch
--
pull_requests: +55
___
Python tracker
<http://bugs.python.org/issue29548>
___
___
Pytho
INADA Naoki added the comment:
> The argument tuple can be NULL for PyEval_CallObject() (it then gets replaced
> with an empty tuple), but will raise a segfault for PyObject_Call().
PyObject_CallObject() accepts NULL as args. Macro vs function is only
difference of them.
On the othe
INADA Naoki added the comment:
> I'm more in favor of modifying PyEval_xxx() to call PyObject_xxx() and
> deprecate them.
That's PR 75 :)
--
___
Python tracker
<http://bugs.py
INADA Naoki added the comment:
> I would suggest to deprecate PyEval_Call*() functions first only in the
> documentation. In 3.8 or 3.9 add the Py_DEPRECATED attribute.
doesn't excluded by PEP 384.
And these 3 functions and 1 macro is outside of "#ifndef Py_LIMITED_API&quo
INADA Naoki added the comment:
I stopped deprecating PyEval_Call APIs, and removing it's usage in PR 75.
Summary of current pull requests:
PR 87 contains fix regression of PyEval_CallObjectWithKeywords for Python 3.6.
PR 75 is for master branch. It contains fix same to PR 87. Additio
Changes by INADA Naoki :
--
pull_requests: +68
___
Python tracker
<http://bugs.python.org/issue29529>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Sorry, I'm not unicode expert.
Important usage of textwrap is printing in terminal.
So I think we should learn from software relating terminal.
tmux uses utf8proc. utf8proc calculates display width by script at here.
https://github.com/JuliaLang/utf8proc
INADA Naoki added the comment:
FYI, I had implemented textwrap respects EAW in Bazaar project.
See here.
http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/revision/5874
--
___
Python tracker
<http://bugs.python.org/issue24
INADA Naoki added the comment:
I've stopped to deprecating. changed issue title.
--
title: deprecate PyEval_Call*() functions. -> Recommend PyObject_Call* APIs
over PyEval_Call*() APIs
___
Python tracker
<http://bugs.python.org
INADA Naoki added the comment:
I think PyObject_Call* APIs should be preferred.
As discussed in #29548, we can't remove or deprecate PyEval_Call*.
But I don't think it's worth enough to document them.
How about keep them undocumented?
--
nos
INADA Naoki added the comment:
In PR 75, I added a comment in Include/ceval.h, right before PyEval_Call* APIs.
https://github.com/python/cpython/pull/75/files#diff-da3df9def5eca3595399b0a5a7698eac
--
___
Python tracker
<http://bugs.python.
Changes by INADA Naoki :
--
pull_requests: +82
___
Python tracker
<http://bugs.python.org/issue29548>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 72dccde884d89586b0cafd990675b7e21720a81f by GitHub in branch
'master':
bpo-29548: Fix some inefficient call API usage (GH-97)
https://github.com/python/cpython/commit/72dccde884d89586b0cafd990675b7
INADA Naoki added the comment:
Would you send pull request on Github?
--
___
Python tracker
<http://bugs.python.org/issue26787>
___
___
Python-bugs-list mailin
INADA Naoki added the comment:
LGTM, but would you send pull request on Github?
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue26
New submission from INADA Naoki:
site.py uses sysconfig (and sysconfigdata, _osx_support) module for user-site
package.
But sysconfig module is not so lightweight, and very rarely used.
Actually speaking, only tests and distutils uses sysconfig in stdlibs.
And it takes about 7% of startup
INADA Naoki added the comment:
Christian: I'm using macOS on office and Linux on home.
sysconfig is imported even on Linux
https://github.com/python/cpython/blob/master/Lib/site.py#L247-L248
https://github.com/python/cpython/blob/master/Lib/site.py#L263
INADA Naoki added the comment:
FYI, here is profile of site:
https://gist.github.com/methane/1f1fe4385dad84f03eb429359f0f917b
--
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
no-site
python_startup_no_site: Median +- std dev: 9.13 ms +- 0.02 ms
default:
python_startup: Median +- std dev: 15.6 ms +- 0.0 ms
GH-136 + skip abs_paths().
python_startup: Median +- std dev: 14.2 ms +- 0.0 ms
profile of GH-136 + skip abs_paths():
https
New submission from INADA Naoki:
abs_paths() is the another most slow parts of site module.
We can speedup it if we have C implementation of os.path.abspath() or
abs_paths().
But before trying it, is abs_paths() really needed for now?
It only rewrite `__file__` and `__cached__` of modules
INADA Naoki added the comment:
I create #29592 for abs_paths(). Let's focus on sysconfig in this issue.
PR 136 ports really needed part of sysconfig into site.py already.
'PYTHONFRAMEWORK' on macOS is the only variable we need import from sysconfig.
Adding `site.cfg` like `p
INADA Naoki added the comment:
path normalization was added by
https://github.com/python/cpython/commit/38cb9f1f174415d3b37fbaeb5d152d65525839d2
--
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
https://github.com/python/cpython/blob/master/Lib/importlib/_bootstrap_external.py#L1089-L1095
While '' in sys.path, it is converted to `getcwd()` before calling PathHook.
Is there any chance relative / not normalized path is in sys.path before site
INADA Naoki added the comment:
$ python2 -S
Python 2.7.12+ (default, Sep 17 2016, 12:08:02)
[GCC 6.2.0 20160914] on linux2
>>> import x
>>> x.__file__
'x.py'
$ python3 -S
Python 3.6.0 (default, Dec 30 2016, 20:49:54)
[GCC 6.2.0 20161005] on linux
>>> i
INADA Naoki added the comment:
PR 136 now adds `sys._framework` and 'PYTHONFRAMEWORK' macro in pyconfig.h.
--
___
Python tracker
<http://bugs.python.o
INADA Naoki added the comment:
I can reproduce it with Sphinx 1.5.2.
Additonally, "None" is shown at top.
--
nosy: +inada.naoki
Added file: http://bugs.python.org/file46648/sphinx-1.5-pydoc.png
___
Python tracker
<http://bugs.python.o
INADA Naoki added the comment:
ref: https://github.com/sphinx-doc/sphinx/issues/2986
--
___
Python tracker
<http://bugs.python.org/issue29520>
___
___
Python-bug
Changes by INADA Naoki :
--
pull_requests: +128
___
Python tracker
<http://bugs.python.org/issue29520>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Could docs.python.org use new Sphinx, after fix this issue?
Travis checks doc with Sphinx 1.5.2, but docs.python.org seems using 1.3.3.
It's too old.
--
___
Python tracker
<http://bugs.python.org/is
Changes by INADA Naoki :
--
pull_requests: +129
___
Python tracker
<http://bugs.python.org/issue29529>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +131
___
Python tracker
<http://bugs.python.org/issue29592>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
I got it.
removeduppaths() may change relpath in sys.path to absolute path.
abs_paths() changes __file__ and __cached__ for consistency with the changed
sys.path.
I updated PR 167 to call abs_paths() only if removeduppaths() modified sys.path.
Strictly speaking
Changes by INADA Naoki :
--
pull_requests: +144
___
Python tracker
<http://bugs.python.org/issue29529>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 3eea8c67fa870c6e2b7a521d292afe7fe3e95f58 by GitHub in branch
'master':
bpo-29520: doc: fix deprecation warning from 'defindex' template (GH-165)
https://github.com/python/cpython/commit/3eea8c67fa870c6e2b7a
Changes by INADA Naoki :
--
pull_requests: +145
___
Python tracker
<http://bugs.python.org/issue29520>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +146
___
Python tracker
<http://bugs.python.org/issue29520>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +147
___
Python tracker
<http://bugs.python.org/issue29520>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +151
___
Python tracker
<http://bugs.python.org/issue29520>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 98604c7683f41f04c633935bb582399c50db838c by GitHub in branch
'2.7':
bpo-29529: Add .travis.yml to 2.7 branch (GH-27)
https://github.com/python/cpython/commit/98604c7683f41f04c633935bb58239
INADA Naoki added the comment:
New changeset cf44d957ec177be62f5349ef88515190dcfccbd6 by GitHub in branch
'2.7':
bpo-29520: doc: fix deprecation warning from 'defindex' template (GH-180)
https://github.com/python/cpython/commit/cf44d957ec177be62f53
INADA Naoki added the comment:
New changeset f0174c69b7b8bd27ee32d96e890d665da29472af by GitHub in branch
'3.5':
bpo-29520: doc: fix deprecation warning from 'defindex' template (GH-179)
https://github.com/python/cpython/commit/f0174c69b7b8bd27ee32
INADA Naoki added the comment:
New changeset 7970cd483346dfd7723da214fb27399ecc574095 by GitHub in branch
'3.6':
bpo-29520: doc: fix deprecation warning from 'defindex' template (GH-178)
https://github.com/python/cpython/commit/7970cd483346dfd7723d
INADA Naoki added the comment:
New changeset e395c4dbe19115aaab315c2a113b172e9fef307a by GitHub in branch
'master':
bpo-29520: doc: add missing dot (GH-182)
https://github.com/python/cpython/commit/e395c4dbe19115aaab315c2a113b17
Changes by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by INADA Naoki :
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue29529>
___
INADA Naoki added the comment:
3.6.0 had been released?
Yury, would you create pull request?
--
___
Python tracker
<http://bugs.python.org/issue28963>
___
___
Changes by INADA Naoki :
--
pull_requests: +161
___
Python tracker
<http://bugs.python.org/issue24274>
___
___
Python-bugs-list mailing list
Unsubscribe:
2601 - 2700 of 3039 matches
Mail list logo
