Change by Inada Naoki :
--
resolution: -> third party
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue37761>
___
___
Inada Naoki added the comment:
I tried without PGO and confirmed performance improved on GCC 7.2.0.
No change on other compiler versions.
$ ./python -m pyperf timeit -s "import json; x = json.dumps({'k': '1' * 2 **
20})" "json.loads(x)"
old: 9211e2
Change by Inada Naoki :
--
pull_requests: +14873
pull_request: https://github.com/python/cpython/pull/15134
___
Python tracker
<https://bugs.python.org/issue37
Inada Naoki added the comment:
And I confirmed performance improvement by my patch (GH-15134) on all of 4
compilers.
$ ./python -m pyperf timeit -s "import json; x = json.dumps({'k': '1' * 2 **
20})" "json.loads(x)"
old: 9211e2
new: 8a758f
opt2: 2
Change by Inada Naoki :
--
components: +IO -Extension Modules
versions: +Python 3.9 -Python 3.8
___
Python tracker
<https://bugs.python.org/issue34488>
___
___
Inada Naoki added the comment:
New changeset 3e41f3cabb661824a1a197116f7f5ead64eb6ced by Inada Naoki (Sergey
Fedoseev) in branch 'master':
bpo-34488: optimize BytesIO.writelines() (GH-8904)
https://github.com/python/cpython/commit/3e41f3cabb661824a1a197116f7f5e
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
New submission from Inada Naoki :
MSVC 2015 supports "z" for size_t format.
I'm not sure about 2013.
AIX support it too. Now "z" is portable enough.
https://mail.python.org/archives/list/python-...@python.org/thread/CAXKWESUIWJNJFLLXXWTQDUWTN3F7KOU/#QVSBYYI
Change by Inada Naoki :
--
versions: +Python 3.9
___
Python tracker
<https://bugs.python.org/issue37781>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Inada Naoki :
--
keywords: +patch
pull_requests: +14890
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/15156
___
Python tracker
<https://bugs.python.org/issu
Inada Naoki added the comment:
I'm conservative about adding public APIs.
But in this case, LIKELY/UNLIKELY macros are battle-tested in the world already.
So I think it's safe to make it public.
But I'll wait to merge for a week for other core
Inada Naoki added the comment:
New changeset 2a570af12ac5e4ac5575a68f8739b31c24d01367 by Inada Naoki in branch
'master':
bpo-37587: optimize json.loads (GH-15134)
https://github.com/python/cpython/commit/2a570af12ac5e4ac5575a68f8739b3
Change by Inada Naoki :
--
resolution: -> fixed
stage: needs patch -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
INADA Naoki added the comment:
You may misunderstood current lookup loop.
In your collisions_count.py, `yield i` must be added right before `while True:`
--
___
Python tracker
<http://bugs.python.org/issue30
INADA Naoki added the comment:
Firstly, do you understand the lookup without perturb?
It's documented here:
https://github.com/python/cpython/blob/258bfc462b1e58689b43f662a10e44ece3a10bef/Objects/dictobject.c#L161-L182
>>> n = 0; L = []
>>> for i in range(16):
... L.app
INADA Naoki added the comment:
It seems not "simplify".
Please try implement it and show benchmark result if you think
it's really worth enough.
--
___
Python tracker
<http://bugs.pyt
New submission from INADA Naoki:
Currently, `make install` doesn't use -jN option of compileall.py
My idea is:
* Use os.cpu_count() for -j0.
* Use the option in Makefile
--
components: Build
messages: 296216
nosy: inada.naoki
priority: normal
severity: normal
status: open
title:
INADA Naoki added the comment:
If this warnings are disabled by default, who enable it?
How about just remove them?
I'm OK to remove them all.
Since it's not ideal, nothing go worse than Python 3.6.
Additionally, if PEP 540 is accepted, we can use UTF-8 for
stdio and filesystem enc
Changes by INADA Naoki :
--
pull_requests: +2315
___
Python tracker
<http://bugs.python.org/issue30686>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +2323
___
Python tracker
<http://bugs.python.org/issue29304>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
pr-2237 only changes lookdict_index() function.
The function is used from only dict_popitem(). So it's not performance
critical part.
And microbench doesn't show significant performance changes:
$ ./python.default -m perf timeit -q -l 2000 -s '
Changes by INADA Naoki :
--
pull_requests: +2387
___
Python tracker
<http://bugs.python.org/issue15216>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 073ae487b3ff9001c69d530c7555ddaa530dee16 by INADA Naoki in branch
'master':
bpo-29304: simplify lookdict_index() function. (GH-2273)
https://github.com/python/cpython/commit/073ae487b3ff9001c69d530c7555dd
Changes by INADA Naoki :
--
pull_requests: +2402
___
Python tracker
<http://bugs.python.org/issue29304>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Antonie:
Thank you for you comment.
Actually speaking, size of instructions are reduced on amd64@gcc.
lookdict_unicode_nodummy and lookdict_split doesn't have any reason to have
duplicated code anymore. I'll do dedupe for them first.
lo
INADA Naoki added the comment:
Tim Peters:
Thanks for your suggestion. But I want to focus on very simple code cleanup
without any change to proving algorithm nor memory access pattern in this issue.
I'll reply about my thought about reducing collision on t
Changes by INADA Naoki :
--
pull_requests: +2455
___
Python tracker
<http://bugs.python.org/issue29304>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Finally, I removed `freeslot` and all lookdict*() functions are much simpler in
GH-2407.
Macro bench doesn't show any significant performance change.
I'll try to write micro benchmarks.
--
___
Python trac
INADA Naoki added the comment:
FYI, issue29585 removes _osx_support dependency from site.py
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue19
INADA Naoki added the comment:
New changeset a8f8d5b4bd30dbe0828550469d98f12d2ebb2ef4 by INADA Naoki in branch
'master':
bpo-29585: optimize site.py startup time (GH-136)
https://github.com/python/cpython/commit/a8f8d5b4bd30dbe0828550469d98f1
Changes by INADA Naoki :
--
pull_requests: +2530
___
Python tracker
<http://bugs.python.org/issue29585>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +2542
___
Python tracker
<http://bugs.python.org/issue29585>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 6b42eb17649bed9615b6e6cecaefdb2f46990b2c by INADA Naoki in branch
'master':
bpo-29585: Fix sysconfig.get_config_var("PYTHONFRAMEWORK") (GH-2483)
https://github.com/python/cpython/commit/6b42eb17649bed9615b6
Changes by INADA Naoki :
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue29585>
___
New submission from INADA Naoki:
>From PEP393, PyUnicode_AsUCS4 is inefficient.
And C implementation of io.StringIO() uses it.
That's why Python 3 is slower than Python 2 on logging_format and
logging_simple benchmarks.
https://mail.python.org/pipermail/speed/2017-February/000503.htm
INADA Naoki added the comment:
I'm sorry, it's my mistake.
I used vmprof on mac and I thought as_ucs4 is bottleneck.
But vmprof on Linux (and perf) shows totally different result.
Maybe, current vmprof doesn't work well for native code on macOS.
--
resolution: -&g
INADA Naoki added the comment:
FYI, https://github.com/python/performance/pull/27 will fix performance
regression.
Python 3 performance is similar to Python 2 after s/warn/warning/
--
___
Python tracker
<http://bugs.python.org/issue30
Changes by INADA Naoki :
--
pull_requests: +2586
___
Python tracker
<http://bugs.python.org/issue30804>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
Sorry, It seems I failed to merge local patch.
--
___
Python tracker
<http://bugs.python.org/issue30804>
___
___
Python-bugs-list m
INADA Naoki added the comment:
#30395 is about forking, but this is about multi threading.
--
___
Python tracker
<http://bugs.python.org/issue28812>
___
___
Pytho
INADA Naoki added the comment:
I like your idea. Would you make pull request?
--
resolution: rejected ->
status: closed -> open
title: Creating namedtuple is too slow to be used in common stdlib (e.g.
functools) -> Optimize namedtuple
INADA Naoki added the comment:
I respect Raymond's rejection. But I want to write down why I like Jelle's
approach.
Currently, functools is the only module which is very popular.
But leaving this means every new namedtuple makes startup time about 0.6ms
slower.
This is also p
INADA Naoki added the comment:
I want to focus on pure Python implementation in this issue.
While "40x faster" is more 10x faster than "4x faster", C implementation
can boost only CPython and makes maintenance more harder.
And sometimes "more 10x faster" is not
INADA Naoki added the comment:
I thought `inplace` option is for debugging/testing without install,
not for installing or packaging.
As far as document [1], `inplace=1` in `setup.cfg` is noticed as:
"which is probably a bad idea for this option, since always building
extensions in-
INADA Naoki added the comment:
I didn't say "let's not do it".
I just want to focus on pure Python implementation at this issue,
because this thread is too long already.
Feel free to open new issue about C implementation.
Even if C implementation is added later, pure Pyt
INADA Naoki added the comment:
Sorry, I'm not packaging expert. Please wait for expert for right direction.
--
___
Python tracker
<http://bugs.python.org/is
Changes by INADA Naoki :
--
pull_requests: +2909
___
Python tracker
<http://bugs.python.org/issue29469>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset ede9084476f88f5a86c7eaaac33cdd938e4cce93 by INADA Naoki (Utkarsh
Upadhyay) in branch 'master':
bpo-31043: fixed test_datetime run twice. (GH-2891)
https://github.com/python/cpython/commit/ede9084476f88f5a86c7eaaac33cdd938e4cce93
-
Changes by INADA Naoki :
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue31043>
___
Changes by INADA Naoki :
--
pull_requests: +2944
___
Python tracker
<http://bugs.python.org/issue30188>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 067931dd95ddfa34297be9c602a796d9f8d9bea3 by INADA Naoki in branch
'master':
bpo-30188: fix TypeError in test_nntplib (GH-2892)
https://github.com/python/cpython/commit/067931dd95ddfa34297be9c602a796d9f8d9bea3
--
nosy: +i
INADA Naoki added the comment:
https://docs.python.org/3.6/library/site.html#site.USER_SITE
> ~/Library/Python/X.Y/lib/python/site-packages for Mac framework builds
So it seems I broke sysconfig.get_path('purelib', 'posix_user').
--
___
INADA Naoki added the comment:
https://github.com/python/cpython/pull/136/files
-if sys.platform == 'darwin':
-from sysconfig import get_config_var
-if get_config_var('PYTHONFRAMEWORK'):
-USER_SITE = get_path('pure
Changes by INADA Naoki :
--
pull_requests: +2980
___
Python tracker
<http://bugs.python.org/issue29585>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset ba9ddb7eea39a651ba7f1ab3eb012e4129c03620 by INADA Naoki in branch
'master':
bpo-29585: fix test fail on macOS Framework build (GH-2928)
https://github.com/python/cpython/commit/ba9ddb7eea39a651ba7f1ab3eb012e
Changes by INADA Naoki :
--
resolution: -> fixed
stage: needs patch -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by INADA Naoki :
--
pull_requests: +3006
___
Python tracker
<http://bugs.python.org/issue29519>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
[gsutil](https://github.com/GoogleCloudPlatform/gsutil) shows massive
"Exception ... ignored" message from this regression.
https://twitter.com/minimum2scp/status/890021408482316289
I think we should backport this fix to Python 2.7.14.
-
INADA Naoki added the comment:
When subclassing, current __repr__ uses `self.__class__.__name__`. So you get
meaningful name already.
When automatic generation, I recommend you to use some wrapper to cache same
namedtuple, since creating namedtuple on the fly is costly job.
I'm a
New submission from INADA Naoki:
like GH-2966, most types with Py_TPFLAGS_HAVE_GC
should call PyObject_GC_UnTrack() at top of the tp_dealloc.
For example, I found lru_cache doesn't call it and I can produce
segmentation fault.
I'll check other types too.
--
components:
INADA Naoki added the comment:
# collection module
dequeiter_dealloc doesn't call Untrack(), but it's safe because it only frees
deque
and deque_dealloc calls Untrack()
dequeiter_dealloc(dequeiterobject *dio)
{
Py_XDECREF(dio->deque);
defdict_dealloc
INADA Naoki added the comment:
> dequeiter_dealloc doesn't call Untrack(), but it's safe because it only frees
> deque
> and deque_dealloc calls Untrack()
It may be not true, while I don't have exploit yet.
--
_
INADA Naoki added the comment:
# _json module
scanner_dealloc()
encoder_dealloc()
# _struct module
unpackiter_dealloc
# _ssl module
context_dealloc()
# Objects/
setiter_dealloc
dictiter_dealloc
dictview_dealloc
# Parser/
ast_dealloc
INADA Naoki added the comment:
> should the base method which calls tp_dealloc do this? Maybe can kill all
> birds with one stone.
It may be possible, but unclear.
Object finalization process is very complicated.
I agree that UnTrack object as soon as refcnt=0, and Track onl
INADA Naoki added the comment:
Docs/extending/newtypes.rst and Docs/include/noddy3.c should be updated too.
But I'm not good English writer. I need help.
--
___
Python tracker
<http://bugs.python.org/is
Changes by INADA Naoki :
--
pull_requests: +3019
___
Python tracker
<http://bugs.python.org/issue31095>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset de34cbe9cdaaf7b85fed86f99c2fd071e1a7b1d2 by INADA Naoki
(Alexander Mohr) in branch 'master':
bpo-31061: fix crash in asyncio speedup module (GH-2966)
https://github.com/python/cpython/commit/de34cbe9cdaaf7b85fed86f99c2fd0
Changes by INADA Naoki :
--
pull_requests: +3023
___
Python tracker
<http://bugs.python.org/issue31061>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset f142e85d22ba135d5205280240f3a2fe1df2649f by INADA Naoki in branch
'3.6':
bpo-31061: fix crash in asyncio speedup module (GH-2984)
https://github.com/python/cpython/commit/f142e85d22ba135d5205280240f3a2
INADA Naoki added the comment:
# microbench
https://gist.github.com/methane/db16224a99d12ad4ed170cf3bd45b819
Slower (2):
- fromkeys_str: 3.85 us +- 0.07 us -> 4.09 us +- 0.09 us: 1.06x slower (+6%)
- fromkeys_int: 4.24 us +- 0.13 us -> 4.48 us +- 0.16 us: 1.06x slower (+6%)
Fas
INADA Naoki added the comment:
New changeset 778928b0c7aa438c282727535814d73df850693a by INADA Naoki in branch
'master':
bpo-29304: Simplify dict lookup functions (GH-2407)
https://github.com/python/cpython/commit/778928b0c7aa438c282727535814d7
Changes by INADA Naoki :
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue29304>
___
INADA Naoki added the comment:
On i386 docker image, pyperformance:
./python -m perf compare_to before.json after.json -G --min-speed=2
Slower (4):
- regex_dna: 279 ms +- 1 ms -> 288 ms +- 3 ms: 1.03x slower (+3%)
- deltablue: 17.9 ms +- 0.3 ms -> 18.3 ms +- 1.0 ms: 1.03x slow
INADA Naoki added the comment:
I think it's false positive of ASAN.
We have dynamically sized block.
https://github.com/python/cpython/blob/3b0f620c1a2a21272a9e2aeca6ca1d1ac10f8162/Objects/dict-common.h#L49-L69
dictobject.c:547 calls memcpy to fill the block and head pointer
is defin
INADA Naoki added the comment:
As Benjamin commented, this is caused by mutating internal dict.
PyType_Lookup() use "method cache", based on "tp_version_tag" in the type
object.
When you modify internal dict directly, namespace is changed without
invalidating tp_version_ta
INADA Naoki added the comment:
But we should check dicts of all parents.
It will has significant penalty, especially for classes having long mro
(inheriting metaclass from typing module cause long mro).
--
___
Python tracker
<http://bugs.python.
INADA Naoki added the comment:
I like idea.
One worrying point is how to deal with dirty dict.
How about do it only when ma_used == keys->dk_nentries?
Slightly off topic. Copy on write can be implemented via dk_refcnt.
Functions just passing `**kwargs` has temporal copy of dict.
And CoW w
INADA Naoki added the comment:
My preference is "Japanese".
After selecting drop down list, we can type "jap" to select Japanese.
On the other hand, we should use IME to input "日本語". It's not so easy as input
alphabets.
--
___
INADA Naoki added the comment:
> Maybe it would help to have a short comment, maybe with a link to this issue,
> on each PyObject_GC_UnTrack().
Is this looks good to you?
/* UnTrack is needed before calling any callbacks (bpo-31095) */
PyObject_GC_UnTrac
INADA Naoki added the comment:
BTW, should this backported to Python 3.5?
--
nosy: +larry
___
Python tracker
<http://bugs.python.org/issue31095>
___
___
Python-bug
New submission from INADA Naoki:
Since dict preserves insertion order, doubly linked list in OrderedDict
can be removed.
There is small performance improvement for odict creation:
$ curl https://api.github.com/orgs/python/repos > repos.json
$ ./py-patched -m perf timeit --compare-to `pwd`
Changes by INADA Naoki :
--
pull_requests: +3232
___
Python tracker
<http://bugs.python.org/issue31265>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
There are some failing tests remaining, and I want to discuss about
some of them here.
Traceback (most recent call last):
File "/home/inada-n/work/python/nodebug/Lib/test/test_ordered_dict.py", line
261, in test_pop
self.assertEqual(m.pop('a&
INADA Naoki added the comment:
I think typical usage is: get, set (incl, creating), and iterate.
* get: there is no difference
* set: inserting new key is little faster since no updating linked list
* iterate: in typical case, new odict is faster because current odict iterator
do lookup for
INADA Naoki added the comment:
od.move_to_end() is slower too:
$ ./py-patched -m perf timeit --compare-to `pwd`/py-default -s 'from
collections import OrderedDict as odict; od =
odict.fromkeys("abcdefghijklmnopqrstuvwxyz")' -- 'od.move_to_end("a"
INADA Naoki added the comment:
Another idea is making "dict preserves insertion order" as language spec.
There are still some difference between dict and odict: .move_to_end() and
__eq__.
But most use cases of odict is just keep insertion order.
For example, parsing config file, js
INADA Naoki added the comment:
When no change happens:
$ ./py-patched -m perf timeit --compare-to `pwd`/py-default -s 'from
collections import Ordedict; od = odict.fromkeys("abcdefghijklmnopqrstuvwxyz")'
--
INADA Naoki added the comment:
New changeset a6296d34a478b4f697ea9db798146195075d496c by INADA Naoki in branch
'master':
bpo-31095: fix potential crash during GC (GH-2974)
https://github.com/python/cpython/commit/a6296d34a478b4f697ea9db7981461
Changes by INADA Naoki :
--
pull_requests: +3234
___
Python tracker
<http://bugs.python.org/issue31095>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +3235
___
Python tracker
<http://bugs.python.org/issue31095>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
pull_requests: +3236
___
Python tracker
<http://bugs.python.org/issue31095>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
I opened backport PR for 3.6, 2.7 and 3.5.
--
versions: +Python 2.7, Python 3.5
___
Python tracker
<http://bugs.python.org/issue31
Changes by INADA Naoki :
--
nosy: +eric.snow
___
Python tracker
<http://bugs.python.org/issue31265>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 2eea952b1b9ebbc2d94fd3faca1536c6b4963725 by INADA Naoki in branch
'3.6':
bpo-31095: fix potential crash during GC (GH-3195)
https://github.com/python/cpython/commit/2eea952b1b9ebbc2d94fd3faca1536
INADA Naoki added the comment:
New changeset 4cde4bdcc86cb08ee3847500a172cc24eba37ffe by INADA Naoki in branch
'2.7':
bpo-31095: Fix potential crash during GC (GH-3197)
https://github.com/python/cpython/commit/4cde4bdcc86cb08ee3847500a172cc
INADA Naoki added the comment:
FYI, my approach is not original, but copied from PyPy.
https://bitbucket.org/pypy/pypy/src/3e52029e9a5a677f7de62ef49f36090465cfbf4c/rpython/rtyper/lltypesystem/rordereddict.py?at=default&fileviewer=file-view-default#rordereddict.py-1437:
INADA Naoki added the comment:
> Hm, indeed, but I see that module 'abc' is in 'sys.modules', probably it is
> then only used by 'collections.abc'/'_collections_abc'. This means that the
> influence of 'ABCMeta' on interpreter start-up
INADA Naoki added the comment:
There is special internal API in dictobject.c
_PyDict_LoadGlobal(PyDictObject *globals, PyDictObject *builtins, PyObject *key)
Maybe, we can have special API like that
_PyDict_LookupMro(PyObject *mro, PyObject *key);
BTW, method cache in _PyType_Lookup is not
INADA Naoki added the comment:
"python_startup_nosite" benchmark imports io module,
and "python_startup" benchmark imports os too.
So no need to additional test for them.
You can see it with `python -v -S -c 'pass&
INADA Naoki added the comment:
Confirmed:
$ ./python-patched -m perf timeit --compare-to `pwd`/python -- 'class C: pass'
python: . 11.9 us +- 0.1 us
python-patched: . 10.3 us +- 0.1 us
Mean +- std dev: [python] 11.9 us +- 0.1 us -> [python-
1101 - 1200 of 3039 matches
Mail list logo