INADA Naoki added the comment:
Uhhh! I'm sorry.
--
___
Python tracker
<http://bugs.python.org/issue20180>
___
___
Python-bugs-list mailing list
Unsubscr
INADA Naoki added the comment:
Additionally, there are not enough METH_FASTCALL CFunctions.
I'm interested in performance difference between METH_VARARGS and
METH_FASTCALL. If METH_FASTCALL is significant faster than VARARGS,
we can convert some common methods from VARARGS to FASTCAL
INADA Naoki added the comment:
Parsing positional arguments for METH_KEYWORDS and METH_FASTCALL can be faster
by
http://bugs.python.org/issue29029
--
___
Python tracker
<http://bugs.python.org/issue29
INADA Naoki added the comment:
This patch makes AC produces more FASTCALL instead of VARARGS.
When looking AC output, one downside is it produces many consts like:
+static const char * const _keywords[] = {"", "", "", "", "", NULL};
-
INADA Naoki added the comment:
LGTM, except pop().
--
___
Python tracker
<http://bugs.python.org/issue29289>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
LGTM again.
--
___
Python tracker
<http://bugs.python.org/issue29029>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from INADA Naoki:
PyDict_GetItemString() is heavily used, especially from keyword argument
parsing.
Current implementation creates temporary string for key object.
This patch avoid the temporary key string when passed C string is ASCII.
This benchmark is based on a8563ef0eb8a
New submission from INADA Naoki:
Median +- std dev: [default] 24.2 ms +- 0.4 ms -> [patched] 19.2 ms +- 0.4 ms:
1.26x faster (-21%)
--
files: print-fastcall.patch
keywords: patch
messages: 285632
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: conv
INADA Naoki added the comment:
This patch checks passed C string is ascii or not.
But I don't want make dict complex too. telco is more faster with issue29296.
Most common builtin functions are not METH_KEYWORDS when it merged.
--
___
P
INADA Naoki added the comment:
Close this issue for now, until profiler shows me PyDict_XString.
--
resolution: -> rejected
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
INADA Naoki added the comment:
I see. AC should support this.
--
dependencies: +Argument Clinic should understand *args and **kwargs parameters
resolution: -> later
status: open -> pending
___
Python tracker
<http://bugs.python.org/i
Changes by INADA Naoki :
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue20291>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
> Maybe it's the expensive price of creating a temporary dictionary?
Please see the patch on issue29295.
PyDict_GetItemString() is bad part, too.
It's create str object, calc hash, lookup dict, and dealloc t
New submission from INADA Naoki:
All lookdict* functions are implemented like pseudo language:
```
lookup()
if not collision:
return result
while True:
perturb_shift()
lookup()
if not collision:
return result
```
This patch changes it as:
```
while True:
lookup
INADA Naoki added the comment:
Raymond, I understand your worries. I won't commit this until I do more
precise survey.
But why I trying this is not only I find duplicated code.
I think benefit of this technique is reduced by historical reason.
In Python 2.7, there are only two l
Changes by INADA Naoki :
Removed file: http://bugs.python.org/file46324/dictlook-refactoring.patch
___
Python tracker
<http://bugs.python.org/issue29304>
___
___
Pytho
INADA Naoki added the comment:
I've attached wrong file. This patch is first version I wanted to post.
It dedupe all five functions.
--
Added file: http://bugs.python.org/file46325/dict-refactor.patch
___
Python tracker
<http://bugs.py
INADA Naoki added the comment:
@haypo, Rietveld didn't accept your patch. (I don't know what format Rietveld
accepts...)
Is PR 65 is same to the third patch?
--
___
Python tracker
<http://bugs.python.o
INADA Naoki added the comment:
I commented at the pull request.
--
___
Python tracker
<http://bugs.python.org/issue29259>
___
___
Python-bugs-list mailin
INADA Naoki added the comment:
I think basic idea is "wrap unknown function call at least once, and preferably
only once".
_PyEval_EvalFrameDefault() calls Py_EnterRecursiveCall("").
So wrapping PyFunction_(Fast)Call* with Py_EnterRecursiveCall() seems redundant
INADA Naoki added the comment:
LGTM
--
___
Python tracker
<http://bugs.python.org/issue29311>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.pyth
INADA Naoki added the comment:
Using FASTCALL for methods accepting **kwargs can't skip creating dict in most
cases. Because they accepts dict.
Even worth, when calling it like `d.update(**config)` (yes, it's abuse of
**, but it can be happen in some C methods), FASTCALL may unpack
INADA Naoki added the comment:
Since mutating kw dict shouldn't affect caller's dict, caller and callee can't
share the dict.
One possible optimization is using PyDict_Copy() to copy the dict.
It can reduce number of hash() calls.
--
___
INADA Naoki added the comment:
> Can we close the issue, or do you prefer to keep it open?
I don't care.
--
___
Python tracker
<http://bugs.python.org
New submission from INADA Naoki:
Tuples consists of None, True, False, str and bytes can be merged safely.
To avoid memory leak, this patch shares such tuples in compile unit (module, in
most case.)
--
files: merge-tuples.patch
keywords: patch
messages: 285933
nosy: haypo, inada.naoki
INADA Naoki added the comment:
I tried this patch with attached script.
```
$ venv/bin/pip install django flask sqlalchemy
$ PYTHONTRACEMALLOC=5 venv/bin/python3 tuplemem.py > tuples.txt
$ sort tuples.txt | uniq -c | sort -nr > tuplecount
```
## default
memory: (32254693, 32292635)
INADA Naoki added the comment:
Oh, thanks. I hadn't checked the warning.
Since bytes are not important in this time, I'll remove bytes support.
--
___
Python tracker
<http://bugs.python.o
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46368/mergetuples2.patch
___
Python tracker
<http://bugs.python.org/issue29336>
___
___
Python-bugs-list m
INADA Naoki added the comment:
TL;DR
Changing positional argument name doesn't break backward compatibility.
After start accepting keyword argument, it's name is part of API and should be
stable.
For example, document says `str.startswith(prefix[, start[, end]])`.
But this patch s
INADA Naoki added the comment:
@ProgVal, could you try following?
I doubt that this difference just shows rlimit behavior, not memory usage.
---
$ cat rlimit_difference_linux_versions.py
import sys
import queue
import multiprocessing
from urllib.request import Request, urlopen
import resource
INADA Naoki added the comment:
On Linux 3.16? 4.7? What is difference between 3.16 and 4.7?
Again, I doubt Python's memory consumption increased by Linux version.
Isn't rlimit more strict for now?
Even if memory usage is really grow, I don't think it's a Python's
INADA Naoki added the comment:
While patch looks safe, some developer may dislike such a large patch
without fixing real issue.
Anyway, Coccinelle seems very interesting tool for refactoring.
--
___
Python tracker
<http://bugs.python.org/issue28
INADA Naoki added the comment:
I don't think this is a regression, bug, bogos naming.
I agree that "_0" is ugly.
But the name describes what is the thread, than "Thread-1".
How about giving default thread_name_prefix less ugly?
e.g. "ThreadPoolExecutor-worker&qu
INADA Naoki added the comment:
LGTM.
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue29314>
___
___
Python-bugs-list mailing list
Unsub
Changes by INADA Naoki :
--
stage: needs patch -> commit review
___
Python tracker
<http://bugs.python.org/issue29314>
___
___
Python-bugs-list mailing list
Un
INADA Naoki added the comment:
Oh, I feel three LGTMs are positive signal.
As I commented on ML, I think "ask forgiveness than permission" is
realistic approach for patches like this.
But it's up to you.
--
___
Python tracker
<http
INADA Naoki added the comment:
"iff" is short form of "if and only if".
--
nosy: +inada.naoki
resolution: -> not a bug
status: open -> closed
___
Python tracker
<http
INADA Naoki added the comment:
LGTM, except 2-space indent.
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue28876>
___
___
Python-bugs-list m
INADA Naoki added the comment:
OK. I'll wait for another opinions.
--
___
Python tracker
<http://bugs.python.org/issue29212>
___
___
Python-bugs-list m
INADA Naoki added the comment:
Thanks. Your patch reduced memory consumption by 2%,
and number of tuples by 15%.
$ cat invtuple.py
import app
import sys
import traceback
import tracemalloc
print(tracemalloc.get_traced_memory())
allobj = sys.getobjects(0, tuple)
print(len(allobj
INADA Naoki added the comment:
LGTM
--
___
Python tracker
<http://bugs.python.org/issue29337>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.pyth
INADA Naoki added the comment:
merge-constants.patch LGTM
--
___
Python tracker
<http://bugs.python.org/issue29336>
___
___
Python-bugs-list mailing list
Unsub
INADA Naoki added the comment:
patch LGTM.
Python 2 supported `dict(**{1:3})`. But I don't know there are any functions
supporting non-string keyword argument.
> PyPy (and other Python implementations?) don't support non-string in type
> dictionary.
Wow! I thought it's
INADA Naoki added the comment:
That's why I want to enable only --with-lto
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue29243>
___
___
INADA Naoki added the comment:
> More generally, would it be possible to share co_consts (None,) tuples
> between code objects of two different modules? Or is it already the case with
> merge-constants.patch?
No. I didn't do it because I don't know it's worth enough.
New submission from INADA Naoki:
While I'm looking Python's memory usage, I found some dicts which key
is not interned "_fields", "_ast" and "__modules__".
--
components: Interpreter Core
files: ast-identifier.patch
keywords: patch
messages: 2
INADA Naoki added the comment:
As I said on ML, now I think embedding some co_* tuples into code object
is better way to reduce number of tuples and memory usage.
So I close this issue for now.
Thanks for review and comments.
--
resolution: -> postponed
status: open -> p
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46414/ast-identifier2.patch
___
Python tracker
<http://bugs.python.org/issue29369>
___
___
Python-bug
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46415/ast-identifier3.patch
___
Python tracker
<http://bugs.python.org/issue29369>
___
___
Python-bug
Changes by INADA Naoki :
Removed file: http://bugs.python.org/file46414/ast-identifier2.patch
___
Python tracker
<http://bugs.python.org/issue29369>
___
___
Python-bug
INADA Naoki added the comment:
> AST objects are supposed to be temporary. Interning strings used in the AST
> processor would make these strings immortal and so increase the memory usage,
> no?
> What is the purpose of the patch? Speedup or reduce the memory usage?
The three ide
Changes by INADA Naoki :
--
resolution: -> fixed
___
Python tracker
<http://bugs.python.org/issue29369>
___
___
Python-bugs-list mailing list
Unsubscrib
INADA Naoki added the comment:
> You should check if _PyUnicode_FromId() returns NULL if it was the first call
> and the UTF-8 decode failed to allocate memory.
thanks. new patch will fix it.
> You might initialize all these identifiers (and check for errors) in
> init_type
INADA Naoki added the comment:
Thanks, Serhiy. You're right.
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue29369>
___
___
Py
INADA Naoki added the comment:
Another run at my machine. (tp_fastcall-6.patch)
+ ../python.default -m perf compare_to default.json patched.json -G
--min-speed=2
Slower (8):
- logging_silent: 736 ns +- 7 ns -> 783 ns +- 12 ns: 1.06x slower (+6%)
- unpickle_pure_python: 829 us +- 13 us -&g
INADA Naoki added the comment:
While I feel your work is great, performance benefit seems very small,
compared complexity of this patch.
I think AST level optimize is more important. co_const and co_stacksize
can be more small if constant folding is done in AST.
And I think global cache
INADA Naoki added the comment:
Digging history, duplicated code is introduced here. (1997-01-17)
https://github.com/python/cpython/commit/99304174680d4c724476dad300ae7fc638842bf0#diff-2131209d0deb0e50c93a88ec6c7b0d52
/* Optimizations based on observations by Jyrki Alakuijala
INADA Naoki added the comment:
BTW, perturb shift uses (i << 2) + i, instead of i*5.
I feel it doesn't make sense in 21th century. I'll change it too.
--
___
Python tracker
<http://bugs.py
INADA Naoki added the comment:
> I think the patch should not be pushed without such analysis. Perhaps
> Raymond will found a time to do this.
Ok, I won't push until expert's LGTM.
> One possible optimization is removing freeslot completely. because:
>
> * f
INADA Naoki added the comment:
dict-refactoring-3.patch:
$ ../python.default -m perf compare_to default.json patched2.json -G
--min-speed=2
Slower (7):
- scimark_lu: 422 ms +- 35 ms -> 442 ms +- 11 ms: 1.05x slower (+5%)
- logging_silent: 736 ns +- 7 ns -> 761 ns +- 21 ns: 1.03x slow
Changes by INADA Naoki :
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue1346238>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by INADA Naoki :
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue11549>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from INADA Naoki:
add_methods(), add_members(), and add_getset() creates PyUnicode from C string
3 times, and calls PyUnicode_InternInplace 2 times.
1. PyDict_GetItemString() at first. (PyUnicode_FromString() is called).
2. In middle, descr_new() calls PyUnicode_InternFromString
INADA Naoki added the comment:
I think I found better way.
Interned string can be get from descripter. Interning can be reduced
without adding private API.
Please don't review the first patch.
--
___
Python tracker
<http://bugs.py
INADA Naoki added the comment:
descr-remove-getitemstring-2.patch is more compact than first patch.
(3 unicode + 2 intern) becomes (2 unicode + 1 intern).
python_startup_no_site: Median +- std dev: 12.5 ms +- 0.1 ms
--
Added file: http://bugs.python.org/file46443/descr-remove
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46444/dict-setitemstring.patch
___
Python tracker
<http://bugs.python.org/issue29383>
___
___
Python-bug
Changes by INADA Naoki :
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue29383>
___
___
Python-bugs-list
INADA Naoki added the comment:
I can't reproduce it because of some reason.
1. While you report it affects Python 3.3~3.7, the script doesn't work on
Python 3.7
2. I've fixed webserver.py to work on Python 3, but it uses very old (draft)
websocket protocol. Current brows
INADA Naoki added the comment:
patch looks OK.
But I prefer `__deepcopy__ = __copy__ = copy`.
I don't know how to support pickling. (should unpickled object
reference os.environ, or copied dict?)
I feel it is different issue.
--
nosy: +inada.
INADA Naoki added the comment:
@haypo, how do you think about ast.Lit and ast.Constant?
Is this patch updated to use ast.Constant?
Or ast.Constant should be used only for some transform like constant folding?
--
versions: +Python 3.7 -Python 3.6
INADA Naoki added the comment:
I agree.
os.environ is not dict, it's a proxy of "current" environment which
is not copyable.
So I agree to copy.copy() shouldn't copy os.environ implicitly.
--
___
Python tracker
<http://bug
INADA Naoki added the comment:
> 1. Changes to AST
I'm working on updating this part. There are some failing tests remains.
But I doubt this stage is worth enough for now.
> a) Merge Num, Str, Bytes and Ellipsis constructors into a single Lit
> (literal) that can hold any
INADA Naoki added the comment:
>> We have already Constant and NameConstant. So it seems there are no need for
>> None, Bool, TupleConst, SetConst nodes.
> Yes, Constant is Victor's version of Lit.
Then, may I remove ast.Lit, and use Constant and NameConstant?
>> I
New submission from INADA Naoki:
Rust and Ruby moved from SipHash-2-4 to SipHash-1-3.
https://github.com/rust-lang/rust/issues/29754
https://bugs.ruby-lang.org/issues/13017
SipHash-1-3 seems faster than 2-4 and secure enough.
--
messages: 286590
nosy: inada.naoki
priority: normal
INADA Naoki added the comment:
Before trying advanced optimizations, I want move suspended obvious
optimizations forwards.
For example, removing unused constants is suspended because constant folding
should be moved from peephole to AST. This is why I found this issue.
After that, I&#
Changes by INADA Naoki :
--
components: +Interpreter Core
type: -> performance
___
Python tracker
<http://bugs.python.org/issue29410>
___
___
Python-bugs-lis
INADA Naoki added the comment:
I'm running pyperformance.
BTW, hashlib doesn't provide siphash24. So can we simply replace siphash24
with siphash13, like ruby?
https://github.com/ruby/ruby/commit/04c94f95d1a1c6a12f5412228a2bcd
INADA Naoki added the comment:
$ ../python.default -m perf compare_to default.json patched4.json -G
Slower (2):
- scimark_sor: 479 ms +- 8 ms -> 485 ms +- 9 ms: 1.01x slower (+1%)
- genshi_xml: 196 ms +- 2 ms -> 196 ms +- 2 ms: 1.00x slower (+0%)
Faster (19):
- json_loads: 62.7 us +-
INADA Naoki added the comment:
> PEP 456 defines an API to add more hashing algorithms and make the selection
> of hash algorithm a compile time option. We can easily add SipHash-1-3 and
> make it the default algorithm. Vendors then can select between FNV2,
> SipHash-1-3 and SipH
INADA Naoki added the comment:
> it seems as if it would make sense to not use a fixed
> hash algorithm for all strings lengths, but instead a
> hybrid one to increase performance for short strings
> (which are used a lot in Python).
>
> Is there a good hash algorithm wi
INADA Naoki added the comment:
Current Py_HASH_CUTOFF implementation seems weak.
```
switch(len) {
/* ((hash << 5) + hash) + *p == hash * 33 + *p */
case 7: hash = ((hash << 5) + hash) + *p++; /* fallthrough */
...
case 1: hash = ((hash
INADA Naoki added the comment:
Py_HASH_CUTOFF uses different secret from siphash already.
The problem is the secret doesn't affects to collision at all.
Attacker can produce large number of collision, without knowing the secret.
BTW, we have FNV already. Let's survey about FNV-1 sh
INADA Naoki added the comment:
https://emboss.github.io/blog/2012/12/14/breaking-murmur-hash-flooding-dos-reloaded/
Murmur may be safe for <16 byte too.
--
___
Python tracker
<http://bugs.python.org/issu
INADA Naoki added the comment:
My motivation is improve speed, reduce memory usage, and quicker
startup time for real world applications. If some optimization in
FAT optimizer has significant speedup, I want to try it.
But this time, my motivation is I felt "everyone think constant fo
INADA Naoki added the comment:
OK, let's forget Py_HASH_CUTOFF for now and focus on SipHash-13.
--
___
Python tracker
<http://bugs.python.org/issue29410>
___
___
INADA Naoki added the comment:
LOAD_METHOD support was based on tp_fastcall.
Without LOAD_METHOD support, methoddescr_get is called and
stack layout after LOAD_METHOD is:
NULL, (bound)PyCFunction, arg1, arg2, ... argN
With LOAD_METHOD support, stack layout is:
PyMethodDescrObject, self
INADA Naoki added the comment:
I'm running pyperformance.
--
Added file: http://bugs.python.org/file46487/loadmethod-methoddescr.patch
___
Python tracker
<http://bugs.python.org/is
INADA Naoki added the comment:
Here is the result. I'll investigate mako result.
+ ../python.default -m perf compare_to default.json patched.json -G
--min-speed=1
Slower (20):
- mako: 40.9 ms +- 0.4 ms -> 44.7 ms +- 0.6 ms: 1.09x slower (+9%)
- chaos: 298 ms +- 3 ms -> 308 ms +-
INADA Naoki added the comment:
microbench:
$ ./python.patched -m perf timeit --compare-to `pwd`/python.default -s "d =
b''" -- "d.decode('ascii')"
python.default: . 109 ns +- 1 ns
python.patched: . 102 ns +- 1 ns
INADA Naoki added the comment:
I confirmed bm_mako performance degrade is caused by L1 cache miss.
(I didn't use --with-optimization)
https://gist.github.com/methane/b33edbf1f123ae026e704b0e005c3606
--
___
Python tracker
<http://bugs.py
Changes by INADA Naoki :
Added file: http://bugs.python.org/file46493/loadmethod-methoddescr-2.patch
___
Python tracker
<http://bugs.python.org/issue29263>
___
___
Pytho
Changes by INADA Naoki :
--
dependencies: -Add tp_fastcall to PyTypeObject: support FASTCALL calling
convention for all callable objects
resolution: -> fixed
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bug
INADA Naoki added the comment:
Why don't you use OrderdDict and reversed()?
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.org/issue29431>
___
___
INADA Naoki added the comment:
I can't understand what is the problem.
If the package produce same binary when dict keeps insertion order,
isn't it a "reproducible build"?
--
___
Python tracker
<http://bug
INADA Naoki added the comment:
For checking compatibility with other implementation, I want to wait
until other implementation compatible with 3.6+ which doesn't
keep insertion order of dict.
For now, there are no 3.6+ compatible Python implementation except CPython.
For checking compatib
INADA Naoki added the comment:
Here is issue tracker of Python.
py2exe and requests are third party project.
Please report an issue to py2exe issue tracker.
--
nosy: +inada.naoki
resolution: -> third party
stage: -> resolved
status: open -&g
New submission from INADA Naoki:
sys.getobjects() contains tuples including NULLs.
Even without pydebug, tuple under construction can be exposed accidentally.
Allowing repr() for such tuples ease investigating what is the tuple.
--
components: Interpreter Core
files: tuple-repr
INADA Naoki added the comment:
The patch missed `continue;`.
And I couldn't reproduce the problem for now.
I'll reopen when I can reproduce the problem.
--
resolution: -> rejected
stage: -> resolved
status: open -> closed
___
Py
INADA Naoki added the comment:
valgrind output is here.
https://gist.github.com/methane/3c010daba71a374fd0b6a41a0d98e3ff
It seems another bug relating to key-sharing dict...
--
nosy: +inada.naoki
___
Python tracker
<http://bugs.python.
INADA Naoki added the comment:
4385 int was_shared = cached == ((PyDictObject *)dict)->ma_keys;
4386 res = PyDict_SetItem(dict, key, value);
4387 if (was_shared && cached != ((PyDictObject *)dict)->ma_keys)
2501 - 2600 of 3039 matches
Mail list logo
