INADA Naoki added the comment:
ref #32911
And it's documented already.
https://docs.python.org/3.7/whatsnew/3.7.html#changes-in-the-python-api
--
nosy: +inada.naoki
___
Python tracker
<https://bugs.python.org/is
INADA Naoki added the comment:
Nice idea!
I think it helps not only sharing more memory for forking application,
but also long running large application.
There are many static objects which is tracked by GC.
It makes full GC time long. And CPU cache is filled by unused data.
For example, web
INADA Naoki added the comment:
AFAIK, Python shutdown process calls full GC.
Don't touching permanent generation makes shutdown faster.
On the other hand, there are some downside:
* Some object may be not freed while shutdown. It looks like "leak" for
application embedding Pyt
INADA Naoki added the comment:
> If this proposition be accepted I would want to see not only cumulated times,
> but also pure times, without time of nested imports.
It can be calculated by script.
I don't want to add more complexity to import.c.
> I guess this feature doesn&#x
INADA Naoki added the comment:
> Actually it is easy. You need just one global integer accumulator and a local
> variable for keeping a temporary copy of it.
You're right! I updated my pull request.
See current output:
https://gist.github.com/methane/185d75a3c8da762d85317
INADA Naoki added the comment:
I don't have any good idea for it.
I usually run `python -X importtime -c 'import functools'` several times to
ensure pyc files are created and it is cached by OS.
On Linux, `strace -c` and `time` command can be used.
But I don't know cross p
INADA Naoki added the comment:
ping?
May I close this issue and pull request?
--
___
Python tracker
<https://bugs.python.org/issue31475>
___
___
Python-bugs-list m
INADA Naoki added the comment:
Oh, the pull request is far larger than I thought!
I doubt that avoiding functools and collections is worth enough, because it is
very common.
For example:
* functools is very commonly imported, especially for wraps().
* collections is imported by functools
INADA Naoki added the comment:
how can I make it more machine readable?
On 2017年9月25日(月) 19:39 Christian Heimes wrote:
>
> Christian Heimes added the comment:
>
> +1 for an option to profile import time.
>
> I have a minor complain: The output format is not machine-friendly
New submission from INADA Naoki:
Makes `import functools` about 1.5 ms faster.
* heapq is used only when Counter.most_common(n).
* types and waakref is used only for singledispatch.
master:
import time: - _functools 76 us (self 76 us)
import time: - _operator 110 us (self 110 us
Changes by INADA Naoki :
--
keywords: +patch
pull_requests: +3743
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue31581>
___
___
Py
INADA Naoki added the comment:
How often uuid1 is used?
I never use it and it looks uuid1 makes uuid.py complicated.
How about split it to _uuid1.py (or uuid/__init__.py and uuid/_uuid1.py)?
I hope PEP 562 is accepted.
It ease splitting out (heavy and slow and dirty) part into submodule
INADA Naoki added the comment:
Sorry, I reject my idea. It clearly overdone. uuid.py is not so huge.
--
versions: +Python 3.5 -Python 3.7
___
Python tracker
<https://bugs.python.org/issue11
INADA Naoki added the comment:
>> Sorry, I reject my idea. It clearly overdone. uuid.py is not so huge.
> Can you please elaborate? Do you think that my PR is wrong?
I looked https://github.com/python/cpython/pull/3684 and
I wonder if I should recommend to split module before rev
INADA Naoki added the comment:
New changeset b24cd055ecb3eea9a15405a6ca72dafc739e6531 by INADA Naoki (James
Sexton) in branch 'master':
bpo-30806 netrc.__repr__() is broken for writing to file (GH-2491)
https://github.com/python/cpython/commit/b24cd055ecb3eea9a15405a6ca72da
INADA Naoki added the comment:
New changeset 9811e80fd0ed9d74c76a66f1dd4e4b8afa9e8f53 by INADA Naoki in branch
'master':
bpo-31581: Reduce the number of imports for functools (GH-3757)
https://github.com/python/cpython/commit/9811e80fd0ed9d74c76a66f1dd4e4b
Change by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
INADA Naoki added the comment:
We already accept PEP 529 and PEP 538.
And there are #15216 (PR 2343).
So I don't think we need more solutions.
--
___
Python tracker
<https://bugs.python.org/is
New submission from INADA Naoki :
Current DER_cert_to_PEM_cert() uses textwrap like this:
def DER_cert_to_PEM_cert(der_cert_bytes):
"""Takes a certificate in binary DER format and returns the
PEM version of it as a string."""
f =
Change by INADA Naoki :
--
components: +Library (Lib)
versions: +Python 3.7
___
Python tracker
<https://bugs.python.org/issue31659>
___
___
Python-bugs-list mailin
Change by INADA Naoki :
--
keywords: +patch
pull_requests: +3829
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue31659>
___
___
Python-
INADA Naoki added the comment:
For the record, this is part of `import requests`.
`import ssl` took 11 ms and `textwrap` weights 2.4 ms of it.
- ipaddress 2048 us (self 2048 us)
- textwrap 2402 us (self 2402 us)
- _ssl 2744 us (self
INADA Naoki added the comment:
New changeset b75a228af8c0553aef44e4e03763af90fbc8737f by INADA Naoki in branch
'master':
bpo-31659: Use simple slicing to format PEM cert (GH-3849)
https://github.com/python/cpython/commit/b75a228af8c0553aef44e4e03763af
Change by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
INADA Naoki added the comment:
New changeset 1a87de7fcfa3c19f08e29047337c350b4a32b259 by INADA Naoki in branch
'master':
bpo-31415: Add `-X importtime` option (GH-3490)
https://github.com/python/cpython/commit/1a87de7fcfa3c19f08e29047337c35
New submission from INADA Naoki :
flags exposed by re module are IntFlag from Python 3.4.
Since it is passed to underlying sre_parse and sre_compile,
tests in loop (e.g. `flags & SRE_FLAG_IGNORECASE`) calls
IntFlag.__and__ and creates new instance everytime.
It makes re.compile() sl
Change by INADA Naoki :
--
keywords: +patch
pull_requests: +3842
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue31671>
___
___
Python-
New submission from INADA Naoki :
Currently, strings.Template uses re.IGNORECASE without re.ASCII:
idpattern = r'[_a-z][_a-z0-9]*'
flags = _re.IGNORECASE
[a-z] matches against 'ı' (0x131, DOTLESS I) and 'ſ' (0x17f, LONG S).
It is not intentional,
Change by INADA Naoki :
--
superseder: -> string.Template should use re.ASCII flag
___
Python tracker
<https://bugs.python.org/issue31669>
___
___
Python-
New submission from INADA Naoki :
email.header has this pattern:
https://github.com/python/cpython/blob/85c0b8941f0c8ef3ed787c9d504712c6ad3eb5d3/Lib/email/header.py#L34-L43
# Match encoded-word strings in the form =?charset?q?Hello_World?=
ecre = re.compile(r
Change by INADA Naoki :
--
keywords: +patch
pull_requests: +3848
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue31677>
___
___
Python-
Change by INADA Naoki :
--
keywords: +patch
pull_requests: +3851
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue31672>
___
___
Python-
INADA Naoki added the comment:
> This means if someone does subclass string.Template and changes the pattern
> to accept Unicode identifiers, then with this change they will also have to
> modify flags, whereas before they didn't.
Thank you for pointing it out.
I removed r
Change by INADA Naoki :
--
stage: -> patch review
type: -> behavior
___
Python tracker
<https://bugs.python.org/issue31672>
___
___
Python-bugs-list
INADA Naoki added the comment:
https://github.com/python/cpython/blob/27c623c845dd6e4b8e1782666ca3a956636da266/Modules/timemodule.c#L80-L112
time module has perf_counter implementation based on QueryPerformanceCounter().
Could we change PyTime_GetMonotonicClock implementation to use it on
INADA Naoki added the comment:
> When proposing to undo recent decisions, please add the people to the nosy
> list who were involved in making that decision in the first place.
I don't propose reverting IntFlag.
I just propose convert IntFlag to int in re module, before pa
INADA Naoki added the comment:
> It's better to keep the re.IGNORECASE since the RFC also says:
>
> Both 'encoding' and 'charset' names are case-independent. Thus the
> charset name "ISO-8859-1" is equivalent to "iso-8859-1", and t
INADA Naoki added the comment:
New changeset bf477a99e0c85258e6573f4ee9eda68fa1f98a31 by INADA Naoki in branch
'master':
bpo-31677: email: Remove re.IGNORECASE flag (GH-3868)
https://github.com/python/cpython/commit/bf477a99e0c85258e6573f4ee9eda6
INADA Naoki added the comment:
> When optimizing, please don't make API changes.
This is not only optimization, but bugfix.
Document of string.Template says:
> By default, "identifier" is restricted to any case-insensitive ASCII
> alphanumeric string (including
INADA Naoki added the comment:
> Yet one way -- make re.ASCII a local flag. Than we could just change the
> idpattern attribute to r'(?a:[_a-z][_a-z0-9]*)', without touching the flags
> attribute.
https://docs.python.org/3.6/library/re.html#regular-expression-synta
INADA Naoki added the comment:
Current pull request override `Template.flags = re.I` after class creation for
backward compatibility, without any API change.
But I'm not sure it's right approach.
How many people who subclass string.Template expect non-ASCII match?
If this change is
INADA Naoki added the comment:
Thank you for benchmarking. I've added news entry about it.
--
___
Python tracker
<https://bugs.python.org/issue31671>
___
___
Change by INADA Naoki :
--
keywords: +patch
pull_requests: +3858
stage: needs patch -> patch review
___
Python tracker
<https://bugs.python.org/issu
INADA Naoki added the comment:
> IntFlag.__and__ does not create a new instance every time -- all new
> instances are cached in the IntFlag machinery (so RegexFlag(7) is only
> created once).
I'm sorry, I misunderstood.
But while new instance is not created each time, 4 Pytho
INADA Naoki added the comment:
New changeset c1c47c166b1012d34f2c6e111ee9ccb5c4d12de7 by INADA Naoki in branch
'master':
bpo-31671: re: Convert RegexFlag to int before compile (GH-3862)
https://github.com/python/cpython/commit/c1c47c166b1012d34f2c6e111ee9cc
Change by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
INADA Naoki added the comment:
Instead caching type(flags), how about this?
if not isinstance(flags, int):
raise TypeError(f"flags must be int or RegexFlag, got {flags!r}")
flags = int(flags)
--
___
Python track
INADA Naoki added the comment:
Japanese HTML has this line:
https://docs.python.org/2/faq/design.html"; />
I suspect this line affects CDN's cache.
But I can't find document about canonical link in fastly's document.
--
INADA Naoki added the comment:
https://twitter.com/miyagawa/status/917629042278359040
Miyagawa-san, the member of Fastly told me they doesn't use HTML
content for cache key, unless we customize VCL.
But I can't find VCL for docs.python.org in github.
INADA Naoki added the comment:
As Instagram's report, disabling cycler GC really helps even if there is
refcont.
All application have some cold data: imported but never used modules, functions.
--
___
Python tracker
<https://bugs.py
INADA Naoki added the comment:
>> This is only useful if the parent process has a lot of memory that's never
>> used by the child processes right? Otherwise, you would lose via refcounting
>> COWs.
>
> What we saw in prod is that memory fragmentation caused b
INADA Naoki added the comment:
Should gc.freeze() do gc.collect() right before freezing?
Or should we document `gc.collect(); gc.freeze();` idiom?
I don't like `gc.freeze(collect=False)`.
So if there is at least one use case of `gc.freeze()` without `gc.collect()`,
I'm +1 on forme
INADA Naoki added the comment:
> So my understand is that gc frees some objects and makes some memory pages
> becomes available to allocate in child process. Allocation on the shared
> memory pages will cause the copy-on-write even without gc.
>
> Though this behavior may h
INADA Naoki added the comment:
New changeset b22273ec5d1992b0cbe078b887427ae9977dfb78 by INADA Naoki in branch
'master':
bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers
(GH-3872)
https://github.com/python/cpython/commit/b22273ec5d1992b0cbe078b887427a
Change by INADA Naoki :
--
pull_requests: +3958
___
Python tracker
<https://bugs.python.org/issue31672>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 7060380d577690a40ebc201c0725076349e977cd by INADA Naoki in branch
'3.6':
bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers
(GH-3872)
https://github.com/python/cpython/commit/7060380d577690a40ebc201c072507
Change by INADA Naoki :
--
pull_requests: +3966
___
Python tracker
<https://bugs.python.org/issue31672>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset 073150db39408c1800e4b9e895ad0b0e195f1056 by INADA Naoki in branch
'master':
bpo-31672: doc: Remove one sentence from library/string.rst (GH-3990)
https://github.com/python/cpython/commit/073150db39408c1800e4b9e895ad0b
INADA Naoki added the comment:
Does it worth enough?
I didn't think it's worth enough because import will be much slower than one
dict lookup.
--
___
Python tracker
<https://bugs.python.o
INADA Naoki added the comment:
Xoptions is not environment variable.
Some modules are imported before xoptions are parsed.
So negative cache for xoptions is little more complex than
negative cache for environment variable.
--
___
Python tracker
INADA Naoki added the comment:
New changeset 5fbe5e161c969bc8a0d44a301152f8bf5afe0fc7 by INADA Naoki (Miss
Islington (bot)) in branch '3.6':
bpo-30806: Fix netrc.__repr__() format (GH-2491)
https://github.com/python/cpython/commit/5fbe5e161c969bc8a0d44a301152f8
Change by INADA Naoki :
--
resolution: -> fixed
versions: +Python 3.7
___
Python tracker
<https://bugs.python.org/issue30806>
___
___
Python-bugs-list mai
Change by INADA Naoki :
--
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue30806>
___
___
Pyth
INADA Naoki added the comment:
When adding environment variable option, it should be documented in
man page and `python -h`.
--
___
Python tracker
<https://bugs.python.org/issue31
INADA Naoki added the comment:
I'm +1 to remove -X option.
Supporting both form makes code ugly and benefit is too small.
And +1 to negative cache too. It's simple for environment varianle.
--
___
Python tracker
<https://bu
INADA Naoki added the comment:
Quick microbench:
```
$ ./python -m perf timeit --compare-to=`pwd`/python.master -s 'd =
dict.fromkeys(range(1000))' -- "for i in range(1000):
del d[i]
d[i]=i
"
python.master: . 124 us +- 9 us
python:
INADA Naoki added the comment:
I don't know why dk_lookup is in dictkeys object. But I think
it's because sharing 1 word from all key-sharing dict.
So ma_clean flag can be in dictkeys object for same reason.
BTW, We use dk_lookup function pointer and it tooks 1 word.
But PyPy use fl
INADA Naoki added the comment:
New changeset 5a66c8a64d180b5f3c80307924adaec53cc8faa3 by INADA Naoki (Julien
Palard) in branch 'master':
bpo-31793: Doc: Specialize smart-quotes for Japanese (GH-4006)
https://github.com/python/cpython/commit/5a66c8a64d180b5f3c80307924adae
INADA Naoki added the comment:
New changeset 47eaaa55247b6ad8ae507c0048509c2e3db79bf0 by INADA Naoki (Miss
Islington (bot)) in branch '2.7':
bpo-31793: Doc: Specialize smart-quotes for Japanese (GH-4006)
https://github.com/python/cpython/commit/47eaaa55247b6ad8ae507c0048509c
INADA Naoki added the comment:
New changeset 64f10492dcda4117ac06399ae46ab645cb09b19e by INADA Naoki (Miss
Islington (bot)) in branch '3.6':
bpo-31793: Doc: Specialize smart-quotes for Japanese (GH-4006)
https://github.com/python/cpython/commit/64f10492dcda4117ac06399ae46ab6
INADA Naoki added the comment:
Thanks, Julien.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 2.7
___
Python tracker
<https://bugs.python.or
INADA Naoki added the comment:
It seems it is merged in 2.7.14.
But I can't find this in changelog of 2.7.14.
https://raw.githubusercontent.com/python/cpython/c707893f9cee870bba8364b3a06eb9cfa3b80e58/Misc/NEWS
--
___
Python tracker
&
INADA Naoki added the comment:
New changeset 3b9173d33adc2903e1af461214333b0052d7b1e9 by INADA Naoki (Steven
Loria) in branch '2.7':
bpo-30806: Fix netrc.__repr__() format (GH-2491)
https://github.com/python/cpython/commit/3b9173d33adc2903e1af461214333b
INADA Naoki added the comment:
Looks nice. Can it speed up aiohttp too?
--
nosy: +inada.naoki
___
Python tracker
<https://bugs.python.org/issue32251>
___
___
INADA Naoki added the comment:
New changeset 7ea143ae795a9fd57eaccf490d316bdc13ee9065 by INADA Naoki in branch
'master':
bpo-29469: Move constant folding to AST optimizer (GH-2858)
https://github.com/python/cpython/commit/7ea143ae795a9fd57eaccf490d316b
Change by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by INADA Naoki :
--
pull_requests: +4753
___
Python tracker
<https://bugs.python.org/issue29469>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
New changeset eadad1b97f64619bfd246b9d3b60d25f456e0592 by INADA Naoki in branch
'master':
bpo-29469: Remove unnecessary peephole optimizer (GH-4863)
https://github.com/python/cpython/commit/eadad1b97f64619bfd246b9d3b60d2
Change by INADA Naoki :
--
pull_requests: +4773
___
Python tracker
<https://bugs.python.org/issue29469>
___
___
Python-bugs-list mailing list
Unsubscribe:
INADA Naoki added the comment:
> ISTM, the constant-tracking macros for peephole.c can also be removed,
> essentially reverting back to the simpler cumlc-style code in Py2.5.
I tried it in GH-4879.
--
___
Python tracker
<https://bugs.p
INADA Naoki added the comment:
Should we backport it to 3.6?
--
___
Python tracker
<https://bugs.python.org/issue32331>
___
___
Python-bugs-list mailin
INADA Naoki added the comment:
Document of socket.type says:
The socket type.
https://docs.python.org/3/library/socket.html#socket.socket.type
OK, so, are SOCK_CLOEXEC and SOCK_NONBLOCK socket type?
These two constants, if defined, can be combined with the socket types and
allow you
INADA Naoki added the comment:
Since normal dict preserves insertion order, OrderedDict usage
will be decreased. So I don't think 2x memory consumption is
big issue.
BTW, functools.lru_cache uses custom doubly-linked list which
is much memory inefficient.
Rewriting it to use OrderedDict
INADA Naoki added the comment:
Please don't do this.
del d[next(iter(d))] is not O(1) on current dict implementation.
OrderedDict is designed for such use cases. Please keep using it.
--
nosy: +inada.naoki
___
Python tracker
&
INADA Naoki added the comment:
New changeset 87010e85cb37192d63b1a30e5fabba307ad5a3f5 by INADA Naoki in branch
'master':
bpo-29469: peephole: Remove const_stack (GH-4879)
https://github.com/python/cpython/commit/87010e85cb37192d63b1a30e5fabba
INADA Naoki added the comment:
> This is surprising. But OrderedDict also can be O(N) here.
Current dict implementation doesn't skip empty entries.
So next(iter(D)) takes time.
On the other hand, OrderedDict uses doubly-linked list to find first entry.
(I fixed it in compact ODict bra
INADA Naoki added the comment:
Hmm, 0.3 μs for each lookup may be negligible compared to re.compile() speed?
--
___
Python tracker
<https://bugs.python.org/issue32
INADA Naoki added the comment:
> We are talking about a dictionary of 512 items in the worst case. On such
> very tiny collection, benchmarking matters more than O(...) complexity ;-)
You're right. Rob Pike said:
"Fancy algorithms are slow when n is small, and n is usua
INADA Naoki added the comment:
* del cache[next(iter(cache))] happens only when sre.compile() is called.
* del cache[next(iter(cache))] is much faster than sre.compile().
OK, performance difference is negligible, surely.
--
___
Python tracker
Change by INADA Naoki :
--
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Save OrderedDict imports in various stdlibs.
___
Python tracker
<https://bugs.python.org/issue32
New submission from INADA Naoki :
csv and re has issues already.
There are some remaining OrderdDict, including docstring.
It looks like some of them can be removed easily without doc update.
This is `ag --exclude-dir=test OrderedDict`, exluding csv, re, pprint and
collections.
typing:
2109
INADA Naoki added the comment:
pyperformance doesn't show significant performance degration. (using commit
5d8d3d1)
$ ./python -m perf compare_to -G --min-speed=2 default.json patched.json
Slower (5):
- pickle_list: 9.40 us +- 0.98 us -> 9.96 us +- 1.20 us: 1.06x slower (+6%)
- pic
INADA Naoki added the comment:
New changeset 507434fd504f3ebc1da72aa77544edc0d73f136e by INADA Naoki in branch
'master':
bpo-15216: io: TextIOWrapper.reconfigure() accepts encoding, errors and newline
(GH-2343)
https://github.com/python/cpython/commit/507434fd504f3ebc1da72aa77544ed
Change by INADA Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
INADA Naoki added the comment:
New slot is really required?
My idea is:
* Copy slots from MRO
* Iterate namespace dict and override slots if dunder is defined
Is it impossible or harder than my thought?
--
nosy: +inada.naoki
___
Python tracker
Change by INADA Naoki :
--
keywords: +patch
pull_requests: +4862
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue32402>
___
___
Python-
INADA Naoki added the comment:
New changeset 4856b0f34a6f4074cd86e66f11a635422374ae98 by INADA Naoki in branch
'master':
bpo-32402: io: Add missing NULL check. (GH-4971)
https://github.com/python/cpython/commit/4856b0f34a6f4074cd86e66f11a635
New submission from INADA Naoki :
Currently, functools.lru_cache implement own doubly-linked list.
But it is inefficient than OrderedDict because each link node is GC object.
So it may eat more memory and take longer GC time.
I added two private C API for OrderedDict and make lru_cache use it
INADA Naoki added the comment:
Current implementation (no news entry yet):
https://github.com/methane/cpython/pull/10/files
--
___
Python tracker
<https://bugs.python.org/issue32
1901 - 2000 of 3039 matches
Mail list logo
