Re: [Python-Dev] [Core-mentorship] C version of weakrefset.py.
Hello, 1)I might be totally wrong here, but even if we go with stripping annotation(in .pyc)...still the "lag" that comes from ABCs needs to be addressed. 2) I been reading for past few days about your fat-optimizer project and the corresponding TO-DO list, if you think that PEP-0511 should be improved, I can work on that. 3)Also I was seeing to existing repos which implement some-kind of optimizaton, like numpy,snake-oil Finally I wanted to decide upon a roadmap, so that I could put more specific efforts.(fat optimizer?) Thank You Regards, Bhavishya On Wed, Jun 21, 2017 at 8:05 PM, Victor Stinner wrote: > 2017-06-21 15:21 GMT+02:00 INADA Naoki : > > ABC slowdown Python startup only 2ms. But importing typing module take > 11ms. > > While typing is not imported from site.py, many new Python application > > will import it. > > It may take over 100ms for applications or libraries heavily depending > on ABCs. > > When typing is not used in the application, only used for static > checks, you can try to "strip" annotations to avoid any overhead on > the application startup. It's not only a matter of "import typing", > it's also the cost of instanciating types like "List[int]" (or even > more complex ones). > > I discussed with Jukka Lehtosalo at Pycon US about stripping > completely annotations. He told me that my PEP 511 may be a good > solution to keep annotation in the .py code, but strip them for > "production code", in the cached .pyc files: > > "PEP 511 -- API for code transformers" > https://www.python.org/dev/peps/pep-0511/ > > This PEP is somehow controversal. Some people fear that it would allow > people to hack the Python language to write their own incompatible > variant of Python. I don't think that my PEP adds anything new, it's > already possible to do that, importlib made it even more easy. I used > my FAT Python optimizer project to sell this PEP. Since FAT Python is > also controversal (it hasn't been proved to be actually faster), the > PEP didn't go far at my last attempt. > > Note: Is core-menthorship the best place for such performance > discussion? :-) Maybe we should open a thread on python-dev@ or speed@ > mailing list. > > Victor > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Core-mentorship] C version of weakrefset.py.
Also I saw your conversation with "Brett Cannon" on lazy-loading some modules at startup, and also doing so using ModuleProxy(also several implementations that exist like importlib's Lazyloader <https://docs.python.org/3/library/importlib.html?highlight=lazy%20import#importlib.util.LazyLoader.factory>, PEAK <http://peak.telecommunity.com/DevCenter/Importing> ). So your suggestion's on this too? Thank You regards, Bhavishya On Thu, Jun 22, 2017 at 7:24 PM, Bhavishya wrote: > Hello, > 1)I might be totally wrong here, but even if we go with stripping > annotation(in .pyc)...still the "lag" that comes from ABCs needs to be > addressed. > > 2) I been reading for past few days about your fat-optimizer project and > the corresponding TO-DO list, if you think that PEP-0511 should be > improved, I can work on that. > > 3)Also I was seeing to existing repos which implement some-kind of > optimizaton, like > numpy,snake-oil > > Finally I wanted to decide upon a roadmap, so that I could put more > specific efforts.(fat optimizer?) > > Thank You > Regards, > Bhavishya > > On Wed, Jun 21, 2017 at 8:05 PM, Victor Stinner > wrote: > >> 2017-06-21 15:21 GMT+02:00 INADA Naoki : >> > ABC slowdown Python startup only 2ms. But importing typing module take >> 11ms. >> > While typing is not imported from site.py, many new Python application >> > will import it. >> > It may take over 100ms for applications or libraries heavily depending >> on ABCs. >> >> When typing is not used in the application, only used for static >> checks, you can try to "strip" annotations to avoid any overhead on >> the application startup. It's not only a matter of "import typing", >> it's also the cost of instanciating types like "List[int]" (or even >> more complex ones). >> >> I discussed with Jukka Lehtosalo at Pycon US about stripping >> completely annotations. He told me that my PEP 511 may be a good >> solution to keep annotation in the .py code, but strip them for >> "production code", in the cached .pyc files: >> >> "PEP 511 -- API for code transformers" >> https://www.python.org/dev/peps/pep-0511/ >> >> This PEP is somehow controversal. Some people fear that it would allow >> people to hack the Python language to write their own incompatible >> variant of Python. I don't think that my PEP adds anything new, it's >> already possible to do that, importlib made it even more easy. I used >> my FAT Python optimizer project to sell this PEP. Since FAT Python is >> also controversal (it hasn't been proved to be actually faster), the >> PEP didn't go far at my last attempt. >> >> Note: Is core-menthorship the best place for such performance >> discussion? :-) Maybe we should open a thread on python-dev@ or speed@ >> mailing list. >> >> Victor >> > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Lazy_loading to pickle/unpickle_pure?
Hi, I added lazy_loading in pickle.py.Here's some statistics if you consider them of any importance: 1)Unpickle_pure: Lazy -> unpickle_pure_python: Mean +- std dev: 728 us +- 24 us Original -> unpickle_pure_python: Mean +- std dev: 771 us +- 22 us 2)Pickle_pure: Lazy-> pickle_pure_python: Mean +- std dev: 919 us +- 18 us Orginal -> pickle_pure_python: Mean +- std dev: 1.04 ms +- 0.03 ms Thanks bhavishya ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pure pickle bechmark.
Hello,
1).I was going through the code of *python pickle* to search any
optimization possibility.But the only thing that I found very alarming was
again the import time(I tried with lazy-import but it didn't helped much.)
I found py3 to be ~45 times slower on* initial imports(very raw
measure..using "time." ) *as compared to py2 on an usual example.
py3->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.009715557098388672(time taken to do initial imports...measured using
*time.time()* )
py2->
./python -c '
favorite_color = { "lion": "yellow", "kitty": "red" }
pickle.dump( favorite_color, open( "save.p", "wb" ) )'
0.000236034393311(time taken to do initial imports...measured using
*time.time()* )
Do you have any thought/ideas on improving this?
Thank You.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
