Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-17 Thread Guido van Rossum
Let's kick this part of the discussion back to python-ideas. On Tue, Oct 17, 2017 at 1:36 PM, Michel Desmoulin wrote: > Maybe it's time to bring back the debate on the "lazy" keyword then ? > Rendering any statement arbitrarily lazy could help with perfs. It would > also make hacks like ugettext

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-17 Thread Michel Desmoulin
Maybe it's time to bring back the debate on the "lazy" keyword then ? Rendering any statement arbitrarily lazy could help with perfs. It would also make hacks like ugettext_lazy in Django useless. And would render moot the extensions of f-strings for lazily rendered ones. And bring lazy imports in

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-17 Thread Neil Schemenauer
Christian Heimes wrote: > That approach could work, but I think that it is the wrong > approach. I'd rather keep Python optimized for long-running > processes and introduce a new mode / option to optimize for > short-running scripts. Another idea is to run a fake transasaction through the process

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-03 Thread Nick Coghlan
On 3 October 2017 at 03:02, Christian Heimes wrote: > On 2017-10-02 16:59, Barry Warsaw wrote: >> On Oct 2, 2017, at 10:48, Christian Heimes wrote: >>> >>> That approach could work, but I think that it is the wrong approach. I'd >>> rather keep Python optimized for long-running processes and intr

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Brett Cannon
On Mon, 2 Oct 2017 at 11:19 Christian Heimes wrote: > On 2017-10-02 19:29, Brett Cannon wrote: > > My current design for an opt-in lazy importing setup includes an > > explicit function for importlib that's mainly targeted for the stdlib > > and it's startup module needs, but could be used by oth

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 19:29, Brett Cannon wrote: > My current design for an opt-in lazy importing setup includes an > explicit function for importlib that's mainly targeted for the stdlib > and it's startup module needs, but could be used by others: > https://notebooks.azure.com/Brett/libraries/di2Btqj7zSI

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Brett Cannon
On Mon, 2 Oct 2017 at 08:00 Christian Heimes wrote: > On 2017-10-02 15:26, Victor Stinner wrote: > > 2017-10-02 13:10 GMT+02:00 INADA Naoki : > >> https://github.com/python/cpython/pull/3796 > >> In this PR, lazy loading only happens when uuid1 is used. > >> But uuid1 is very uncommon for nowdays

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 16:59, Barry Warsaw wrote: > On Oct 2, 2017, at 10:48, Christian Heimes wrote: >> >> That approach could work, but I think that it is the wrong approach. I'd >> rather keep Python optimized for long-running processes and introduce a >> new mode / option to optimize for short-running

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Gregory P. Smith
On Mon, Oct 2, 2017 at 8:03 AM Victor Stinner wrote: > 2017-10-02 16:48 GMT+02:00 Christian Heimes : > > That approach could work, but I think that it is the wrong approach. I'd > > rather keep Python optimized for long-running processes and introduce a > > new mode / option to optimize for short

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Serhiy Storchaka
02.10.17 16:26, Victor Stinner пише: While "import module" is fast, maybe we should use sometimes a global variable to cache the import. module = None def func(): global module if module is None: import module ... I optimized "import module", and I think it can be optimized even mo

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread George King
Fair, but can you justify your preference? From my perspective, I write many small command line scripts, and all of them would benefit from faster load time. Am I going to have to stick mode-setting incantations at the top of every single one? I occasionally write simple servers, and none of the

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Victor Stinner
2017-10-02 16:48 GMT+02:00 Christian Heimes : > That approach could work, but I think that it is the wrong approach. I'd > rather keep Python optimized for long-running processes and introduce a > new mode / option to optimize for short-running scripts. "Filling caches on demand" is an old pattern

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 15:26, Victor Stinner wrote: > 2017-10-02 13:10 GMT+02:00 INADA Naoki : >> https://github.com/python/cpython/pull/3796 >> In this PR, lazy loading only happens when uuid1 is used. >> But uuid1 is very uncommon for nowdays. > > Antoine Pitrou added a new C extension _uuid which is imp

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Barry Warsaw
On Oct 2, 2017, at 10:48, Christian Heimes wrote: > > That approach could work, but I think that it is the wrong approach. I'd > rather keep Python optimized for long-running processes and introduce a > new mode / option to optimize for short-running scripts. What would that look like, how would

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread George King
I’m new to this issue, but curious: could the long-running server mitigate lazy loading problems simply by explicitly importing the deferred modules, e.g. at the top of __main__.py? It would require some performance tracing or other analysis to figure out what needed to be imported, but this mig

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 14:05, George King wrote: > I’m new to this issue, but curious: could the long-running server > mitigate lazy loading problems simply by explicitly importing the > deferred modules, e.g. at the top of __main__.py? It would require some > performance tracing or other analysis to figure

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Victor Stinner
2017-10-02 13:10 GMT+02:00 INADA Naoki : > https://github.com/python/cpython/pull/3796 > In this PR, lazy loading only happens when uuid1 is used. > But uuid1 is very uncommon for nowdays. Antoine Pitrou added a new C extension _uuid which is imported as soon as uuid(.py) is imported. On Linux at

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread INADA Naoki
Hi. My company is using Python for web service. So I understand what you're worrying. I'm against fine grained, massive lazy loading too. But I think we're careful enough for lazy importing. https://github.com/python/cpython/pull/3849 In this PR, I stop using textwrap entirely, instead of lazy i