[issue31580] Defer compiling regular expressions

2017-09-27 Thread Barry A. Warsaw
Barry A. Warsaw added the comment: I'm closing this experiment. I'm not convinced that even if we can make start up time faster for module global regular expressions, we'll ever get enough buy-in from the ecosystem to make this worth it, because you'd really want to get all your dependencies

[issue31580] Defer compiling regular expressions

2017-09-26 Thread Ezio Melotti
Ezio Melotti added the comment: What about adding a lazy_compile() function? It will leave the current behavior unchanged, it's explicit, and it's easier to use cross version (if importing re.lazy_compile fails, use re.compile). FWIW I'm -1 on changing re.compile, -1 on adding re.IMMEDIATE, -

[issue31580] Defer compiling regular expressions

2017-09-26 Thread Stefan Behnel
Stefan Behnel added the comment: I'm also against changing re.compile() to not compile. And I often write code like this: replace_whitespace = re.compile(r"\s+").sub which is not covered by your current proposed change. -- nosy: +scoder ___ Pyt

[issue31580] Defer compiling regular expressions

2017-09-26 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: Barry, please test re.search() with your changes. $ ./python -m timeit -s 'import re; s = "a" * 100' 're.search("b", s)' Unpatched: 50 loops, best of 5: 529 nsec per loop Patched:5 loops, best of 5: 7.46 usec per loop --

[issue31580] Defer compiling regular expressions

2017-09-26 Thread Raymond Hettinger
Raymond Hettinger added the comment: I'm flat-out opposed to changing the default behavior. If some API change gets make it needs to strictly be opt-in. Even then, I don't think this is a great idea. We already have ways to do it if people actually cared about this. FWIW, other languages

[issue31580] Defer compiling regular expressions

2017-09-26 Thread Barry A. Warsaw
Barry A. Warsaw added the comment: On Sep 26, 2017, at 11:27, R. David Murray wrote: > > Precompiling as a compile-time optimization would be cool. I think we are > currently favoring doing that kind of thing as an AST optimization step? I was thinking about that too. > I think Raymond and

[issue31580] Defer compiling regular expressions

2017-09-26 Thread R. David Murray
R. David Murray added the comment: Precompiling as a compile-time optimization would be cool. I think we are currently favoring doing that kind of thing as an AST optimization step? I think Raymond and my point was that the current behavior should remain unchanged by default. So a re.DEFERRE

[issue31580] Defer compiling regular expressions

2017-09-26 Thread Barry A. Warsaw
Barry A. Warsaw added the comment: Let's separate the use of lru_cache from the deferred compilation. I think I'll just revert the change to use lru_cache, although I'll note that the impetus for this was the observation that once MAXCACHE is reached the entire regexp cache is purged. That s

[issue31580] Defer compiling regular expressions

2017-09-25 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: lru_cache() is not used for reasons. The patch makes using cached patterns slower by adding an overhead of 4 additional checks. It makes resolving attributes of a deferred patterns slower. All this slows down a straight call to re.search(pattern, s). Errors

[issue31580] Defer compiling regular expressions

2017-09-25 Thread R. David Murray
R. David Murray added the comment: I agree with Raymond. It would be strange to have the API that is obviously designed to pre-compile the regex not pre-compile the regex. If the concern is that a non-precompiled regex might get bumped out of the cache but you want a way to only compile a "bu

[issue31580] Defer compiling regular expressions

2017-09-25 Thread Raymond Hettinger
Raymond Hettinger added the comment: ISTM, the whole point is to compile in advance. When I worked during high frequency trading, that was essential to news trading where you *really* didn't want to pay the compilation cost at the time the regex was used. This proposal takes away the user's

[issue31580] Defer compiling regular expressions

2017-09-25 Thread Barry A. Warsaw
Changes by Barry A. Warsaw : -- keywords: +patch pull_requests: +3742 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-

[issue31580] Defer compiling regular expressions

2017-09-25 Thread Barry A. Warsaw
New submission from Barry A. Warsaw: It's a very common pattern to see the following at module scope: cre_a = re.compile('some pattern') cre_b = re.compile('other pattern') and so on. This can cost you at start up time because all those regular expressions are compiled at import time, even if