[issue13169] Regular expressions with 0 to 65536 repetitions and above makes Python crash

[Maurice de Rooij]

New submission from Maurice de Rooij :

Regular expressions with 0 to 65536 repetitions and above makes Python crash 
with a "OverflowError: regular expression code size limit exceeded" exception.
65535 repetitions do not raise this issue.

Tested and confirmed this with versions 2.7.1 and 3.2.2.

C:\Python27>python.exe
Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> re.search('(?s)\A.{0,65535}test', 'test')
<_sre.SRE_Match object at 0x00B4E4B8>
>>> re.search('(?s)\A.{0,65536}test', 'test')
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\re.py", line 142, in search
return _compile(pattern, flags).search(string)
  File "C:\Python27\lib\re.py", line 243, in _compile
p = sre_compile.compile(pattern, flags)
  File "C:\Python27\lib\sre_compile.py", line 523, in compile
groupindex, indexgroup
OverflowError: regular expression code size limit exceeded
>>>

C:\Python32>python.exe
Python 3.2.2 (default, Sep  4 2011, 09:51:08) [MSC v.1500 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> re.search('(?s)\A.{0,65535}test', 'test')
<_sre.SRE_Match object at 0x00A6F250>
>>> re.search('(?s)\A.{0,65536}test', 'test')
Traceback (most recent call last):
  File "C:\Python32\lib\functools.py", line 176, in wrapper
result = cache[key]
KeyError: (, '(?s)\\A.{0,65536}test', 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python32\lib\re.py", line 158, in search
return _compile(pattern, flags).search(string)
  File "C:\Python32\lib\re.py", line 255, in _compile
return _compile_typed(type(pattern), pattern, flags)
  File "C:\Python32\lib\functools.py", line 180, in wrapper
result = user_function(*args, **kwds)
  File "C:\Python32\lib\re.py", line 267, in _compile_typed
return sre_compile.compile(pattern, flags)
  File "C:\Python32\lib\sre_compile.py", line 514, in compile
groupindex, indexgroup
OverflowError: regular expression code size limit exceeded
>>>

--
components: Library (Lib)
messages: 145469
nosy: techmaurice
priority: normal
severity: normal
status: open
title: Regular expressions with 0 to 65536 repetitions and above makes Python 
crash
type: crash
versions: Python 2.7, Python 3.2

___
Python tracker 
<http://bugs.python.org/issue13169>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13169] Regular expressions with 0 to 65536 repetitions raises OverflowError

[Maurice de Rooij]

Maurice de Rooij  added the comment:

So if I understand correctly, the maximum of 65535 repetitions is by design?

Have tried a workaround by repeating the repetitions by placing it inside a 
capturing group, which is perfectly legal with Perl regular expressions:

$mystring = "test";
if($mystring =~ m/^(.{0,32766}){0,3}test/s) { print "Yes\n"; }
(32766 being the max repetitions in Perl)

Unfortunately, in Python this does not work and raises a "nothing to repeat" 
sre_constants error:
re.search('(?s)\A(.{0,65535}){0,3}test', 'test')

This, however works, which yields 65536 repetitions of DOTALL:
re.search('(?s)\A.{0,65535}.{0,1}test', 'test')

In the end this solves my problem sort or less, but requires extra logic in my 
script and complicates stuff unnecessary.

A suggestion might be to make repetitions of repeats possible?

--

___
Python tracker 
<http://bugs.python.org/issue13169>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com