Re: [Python-Dev] cpython (3.2): Issue #16335: Fix integer overflow in unicode-escape decoder.

[Christian Heimes]

Am 21.01.2013 10:46, schrieb serhiy.storchaka:
> http://hg.python.org/cpython/rev/7625866f8127
> changeset:   81622:7625866f8127
> branch:  3.2
> parent:  81610:260a9afd999a
> user:Serhiy Storchaka 
> date:Mon Jan 21 11:38:00 2013 +0200
> summary:
>   Issue #16335: Fix integer overflow in unicode-escape decoder.
> 
> files:
>   Lib/test/test_ucn.py|  16 
>   Objects/unicodeobject.c |   3 ++-
>   2 files changed, 18 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Lib/test/test_ucn.py b/Lib/test/test_ucn.py
> --- a/Lib/test/test_ucn.py
> +++ b/Lib/test/test_ucn.py
> @@ -8,6 +8,7 @@
>  """#"
>  
>  import unittest
> +import _testcapi
>  
>  from test import support
>  
> @@ -141,6 +142,21 @@
>  str, b"\\NSPACE", 'unicode-escape', 'strict'
>  )
>  
> +@unittest.skipUnless(_testcapi.INT_MAX < _testcapi.PY_SSIZE_T_MAX,
> + "needs UINT_MAX < SIZE_MAX")
> +def test_issue16335(self):
> +# very very long bogus character name
> +try:
> +x = b'\\N{SPACE' + b'x' * (_testcapi.UINT_MAX + 1) + b'}'
> +except MemoryError:
> +raise unittest.SkipTest("not enough memory")
> +self.assertEqual(len(x), len(b'\\N{SPACE}') + (_testcapi.UINT_MAX + 
> 1))
> +self.assertRaisesRegex(UnicodeError,
> +'unknown Unicode character name',
> +x.decode, 'unicode-escape'
> +)
> +
> +

The test requires a lot of memory on my 64bit Linux box. Two of three
times the test process was killed by the Kernel's out of memory manager
although I have 8 GB physical RAM and 4 GB swap. Can you rewrite the
test to use less memory?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython (2.7): Fix memory error in test_ucn.

[Antoine Pitrou]

On Mon, 21 Jan 2013 12:06:25 +0100 (CET)
serhiy.storchaka  wrote:
> diff --git a/Lib/test/test_ucn.py b/Lib/test/test_ucn.py
> --- a/Lib/test/test_ucn.py
> +++ b/Lib/test/test_ucn.py
> @@ -144,13 +144,14 @@
>  # very very long bogus character name
>  try:
>  x = b'\\N{SPACE' + b'x' * int(_testcapi.UINT_MAX + 1) + b'}'
> +self.assertEqual(len(x), len(b'\\N{SPACE}') +
> + (_testcapi.UINT_MAX + 1))
> +self.assertRaisesRegexp(UnicodeError,
> +'unknown Unicode character name',
> +x.decode, 'unicode-escape'
> +)
>  except MemoryError:
>  raise unittest.SkipTest("not enough memory")
> -self.assertEqual(len(x), len(b'\\N{SPACE}') + (_testcapi.UINT_MAX + 
> 1))
> -self.assertRaisesRegexp(UnicodeError,
> -'unknown Unicode character name',
> -x.decode, 'unicode-escape'
> -)

You can't just do that. The test may end up swapping a lot and make the
machine grind to a halt, rather than raise MemoryError. This threatens
to make life miserable for anyone who hasn't enough RAM, but has enough
swap to fit the test's working set.

Really, you have to use a bigmem decorator.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com