[issue4935] Segmentation fault in bytearray tests

Antoine Pitrou added the comment: Committed in trunk, py3k, 2.6 and 3.0. Thanks! -- resolution: accepted -> fixed status: open -> closed ___ Python tracker ___ __

[issue4935] Segmentation fault in bytearray tests

Mark Dickinson added the comment: Looks fine; I think this should be applied. It seems as though your reindentation left trailing whitespace on the blank lines in the function: the svn commit hook might complain about this... -- resolution: -> accepted _

[issue4935] Segmentation fault in bytearray tests

Antoine Pitrou added the comment: You are right, old_i and old_j are unused, they were part of another approach I tried and which failed. Attaching new patch with these 2 variables removed, and the function cleanly reindented (of course the patch is messier because of this). Added file: http://

[issue4935] Segmentation fault in bytearray tests

Mark Dickinson added the comment: Actually, what's the purpose of old_j? It looks like that's not needed any more, either! ___ Python tracker ___ ___

[issue4935] Segmentation fault in bytearray tests

Mark Dickinson added the comment: What's the purpose of old_i? It looks like it's never used for anything. Other than than, the patch looks good to me. I'd guess that the "if (i < 0)" was simply optimized away. This isn't necessarily a compiler bug: if I understand correctly, a strict readi

[issue4935] Segmentation fault in bytearray tests

Antoine Pitrou added the comment: Here is a patch. The idea is to use unsigned arithmetic instead of signed, and to check for overflows by comparing with PY_SSIZE_T_MAX. (the exact reason of the crash is unknown, it looks like either a compiler bug or a mis-optimization as (i < 0) returns 0 whi

[issue4935] Segmentation fault in bytearray tests

New submission from Antoine Pitrou : This happens on a 32-bit build on a 64-bit system, which happens to have some interesting properties: for example, malloc() will happily allocate memory larger than Py_SSIZE_T_MAX. The crash is exactly triggered by the following snippet: if sys.maxsi