[issue18596] enable usage of AddressSanitizer in CPython [PATCH]

[halfie]

New submission from halfie:

Hi,

When trying to build CPython "tip" with AddressSanitizer enabled, I get the 
following crash,

make Parser/pgen
make[1]: Entering directory `/scratch/repos/cpython'
make[1]: `Parser/pgen' is up to date.
make[1]: Leaving directory `/scratch/repos/cpython'
Parser/pgen ./Grammar/Grammar Include/graminit.h Python/graminit.c
=
==1375== ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x6038f020 at pc 0x40b260 bp 0x7fff6e6cbbf0 sp 0x7fff6e6cbbe8
READ of size 4 at 0x6038f020 thread T0
#0 0x40b25f in _PyObject_Realloc 
/scratch/repos/cpython/Objects/obmalloc.c:1551
#1 0x403048 in PyNode_AddChild /scratch/repos/cpython/Parser/node.c:98
#2 0x4037fe in push /scratch/repos/cpython/Parser/parser.c:126
#3 0x4121bd in parsetok /scratch/repos/cpython/Parser/parsetok.c:211
#4 0x413344 in PyParser_ParseFileFlags 
/scratch/repos/cpython/Parser/parsetok.c:86
#5 0x413422 in getgrammar /scratch/repos/cpython/Parser/pgenmain.c:92
#6 0x401860 in main /scratch/repos/cpython/Parser/pgenmain.c:55
#7 0x3cb2221b74 in ?? ??:0
#8 0x401994 in _start ??:?
0x6038f020 is located 96 bytes to the left of 0-byte region 
[0x6038f080,0x6038f080)
==1375== AddressSanitizer CHECK failed: 
../../../../libsanitizer/asan/asan_allocator2.cc:216 "((id)) != (0)" (0x0, 0x0)
#0 0x3ab681237d in ?? ??:0
#1 0x3ab68191b3 in ?? ??:0
#2 0x3ab680563b in ?? ??:0
#3 0x3ab6816e8f in ?? ??:0
#4 0x3ab6817ff1 in ?? ??:0
#5 0x3ab6812763 in ?? ??:0
#6 0x40b25f in _PyObject_Realloc 
/scratch/repos/cpython/Objects/obmalloc.c:1551
#7 0x403048 in PyNode_AddChild /scratch/repos/cpython/Parser/node.c:98
#8 0x4037fe in push /scratch/repos/cpython/Parser/parser.c:126
#9 0x4121bd in parsetok /scratch/repos/cpython/Parser/parsetok.c:211
#10 0x413344 in PyParser_ParseFileFlags 
/scratch/repos/cpython/Parser/parsetok.c:86
#11 0x413422 in getgrammar /scratch/repos/cpython/Parser/pgenmain.c:92
#12 0x401860 in main /scratch/repos/cpython/Parser/pgenmain.c:55
#13 0x3cb2221b74 in ?? ??:0
#14 0x401994 in _start ??:?
make: *** [Include/graminit.h] Error 1

...

AddressSanitizer is a fast memory error detector. See 
http://clang.llvm.org/docs/AddressSanitizer.html for details.


Build options used
==

export CFLAGS="-fsanitize=address -g"

export LDFLAGS="-fsanitize=address"

./configure --prefix=/opt/python --with-valgrind


Software Versions
==

$ hg log

changeset:   84902:6e1dd1ce95b8
branch:  2.7
tag: tip
parent:  84893:1dbcb0299088
user:Terry Jan Reedy 
date:Tue Jul 30 01:37:36 2013 -0400
summary: Make all idle test case names end with 'Test'.

$ gcc --version
gcc (GCC) 4.8.1 20130603 (Red Hat 4.8.1-1)

$ cat /etc/redhat-release 
Fedora release 19 (Schrödinger’s Cat)

Fix / Work-around
=

See 
http://code.google.com/p/address-sanitizer/wiki/AddressSanitizer#Turning_off_instrumentation

Turning off the instrumentation for _PyObject_Free and  _PyObject_Realloc is 
required.

Advantages
==

AddressSanitizer can *now* be used to test and debug CPython.

AddressSanitizer has helped me find a large number of bugs in other softwares.

--
components: Interpreter Core
files: enable-ASAN-in-CPython.diff
keywords: patch
messages: 193917
nosy: halfie
priority: normal
severity: normal
status: open
title: enable usage of AddressSanitizer in CPython [PATCH]
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file31083/enable-ASAN-in-CPython.diff

___
Python tracker 
<http://bugs.python.org/issue18596>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18596] enable usage of AddressSanitizer in CPython [PATCH]

[halfie]

halfie added the comment:

This patch should only be applied if we are sure that the allocator is fine and 
doing such things by design.

--
type: enhancement -> crash

___
Python tracker 
<http://bugs.python.org/issue18596>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18596] enable usage of AddressSanitizer in CPython [PATCH]

[halfie]

halfie added the comment:

Using,

# define ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS 
__attribute__((no_address_safety_analysis)) __attribute__ ((noinline))

instead of,

# define ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS 
__attribute__((no_address_safety_analysis))

seems to be a more future-proof (against inlining) solution.

--

___
Python tracker 
<http://bugs.python.org/issue18596>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14748] spwd.getspall() is returning LDAP (non local) users too

[halfie]

New submission from halfie :

spwd.getspall() is returning LDAP (non local) users too. 

On RHEL 6.2 machine with LDAP authentication configured, spwd.getspall() 
is returning LDAP (non local) users too. On a similarly configured CentOS 6.2 
machine, spwd.getspall() is returning only local users.

Is spwd.getspall() supposed to return LDAP users? (If yes, this should to 
documented). Why is spwd.getspall() behavior different on different Linux OSes?

--
components: Library (Lib)
messages: 160188
nosy: halfie
priority: normal
severity: normal
status: open
title: spwd.getspall() is returning LDAP (non local) users too
type: behavior
versions: Python 2.6

___
Python tracker 
<http://bugs.python.org/issue14748>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18596] enable usage of AddressSanitizer in CPython [PATCH]

[halfie]

halfie added the comment:

I am attaching the latest tested patch against tip.

--
Added file: http://bugs.python.org/file31866/ASAN-compat-35da5d848ffd-v3.patch

___
Python tracker 
<http://bugs.python.org/issue18596>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com