On Thu, Nov 01, 2012 at 08:52:33PM +0100, do...@redhat.com wrote:
> From: Dodji Seketeli <do...@seketeli.org>
>
> Hello,
>
> The set of patches following this message represents the work that
> happened on the asan branch to build up the Address Sanitizer work
> started in the Google branch.
>
> Address Sanitizer (aka asan) is a memory error detector. It finds
> use-after-free and {heap,stack,global}-buffer overflow bugs in C/C++
> programs.
>
> One can learn about the way it works by reading the pdf slides at [1],
> or by reading the documentation on the wiki page of the project at [2].
>
> To make a long story short, it works by associating each memory region
> of eight consecutive bytes with a shadow byte that tells whether if
> each byte of the memory region is addressable or not. So,
> conceptually, there is a function 'MemToShadow' which, for each set of
> contiguous eight bytes of memory returns a shadow byte that tells
> whether if each byte is accessible or not.
>
> Then, each memory access is instrumented by the asan pass to retrieve
> the shadow byte of the accessed memory; if the access is to a memory
> address that is deemed non-accessible, a call to an asan runtime
> library function is issued to report a meaningful error to the user,
> and the access is performed, letting the user program proceed despite
> the error.
>
> The advantage of this approach, compared to say, Valgrind[4] is the
> lower time and space overhead. Eventually, when this tool becomes
> more solid, it'll become complementary to Valgrind.
>
> Apart from the compiler components, asan needs a runtime library to
> function. We share that library with the LLVM implementation of asan
> that is described at [3]. The last patch of the set imports this
> library in its pristine form into our tree. The plan is to regularly
> synchronize it with its LLVM upstream repository.
>
> On behalf of the GCC asan developers listed below, I am thus proposing
> these patches for inclusion into trunk. I chose to follow the
> chronological commits that happened on the [asan] branch, to ease the
> authorship propagation. Except for some few exceptions, each of these
> commits are reasonably logically atomic, so they hopefully shouldn't
> be too hard to review.
>
> The first patch is the initial import of the asan state from the
> Google branch into the [asan] branch. Subsequent patches clean the
> code up, add features like protection of stack and global variables,
> instrumentation of memory access through built-in functions, and, last
> but not least, the import of the runtime library.
>
> Please note that the ChangeLog.asan is meant to disappear at commit
> time, as its content will be updated (for the dates) and prepended to
> the normal ChangeLog file.
>
> One noticeable shortcoming that we have at the moment is the lack of a
> DejaGNU test harness for this. This is planned to be addressed as
> soon as possible.
>
> Please find below is a summary of the patches of the set.
>
> Thanks.
>
> [1]:
> http://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=get&target=kcc.pdf
> [2]: http://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm
> [3]: http://code.google.com/p/address-sanitizer/w/list
> [4]: http://www.valgrind.org
>
> Diego Novillo (2):
> Initial import of asan from the Google branch
> Rename tree-asan.[ch] to asan.[ch]
>
> Dodji Seketeli (3):
> Make build_check_stmt accept an SSA_NAME for its base
> Factorize condition insertion code out of build_check_stmt
> Instrument built-in memory access function calls
>
> Jakub Jelinek (6):
> Initial asan cleanups
> Emit GIMPLE directly instead of gimplifying GENERIC.
> Allow asan at -O0
> Implement protection of stack variables
> Implement protection of global variables
> Fix a couple of ICEs.
>
> Wei Mi (2):
> Don't forget to protect 32 bytes aligned global variables.
> Import the asan runtime library into GCC tree
>
> ChangeLog.asan | 7 +
> Makefile.def | 2 +
> Makefile.in | 487 +-
> configure | 1 +
> configure.ac | 1 +
> gcc/ChangeLog.asan | 175 +
> gcc/Makefile.in | 10 +-
> gcc/asan.c | 1495 ++
> gcc/asan.h | 70 +
> gcc/cfgexpand.c | 165 +-
> gcc/common.opt | 4 +
> gcc/config/i386/i386.c | 11 +
> gcc/doc/invoke.texi | 8 +-
> gcc/doc/tm.texi | 6 +
> gcc/doc/tm.texi.in | 2 +
> gcc/gcc.c | 1 +
> gcc/passes.c | 2 +
> gcc/target.def | 11 +
> gcc/toplev.c | 14 +
> gcc/tree-pass.h | 2 +
> gcc/varasm.c | 22 +
> libasan/ChangeLog.asan | 3 +
> libasan/LICENSE.TXT | 97 +
> libasan/Makefile.am | 98 +
> libasan/Makefile.in | 992 ++
> libasan/README.gcc | 4 +
> libasan/aclocal.m4 | 9645 ++++++++++
> libasan/asan_allocator.cc | 1045 ++
> libasan/asan_allocator.h | 177 +
> libasan/asan_flags.h | 103 +
> libasan/asan_globals.cc | 206 +
> libasan/asan_intercepted_functions.h | 217 +
> libasan/asan_interceptors.cc | 704 +
> libasan/asan_interceptors.h | 39 +
> libasan/asan_internal.h | 169 +
> libasan/asan_linux.cc | 150 +
> libasan/asan_lock.h | 40 +
> libasan/asan_mac.cc | 526 +
> libasan/asan_mac.h | 54 +
> libasan/asan_malloc_linux.cc | 142 +
> libasan/asan_malloc_mac.cc | 427 +
> libasan/asan_malloc_win.cc | 140 +
> libasan/asan_mapping.h | 120 +
> libasan/asan_new_delete.cc | 54 +
> libasan/asan_poisoning.cc | 151 +
> libasan/asan_posix.cc | 118 +
> libasan/asan_report.cc | 492 +
> libasan/asan_report.h | 51 +
> libasan/asan_rtl.cc | 404 +
> libasan/asan_stack.cc | 35 +
> libasan/asan_stack.h | 52 +
> libasan/asan_stats.cc | 86 +
> libasan/asan_stats.h | 65 +
> libasan/asan_thread.cc | 153 +
> libasan/asan_thread.h | 103 +
> libasan/asan_thread_registry.cc | 188 +
> libasan/asan_thread_registry.h | 83 +
> libasan/asan_win.cc | 190 +
> libasan/config.guess | 1530 ++
> libasan/config.sub | 1773 ++
> libasan/configure | 17515
> +++++++++++++++++++
> libasan/configure.ac | 67 +
> libasan/depcomp | 630 +
> libasan/include/sanitizer/asan_interface.h | 197 +
> libasan/include/sanitizer/common_interface_defs.h | 66 +
> libasan/install-sh | 527 +
> libasan/interception/interception.h | 195 +
> libasan/interception/interception_linux.cc | 28 +
> libasan/interception/interception_linux.h | 35 +
> libasan/interception/interception_mac.cc | 29 +
> libasan/interception/interception_mac.h | 47 +
> libasan/interception/interception_win.cc | 149 +
> libasan/interception/interception_win.h | 43 +
> libasan/libtool-version | 6 +
> libasan/ltmain.sh | 9661 ++++++++++
> libasan/missing | 376 +
> libasan/sanitizer_common/sanitizer_allocator.cc | 83 +
> libasan/sanitizer_common/sanitizer_allocator64.h | 573 +
> libasan/sanitizer_common/sanitizer_atomic.h | 63 +
> libasan/sanitizer_common/sanitizer_atomic_clang.h | 120 +
> libasan/sanitizer_common/sanitizer_atomic_msvc.h | 134 +
> libasan/sanitizer_common/sanitizer_common.cc | 151 +
> libasan/sanitizer_common/sanitizer_common.h | 181 +
> libasan/sanitizer_common/sanitizer_flags.cc | 95 +
> libasan/sanitizer_common/sanitizer_flags.h | 25 +
> libasan/sanitizer_common/sanitizer_internal_defs.h | 186 +
> libasan/sanitizer_common/sanitizer_libc.cc | 189 +
> libasan/sanitizer_common/sanitizer_libc.h | 69 +
> libasan/sanitizer_common/sanitizer_linux.cc | 296 +
> libasan/sanitizer_common/sanitizer_list.h | 118 +
> libasan/sanitizer_common/sanitizer_mac.cc | 249 +
> libasan/sanitizer_common/sanitizer_mutex.h | 106 +
> libasan/sanitizer_common/sanitizer_placement_new.h | 31 +
> libasan/sanitizer_common/sanitizer_posix.cc | 187 +
> libasan/sanitizer_common/sanitizer_printf.cc | 196 +
> libasan/sanitizer_common/sanitizer_procmaps.h | 95 +
> libasan/sanitizer_common/sanitizer_stackdepot.cc | 194 +
> libasan/sanitizer_common/sanitizer_stackdepot.h | 27 +
> libasan/sanitizer_common/sanitizer_stacktrace.cc | 245 +
> libasan/sanitizer_common/sanitizer_stacktrace.h | 73 +
> libasan/sanitizer_common/sanitizer_symbolizer.cc | 311 +
> libasan/sanitizer_common/sanitizer_symbolizer.h | 97 +
> .../sanitizer_common/sanitizer_symbolizer_linux.cc | 162 +
> .../sanitizer_common/sanitizer_symbolizer_mac.cc | 31 +
> .../sanitizer_common/sanitizer_symbolizer_win.cc | 33 +
> libasan/sanitizer_common/sanitizer_win.cc | 205 +
> 106 files changed, 57193 insertions(+), 25 deletions(-)
> create mode 100644 ChangeLog.asan
> create mode 100644 gcc/ChangeLog.asan
> create mode 100644 gcc/asan.c
> create mode 100644 gcc/asan.h
> create mode 100644 libasan/ChangeLog.asan
> create mode 100644 libasan/LICENSE.TXT
> create mode 100644 libasan/Makefile.am
> create mode 100644 libasan/Makefile.in
> create mode 100644 libasan/README.gcc
> create mode 100644 libasan/aclocal.m4
> create mode 100644 libasan/asan_allocator.cc
> create mode 100644 libasan/asan_allocator.h
> create mode 100644 libasan/asan_flags.h
> create mode 100644 libasan/asan_globals.cc
> create mode 100644 libasan/asan_intercepted_functions.h
> create mode 100644 libasan/asan_interceptors.cc
> create mode 100644 libasan/asan_interceptors.h
> create mode 100644 libasan/asan_internal.h
> create mode 100644 libasan/asan_linux.cc
> create mode 100644 libasan/asan_lock.h
> create mode 100644 libasan/asan_mac.cc
> create mode 100644 libasan/asan_mac.h
> create mode 100644 libasan/asan_malloc_linux.cc
> create mode 100644 libasan/asan_malloc_mac.cc
> create mode 100644 libasan/asan_malloc_win.cc
> create mode 100644 libasan/asan_mapping.h
> create mode 100644 libasan/asan_new_delete.cc
> create mode 100644 libasan/asan_poisoning.cc
> create mode 100644 libasan/asan_posix.cc
> create mode 100644 libasan/asan_report.cc
> create mode 100644 libasan/asan_report.h
> create mode 100644 libasan/asan_rtl.cc
> create mode 100644 libasan/asan_stack.cc
> create mode 100644 libasan/asan_stack.h
> create mode 100644 libasan/asan_stats.cc
> create mode 100644 libasan/asan_stats.h
> create mode 100644 libasan/asan_thread.cc
> create mode 100644 libasan/asan_thread.h
> create mode 100644 libasan/asan_thread_registry.cc
> create mode 100644 libasan/asan_thread_registry.h
> create mode 100644 libasan/asan_win.cc
> create mode 100644 libasan/config.guess
> create mode 100644 libasan/config.sub
> create mode 100644 libasan/configure
> create mode 100644 libasan/configure.ac
> create mode 100644 libasan/depcomp
> create mode 100644 libasan/include/sanitizer/asan_interface.h
> create mode 100644 libasan/include/sanitizer/common_interface_defs.h
> create mode 100644 libasan/install-sh
> create mode 100644 libasan/interception/interception.h
> create mode 100644 libasan/interception/interception_linux.cc
> create mode 100644 libasan/interception/interception_linux.h
> create mode 100644 libasan/interception/interception_mac.cc
> create mode 100644 libasan/interception/interception_mac.h
> create mode 100644 libasan/interception/interception_win.cc
> create mode 100644 libasan/interception/interception_win.h
> create mode 100644 libasan/libtool-version
> create mode 100644 libasan/ltmain.sh
> create mode 100644 libasan/missing
> create mode 100644 libasan/sanitizer_common/sanitizer_allocator.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_allocator64.h
> create mode 100644 libasan/sanitizer_common/sanitizer_atomic.h
> create mode 100644 libasan/sanitizer_common/sanitizer_atomic_clang.h
> create mode 100644 libasan/sanitizer_common/sanitizer_atomic_msvc.h
> create mode 100644 libasan/sanitizer_common/sanitizer_common.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_common.h
> create mode 100644 libasan/sanitizer_common/sanitizer_flags.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_flags.h
> create mode 100644 libasan/sanitizer_common/sanitizer_internal_defs.h
> create mode 100644 libasan/sanitizer_common/sanitizer_libc.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_libc.h
> create mode 100644 libasan/sanitizer_common/sanitizer_linux.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_list.h
> create mode 100644 libasan/sanitizer_common/sanitizer_mac.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_mutex.h
> create mode 100644 libasan/sanitizer_common/sanitizer_placement_new.h
> create mode 100644 libasan/sanitizer_common/sanitizer_posix.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_printf.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_procmaps.h
> create mode 100644 libasan/sanitizer_common/sanitizer_stackdepot.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_stackdepot.h
> create mode 100644 libasan/sanitizer_common/sanitizer_stacktrace.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_stacktrace.h
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer.h
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer_linux.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer_mac.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer_win.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_win.cc
>
Dodji,
The Google branch is missing the required
interception/mach_override/mach_override.h
and interception/mach_override/mach_override.c files from compiler-rt svn for
darwin. I have
posted what I believe to be the final patch which eanbles libsanitizer on
darwin...
http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01285.html
which has been tested with the existing asan testsuite, the use-after-free.c
testcase as
well as the Polyhedron 2005 benchmarks for -O1 -g -fno-omit-frame-pointer
-faddress-sanitizer
and -O3 -funroll-loops -ffast-math -g -fno-omit-frame-pointer
-faddress-sanitizer
to prove that the current mach_override from upstream is sufficient for darwin
to use.
Due to the large number of maintainers for libsanitizer, it is unclear who is
the person
responsible for upstream merges to lobby for these files to be ported into gcc
trunk.
With Alexander Potapenko's commit of the bug fix to
mach_override/mach_override.c
required for FSF gcc...
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20121112/155989.html
...there really is no reason to continue to delay (as the interpose code simply
won't
be completed in time for gcc 4.8.0). Can we please get some movement on
importing
these missing files from upstream? Thanks.
Jack
