I was using valgrind and found an out of bounds error reading 8 bytes off an array of 3 byte data structures where the extra 5 bytes being read were out of the array bounds. I attached a program that ends the array at the end of a page so reading beyond the end of the array would cause a crash, and this is. x86-64 crashes, x86-32 doesn't.
Before I file a bug report I wanted to check to see if my expectations are wrong or if this is a compiler bug. Is there anything that allows the compiler to generate instructions that would read beyond the end of an array potentially causing a crash if the page isn't accessible? Or is this program somehow invalid? tested gcc and g++ 4.7.2 and from svn, gcc (GCC) 4.9.0 20140221 (experimental) While both lines read an array entry, only the second crashes. dup = c[i]; fun(c[i]); The attached program sets up and reads through the array with extra padding at the of the array from 8 bytes to 0 bytes. Padding from 4 to 0 crashes. There are some #defines to make it easy to use malloc vs mmap, use munmap or mprotect to make the next page not accessible. -- David Fries <da...@fries.net> PGP pub CB1EE8F0 http://fries.net/~david/
//#define __USE_MISC //#define __USE_GNU #include <sys/mman.h> #include <sys/stat.h> #include <fcntl.h> #include <string.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> typedef struct { char r, g, b; /* char a; char w; */ } rgb; static char rtn; char fun(rgb r) { rtn = r.r/2 + r.g/3 + r.b/3; return rtn; } //#define USE_MALLOC #define SPECIFY_ADDRESS //#define MPROTECT /* g++ 4.7.2 generates an 8 byte read for rgb in fun(c[i]); which for the * last 2 pixels (3 bytes each), is out of the array bounds. * valgrind will flag the problem in all cases. * just mmap runs without crashing, /proc/pid/maps shows there happens to * be an adjacent mapping and additional mmaps expands down * with SPECIFY_ADDRESS, it maps, unmaps, and maps one page down to leave * an whole, which results in a crash * with MPROTECT out of bounds is protected and it crashes * Linux 3.2.0 x86-64 * libc6 2.13 * g++ 4.7.2 and 4.9.0 20140221 (experimental) * * movq (%rax), %rdi * call _Z3fun3rgb */ char test(int buffer) { const int page = sysconf(_SC_PAGE_SIZE); const int count = 3216360; printf("buffer %d\n", buffer); #ifdef USE_MALLOC rgb *c = (rgb*)malloc(sizeof(rgb) * count + buffer); #else // 3 extra pages worth, 1 rounding, 1 mprotect, 1 align size_t size = (sizeof(rgb)*count + buffer + 3*page)/page * page; char *p = (char*)mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); #ifdef SPECIFY_ADDRESS char *p1 = p; munmap(p1, size); p = (char*)mmap(p1 - page, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); #endif char *none = NULL; #ifdef MPROTECT none = p + size - page; buffer += page; if(mprotect(none, page, PROT_NONE) == -1) perror("mprotect failed"); #endif rgb *c = (rgb*)(p + size - sizeof(rgb)*count - buffer); printf("mmap size %llu, start %p end %p rgb * %p protect at %p\n", (long long)size, p, p+size, c, none); #endif memset(c, 0, sizeof(rgb)*count); rgb dup; int i; for(i=0; i<count; ++i) { dup = c[i]; //fflush(stdout); fun(c[i]); //fflush(stdout); } #ifdef CLEANUP #ifdef USE_MALLOC free(c); #else munmap(p, size); #endif #endif return dup.r; } int main(int argc, char **argv) { if(argc == 2) test(atoi(argv[1])); int i; for(i=8; i>=0; --i) test(i); return rtn; }