[Python-Dev] valgrind
Hi! I've embedded python into my application. Using valgrind I got a lot of errors. I understand that "Conditional jump or move depends on uninitialised value(s)" errors are completely ok (from Misc/README.valgrind). However, I don't understand why "Invalid read"'s are legal, like this: ==21737== Invalid read of size 4 ==21737==at 0x408DDDF: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==21737==by 0x4096F67: (within /usr/lib/libpython2.4.so.1.0) ==21737==by 0x408A5AC: PyCFunction_Call (in /usr/lib/libpython2.4.so.1.0) ==21737==by 0x40C65F8: PyEval_EvalFrame (in /usr/lib/libpython2.4.so.1.0) ==21737== Address 0xC02E010 is 32 bytes inside a block of size 40 free'd ==21737==at 0x401D139: free (vg_replace_malloc.c:233) ==21737==by 0x408DE00: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==21737==by 0x407BB4D: (within /usr/lib/libpython2.4.so.1.0) ==21737==by 0x407A3D6: (within /usr/lib/libpython2.4.so.1.0) Here python reads from an already-freed memory area, right? (I don't think that Misc/README.valgrind answers this question). Or is it a false alarm? Thanks, Geza Herman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] valgrind
On Tue, 7 Nov 2006, Tim Peters wrote: > When PyObject_Free is handed an address it doesn't control, the "arena > base address" it derives from that address may point at anything the > system malloc controls, including uninitialized memory, memory the > system malloc has allocated to something, memory the system malloc has > freed, or internal system malloc bookkeeping bytes. The > Py_ADDRESS_IN_RANGE macro has no way to know before reading it up. > > So figure out which line of code valgrind is complaining about > (doesn't valgrind usually produce that?). If it's coming from the > expansion of Py_ADDRESS_IN_RANGE, it's not worth more thought. Hmm. I don't think that way. What if free() does other things? For example if free(addr) sees that the memory block at addr is the last block then it may call brk with a decreased end_data_segment. Or the last block in an mmap'd area - it calls unmap. So when Py_ADDRESS_IN_RANGE tries to read from this freed memory block it gets SIGSEGV. However, I've never got SIGSEGV from python. I don't really think that reading from an already-freed block is ever legal. I asked my original question because I saw that I'm not the only one who sees "Illegal reads" from python. Is valgrind wrong in this case? I just want to be sure that I'll never get SIGSEGV from python. Note that Misc/valgrind-python.supp contains suppressions "Invalid read"'s at Py_ADDRESS_IN_RANGE. Geza Herman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] valgrind
> > For example > > if free(addr) sees that the memory block at addr is the last block then it > > may call brk with a decreased end_data_segment. > > It can't. In brk, you can only manage memory in chunks of "one page" > (i.e. 4kiB on x86). Since we only access memory on the same page, > access is guaranteed to succeed. Yes, I'm aware of it. But logically, it is possible, isn't it? At malloc(), libc recognizes that brk needed, it calls sbrk(4096). Suppose that python releases this very same block immediately. At free(), libc recognizes that sbrk(-4096) could be executed, so the freed block not available anymore (even for reading) > > Or the last block > > in an mmap'd area - it calls unmap. So when Py_ADDRESS_IN_RANGE tries > > to read from this freed memory block it gets SIGSEGV. However, I've never > > got SIGSEGV from python. > > Likewise. This is guaranteed to work, by the processor manufacturers. The same: if the freed block is the last one in the mmap'd area, libc may unmap it, doesn't it? > > I don't really think that reading from an already-freed block is ever > > legal. > > Define "legal". There is no law against it; you don't go to jail for > doing it. What other penalties would you expect (other than valgrind > spitting out error messages, and users complaining from time to time > that it's "illegal")? Ok, sorry about the strong word "legal". > > I asked my original question because I saw that I'm not the only > > one who sees "Illegal reads" from python. Is valgrind wrong in this case? > > If it is this case, then no, valgrind is right. Notice that valgrind > doesn't call them "illegal"; it calls them "invalid". > > > I just want to be sure that I'll never get SIGSEGV from python. > > You least won't get SIGSEGVs from that part of the code. That's what I still don't understand. If valgrind is right then how can python be sure that it can still reach a freed block? > > Note that Misc/valgrind-python.supp contains suppressions "Invalid read"'s > > at Py_ADDRESS_IN_RANGE. > > Right. This is to tell valgrind that these reads are known to work > as designed. Does this mean that python strongly depends on libc? If I want to port python to another platform which uses a totally different malloc, is Py_ADDRESS_IN_RANGE guaranteed to work or do I have to make some changes? (actually I'm porting python to another platfrom that's why I'm asking these questions, not becaue I'm finical or something) Thanks, Geza Herman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] valgrind
Thanks Martin, now everything is clear. Python always reads from the page where the about-to-be-freed block is located (that was the information that I missed) - as such, never causes a SIGSEGV. Geza Herman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
