[valgrind] [Bug 418756] New: MAP_FIXED_NOREPLACE mmap flag unsupported
https://bugs.kde.org/show_bug.cgi?id=418756 Bug ID: 418756 Summary: MAP_FIXED_NOREPLACE mmap flag unsupported Product: valgrind Version: 3.15 SVN Platform: Other OS: Linux Status: REPORTED Severity: normal Priority: NOR Component: general Assignee: jsew...@acm.org Reporter: st...@yandex.ru Target Milestone: --- SUMMARY valgrind doesn't support MAP_FIXED_NOREPLACE mmap flag, causing the programs that use it, to misbehave. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] New: uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 Bug ID: 398445 Summary: uninitialized memory false reports on shared memory Product: valgrind Version: unspecified Platform: Other OS: Linux Status: UNCONFIRMED Severity: normal Priority: NOR Component: memcheck Assignee: jsew...@acm.org Reporter: s...@list.ru Target Milestone: --- Created attachment 114868 --> https://bugs.kde.org/attachment.cgi?id=114868&action=edit test case Hello. The attached test-case demonstrates the problem. It creates 2 pages of shared memory, uninitializes one page and initializes another. Even though both pages contain valid values, valgrind doesn't understand shared memory and reports an error. I have the program that uses shared memory a lot, and gets hundreds of false errors from valgrind. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 Stas Sergeev changed: What|Removed |Added Version|unspecified |3.13.0 -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #2 from Stas Sergeev --- Created attachment 114889 --> https://bugs.kde.org/attachment.cgi?id=114889&action=edit 2-process test-case Hi, thanks for your reply. Here is the test-case that does the same thing but with 2 processes scheme. Same valgrind error. While the first test-case represents my real program more, I really see no difference. As for the difficulty of fixing this - yes, I perfectly understand it. While I can think of some ugly work-arounds (like, for example, checking if the values have changed, and consider them initialized in that case), nothing good can probably be done. How about downgrading the error to warning if it comes from the shared mem? -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #4 from Stas Sergeev --- Created attachment 114906 --> https://bugs.kde.org/attachment.cgi?id=114906&action=edit 2 processes and VALGRIND_MAKE_MEM_DEFINED (In reply to Philippe Waroquiers from comment #3) > It is not very clear to me what is the reason > to map twice the same file. Mapping file twice may not be needed, but mapping shared memory twice (which is what my test-case does) allows you to create the mem aliases with different access rights. Consider writing the JIT. I am sure you, as a valgrind developer, write JIT every day actually. :) To trap self-modified code, you would execute the JITted code in PROT_EXEC window (no PROT_WRITE), and the JIT itself will access it via PROT_READ|PROT_WRITE window (very rough picture, it functions differently of course, but multiple windows with different perms are needed). But I would suggest to forget this exotic case and assume I use 2 processes as my second test-case does. This is because I simply don't see any difference. If you explain me why 2 processes+shmem are safe, I guess I'll deal with the rest on my own. Since valgrind doesn't know how the actions on shmem in one process affects another, it just exactly the same way doesn't know how writes within a single process can affect another mem window in the same process. > At my work, we are using shared memory between > several processes. We just declare the shared memory > as initialised just after mmap. See my next test-case where I did exactly that. I don't think it was needed, as AFAIK mmap()ed memory is always zeroed and valgrind knows this already, but I did that to follow your suggestion. > There is no concept of 'undefined error' and 'undefined warning', > and introducing this distinction seems not too desirable > (e.g. impact on suppression files, etc). Then you should at least document the possible sources of false-positives, including this one. I spent many hours trying to understand what valgrind is up to. A very long route was passed to track things back to the shared memory use (--track-origins just points to the stack allocation that didn't give me any clue). If that would have been documented, I would immediately understand that I hit exactly that case. > or disable error reporting for the mmap-ed address > range using > VALGRIND_DISABLE_ADDR_ERROR_REPORTING_IN_RANGE Thanks. If nothing else, I can do that. But since I don't want to deal with valgrind instrumentation headers, I can as well add an extra initializers just to silence down the errors. So basically I _will_ find the ways to silence down the errors, but getting things improved (or at least documented) on valgrind side would be a good result after my many-hours research of those false-positives. :) -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #5 from Stas Sergeev --- Think of the following use-case. Consider a very silly shm IPC that uses just one large struct with many input and output fields. Client fills in the output fields and copies the entire struct to the shm buffer. Server reads it, fills in the input fields and puts it back to shm. Client reads the struct back with all fields now properly initialized. valgrind in this scenario would still think some fields are uninitialized because it can't understand that server have already replaced the entire content in this buffer. This is what my test-case is trying to demonstrate. It doesn't matter whether the server is in the same process or another, as valgrind will behave the same way in both cases. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #8 from Stas Sergeev --- (In reply to Philippe Waroquiers from comment #7) > If mmap would necessarily zero out memory, then > that would create major difficulties to use > shared memory. > I think that what the kernel guarantees is that > the first process that creates new memory > with mmap gets zero-ed memory Yes but I don't think valgrind knows if this shm page is mapped first time or N-th time. It simply doesn't have such info, unless I am missing something obvious. So it should probably assume the memory is initialized. Or does it check if the mapping is MAP_SHARED and assumes uninitialized only in that case? In either case, my latest test-case does mmap() then VALGRIND_MAKE_MEM_DEFINED() then fork(). No second mapping any more. > Initialising yourself the memory via one ptr > but accessing it via another mapping > is not the same as declaring the memory > defined after *each* mmap. > In the first case, valgrind can just detect that > the memory at ptr1 address is initialised, > but it cannot see this initialisation via > the second mapping. Well, my last test-case does only one mmap() and then fork(). So even if I think hard I don't see how can I satisfy your current suggestion. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #9 from Stas Sergeev --- (In reply to Ivo Raisr from comment #6) > Yes, indeed. That's why we have syscall and ioctl wrappers in Valgrind. > They describe what the other side (kernel) expects from the buffers sent > there > and in what shape they come back with buffers received. > > It should be quite easy to extend this mechanism to an arbitrary > client-server protocol, be it via shm, sockets or any other IPC. I don't see a big use of it. Unlike the kernel ioctls, in this case users will have to create these protocol wrappers on their own for every prog. I don't think too many people would be interested in doing so. I think the mission of valgrind is to make it easier for the people to track the source of the problem. As in this case the source of the problem is valgrind itself, I think the adequate help for people would be the proper documentation. It will allow them to track the problems to valgrind instead of poking their own code in vain. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #11 from Stas Sergeev --- (In reply to Philippe Waroquiers from comment #10) > If you declare the memory initialised before forking, > using VALGRIND_MAKE_MEM_DEFINED() ... which is what my latest test-case already does. Please suggest how to improve the test-case. > But in any case, as you suggested, we should > improve the doc to mention the multi-mapping. > > Any suggestion where to put this information ? > (or asked otherwise, where did you search in > the doc ?) In google, of course. :) "valgrind memcheck false positives" or "valgrind uninitialized false positives" etc. google will find it in a wiki or in off-line docs, so whatever place you prefer, is only a question of the valgrind-established practices. google will find it anywhere. Since valgrind knows that the error comes from the shared mem, it can print more description itself, referring to the docs. In this case the user will certainly not miss it. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #13 from Stas Sergeev --- Will it help to print an additional warning when the uninitialized mem is copied to the shared region? The chances are very high this will result in a false-positives later, so maybe catching it at that stage would be better? If such warning is printed, valgrind may just consider them initialized from now on, to avoid the false-positive error later. What do you think about such strategy? -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #15 from Stas Sergeev --- (In reply to Philippe Waroquiers from comment #14) > So, IMO, the easiest is to use the client requests Which ones? I did all the modifications you asked for, they all in my latest test-case. > (the downside of this is that this implies modification of the code). No, that downside is that you didn't suggest any further modifications. :) -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 398445] uninitialized memory false reports on shared memory
https://bugs.kde.org/show_bug.cgi?id=398445 --- Comment #17 from Stas Sergeev --- (In reply to Philippe Waroquiers from comment #16) > That might not be straightforward to do, in particular if you have > multiple threads (you probably need a critical section around the > write operations) or processes (this will even be more complex, as you > will probably need a side channel to inform all other processes about what > was done). Exactly, that's the point. You need a side-channel that also knows about the relationship between the pointers to shm in different processes - something only the kernel knows about normally (a bit simpler if shm is mapped before fork() - then the pointers are equal, but this is not always the case). Thant's why I think VALGRIND_MAKE_MEM_DEFINED doesn't actually work for this case, and yet you suggested it all along the way here (and I was updating the test-case) and also in the docs. So I think there is some mistake. This all seems to suggest it might be better to enable the errors from shm with some extra command-line switch, leaving it disabled by default... -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 253657] missing some amd64 to let wine/amd64 run on valgrind/amd64
https://bugs.kde.org/show_bug.cgi?id=253657 --- Comment #16 from Stas Sergeev --- Created attachment 97737 --> https://bugs.kde.org/attachment.cgi?id=97737&action=edit a test case Sorry for delay. Here's the segregs test case. -- You are receiving this mail because: You are watching all bug changes.
