Re: Uninit warnings due to optimizing short-circuit conditionals
Hi Richard,
On Tue, 2022-02-15 at 08:25 +0100, Richard Biener wrote:
> On Mon, Feb 14, 2022 at 6:38 PM Mark Wielaard wrote:
> > Yes. valgrind keeps track of uninitialized bits and propagates them
> > around till "use". Where use is anything that might alter the
> > observable behavior of the program. Which is control flow
> > transfers, conditional moves, addresses used in memory accesses,
> > and data passed to system calls.
> >
> > This paper describes some of the memcheck tricks:
> > https://valgrind.org/docs/memcheck2005.pdf
>
> That probably means bugs like
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63311
> could be resolved as fixed (in valgrind).
I just tried the testcase from that bug and it still replicates with
gcc 11.2.1 and valgrind 3.18.1. And as far as I can see it really
cannot be fixed in valgrind since gcc really generates a conditional
jump based on an uninit variable in this case.
It does look a bit like what Julian described in:
Memcheck Reloaded
dealing with compiler-generated branches on undefined values
https://archive.fosdem.org/2020/schedule/event/debugging_memcheck_reloaded/
Which should be able to recover/reconstruct the original control flow.
In cases like:
int result
bool ok = compute_something(&result)
if (ok && result == 42) { ... }
where gcc turns that last line upside down:
if (result == 42 && ok) { ... }
But it doesn't work in this case. Probably because this is a slightly
more complex case involving 3 distinct variables instead of 2.
Cheers,
Mark
Request easy bug fix
Hi, I hope it is not inappropriate to call attention to a specific bug. https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85487. I tried to do it myself, but got lost on the part where I needed to compile gcc 3 times and compare test results to some (un)known test results. It was too much time and setup for a fix that will probably take 2 minutes to implement, so I am asking if someone that already contributes to gcc can please look at this. I think it should only take a couple of minutes to implement. Thank you
Re: Request easy bug fix
On Tue, 15 Feb 2022 at 12:34, Baruch Burstein via Gcc wrote: > Hi, > > I hope it is not inappropriate to call attention to a specific bug. > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85487. > I tried to do it myself, but got lost on the part where I needed to > compile gcc 3 times and compare test results to some (un)known test > results. You compare the test results of your patched gcc to the test results of an unpatched gcc. How did you get lost? To "compile gcc 3 times" you just run "make", and it does that automatically (unless you configured with --disable-bootstrap, in which case it just compiles once). > It was too much time and setup for a fix that will probably > take 2 minutes to implement, so I am asking if someone that already > contributes to gcc can please look at this. I think it should only > take a couple of minutes to implement. > No change ever takes two minutes, because you have to write tests at a minimum.
Re: Uninit warnings due to optimizing short-circuit conditionals
Sorry for the delayed response. I've been paging this all back in.
I first saw this problem when memcheck-ing Firefox as compiled by Clang, some
years back. Not long after GCC was also at it. The transformation in
question is (at the C level):
A && B ==> B && A if it can be proved that A
is always false whenever B is undefined
and (I assume) that B is provably exception-free
where && means the standard lazy left-first C logical-AND. I believe this
might have become more prevalent due to ever-more aggressive inlining (at
least for Firefox), which presented the compilers with greater opportunities
to make the required proofs.
After wondering what to do about this viz-a-viz Memcheck, I realised after a
while that, because Memcheck does know how to do exact definedness propagation
through bitwise and/or, I could "fix" it by recovering the underlying &&
expression through analysis of local fragments of the control flow graph. So
it now looks for
if !A goto X
if !B goto X
X:
and generates IR for analysis as if it had seen
"if (A bitwise-& B) ".
Note that the bitwise vs logical-AND distinction isn't really correct; what
we're really talking about is non-lazy vs lazy AND. The number of bits
involved in the representation is irrelevant.
A couple of other notes:
* Valgrind only deals with 2-argument &&s; that is all that seemed necessary.
In principle though the transformation generalises to any number of terms
&&-ed together, not just 2.
* For reasons I don't really remember now, I didn't need to deal with the
equivalent OR case. It's all the same at the machine code level. (Waves
hands and mumbles something about De Morgan ..)
I'm sure all the above info is in the slides of the Fosdem talk that Mark
mentioned. I don't think the above contributes anything new.
On Tue, Feb 15, 2022 at 1:29 PM Mark Wielaard wrote:
>
> Hi Richard,
>
> On Tue, 2022-02-15 at 08:25 +0100, Richard Biener wrote:
> > On Mon, Feb 14, 2022 at 6:38 PM Mark Wielaard wrote:
> > > Yes. valgrind keeps track of uninitialized bits and propagates them
> > > around till "use". Where use is anything that might alter the
> > > observable behavior of the program. Which is control flow
> > > transfers, conditional moves, addresses used in memory accesses,
> > > and data passed to system calls.
> > >
> > > This paper describes some of the memcheck tricks:
> > > https://valgrind.org/docs/memcheck2005.pdf
> >
> > That probably means bugs like
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63311
> > could be resolved as fixed (in valgrind).
>
> I just tried the testcase from that bug and it still replicates with
> gcc 11.2.1 and valgrind 3.18.1. And as far as I can see it really
> cannot be fixed in valgrind since gcc really generates a conditional
> jump based on an uninit variable in this case.
>
> It does look a bit like what Julian described in:
>
> Memcheck Reloaded
> dealing with compiler-generated branches on undefined values
> https://archive.fosdem.org/2020/schedule/event/debugging_memcheck_reloaded/
>
> Which should be able to recover/reconstruct the original control flow.
> In cases like:
>
> int result
> bool ok = compute_something(&result)
> if (ok && result == 42) { ... }
>
> where gcc turns that last line upside down:
>
> if (result == 42 && ok) { ... }
>
> But it doesn't work in this case. Probably because this is a slightly
> more complex case involving 3 distinct variables instead of 2.
>
> Cheers,
>
> Mark
Re: Uninit warnings due to optimizing short-circuit conditionals
On Tue, Feb 15, 2022 at 2:00 PM Julian Seward wrote: > > Sorry for the delayed response. I've been paging this all back in. > > I first saw this problem when memcheck-ing Firefox as compiled by Clang, some > years back. Not long after GCC was also at it. The transformation in > question is (at the C level): > > A && B ==> B && A if it can be proved that A > is always false whenever B is undefined > and (I assume) that B is provably exception-free > > where && means the standard lazy left-first C logical-AND. I believe this > might have become more prevalent due to ever-more aggressive inlining (at > least for Firefox), which presented the compilers with greater opportunities > to make the required proofs. Note GCC doesn't try to prove this, instead it reasons that when B is undefined it takes an indeterminate value and if A is _not_ always false then the program would have invoked undefined behavior, so we can disregard this possibility and assume B is not undefined. So either B is not undefined and everything is OK, or B is undefined but then A must be always false. Note that when A is always false we may have transformed a valid program (does not access B) into a program invoking undefined behavior (in C language terms). We don't treat undefined uses as "very" undefined behavior but I do remember we've shot ourselves in the foot with this transform - in this case we'd have to make the use of B determinate somehow, something we cannot yet do. So we'd want a transform like A && B ==> OK(B) && A where 'OK' sanitizes B in case it is undefined. The error we can run into is that two of the uninit Bs can be equated to two different values, breaking the B == B invariant (technically also OK, but not if it was us that introduced the undefinedness in the first place). Richard.
Re: Request easy bug fix
On Tue, 2022-02-15 at 12:55 +, Jonathan Wakely via Gcc wrote: > On Tue, 15 Feb 2022 at 12:34, Baruch Burstein via Gcc < > gcc@gcc.gnu.org> > wrote: > > > Hi, > > > > I hope it is not inappropriate to call attention to a specific bug. > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85487. > > I tried to do it myself, but got lost on the part where I needed to > > compile gcc 3 times and compare test results to some (un)known test > > results. Thanks for trying to fix the bug. > > > You compare the test results of your patched gcc to the test results > of an > unpatched gcc. > > How did you get lost? To "compile gcc 3 times" you just run "make", > and it > does that automatically (unless you configured with --disable- > bootstrap, in > which case it just compiles once). Jonathan, if I may: you're extremely familiar with hacking on GCC, and I think that familiarity is leading you to underestimate the learning curve for someone new getting involved in GCC development. As you say, --disable-bootstrap is the configure-time option to use when working on a new patch, since it avoids the "compile 3 times" cycle. We could probably document that better. > > > > It was too much time and setup for a fix that will probably > > take 2 minutes to implement, so I am asking if someone that already > > contributes to gcc can please look at this. I think it should only > > take a couple of minutes to implement. Baruch: here you are underestimating the time that adding a new feature takes; yes, it perhaps could take about 2 minutes to get a minimal proof-of-concept working, but once you start adding documentation, test-cases, etc it becomes more than that. Also, looking at the discussion now happening in the bug report, it's not clear that the absolute minimum implementation is the correct one I'm guessing that you care because you're working in a mixed Visual Studio/GCC environment, and have a codebase with these pragmas. Does Visual Studio complain about mismatches, or incorrect nesting? If so, can you give some more information about these interoperability issues being discussed in the bug report? (I used to work in such an environment, but that was over 20 years ago; my knowledge of Visual Studio is *very* out of date, sorry) > > > > No change ever takes two minutes, because you have to write tests at > a > minimum. Hope this is constructive Dave
Benchmark recommendations needed
Hello all! I'm working on porting GCC to a new processor architecture. I think I've finally got to a fairly stable stage, so the next logical step would be to test and optimize. For that, I would need some benchmarks, and this is where I'm seeking your help. This being a hobby project, I can't shell out $1000+ for the spec suite. On top of that, I only have newlib ported as the runtime, which means very limited support for OS facilities. I already have dhrystone, but what else would you recommend using? I'm looking for 'general purpose' payloads, things, where I can judge object code size, instruction set utilization, look for tuning opportunities, etc. I would like to also be able to compare the results with other architectures (FPGA cores, such as nios2 as well as some low-end cores, such as 32-bit arm/thumb and riscv-RV32IMFC). So, can you suggest some benchmarks or applications to be used as ones? Thanks a bunch! Andras
Re: Benchmark recommendations needed
On Tue, Feb 15, 2022 at 3:25 PM Andras Tantos wrote: > > Hello all! > > I'm working on porting GCC to a new processor architecture. I think > I've finally got to a fairly stable stage, so the next logical step > would be to test and optimize. For that, I would need some benchmarks, > and this is where I'm seeking your help. > > This being a hobby project, I can't shell out $1000+ for the spec > suite. On top of that, I only have newlib ported as the runtime, which > means very limited support for OS facilities. > > I already have dhrystone, but what else would you recommend using? > > I'm looking for 'general purpose' payloads, things, where I can judge > object code size, instruction set utilization, look for tuning > opportunities, etc. > > I would like to also be able to compare the results with other > architectures (FPGA cores, such as nios2 as well as some low-end cores, > such as 32-bit arm/thumb and riscv-RV32IMFC). > > So, can you suggest some benchmarks or applications to be used as ones? There's scimark and nbench and for scientific there's fortran polyhedron. The botan crypto library also has some extensive benchmark setup for crypto kernels. Richard. > Thanks a bunch! > Andras > >
Re: Request easy bug fix
On Tue, 15 Feb 2022 at 13:58, David Malcolm wrote: > On Tue, 2022-02-15 at 12:55 +, Jonathan Wakely via Gcc wrote: > > On Tue, 15 Feb 2022 at 12:34, Baruch Burstein via Gcc < > > gcc@gcc.gnu.org> > > wrote: > > > > > Hi, > > > > > > I hope it is not inappropriate to call attention to a specific bug. > > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85487. > > > I tried to do it myself, but got lost on the part where I needed to > > > compile gcc 3 times and compare test results to some (un)known test > > > results. > > Thanks for trying to fix the bug. > > > > > > > You compare the test results of your patched gcc to the test results > > of an > > unpatched gcc. > > > > How did you get lost? To "compile gcc 3 times" you just run "make", > > and it > > does that automatically (unless you configured with --disable- > > bootstrap, in > > which case it just compiles once). > > Jonathan, if I may: you're extremely familiar with hacking on GCC, and > I think that familiarity is leading you to underestimate the learning > curve for someone new getting involved in GCC development. > > As you say, --disable-bootstrap is the configure-time option to use > when working on a new patch, since it avoids the "compile 3 times" > cycle. We could probably document that better. > Agreed. So it would help to know which docs Baruch was looking at when getting lost. There's no point adding *more* docs if we already have some that are doing more harm than good. > > > > > > > > It was too much time and setup for a fix that will probably > > > take 2 minutes to implement, so I am asking if someone that already > > > contributes to gcc can please look at this. I think it should only > > > take a couple of minutes to implement. > > Baruch: here you are underestimating the time that adding a new feature > takes; yes, it perhaps could take about 2 minutes to get a minimal > proof-of-concept working, but once you start adding documentation, > test-cases, etc it becomes more than that. Also, looking at the > discussion now happening in the bug report, it's not clear that the > absolute minimum implementation is the correct one > > I'm guessing that you care because you're working in a mixed Visual > Studio/GCC environment, and have a codebase with these pragmas. > Does Visual Studio complain about mismatches, or incorrect nesting? > If so, can you give some more information about these interoperability > issues being discussed in the bug report? > (I used to work in such an environment, but that was over 20 years ago; > my knowledge of Visual Studio is *very* out of date, sorry) > Clang and the MSVC compiler both ignore any tokens after the pragma, so that seems good enough for GCC too: https://godbolt.org/z/norv947a5
Invitation to join globaldce-newsletter
Hi gcc@gcc.gnu.org, han.zhang...@gmail.com invited you to join the globaldce-newsletter group. https://groups.google.com/d/forum/globaldce-newsletter Message from han.zhang...@gmail.com: You are hereby invited to join globaldce-newsletter googlegroup to get regular updates on our giveaways, events and software releases from the globaldce project: https://github.com/globaldce About this group: A googlegroup for discussions around the global decentralized collaboration environment. Google Groups allows you to create and participate in online forums and email-based groups with a rich community experience. You can also use your Group to share documents, pictures, calendars, invitations, and other resources. Accept this invitation at: https://groups.google.com/forum/subscribe?token=AHZ7KVOLz4HMYzFBl_wZ1Jnbdxb2kksgxtCCsjqBh2UGbe3ZOZLUhG0P439lfo-AZhpPKTw7WBa5DoUpeq-VsJ_acmaZBP1Fww If you do not wish to be a member of this group or believe this group may contain spam: * You can report this group for abuse at https://groups.google.com/d/abuse/AJmrmCtutnVg6-QvlGAjK18NPxKY_ERl8o86H60WrgHgpTWwW7Ps53e-RBCl_EEh0ggSNY52qvRgim-fPjC2jnKu-OuVy-WI1GvpukK7XHWAX4EQbiSuxgw * You can opt out of all future Google Groups activity at https://groups.google.com/d/optout Visit Google Groups Help Center at https://support.google.com/groups/answer/46601?hl=en.
Re: Uninit warnings due to optimizing short-circuit conditionals
On Tue, 2022-02-15 at 14:28 +0100, Richard Biener wrote: > On Tue, Feb 15, 2022 at 2:00 PM Julian Seward > wrote: > > > > Sorry for the delayed response. I've been paging this all back in. > > > > I first saw this problem when memcheck-ing Firefox as compiled by > > Clang, some > > years back. Not long after GCC was also at it. The transformation > > in > > question is (at the C level): > > > > A && B ==> B && A if it can be proved that A > > is always false whenever B is undefined > > and (I assume) that B is provably exception- > > free > > > > where && means the standard lazy left-first C logical-AND. I > > believe this > > might have become more prevalent due to ever-more aggressive > > inlining (at > > least for Firefox), which presented the compilers with greater > > opportunities > > to make the required proofs. > > Note GCC doesn't try to prove this, instead it reasons that when > B is undefined it takes an indeterminate value and if A is _not_ > always > false then the program would have invoked undefined behavior, so we > can disregard this possibility and assume B is not undefined. So > either B is not undefined and everything is OK, or B is undefined but > then A must be always false. > > Note that when A is always false we may have transformed a valid > program (does not access B) into a program invoking undefined > behavior > (in C language terms). We don't treat undefined uses as "very" > undefined > behavior but I do remember we've shot ourselves in the foot with this > transform - in this case we'd have to make the use of B determinate > somehow, something we cannot yet do. So we'd want a transform > like > > A && B ==> OK(B) && A > > where 'OK' sanitizes B in case it is undefined. The error we can run > into > is that two of the uninit Bs can be equated to two different values, > breaking the B == B invariant (technically also OK, but not if it was > us > that introduced the undefinedness in the first place). > > Richard. Thanks everyone for the various insights. I've gone ahead and committed my workaround for the -fanalyzer uninit false positive to trunk (as r12-7251- g1e2fe6715a949f80c1204ae244baad3cd80ffaf0). Dave
