Re: US-CERT Vulnerability Note VU#162289
BTW,
It occurs to me that the optimisation is just as likely (if not more
likely) to remove security holes as it is to introduce them.
If someone writes comparison involving 'buf+len' because they
incorrectly ignored a possibility of 'buf+len' wrapping, then
the optimisation fixes, not breaks, the code.
My evidence-free opinion is that this scenario is in fact more likely
than the apparently contrived example in the CERT note.
Ralph.
PS The CERT note refers to "existing programs" broken by the
optimisation. I wrote to their feedback address asking about the
"existing programs" and heard nothing back, which makes me suspect that
the authors of the note don't know of any, despite their claim.
> The GCC SC was aware of this CERT posting before it was public. Our
> feeling is that this is not a GCC bug, although it is something that
> we would like GCC to warn about. I talked to Ian Taylor and he
> agreed to work on the warning.
>
> Some other compilers do this optimization too, so compiling similar
> applications with them will also lead to the removal of the test.
> We've identified one specific highly-optimizing compiler that does
> this optimization. If you have information about other such
> compilers, please send it to me, and I will forward to CERT. We
> would like to counter the idea that GCC is unique in this regard.
> (In fact, I expect that the reason that this issue was found with GCC
> is that GCC is used so widely!)
>
> Here is the test program:
>
> int f(char *buf, int len) {
>len = 1 << 30;
>if (buf + len < buf)
> return 1;
>
>return 0;
> }
>
> If you know of a non-GCC compiler that optimizes away the test (so
> that the function always returns 0), please post here, and let me
> know the name, version number, command-line options, etc. you used to
> demonstrate that.
>
> Thanks,
>
Security vulernarability or security feature?
Dear "Cert", I originally raised this with you privately, but you are slow to respond, so I am raising this again, more concisely, and CC'd to a less private forum. (a) Arithmetic overflows have historically been a significant source of security vulnerabilities. (b) Recent versions of gcc (along with other compilers) contain an optimisation that can *REMOVE* arithmetic overflows. Why is Cert advising people to avoid an optimisation that can --- realistically, although probably rarely --- remove security vulnerabilities? [I also note that the example you claim is a "length check" in your advisory, is nothing of the sort. It is an oddly written test of the absolute position of a pointer. I don't actually see how the optimisation in question could remove a check on the length of something. And even more, I don't see how such a hypothetical length check could not also avoid being broken by other 101 other things, such as variations in OS memory layout, which may vary even between successive runs of identical binarys] Ralph.
Re: Security vulernarability or security feature?
> I am very interested in seeing how this optimization can remove
> arithmetic overflows.
int foo (char * buf, int n)
{
// buf+n may overflow of the programmer incorrectly passes
// a large value of n. But recent versions of gcc optimise
// to 'n < 100', removing the overflow.
return buf + n < buf + 100;
}
Compiled on i386, gcc-4.3.0 with -O2 gives:
foo:
xorl%eax, %eax
cmpl$99, 8(%esp)
setle %al
ret
E.g., calling foo with:
#include
int main()
{
char buf[100];
printf ("%d\n", foo (buf, 15));
return 0;
}
on my PC (where the stack is just below the 3Gig position).
> > Why is Cert advising people to avoid an optimisation that can ---
> > realistically, although probably rarely --- remove security
> > vulnerabilities?
> >
> If you are referring to VU#694123, this refers to an optimization
I'm talking about 162289.
Ralph.
> that removes checks pointer arithmetic wrapping. The optimization
> doesn't actually eliminate the wrapping behavior; this still occurs.
> It does, however, eliminate certain kinds of checks (that depend upon
> undefined behavior).
>
> Thanks,
> rCs
Re: Security vulernarability or security feature?
Robert, You have failed to answer my original question, and I think have failed to understand the point of the example. The example shows that what you are claiming is a vulnerability in 162289 is in fact a security feature. > that removes checks pointer arithmetic wrapping. Just to be 100% clear, the optimisation I gave you an example of, and which you think is "pretty cool" is precisely the same optimisation you are complaining about in 162289, specifically, a pointer value may be canceled from both sides of a comparison. This is slightly confused by the fact that in the 162289 example, two optimisations take place [gcc gurus please correct if I am wrong]: (a) a pointer value is canceled from both sides of a comparison, changing (buf+len The optimization > doesn't actually eliminate the wrapping behavior; this still occurs. > It does, however, eliminate certain kinds of checks (that depend upon > undefined behavior). > > Thanks, > rCs >
Re: Security vulernarability or security feature?
> This clearly is insecure coding.
Yes, it was intended to be an example of what gcc does with bad code.
> When such assumptions are made ... the compiler may eliminate the bug
> in some cases giving the programmer a false feeling that "Oh! My code
> is bug free". The problem is that when the code is compiled on a
> different platform, with different switches, the bug may reappear.
Yes. I was not intending to advocate reliance on the optimisation. I
was attempting to illustrate that things are not as simple as Cert is
claiming.
> I
> just wanted to bring out the point about the assumption ... may be a
> diagnostic should be issued.
I believe this is being done as we talk.
Ralph.
Or the compiler is smart enough to figure
> out the values of "buf" and only optimise on cases which are safe.
>
> >
> > Compiled on i386, gcc-4.3.0 with -O2 gives:
> >
> > foo:
> > xorl%eax, %eax
> > cmpl$99, 8(%esp)
> > setle %al
> > ret
> >
> > E.g., calling foo with:
> >
> > #include
> > int main()
> > {
> > char buf[100];
> > printf ("%d\n", foo (buf, 15));
> > return 0;
> > }
> >
> > on my PC (where the stack is just below the 3Gig position).
> >
> >
> >
> > > > Why is Cert advising people to avoid an optimisation that can
> > > > --- realistically, although probably rarely --- remove security
> > > > vulnerabilities?
> > > >
> > > If you are referring to VU#694123, this refers to an optimization
> >
> > I'm talking about 162289.
> >
> > Ralph.
> >
> >
> >
> > > that removes checks pointer arithmetic wrapping. The
> > > optimization doesn't actually eliminate the wrapping behavior;
> > > this still occurs. It does, however, eliminate certain kinds of
> > > checks (that depend upon undefined behavior).
> > >
> > > Thanks,
> > > rCs
> >
Re: Security vulernarability or security feature? VU#162289
> Thanks for your further explanation of this optimization. Here is
> what I understand. Please correct me if I am wrong on any of these
> points:
Points 1...5, no quibble.
Now that you appear to admit that the issue is wrap-around and not a
length-check, this raises the question of whether there is any security
problem with the optimisation at all, even for badly written
applications.
Length-checks are directly related to security, because they protect
against buffer-overruns which are often directly exploited by attackers.
It is much harder to see how reliance on wrap-around could contribute to
the security of an application. Where pointer arithmetic is invalid,
and this is carried out in context where security matters, I would
expect it to not matter whether or not invalid arithmetic used
wrap-around or not.
Of course, for just about any compiler feature, it is probably possible
to contrive badly written code that is exploitably insecure with that
feature, and --- purely by chance --- not exploitable without the
feature.
But it is hard to see what reason you have for picking on this
particular feature of this particular compiler.
> 6. The Overview of VU#162289 states "Some C compilers optimize away
> pointer arithmetic overflow tests that depend on undefined behavior
> without providing a diagnostic (a warning)."
>
> The point being is that we are not objecting to the optimization, we
> are objecting to the lack of any diagnostic.
Yes, you have dramatically improved the wording from previous
versions. However, you still say:
"avoid using compiler implementations that perform the offending
optimization"
without any warning that in some cases the "offending" optimization may
be covering-up a security issues, and that avoiding the optimization
without fixing the underlying problems in the application may actually
be harmful. (In fact, as far as I can, more likely to be harmful than
helpful).
I must admit, given the problems that have been identified with the
advisory (how many versions has it been through?), I would be far
happier if you subjected it to independent third-party expert review,
and withdrew the advisory until that is completed in a satisfactory
manner (rather than repeatedly incrementally tweaks).
Ralph.
>
> Because both examples could benefit from being rewritten (this is
> what the compiler is doing) I don't see the harm in issuing a
> diagnostic in both cases.
>
> rCs
>
> > Robert,
> >
> > You have failed to answer my original question, and I think have
> > failed to understand the point of the example.
> >
> > The example shows that what you are claiming is a vulnerability in
> > 162289 is in fact a security feature.
> >
> >
> >> that removes checks pointer arithmetic wrapping.
> >>
> >
> > Just to be 100% clear, the optimisation I gave you an example of,
> > and which you think is "pretty cool" is precisely the same
> > optimisation you are complaining about in 162289, specifically, a
> > pointer value may be canceled from both sides of a comparison.
> >
> > This is slightly confused by the fact that in the 162289 example,
> > two optimisations take place [gcc gurus please correct if I am
> > wrong]:
> >
> > (a) a pointer value is canceled from both sides of a comparison,
> > changing (buf+len > observable behaviour of the code, and is what the debate is about.
> >
> > (b) in the example given in 162289, (len<0) can then be evaluated at
> > compile time, removing the test entirely. This does not change the
> > runtime behaviour of the code an is completely irrelevant.
> >
> > To make it even clearer, we can disable optimisation (b) while
> > leaving optimisation (a), by making 'len' volatile:
> >
> > int foo (char * buf)
> > {
> > volatile int len = 1<<30;
> > if (buf+len < buf)
> > return 1;
> > else
> > return 0;
> > }
> >
> > gcc then generates:
> >
> > foo:
> > subl$16, %esp
> > movl$1073741824, 12(%esp)
> > movl12(%esp), %eax
> > addl$16, %esp
> > shrl$31, %eax
> > ret
> >
> > This has not completely removed the test, but when executed, it will
> > still always return 0, giving precisely the same run-time behaviour
> > as without the 'volatile'.
> >
> > To re-iterate:
> >
> > (a) Why does Cert advise against an optimisation that, under
> > the right circumstances, can remove examples of a historically
> > significant class of security holes.
> >
> > (b) The example you claim is a length check in 162289 is not a
> > length check and does not support the conclusions you draw from it.
> >
> > Ralph.
> >
> >
> >
> >> The optimization
> >> doesn't actually eliminate the wrapping behavior; this still
> >> occurs. It does, however, eliminate certain kinds of checks (that
> >> depend upon undefined behavior).
> >>
> >> Thanks,
> >> rCs
> >>
> >>
>
