Re: Integer overflow in operator new
Joe Buck writes: >If a check were to be implemented, the right thing to do would be to throw >bad_alloc (for the default new) or return 0 (for the nothrow new). What do you do if the user has defined his own operator new that does something else? >There cases where the penalty for this check could have >an impact, like for pool allocators that are otherwise very cheap. >If so, there could be a flag to suppress the check. Excessive code size growth could also be problem for some programs. Ross Ridge
Re: x86 inc/dec on core2
Hello! > I was wondering, if: > > /* X86_TUNE_USE_INCDEC */ > ~(m_PENT4 | m_NOCONA | m_CORE2 | m_GENERIC), > > is correct. Should it be: > > /* X86_TUNE_USE_INCDEC */ > ~(m_PENT4 | m_NOCONA | m_GENERIC), > > ? inc/dec has the same performance as add/sub on Core 2 Duo. But inc/dec is shorter. What about partial flag register dependency of inc/dec? Uros.
Re: Integer overflow in operator new
* Karl Chen: > "4 * n", unchecked, is vulnerable to integer overflow. On IA-32, > "new int[0x4001]" becomes equivalent to "new int[1]". I've > verified this on gcc-2.95 through 4.1. For larger objects the > effects are exaggerated; smaller counts are needed to overflow. This PR19351, by the way. The most widespread interpretation of the standard is that conforming implementations aren't allowed to raise an exception in this case: the arithmetic is defined to occur in terms of an unsigned type. See the 2005 discussion on comp.std.c++ on this topic. > This is similar to the calloc integer overflow vulnerability in > glibc, which was fixed back in 2002. Interestingly, RUS-CERT > 2002-08:02 did mention 'operator new', and so did Bugtraq 5398. > http://cert.uni-stuttgart.de/advisories/calloc.php > http://www.securityfocus.com/bid/5398/discuss Yeah, I've essentially given up on this one. The official response from the C++ folks is here: http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_closed.html#256 | Each implementation is required to document the maximum size of an | object (Annex B limits). It is not difficult for a program to check | array allocations to ensure that they are smaller than this | quantity. Implementations can provide a mechanism in which users | concerned with this problem can request extra checking before array | allocations, just as some implementations provide checking for array | index and pointer validity. However, it would not be appropriate to | require this overhead for every array allocation in every program.
Re: Integer overflow in operator new
Florian Weimer wrote: This PR19351, by the way. The most widespread interpretation of the standard is that conforming implementations aren't allowed to raise an exception in this case: the arithmetic is defined to occur in terms of an unsigned type. Well for sure the standard does not allow you to generate junk code silently, so given the two choices of bad code or an exception I think the interpretation of the standard is not the relevent criterion. Anyway, you can always raise an exception if you run out of storage, and the standard has nothing formal to say on that topic. The official response from the C++ folks is here: http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_closed.html#256 | Each implementation is required to document the maximum size of an | object (Annex B limits). It is not difficult for a program to check | array allocations to ensure that they are smaller than this | quantity. Implementations can provide a mechanism in which users | concerned with this problem can request extra checking before array | allocations, just as some implementations provide checking for array | index and pointer validity. However, it would not be appropriate to | require this overhead for every array allocation in every program. You are quoting rationale. Surely that has no normative force. A standard cannot meaningfully talk about what is or what is not appropriate code generation anyway.
Re: Integer overflow in operator new
2007/4/7, Ross Ridge <[EMAIL PROTECTED]>: Joe Buck writes: >If a check were to be implemented, the right thing to do would be to throw >bad_alloc (for the default new) or return 0 (for the nothrow new). What do you do if the user has defined his own operator new that does something else? The callees checkers should to be with optional stubs, by example, the user wants to catch the error, log and send an e-mail to him and to data center. >There cases where the penalty for this check could have >an impact, like for pool allocators that are otherwise very cheap. >If so, there could be a flag to suppress the check. Excessive code size growth could also be problem for some programs. A solution is using the -shared option to generate ".so" library. Another future solution is pack the big ".so" library with UPX (Ultimate Packer for eXecutables) or extend the ELF format to permit pack the sections with GZ, BZ2 or LZMA. Ross Ridge J.C. Pizarro.
Re: Integer overflow in operator new
J.C. Pizarro wrote: A solution is using the -shared option to generate ".so" library. That does not solve things in environments like embedded environments where there are no shared libraries. Another future solution is pack the big ".so" library with UPX (Ultimate Packer for eXecutables) or extend the ELF format to permit pack the sections with GZ, BZ2 or LZMA. We are worried about code space in memory, not space on disk!
Re: Integer overflow in operator new
2007/4/7, Robert Dewar <[EMAIL PROTECTED]>: > A solution is using the -shared option to generate ".so" library. That does not solve things in environments like embedded environments where there are no shared libraries. Use -Os and "strip --strip-all". And remove code if you don't like it. > Another future solution is pack the big ".so" library with UPX > (Ultimate Packer for eXecutables) or extend the ELF format to > permit pack the sections with GZ, BZ2 or LZMA. We are worried about code space in memory, not space on disk! Or extend the ELF format to permit pack non-solidly or solidly (=> slower stream) the many subsections with GZ, BZ2, UPX or LZMA (<=1MiB to uncompress e.g.). Theirs buffers are very small to permit to raise an exception. Like squashfs for embedded systems instead cramfs.
Re: Integer overflow in operator new
2007/4/7, Robert Dewar <[EMAIL PROTECTED]>: > > A solution is using the -shared option to generate ".so" library. > > That does not solve things in environments like embedded > environments where there are no shared libraries. Use -Os and "strip --strip-all". And remove code if you don't like it. > > Another future solution is pack the big ".so" library with UPX > > (Ultimate Packer for eXecutables) or extend the ELF format to > > permit pack the sections with GZ, BZ2 or LZMA. > > We are worried about code space in memory, not space on disk! Or extend the ELF format to permit pack non-solidly the many subsections with GZ, BZ2, UPX or LZMA (<=1MiB to uncompress e.g.). Theirs buffers are very small to permit to raise an exception. Like squashfs for embedded systems instead cramfs. This same idea is applicable to pack the gigant /usr/lib/libgcj.so that its current non-packed size on disk is >=9 MiB, sometimes >=50 MiB. Remember, Java is generated to C++ with gcj and compiled with g++.
Re: Integer overflow in operator new
[EMAIL PROTECTED] (Ross Ridge) writes: | Joe Buck writes: | >If a check were to be implemented, the right thing to do would be to throw | >bad_alloc (for the default new) or return 0 (for the nothrow new). | | What do you do if the user has defined his own operator new that does | something else? More precisely? | >There cases where the penalty for this check could have | >an impact, like for pool allocators that are otherwise very cheap. | >If so, there could be a flag to suppress the check. | | Excessive code size growth could also be problem for some programs. Those programs willing to do anything to avoid imagined or perceived "excessive code size growth" may use the suggested switch. -- Gaby
Re: x86 inc/dec on core2
On Sat, Apr 07, 2007 at 11:29:46AM +0200, Uros Bizjak wrote: > Hello! > > >> I was wondering, if: > >> > >> /* X86_TUNE_USE_INCDEC */ > >> ~(m_PENT4 | m_NOCONA | m_CORE2 | m_GENERIC), > >> > >> is correct. Should it be: > >> > >> /* X86_TUNE_USE_INCDEC */ > >> ~(m_PENT4 | m_NOCONA | m_GENERIC), > >> > >> ? > > > >inc/dec has the same performance as add/sub on Core 2 Duo. But > >inc/dec is shorter. > > > > What about partial flag register dependency of inc/dec? There is no partial flag register dependency on inc/dec. H.J.
bugzilla admin request
Hi there, I'm actively working on a patch for enhancement request PR14331, which is still currently in 'NEW' state. Would some kindly admin like to assign it to me? cheers, DaveK -- Can't think of a witty .sigline today
Re: bugzilla admin request
On Saturday 07 April 2007 15:28, Dave Korn wrote: > Hi there, > > I'm actively working on a patch for enhancement request PR14331, which is > still currently in 'NEW' state. Would some kindly admin like to assign it > to me? Didn't you get bugzilla super-powers with your write after approval account? Paul
RE: bugzilla admin request
On 07 April 2007 15:37, Paul Brook wrote: > On Saturday 07 April 2007 15:28, Dave Korn wrote: >> Hi there, >> >> I'm actively working on a patch for enhancement request PR14331, which is >> still currently in 'NEW' state. Would some kindly admin like to assign it >> to me? > > Didn't you get bugzilla super-powers with your write after approval account? > > Paul No, looks like an oversight; there are no permission bits listed on my user prefs. I logged out and logged back in to get a fresh cookie, still no difference. Relevant list Cc'd. cheers, DaveK -- Can't think of a witty .sigline today
Re: bugzilla admin request
On 4/7/07, Dave Korn <[EMAIL PROTECTED]> wrote: No, looks like an oversight; there are no permission bits listed on my user prefs. I logged out and logged back in to get a fresh cookie, still no difference. Including your @gcc.gnu.org account? -- Pinski
RE: bugzilla admin request
On 07 April 2007 16:01, Andrew Pinski wrote: > On 4/7/07, Dave Korn <[EMAIL PROTECTED]> wrote: >> No, looks like an oversight; there are no permission bits listed on my >> user prefs. I logged out and logged back in to get a fresh cookie, still >> no difference. > > Including your @gcc.gnu.org account? > > -- Pinski That'll be it! Thanks Andrew. cheers, DaveK -- Can't think of a witty .sigline today
Re: Variable scope debug info
On 06/04/07, Joe Buck <[EMAIL PROTECTED]> wrote:
On Fri, Apr 06, 2007 at 11:38:50AM +0100, Rob Quill wrote:
> So the general concensus is that's it's not worth doing?
> Hypothetically, if I did it and it didn't make much difference, would
> it be worth submitting a patch? Or should I just give up before I
> start?
It might be worth doing. I think that, in addition to a patch,
I'd like to see measurements (maybe just the size increase in
libstdc++.{a,so}). If the cost is small, I will not object.
Seeing what the debugging sessions look like before and after
will help assure that we're getting it right, especially at
the boundary: if we have
23 Foo foo = func();
24 int bar = 0;
then if a breakpoint is set at line 23, foo should not be in scope,
but at line 24, it should be.
OK, that sounds good to me. As you say, there is no way to know until
it is actually implemented how muh affect it will have. However, I
won't be able to start on it until summer as I have to worry about
sitting my finals, so I'll let you know when I get started.
Thanks all for your help :)
Rob
Re: Integer overflow in operator new
On Sat, Apr 07, 2007 at 12:15:10PM +0200, Florian Weimer wrote: > * Karl Chen: > > > "4 * n", unchecked, is vulnerable to integer overflow. On IA-32, > > "new int[0x4001]" becomes equivalent to "new int[1]". I've > > verified this on gcc-2.95 through 4.1. For larger objects the > > effects are exaggerated; smaller counts are needed to overflow. > > This PR19351, by the way. Also by the way, it's triggered in the libstdc++ testsuite in at least one place :-) I can't remember which test any more, but last month I discovered that there is a use of operator new[] with a subscript of INT_MAX - 1 (INT_MAX is handled specially). In general this still works out to be more memory than can be allocated and the test tests what it wanted to (bad_alloc). -- Daniel Jacobowitz CodeSourcery
Re: Variable scope debug info
Rob Quill wrote: OK, that sounds good to me. As you say, there is no way to know until it is actually implemented how muh affect it will have. However, I won't be able to start on it until summer as I have to worry about sitting my finals, so I'll let you know when I get started. I would say this is only worth a minimal penalty in debug info size. It is one thing to deal with cases where something is visible in terms of canonical language semantics, and invisible because of optimization. It is quite another to deal with error checks on debug requests that just make no sense at all because the entity is not visible because of language semantics. This would just be guarding against (a rather peculiar) pilot error.
Documenting -fargument-noalias-anything in gcc-4.2/changes.html
Gerald, I do not have easy access to the HTML repository anymore. Martin Michlmayr asked me to add to the 4.2 changes list the inclusion of the new compile time option -fargument-noalias-anything. I constructed the following out of the invoke.texi document, to be included in gcc-4.2/changes.html, after "General Optimizer Improvements": -fargument-noalias-anything specifies that arguments do not alias any other storage. Each language will automatically use whatever option is required by the language standard. You should not need to use these options yourself. I would be glad if you could commit it. Thanks, -- Toon Moene - e-mail: [EMAIL PROTECTED] - phone: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands At home: http://moene.indiv.nluug.nl/~toon/ Who's working on GNU Fortran: http://gcc.gnu.org/ml/gcc/2007-01/msg00059.html
Re: Integer overflow in operator new
Joe Buck writes: >If a check were to be implemented, the right thing to do would be to throw >bad_alloc (for the default new) or return 0 (for the nothrow new). Ross Ridge writes: >What do you do if the user has defined his own operator new that does >something else? Gabriel Dos Reis writes: >More precisely? Well, for example, like all other things that a new_handler can do, like throwing an exception derived from bad_alloc or calling exit(). In addition, any number of side effects are possible, like printing error messages or setting flags. >Those programs willing to do anything to avoid imagined or perceived >"excessive code size growth" may use the suggested switch. The code size growth would be real, and there are enough applications out there that would consider any unnecessary growth in code excessive. The switch would be required both for that reason, and for Standard conformance. Ross Ridge
Re: Integer overflow in operator new
[EMAIL PROTECTED] (Ross Ridge) writes: | Joe Buck writes: | >If a check were to be implemented, the right thing to do would be to throw | >bad_alloc (for the default new) or return 0 (for the nothrow new). | | Ross Ridge writes: | >What do you do if the user has defined his own operator new that does | >something else? | | Gabriel Dos Reis writes: | >More precisely? | | Well, for example, like all other things that a new_handler can do, | like throwing an exception derived from bad_alloc or calling exit(). | In addition, any number of side effects are possible, like printing | error messages or setting flags. I believe you're confused about the semantics. The issue here is that the *size of object* requested can be represented. That is independent of whether the machine has enough memory or not. So, new_handler is a red herring. -- Gaby
Re: Integer overflow in operator new
[EMAIL PROTECTED] (Ross Ridge) writes: > Well, for example, like all other things that a new_handler can do, > like throwing an exception derived from bad_alloc or calling exit(). > In addition, any number of side effects are possible, like printing > error messages or setting flags. Gabriel Dos Reis writes: >I believe you're confused about the semantics. >The issue here is that the *size of object* requested can be >represented. That is independent of whether the machine has enough >memory or not. So, new_handler is a red herring The issue is what GCC should do when the calculation of the size of memory to allocate with operator new() results in unsigned wrapping. Currently, GCC's behavior is standard conforming but probably isn't the expected result. If GCC does something other than what operator new() does when there isn't enough memory available then it will be doing something that is both non-conforming and probably not what was expected. Ross Ridge
Re: Integer overflow in operator new
Gabriel Dos Reis writes:
> >I believe you're confused about the semantics.
> >The issue here is that the *size of object* requested can be
> >represented. That is independent of whether the machine has enough
> >memory or not. So, new_handler is a red herring
On Sat, Apr 07, 2007 at 06:05:35PM -0400, Ross Ridge wrote:
> The issue is what GCC should do when the calculation of the size of
> memory to allocate with operator new() results in unsigned wrapping.
> Currently, GCC's behavior is standard conforming but probably isn't the
> expected result. If GCC does something other than what operator new()
> does when there isn't enough memory available then it will be doing
> something that is both non-conforming and probably not what was expected.
Consider an implementation that, when given
Foo* array_of_foo = new Foo[n_elements];
passes __compute_size(elements, sizeof Foo) instead of n_elements*sizeof Foo
to operator new, where __compute_size is
inline size_t __compute_size(size_t num, size_t size) {
size_t product = num * size;
return product >= num ? product : ~size_t(0);
}
This counts on the fact that any operator new implementation has to fail
when asked to supply every single addressible byte, less one. It would
appear that the extra cost, for the non-overflow case, is two
instructions (on most architectures): the compare and the branch, which
can be arranged so that the prediction is not-taken.
I haven't memorized the standard, but I don't believe that this
implementation would violate it. The behavior differs only when more
memory is requested than can be delivered.
Re: Integer overflow in operator new
On Sat, Apr 07, 2007 at 04:01:57PM -0500, Gabriel Dos Reis wrote: > [EMAIL PROTECTED] (Ross Ridge) writes: > > | Joe Buck writes: > | >If a check were to be implemented, the right thing to do would be to throw > | >bad_alloc (for the default new) or return 0 (for the nothrow new). > | > | Ross Ridge writes: > | >What do you do if the user has defined his own operator new that does > | >something else? > | > | Gabriel Dos Reis writes: > | >More precisely? > | > | Well, for example, like all other things that a new_handler can do, > | like throwing an exception derived from bad_alloc or calling exit(). > | In addition, any number of side effects are possible, like printing > | error messages or setting flags. > > I believe you're confused about the semantics. > The issue here is that the *size of object* requested can be > represented. That is independent of whether the machine has enough > memory or not. So, new_handler is a red herring. The user's new function will be passed a size that has been converted into bytes. If gcc's implementation does multiply-with-saturation instead of straight multiply to get this size, user-defined new operators will work as expected.
Re: Integer overflow in operator new
On Sat, Apr 07, 2007 at 07:41:59AM -0400, Robert Dewar wrote: > J.C. Pizarro wrote: > > >A solution is using the -shared option to generate ".so" library. > > That does not solve things in environments like embedded > environments where there are no shared libraries. > > > >Another future solution is pack the big ".so" library with UPX > >(Ultimate Packer for eXecutables) or extend the ELF format to > >permit pack the sections with GZ, BZ2 or LZMA. > > We are worried about code space in memory, not space on disk! This is why I suggested that, should we implement a better check, there should be an option to turn it off, so programmers who cannot afford an extra byte are taken care of. However, the extra code might amount to 2-4 instructions per call to array-new (for objects of size > 1), and this is not that common an operation in any case.
Re: Integer overflow in operator new
Joe Buck <[EMAIL PROTECTED]> writes: | This is why I suggested that, should we implement a better check, | there should be an option to turn it off, so programmers who cannot | afford an extra byte are taken care of. I agree. -- Gaby
Re: Integer overflow in operator new
[EMAIL PROTECTED] (Ross Ridge) writes: [...] | Gabriel Dos Reis writes: | >I believe you're confused about the semantics. | >The issue here is that the *size of object* requested can be | >represented. That is independent of whether the machine has enough | >memory or not. So, new_handler is a red herring | | The issue is what GCC should do when the calculation of the size of | memory to allocate with operator new() results in unsigned wrapping. | Currently, GCC's behavior is standard conforming but probably isn't the | expected result. I don't understand this. -- Gaby
Re: Integer overflow in operator new
Joe Buck writes:
>Consider an implementation that, when given
>
>Foo* array_of_foo = new Foo[n_elements];
>
>passes __compute_size(elements, sizeof Foo) instead of n_elements*sizeof Foo
>to operator new, where __compute_size is
>
>inline size_t __compute_size(size_t num, size_t size) {
>size_t product = num * size;
>return product >= num ? product : ~size_t(0);
>}
Yes, doing something like this instead would largely answer my concerns.
>This counts on the fact that any operator new implementation has to fail
>when asked to supply every single addressible byte, less one.
I don't know if you can assume "~size_t(0)" is equal to the number of
addressable bytes, less one. A counter example would be 16-bit 80x86
compilers where size_t is 16-bits and an allocation of 65535 bytes can
succeed, but I don't know if GCC supports any targets where something
similar can happen.
>I haven't memorized the standard, but I don't believe that this
>implementation would violate it. The behavior differs only when more
>memory is requested than can be delivered.
It differs because the actual amount of memory requested is the result
of the unsigned multiplication of "n_elements * sizeof Foo", using your
example above. Since this result of this caclulation isn't undefined,
even if it "overflows", there's no room for the compiler to calculate
a different value to pass to operator new().
Ross Ridge
