Setting up a wiki for GNU Project volunteers?
A wiki for GNU Project volunteers would provide a central place for cross-project collaboration, including collaboration between gcc, glibc, gdb, and binutils. This is an invitation to discuss having a high-level wiki for GNU Project volunteers. The conversation is on gnu-misc-discuss: https://lists.gnu.org/archive/html/gnu-misc-discuss/2019-12/msg2.html Please feel free to send me your input directly if you like. Thank you for your input. -- Cheers, Carlos.
Re: Errors building libgcc for powerpc64le-linux-gnu
On Sat, Dec 14, 2019 at 11:25 PM Segher Boessenkool wrote: > > On Sat, Dec 14, 2019 at 10:51:50AM -0800, Ian Lance Taylor via gcc wrote: > > I'm seeing compiler crashes building libgcc for powerpc64le-linux-gnu, > > cross-compiling from x86_64-pc-linux-gnu. I'm at SVN revision 279830. > > I'm seeing the following. Is anybody else seeing this crash? Thanks. > > No, and that makes me wonder what is going on. The error is simple enough > of course, as you note in a later message; but why do we not see it on > every other build? I think it's because clang treats a left shift by a negative number as undefined behavior but GCC does not. So GCC is consistently producing some number, and clang is producing different numbers. I should note that I don't really understand what purpose that constant is serving anyhow. Ian
Re: Usage of C11 Annex K Bounds-checking interfaces on GCC
On 15/12/2019 02:57, Jeffrey Walton wrote:
On Sat, Dec 14, 2019 at 12:36 PM Martin Sebor wrote:
On 12/9/19 8:15 PM, li zi wrote:
Hi All,
We are using gcc in our projects and we found some of the C standard functions
(like memcpy, strcpy) used in gcc may induce security vulnerablities like
buffer overflow. Currently we have not found any instances which causes such
issues.
(This post is all "In My Humble Opinion".)
The correct use of memcpy, strcpy, etc., does not introduce any security
vulnerabilities. /Incorrect/ use does - just as incorrect use of any
function can risk bugs, and therefore security vulnerabilities.
But we feel better to change these calls to Cll Annex K Bounds-checking
interfaces like memcpy_s, strcpy_s etc. By defining a secure calls method (list
of func pointers) and allowing application to register the method. I understand
that this affects performance because of return value check added for _s
calls, but this will relieve overflow kind of issues from code. And also
currently using bounds-checking interfaces is a general industry practice.
Please share your opinion on it, and if any discussion happened in community to
do some changes in future.
Thoughtless "change all standard functions to Annex K
functions" is management arse-covering as an alternative to proper
quality development techniques. It adds nothing to the software
quality, it has no reduction in the risk of errors or their
consequences. It is nothing more than a way to try to blame other
people when something has gone wrong.
If you find that using these functions really does give you better
software with lower risks of problems, then by all means use them. But
don't use them blindly just because someone says they are "safer".
GCC's Object Size Checking is a non-intrusive solution to
the problem. It avoids the considerable risk of introducing
bugs while replacing existing calls with those to the _s
functions. The implementation is restricted to constant
sizes so its effectiveness is a limited, but we have been
discussing enhancing it to non-constant sizes as well, as
Clang already does. With that, it should provide protection
with an effectiveness comparable to the _s functions but
without any of the downsides. (Note that GCC's buffer
overflow warnings are not subject to the same limitation.)
Besides Object Size Checking, I would suggest making use of
the new attribute access. It lets GCC detect (though not
prevent) out-of-bounds accesses by calls to user-defined
functions decorated with the attribute.
The safer functions have three or four security goals. The workarounds
don't meet the goals.
Let's call them "Annex K" functions, rather than "safer functions",
until it is demonstrated that they actually /are/ safer in some way.
And let's talk about other tools in the toolbox, to use in addition or
as alternatives, rather than calling them "workarounds".
The safer functions require the destination size to ensure a buffer
overflow does not occur.
That is useless unless you know the destination size. And if you know
the destination size, you can use that to make sure your calls to
memcpy, strcpy, etc., are safe and correct.
The use of the Annex K functions therefore gives you no technical
benefits in this aspect. They may, however, give you the non-technical
benefit of forcing the programmer to think about destination sizes -
they can't be lazy about it. However, if you are dealing with code that
needs to be of good quality then you should already have development
practices that cover this - such as code reviews, programmer training,
code standards, etc.
The safe functions always terminate the destination buffer.
So do the standard functions when used correctly.
I'm quite happy to agree that strncpy has a silly specification, and it
can surprise people both in its inefficiency and in the fact that it
does not always null-terminate the destination. Creating a "strncpy_s"
with redundant parameters and confusingly different semantics is /not/
the answer. The right solution would be to deprecate strncpy, and make
a replacement with a different name and better semantics, such as:
char * strscpy(char * restrict s1, const char * restrict s2, size_t n)
{
if (n > 0) {
*s1 = '\0';
strncat(s1, s2, n - 1);
}
return s1;
}
/That/ would have been a useful, clear, and consistent function that
copies at most "n" characters from the string, and always terminates the
copy.
The safe functions provide a consistent return value. There is only
one success code.
There is only one success code from the Annex K functions - but no
guarantees about failures. It will call the constraint handler - which
might return, or might not. Frankly, you have no idea what it might do
in the general case, since the constraint handler is a global state
variable and not even thread-safe.
The only way to write good, safe and reliable
Re: Errors building libgcc for powerpc64le-linux-gnu
On Sun, Dec 15, 2019 at 09:43:20AM -0800, Ian Lance Taylor wrote: > On Sat, Dec 14, 2019 at 11:25 PM Segher Boessenkool > wrote: > > > > On Sat, Dec 14, 2019 at 10:51:50AM -0800, Ian Lance Taylor via gcc wrote: > > > I'm seeing compiler crashes building libgcc for powerpc64le-linux-gnu, > > > cross-compiling from x86_64-pc-linux-gnu. I'm at SVN revision 279830. > > > I'm seeing the following. Is anybody else seeing this crash? Thanks. > > > > No, and that makes me wonder what is going on. The error is simple enough > > of course, as you note in a later message; but why do we not see it on > > every other build? > > I think it's because clang treats a left shift by a negative number as > undefined behavior but GCC does not. So GCC is consistently producing > some number, and clang is producing different numbers. Hrm. Why did that not show up with ubsan then though? (Or maybe it did, and I just never heard). > I should note that I don't really understand what purpose that > constant is serving anyhow. - "operands[2] = GEN_INT (1 << (75 - REGNO (operands[0])));") + "operands[2] = GEN_INT (1 << (7 - (REGNO (operands[0]) - CR0_REGNO)));") The constant is the bitmask of which CR fields to save/restore (always one here, but the insn allows any combination). Committing that patch later today. Thanks for the report! Segher
Re: Usage of C11 Annex K Bounds-checking interfaces on GCC
On Sun, Dec 15, 2019 at 1:25 PM David Brown wrote: > > On 15/12/2019 02:57, Jeffrey Walton wrote: > > On Sat, Dec 14, 2019 at 12:36 PM Martin Sebor wrote: > >> > >> On 12/9/19 8:15 PM, li zi wrote: > >>> Hi All, > >>> We are using gcc in our projects and we found some of the C standard > >>> functions (like memcpy, strcpy) used in gcc may induce security > >>> vulnerablities like buffer overflow. Currently we have not found any > >>> instances which causes such issues. > > (This post is all "In My Humble Opinion".) > > The correct use of memcpy, strcpy, etc., does not introduce any security > vulnerabilities. /Incorrect/ use does - just as incorrect use of any > function can risk bugs, and therefore security vulnerabilities. > > >>> But we feel better to change these calls to Cll Annex K Bounds-checking > >>> interfaces like memcpy_s, strcpy_s etc. By defining a secure calls method > >>> (list of func pointers) and allowing application to register the method. > >>> I understand that this affects performance because of return value check > >>> added for _s calls, but this will relieve overflow kind of issues > >>> from code. And also currently using bounds-checking interfaces is a > >>> general industry practice. > >>> Please share your opinion on it, and if any discussion happened in > >>> community to do some changes in future. > > Thoughtless "change all standard functions to Annex K > functions" is management arse-covering as an alternative to proper > quality development techniques. It adds nothing to the software > quality, it has no reduction in the risk of errors or their > consequences. It is nothing more than a way to try to blame other > people when something has gone wrong. If RTFM was going to work, then it would have happened in the last 50 years or so. If error free programming was going to happen, then it would have happened in the last 50 years or so. Come back to reality. > If you find that using these functions really does give you better > software with lower risks of problems, then by all means use them. But > don't use them blindly just because someone says they are "safer". Hard to do when they are missing. > >> GCC's Object Size Checking is a non-intrusive solution to > >> the problem. It avoids the considerable risk of introducing > >> bugs while replacing existing calls with those to the _s > >> functions. The implementation is restricted to constant > >> sizes so its effectiveness is a limited, but we have been > >> discussing enhancing it to non-constant sizes as well, as > >> Clang already does. With that, it should provide protection > >> with an effectiveness comparable to the _s functions but > >> without any of the downsides. (Note that GCC's buffer > >> overflow warnings are not subject to the same limitation.) > >> > >> Besides Object Size Checking, I would suggest making use of > >> the new attribute access. It lets GCC detect (though not > >> prevent) out-of-bounds accesses by calls to user-defined > >> functions decorated with the attribute. > > > > The safer functions have three or four security goals. The workarounds > > don't meet the goals. > > Let's call them "Annex K" functions, rather than "safer functions", > until it is demonstrated that they actually /are/ safer in some way. > And let's talk about other tools in the toolbox, to use in addition or > as alternatives, rather than calling them "workarounds". Microsoft calls them "safer" functions. They are "safer" then the original C functions they are supplementing. For completeness, Microsoft does not claim they are completely safe. Runtime crashes due to unwanted terminates are a real problem if the program handles sensitive data, like passwords and private keys. We can't have crash dumps with passwords and private keys written to the file system in a core dump, or shipped to an error reporting service like Crash Reporter or Windows Error Reporting. That's a different [policy] problem. > > The safer functions require the destination size to ensure a buffer > > overflow does not occur. > > That is useless unless you know the destination size. And if you know > the destination size, you can use that to make sure your calls to > memcpy, strcpy, etc., are safe and correct. Hugh? Are you begging the argument: char* ptr = malloc (50); And then claiming you don't know the size? > The use of the Annex K functions therefore gives you no technical > benefits in this aspect. They may, however, give you the non-technical > benefit of forcing the programmer to think about destination sizes - > they can't be lazy about it. However, if you are dealing with code that > needs to be of good quality then you should already have development > practices that cover this - such as code reviews, programmer training, > code standards, etc. Developer training does not work. If it was going to work, then it would have happened in the last 50 years or so. Microsoft recognized the fact years
gcc-10-20191215 is now available
Snapshot gcc-10-20191215 is now available on https://gcc.gnu.org/pub/gcc/snapshots/10-20191215/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 10 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/trunk revision 279405 You'll find: gcc-10-20191215.tar.xz Complete GCC SHA256=f577e49a317d7a2009c75664e66a5d2111c8ee948a3436a56a0217d6ee8e9229 SHA1=9f62025b2191d5ac0e860cc74dec026698930b54 Diffs from 10-20191208 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-10 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
Re: Proposal for the transition timetable for the move to GIT
On Wed, 11 Dec 2019, Joseph Myers wrote: > Concretely: when I did a comparison of the tip of trunk against master > from a reposurgeon conversion on 29 November, there were 1421 differences > (files or directories only present in one of SVN or git or with different > contents). As of today with the SVN dump reader rewrite, this is down to > just two differences (plus two empty directories present in SVN as git > doesn't represent empty directories), and we understand exactly where the > problem arises with a trunk deletion and recreation and what's odd about > that particular trunk deletion and recreation. Update: having done comparisons for every branch tip and tag, and investigated all the problems found and collectively fixed the bugs in question based on reduced testcases from that investigation, I believe all such problems causing comparison failures have now been fixed (and all problems causing tags or branches to be missing); I'm running a fresh conversion and comparisons to confirm. If those comparisons are clean I may also compare the refs/deleted tags and branches to provide additional points at which the tree contents are verified correct. Note: these comparisons are after deleting empty directories from the SVN checkout, because git doesn't represent empty directories. There are also two branches (c++-modules and melt-branch) where some files have SVN keyword expansion enabled, and the files with keyword expansion enabled need excluding manually from the comparison process. -- Joseph S. Myers jos...@codesourcery.com
Re: Usage of C11 Annex K Bounds-checking interfaces on GCC
在 2019/12/16 4:00, Jeffrey Walton 写道:
>
> If RTFM was going to work, then it would have happened in the last 50
> years or so.
>
> If error free programming was going to happen, then it would have
> happened in the last 50 years or so.
>
> Come back to reality.
>
What's your point? Don't RTFM then don't code, period.
>
> Microsoft calls them "safer" functions. They are "safer" then the
> original C functions they are supplementing. For completeness,
> Microsoft does not claim they are completely safe.
>
They are of course not 'safer' for two reasons:
One is that by having an additional parameter you ask for an additional
size argument, but it is still possible that the user passed a wrong
size, such as when you want the number of `wchar_t`s but your user
supplied the number of bytes, which you have no clue about. The best
advice would be using C++ templates to deduce the size of output buffer,
but it doesn't work in C, and even in C++ it works only when the
argument is an array, string, vector, etc. It doesn't work if the
argument is a pointer, in which case you still have to pass the size
yourself.
The other reason is that by requiring more arguments you increase the
probability of bugs. Let's say there is a 1% chance that you pass a
wrong argument. Then if there is 1 argument, the probability that you do
everything right is 99%. If there are 2 arguments, it is 98.01%. If
there are 10 arguments, it is 97.0299%. If there are 100 arguments, it
is about 36.6%. It is not something we would like.
> Hugh? Are you begging the argument:
>
> char* ptr = malloc (50);
>
> And then claiming you don't know the size?
>
Why don't you use Java which keeps tracking of allocated arrays and
throws exceptions in case of out-of-bound access?
> Developer training does not work. If it was going to work, then it
> would have happened in the last 50 years or so.
>
> Microsoft recognized the fact years ago. You have to force developers
> to use something safer.
>
Let's take a C++ example. A lot of people still prefer `operator[]` to
`.at()`, which is prone to errors; but they have been taught and have
got used to that for decades. It is not developer training that does not
work. It is /bad/ developer training that does not work.
>
> I don't think a known dangerous and banned function is a good example.
> strcpy() is banned by both Microsoft and APple. Only Linux still
> embraces strcpy(). strcpy() still suffers the problem that is trying
> to be corrected.
>
I don't think `strcpy()` is unsafe. The only real issue of it is that it
copies strings without returning its length. So in order to get the
length you have to scan the copied string again. `stpcpy()` would be a
much better alternative.
>
> I generally consider the Glibc folks better trained in C and more
> knowledgeable of the C standard then me. If the Glibc folks are making
> the mistakes, then there is no hope in practice for folks like me or
> those who are just starting in C. There are too many sharp edges.
>
Yes yes why don't you use Java? If you write C you are supposed to have
been well educated ('well educated' means at least you should RTFM
before ask). C is not for beginners.
--
Best regards,
LH_Mouse
signature.asc
Description: OpenPGP digital signature
Re: Usage of C11 Annex K Bounds-checking interfaces on GCC
On Sun, Dec 15, 2019 at 9:43 PM Liu Hao wrote: > > 在 2019/12/16 4:00, Jeffrey Walton 写道: > > > > If RTFM was going to work, then it would have happened in the last 50 > > years or so. > > > > If error free programming was going to happen, then it would have > > happened in the last 50 years or so. > > > > Come back to reality. > > > > What's your point? Don't RTFM then don't code, period. > > > > > Microsoft calls them "safer" functions. They are "safer" then the > > original C functions they are supplementing. For completeness, > > Microsoft does not claim they are completely safe. > > > > They are of course not 'safer' for two reasons: > > One is that by having an additional parameter you ask for an additional > size argument, but it is still possible that the user passed a wrong > size, such as when you want the number of `wchar_t`s but your user > supplied the number of bytes, which you have no clue about. The best > advice would be using C++ templates to deduce the size of output buffer, > but it doesn't work in C, and even in C++ it works only when the > argument is an array, string, vector, etc. It doesn't work if the > argument is a pointer, in which case you still have to pass the size > yourself. > > The other reason is that by requiring more arguments you increase the > probability of bugs. Let's say there is a 1% chance that you pass a > wrong argument. Then if there is 1 argument, the probability that you do > everything right is 99%. If there are 2 arguments, it is 98.01%. If > there are 10 arguments, it is 97.0299%. If there are 100 arguments, it > is about 36.6%. It is not something we would like. Typical of engineers... Now you are arguing for problems that don't exist. Perhaps you should stick with the problems that do exist. > > Hugh? Are you begging the argument: > > > > char* ptr = malloc (50); > > > > And then claiming you don't know the size? > > Why don't you use Java which keeps tracking of allocated arrays and > throws exceptions in case of out-of-bound access? Yeah, that's the answer. We could write the whole OS in JavaScript. > > Developer training does not work. If it was going to work, then it > > would have happened in the last 50 years or so. > > > > Microsoft recognized the fact years ago. You have to force developers > > to use something safer. [More useless shit snipped]. Jeff
Re: Usage of C11 Annex K Bounds-checking interfaces on GCC
Le 16/12/2019 à 03:43, Liu Hao a écrit :
I generally consider the Glibc folks better trained in C and more
knowledgeable of the C standard then me. If the Glibc folks are making
the mistakes, then there is no hope in practice for folks like me or
those who are just starting in C. There are too many sharp edges.
Yes yes why don't you use Java? If you write C you are supposed to have
been well educated ('well educated' means at least you should RTFM
before ask). C is not for beginners.
C is a low-level language and the C programmer is just "supposed to
know what (s)he does."
If this is critical for you, then start learning a higher level
language (Java as suggested or Ada). You will love it and write safe
programs. The solution isn't in a library, it is in the language
allowing the compiler or run-time to detect these errors and/or forbid
dangerous constructs. Note that the later feature doesn't forbid you to
do what you want; it forces you to do it well.
Didier (just a lurker on this list)
