compiling gcc 5.x from the scratch: issues
Hello, gcc team. At my work I would like to have recent gcc installed but i have no sudo rights to update the current gcc (its 4.4.7, and OS is redhat linux). So I checked out latest version of gcc via svn, and following guidelines given at gcc webpage, tried to download prerequisites, configure gcc to be compiled into a separate folder and to start make. But got a lot of errors including errors in test files (that are easy to fix), errors during linkage (seems to be almost final step); eventually failed to build it up. As it is supposed that this SW is extensively tested, could you tell me: whether someone tried to build latest gcc (say, gcc 5.1 or 5.2) from scratch and was succeed in this (if he/she follows the guidelines given at the gcc web-site? If possible, could you check -whether it is still possible to compile gcc or not and update guidelines, if something has been changed when time passes by? Thanks a lot, Best regards, Mikhail Mishurovskiy, PhD Senior Engineer, Advanced Media Lab, DMC R& Center @Samsung Electronics Co.LTD. Deputy Group Leader, Samsung R&D Russia Institute Cell (Local Korean): 010-2996-1979
Re: compiling gcc 5.x from the scratch: issues
On 2015.09.09 at 08:36 +, Michael Mishourovsky wrote: > At my work I would like to have recent gcc installed but i have no > sudo rights to update the current gcc (its 4.4.7, and OS is redhat > linux). > > So I checked out latest version of gcc via svn, and following > guidelines given at gcc webpage, tried to download prerequisites, > configure gcc to be compiled into a separate folder and to start > make. But got a lot of errors including errors in test files (that > are easy to fix), errors during linkage (seems to be almost final > step); eventually failed to build it up. As it is supposed that this > SW is extensively tested, could you tell me: whether someone tried to > build latest gcc (say, gcc 5.1 or 5.2) from scratch and was succeed > in this (if he/she follows the guidelines given at the gcc web-site? > If possible, could you check -whether it is still possible to compile > gcc or not and update guidelines, if something has been changed when > time passes by? Please make sure that you use a build directory outside the gcc source tree. You may simply follow: https://gcc.gnu.org/install/ If the issue still persists, please post the exact errors that you encounter. -- Markus
Fwd: Successful GCC 5.2 installation
Hi, The GCC 5.2 installation was successful on an HP Pavilion x360. The outputs required are: Output of config.guess: x86_64-unknown-linux-gnu Output of gcc -v: Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/5.2.0/lto-wrapper Target: x86_64-unknown-linux-gnu Configured with: /home/andy356/Source/gcc-5.2.0/configure --disable-multilib Thread model: posix gcc version 5.2.0 (GCC) Distribution: Debian GNU/Linux 8.1 (installed at this version, currently at 8.2) Linux kernel: Linux debian 3.16.0-4-amd64 glibc version: 2.19-18+deb8u1
Advertisement in the GCC mirrors list
Hi, http://mirrors-ru.go-parts.com/gcc - Online Shop ftp://mirrors-ru.go-parts.com/gcc - bad rsync://mirrors-ru.go-parts.com/gcc - bad http://mirrors-uk.go-parts.com/gcc/ - Online Shop ftp://mirrors-uk.go-parts.com/gcc - bad rsync://mirrors-uk.go-parts.com/gcc - bad
Compiler support for erasure of sensitive data
Over the past couple months I have been working on adding a function to
glibc for erasure of sensitive data after it's used. This function
already exists in various other libraries under various names --
explicit_bzero, explicit_memset, SecureZeroMemory, memset_s,
OPENSSL_cleanse, etc -- and the problem it solves is, in this code:
void encrypt_with_phrase(const char *phrase, const char *in,
char *out, size_t n)
{
char key[16];
genkey(phrase, key);
encrypt(key, in, out, n);
memset(key, 0, 16);
}
the compiler is entitled to delete the final call to memset, on the
grounds that `key` is dead after the call, so no conforming C program
can access that memory to prove that it has been cleared. However, a
crash dump, a debugger, or a plain old bug in the program might expose
that region of the stack to prying eyes. This is listed at
https://cwe.mitre.org/data/definitions/14.html as a "common weakness".
explicit_bzero (under whatever name) is just bzero with an additional,
documented guarantee that the compiler will not optimize it out even
if the memory region is dead afterward. To further underline that
this is a common scenario, there are 152 uses of either
OPENSSL_cleanse or explicit_bzero in libressl 2.2.3 (they appear to be
in the middle of changing the name).
During development of my patches, we encountered two problems which
cannot be fully resolved without compiler assistance, so I want to
open a conversation about getting that assistance added to GCC and
Clang.
The first, simpler problem is strictly optimization. explicit_bzero
can be optimized to memset followed by a vacuous use of the memory
region (generating no machine instructions, but preventing the stores
from being deleted as dead); this is valuable because the sensitive
data is often small and fixed-size, so the memset can in turn be
replaced by inline code. (This also facilitates implementation of
-D_FORTIFY_SOURCE checks for explicit_bzero.) Again looking at
libressl, 92 of those 152 uses are improved by a crude version of this
optimization:
void explicit_bzero(void *, size_t);
extern inline __attribute__((gnu_inline, always_inline))
void explicit_bzero_constn(void *ptr, size_t len)
{
typedef struct {char x[len];} memblk;
memset(ptr, 0, len);
asm("" : : "m" (*(memblk __attribute__((may_alias)) *)ptr));
}
#define explicit_bzero(s, n) \
(__extension__(__builtin_constant_p(n) && (n) > 0 \
? explicit_bzero_constn(s, n)\
: explicit_bzero(s, n)))
I call this "crude" because it only works in GCC, when compiling C,
and when the length parameter is compile-time constant. GCC issues no
error for this code when 'len' is not compile-time constant, but it is
not documented to work reliably. When compiling C++, GCC does not
accept a structure containing an array whose size is not *lexically*
constant; even if the body of explicit_bzero_constn is moved into the
macro so that the whole thing is guarded by __builtin_constant_p,
using explicit_bzero with a non-constant size will cause a compile
error. The same is true for Clang whether compiling C or C++.
This problem could be solved with a very simple feature addition:
extern inline __attribute__((gnu_inline, always_inline))
void explicit_bzero(void *ptr, size_t len)
{
memset(ptr, 0, len);
__builtin_use_memory(ptr, len);
}
where __builtin_use_memory compiles to no machine instructions, but is
treated as a non-deletable use of all LEN bytes starting at PTR,
whether or not LEN is constant.
The harder problem, unfortunately, is one of correctness. Consider
now this example (courtesy Florian Weimer):
extern void explicit_bzero(void *, size_t);
struct key { unsigned long long hi, lo; };
extern struct key get_key(void);
extern void use_key(struct key);
void without_clear(void)
{
struct key k = get_key();
use_key(k);
}
void with_clear(void)
{
struct key k = get_key();
use_key(k);
explicit_bzero(&k, sizeof k);
}
On AMD64 this will be compiled to something like
without_clear:
subq$8, %rsp
callget_key
movq%rdx, %rsi
movq%rax, %rdi
calluse_key
addq$8, %rsp
ret
with_clear:
subq$24, %rsp
callget_key
movq%rax, %rdi
movq%rdx, %rsi
movq%rax, (%rsp)
movq%rdx, 8(%rsp)
calluse_key
movq%rsp, %rdi
movl$16, %esi
callexplicit_bzero
addq$24, %rsp
ret
The ABI dictates basically everything you see. The call to
explicit_bzero has forced the compiler to *create* a second copy of
the variable `k` on the stack, just so it can be erased -- and the
copy in registers survives (at least for a sh
Fwd: Compiler support for erasure of sensitive data
[Sorry for the double-post, but I got the LLVM mailing list address wrong,
and I want to make sure that everyone sees the entire conversation.]
Over the past couple months I have been working on adding a function to
glibc for erasure of sensitive data after it's used. This function
already exists in various other libraries under various names --
explicit_bzero, explicit_memset, SecureZeroMemory, memset_s,
OPENSSL_cleanse, etc -- and the problem it solves is, in this code:
void encrypt_with_phrase(const char *phrase, const char *in,
char *out, size_t n)
{
char key[16];
genkey(phrase, key);
encrypt(key, in, out, n);
memset(key, 0, 16);
}
the compiler is entitled to delete the final call to memset, on the
grounds that `key` is dead after the call, so no conforming C program
can access that memory to prove that it has been cleared. However, a
crash dump, a debugger, or a plain old bug in the program might expose
that region of the stack to prying eyes. This is listed at
https://cwe.mitre.org/data/definitions/14.html as a "common weakness".
explicit_bzero (under whatever name) is just bzero with an additional,
documented guarantee that the compiler will not optimize it out even
if the memory region is dead afterward. To further underline that
this is a common scenario, there are 152 uses of either
OPENSSL_cleanse or explicit_bzero in libressl 2.2.3 (they appear to be
in the middle of changing the name).
During development of my patches, we encountered two problems which
cannot be fully resolved without compiler assistance, so I want to
open a conversation about getting that assistance added to GCC and
Clang.
The first, simpler problem is strictly optimization. explicit_bzero
can be optimized to memset followed by a vacuous use of the memory
region (generating no machine instructions, but preventing the stores
from being deleted as dead); this is valuable because the sensitive
data is often small and fixed-size, so the memset can in turn be
replaced by inline code. (This also facilitates implementation of
-D_FORTIFY_SOURCE checks for explicit_bzero.) Again looking at
libressl, 92 of those 152 uses are improved by a crude version of this
optimization:
void explicit_bzero(void *, size_t);
extern inline __attribute__((gnu_inline, always_inline))
void explicit_bzero_constn(void *ptr, size_t len)
{
typedef struct {char x[len];} memblk;
memset(ptr, 0, len);
asm("" : : "m" (*(memblk __attribute__((may_alias)) *)ptr));
}
#define explicit_bzero(s, n) \
(__extension__(__builtin_constant_p(n) && (n) > 0 \
? explicit_bzero_constn(s, n)\
: explicit_bzero(s, n)))
I call this "crude" because it only works in GCC, when compiling C,
and when the length parameter is compile-time constant. GCC issues no
error for this code when 'len' is not compile-time constant, but it is
not documented to work reliably. When compiling C++, GCC does not
accept a structure containing an array whose size is not *lexically*
constant; even if the body of explicit_bzero_constn is moved into the
macro so that the whole thing is guarded by __builtin_constant_p,
using explicit_bzero with a non-constant size will cause a compile
error. The same is true for Clang whether compiling C or C++.
This problem could be solved with a very simple feature addition:
extern inline __attribute__((gnu_inline, always_inline))
void explicit_bzero(void *ptr, size_t len)
{
memset(ptr, 0, len);
__builtin_use_memory(ptr, len);
}
where __builtin_use_memory compiles to no machine instructions, but is
treated as a non-deletable use of all LEN bytes starting at PTR,
whether or not LEN is constant.
The harder problem, unfortunately, is one of correctness. Consider
now this example (courtesy Florian Weimer):
extern void explicit_bzero(void *, size_t);
struct key { unsigned long long hi, lo; };
extern struct key get_key(void);
extern void use_key(struct key);
void without_clear(void)
{
struct key k = get_key();
use_key(k);
}
void with_clear(void)
{
struct key k = get_key();
use_key(k);
explicit_bzero(&k, sizeof k);
}
On AMD64 this will be compiled to something like
without_clear:
subq$8, %rsp
callget_key
movq%rdx, %rsi
movq%rax, %rdi
calluse_key
addq$8, %rsp
ret
with_clear:
subq$24, %rsp
callget_key
movq%rax, %rdi
movq%rdx, %rsi
movq%rax, (%rsp)
movq%rdx, 8(%rsp)
calluse_key
movq%rsp, %rdi
movl$16, %esi
callexplicit_bzero
addq$24, %rsp
ret
The ABI dictates basically everything you see. The call to
explicit_bzero has forced the compi
Re: Advertisement in the GCC mirrors list
Gerald, I think we've had similar issues with these mirrors in the past as well, shall we just remove them from the list? On 9 September 2015 at 17:28, niXman wrote: > > Hi, > > http://mirrors-ru.go-parts.com/gcc - Online Shop > ftp://mirrors-ru.go-parts.com/gcc - bad > rsync://mirrors-ru.go-parts.com/gcc - bad > > http://mirrors-uk.go-parts.com/gcc/ - Online Shop > ftp://mirrors-uk.go-parts.com/gcc - bad > rsync://mirrors-uk.go-parts.com/gcc - bad > >
Re: Compiler support for erasure of sensitive data
OK, I have now failed to find the LLVM development list twice in a row. Could some kind soul please forward the message to whereever the heck the proper address is? zw
Re: [musl] Compiler support for erasure of sensitive data
On Wed, Sep 09, 2015 at 12:36:01PM -0400, Zack Weinberg wrote:
> The first, simpler problem is strictly optimization. explicit_bzero
> can be optimized to memset followed by a vacuous use of the memory
> region (generating no machine instructions, but preventing the stores
> from being deleted as dead); this is valuable because the sensitive
> data is often small and fixed-size, so the memset can in turn be
> replaced by inline code. (This also facilitates implementation of
> -D_FORTIFY_SOURCE checks for explicit_bzero.) Again looking at
> libressl, 92 of those 152 uses are improved by a crude version of this
> optimization:
>
> void explicit_bzero(void *, size_t);
> extern inline __attribute__((gnu_inline, always_inline))
> void explicit_bzero_constn(void *ptr, size_t len)
> {
> typedef struct {char x[len];} memblk;
> memset(ptr, 0, len);
> asm("" : : "m" (*(memblk __attribute__((may_alias)) *)ptr));
> }
> #define explicit_bzero(s, n) \
> (__extension__(__builtin_constant_p(n) && (n) > 0 \
> ? explicit_bzero_constn(s, n)\
> : explicit_bzero(s, n)))
>
> I call this "crude" because it only works in GCC, when compiling C,
> and when the length parameter is compile-time constant. GCC issues no
> error for this code when 'len' is not compile-time constant, but it is
> not documented to work reliably. When compiling C++, GCC does not
> accept a structure containing an array whose size is not *lexically*
> constant; even if the body of explicit_bzero_constn is moved into the
> macro so that the whole thing is guarded by __builtin_constant_p,
> using explicit_bzero with a non-constant size will cause a compile
> error. The same is true for Clang whether compiling C or C++.
>
> This problem could be solved with a very simple feature addition:
>
> extern inline __attribute__((gnu_inline, always_inline))
> void explicit_bzero(void *ptr, size_t len)
> {
> memset(ptr, 0, len);
> __builtin_use_memory(ptr, len);
> }
You're making this harder than it needs to be. The "m" constraint is
the wrong thing to use here. Simply use:
__asm__(""::"r"(ptr):"memory");
The memory constraint implies that the asm can read or write any
memory that's reachable by it. The lack of output constraints implies
__volatile__ which is also needed.
Rich
Re: Advertisement in the GCC mirrors list
On 09/09/2015 10:41 AM, Jonathan Wakely wrote: Gerald, I think we've had similar issues with these mirrors in the past as well, shall we just remove them from the list? Please do. jeff
Re: [musl] Compiler support for erasure of sensitive data
On Wed, Sep 9, 2015 at 12:42 PM, Rich Felker wrote:
> You're making this harder than it needs to be. The "m" constraint is
> the wrong thing to use here. Simply use:
>
> __asm__(""::"r"(ptr):"memory");
Please review my earlier conversation with Adhemerval on exactly this point.
zw
Re: Compiler support for erasure of sensitive data
> On Sep 9, 2015, at 12:36 PM, Zack Weinberg wrote:
>
> ...
> I think the ideal feature addition to address this would be
>
>void safe(void)
>{
>struct key __attribute__((sensitive)) k = get_key();
>use_key(k);
>}
That certainly is a cleaner answer. What is attractive about it is that it
expresses the need for variables (data) to be given different treatment, rather
than expecting the programmer to code that special treatment in every place
where that data becomes dead. It's also likely to be a whole lot harder to
implement, unfortunately.
Then again, suppose all you had is explicit_bzero, and an annotation on the
data saying it's sensitive. Can static code analyzers take care of the rest?
If so, this sort of thing doesn't need to be in the compiler.
paul
[wwwdocs] PATCH forRe: Advertisement in the GCC mirrors list
On Wed, 9 Sep 2015, Jonathan Wakely wrote: > Gerald, I think we've had similar issues with these mirrors in the > past as well, shall we just remove them from the list? > >> http://mirrors-ru.go-parts.com/gcc - Online Shop >> ftp://mirrors-ru.go-parts.com/gcc - bad >> rsync://mirrors-ru.go-parts.com/gcc - bad >> >> http://mirrors-uk.go-parts.com/gcc/ - Online Shop >> ftp://mirrors-uk.go-parts.com/gcc - bad >> rsync://mirrors-uk.go-parts.com/gcc - bad Yes. Immediately done with the patch below. Dan, heads up. I hope this was not the plan from the beginning? Gerald Index: mirrors.html === RCS file: /cvs/gcc/wwwdocs/htdocs/mirrors.html,v retrieving revision 1.230 diff -u -r1.230 mirrors.html --- mirrors.html23 Apr 2015 21:45:38 - 1.230 +++ mirrors.html9 Sep 2015 16:56:28 - @@ -45,17 +45,9 @@ rsync://mirror2.babylon.network/gcc/, thanks to Tim Semeijn (noc@babylon.network) at Babylon Network. The Netherlands, Nijmegen: ftp://ftp.nluug.nl/mirror/languages/gcc";>ftp.nluug.nl, thanks to Jan Cristiaan van Winkel (jc at ATComputing.nl) -Russia: - http://mirrors-ru.go-parts.com/gcc/";>http://mirrors-ru.go-parts.com/gcc -| ftp://mirrors-ru.go-parts.com/gcc";>ftp://mirrors-ru.go-parts.com/gcc -| rsync://mirrors-ru.go-parts.com/gcc, thanks to Dan Derebenskiy (dderebens...@go-parts.com) at Go-Parts. Slovakia, Bratislava: http://gcc.fyxm.net/";>gcc.fyxm.net, thanks to Jan Teluch (admin at 2600.sk) UK: ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/";>ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/, thanks to mirror at mirrorservice.org UK, London: http://gcc-uk.internet.bs";>http://gcc-uk.internet.bs, thanks to Internet.bs (info at internet.bs) -UK: - http://mirrors-uk.go-parts.com/gcc/";>http://mirrors-uk.go-parts.com/gcc/ -| ftp://mirrors-uk.go-parts.com/gcc";>ftp://mirrors-uk.go-parts.com/gcc -| rsync://mirrors-uk.go-parts.com/gcc US, Saint Louis: http://gcc.petsads.us";>http://gcc.petsads.us, thanks to Sergey Kutserey (s.kutserey at gmail.com) US, San Jose: http://www.netgull.com/gcc/";>http://www.netgull.com, thanks to admin at netgull.com US:
Re: Compiler support for erasure of sensitive data
On 09/09/2015 12:52 PM, paul_kon...@dell.com wrote: > Then again, suppose all you had is explicit_bzero, and an annotation > on the data saying it's sensitive. Can static code analyzers take > care of the rest? If so, this sort of thing doesn't need to be in > the compiler. The thing that absolutely has to be implemented in the compiler (AFAICT) is register clearing. I'm undecided as to how *necessary* that is. There certainly can be a lot of sensitive data in registers (e.g. AESNI puts an entire AES key schedule in xmm registers). I don't know of any exploits that depended on salvaging such data from registers, but I don't follow exploit research closely. zw
Re: [wwwdocs] PATCH forRe: Advertisement in the GCC mirrors list
Hi Gerald, It definitely was not. If it was, then we wouldn't keep the servers up for a year+ (for some other mirrors, 2+ years)We just discontinued our mirrors project a week or two ago, and haven't had time to contact everyone yet to take the links down. We will not be returning. Thank you. On 9/9/15 9:58 AM, Gerald Pfeifer wrote: On Wed, 9 Sep 2015, Jonathan Wakely wrote: Gerald, I think we've had similar issues with these mirrors in the past as well, shall we just remove them from the list? http://mirrors-ru.go-parts.com/gcc - Online Shop ftp://mirrors-ru.go-parts.com/gcc - bad rsync://mirrors-ru.go-parts.com/gcc - bad http://mirrors-uk.go-parts.com/gcc/ - Online Shop ftp://mirrors-uk.go-parts.com/gcc - bad rsync://mirrors-uk.go-parts.com/gcc - bad Yes. Immediately done with the patch below. Dan, heads up. I hope this was not the plan from the beginning? Gerald Index: mirrors.html === RCS file: /cvs/gcc/wwwdocs/htdocs/mirrors.html,v retrieving revision 1.230 diff -u -r1.230 mirrors.html --- mirrors.html23 Apr 2015 21:45:38 - 1.230 +++ mirrors.html9 Sep 2015 16:56:28 - @@ -45,17 +45,9 @@ rsync://mirror2.babylon.network/gcc/, thanks to Tim Semeijn (noc@babylon.network) at Babylon Network. The Netherlands, Nijmegen: ftp://ftp.nluug.nl/mirror/languages/gcc";>ftp.nluug.nl, thanks to Jan Cristiaan van Winkel (jc at ATComputing.nl) -Russia: - http://mirrors-ru.go-parts.com/gcc/";>http://mirrors-ru.go-parts.com/gcc -| ftp://mirrors-ru.go-parts.com/gcc";>ftp://mirrors-ru.go-parts.com/gcc -| rsync://mirrors-ru.go-parts.com/gcc, thanks to Dan Derebenskiy (dderebens...@go-parts.com) at Go-Parts. Slovakia, Bratislava: http://gcc.fyxm.net/";>gcc.fyxm.net, thanks to Jan Teluch (admin at 2600.sk) UK: ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/";>ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/, thanks to mirror at mirrorservice.org UK, London: http://gcc-uk.internet.bs";>http://gcc-uk.internet.bs, thanks to Internet.bs (info at internet.bs) -UK: - http://mirrors-uk.go-parts.com/gcc/";>http://mirrors-uk.go-parts.com/gcc/ -| ftp://mirrors-uk.go-parts.com/gcc";>ftp://mirrors-uk.go-parts.com/gcc -| rsync://mirrors-uk.go-parts.com/gcc US, Saint Louis: http://gcc.petsads.us";>http://gcc.petsads.us, thanks to Sergey Kutserey (s.kutserey at gmail.com) US, San Jose: http://www.netgull.com/gcc/";>http://www.netgull.com, thanks to admin at netgull.com US: -- Thank You, Dan Derebenskiy President, Parts Dynasty Corp. (916) 396-9118
Re: [wwwdocs] PATCH forRe: Advertisement in the GCC mirrors list
But we will keep the USA mirror up indefinitely. On 9/9/15 10:00 AM, Dan Derebenskiy wrote: Hi Gerald, It definitely was not. If it was, then we wouldn't keep the servers up for a year+ (for some other mirrors, 2+ years)We just discontinued our mirrors project a week or two ago, and haven't had time to contact everyone yet to take the links down. We will not be returning. Thank you. On 9/9/15 9:58 AM, Gerald Pfeifer wrote: On Wed, 9 Sep 2015, Jonathan Wakely wrote: Gerald, I think we've had similar issues with these mirrors in the past as well, shall we just remove them from the list? http://mirrors-ru.go-parts.com/gcc - Online Shop ftp://mirrors-ru.go-parts.com/gcc - bad rsync://mirrors-ru.go-parts.com/gcc - bad http://mirrors-uk.go-parts.com/gcc/ - Online Shop ftp://mirrors-uk.go-parts.com/gcc - bad rsync://mirrors-uk.go-parts.com/gcc - bad Yes. Immediately done with the patch below. Dan, heads up. I hope this was not the plan from the beginning? Gerald Index: mirrors.html === RCS file: /cvs/gcc/wwwdocs/htdocs/mirrors.html,v retrieving revision 1.230 diff -u -r1.230 mirrors.html --- mirrors.html23 Apr 2015 21:45:38 -1.230 +++ mirrors.html9 Sep 2015 16:56:28 - @@ -45,17 +45,9 @@ href="rsync://mirror2.babylon.network/gcc/">rsync://mirror2.babylon.network/gcc/, thanks to Tim Semeijn (noc@babylon.network) at Babylon Network. The Netherlands, Nijmegen: href="ftp://ftp.nluug.nl/mirror/languages/gcc";>ftp.nluug.nl, thanks to Jan Cristiaan van Winkel (jc at ATComputing.nl) -Russia: - href="http://mirrors-ru.go-parts.com/gcc/";>http://mirrors-ru.go-parts.com/gcc -| href="ftp://mirrors-ru.go-parts.com/gcc";>ftp://mirrors-ru.go-parts.com/gcc -| href="rsync://mirrors-ru.go-parts.com/gcc">rsync://mirrors-ru.go-parts.com/gcc, thanks to Dan Derebenskiy (dderebens...@go-parts.com) at Go-Parts. Slovakia, Bratislava: href="http://gcc.fyxm.net/";>gcc.fyxm.net, thanks to Jan Teluch (admin at 2600.sk) UK: href="ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/";>ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/, thanks to mirror at mirrorservice.org UK, London: href="http://gcc-uk.internet.bs";>http://gcc-uk.internet.bs, thanks to Internet.bs (info at internet.bs) -UK: - href="http://mirrors-uk.go-parts.com/gcc/";>http://mirrors-uk.go-parts.com/gcc/ -| href="ftp://mirrors-uk.go-parts.com/gcc";>ftp://mirrors-uk.go-parts.com/gcc -| href="rsync://mirrors-uk.go-parts.com/gcc">rsync://mirrors-uk.go-parts.com/gcc US, Saint Louis: href="http://gcc.petsads.us";>http://gcc.petsads.us, thanks to Sergey Kutserey (s.kutserey at gmail.com) US, San Jose: href="http://www.netgull.com/gcc/";>http://www.netgull.com, thanks to admin at netgull.com US: -- Thank You, Dan Derebenskiy President, Parts Dynasty Corp. (916) 396-9118
Re: [musl] Compiler support for erasure of sensitive data
On Wed, Sep 09, 2015 at 12:47:10PM -0400, Zack Weinberg wrote:
> On Wed, Sep 9, 2015 at 12:42 PM, Rich Felker wrote:
> > You're making this harder than it needs to be. The "m" constraint is
> > the wrong thing to use here. Simply use:
> >
> > __asm__(""::"r"(ptr):"memory");
>
> Please review my earlier conversation with Adhemerval on exactly this point.
My understanding is that you consider this a "big hammer". Does that
really matter if the intent is that it only be used in isolated,
sensitive contexts? Are you just unhappy with the performance cost, or
concerned that the clobber will cause more spilling of sensitive data?
I'm doubtful that this would happen because a "memory" clobber does
not affect all data cached in registers, only data which is
potentially reachable by the asm.
In any case, I think the intent of my reply was unclear. I did not
mean to detract from the idea of compiler support for handling of
sensitive data, just to point out that the hack with the "m"
constraint is wrong and easily fixed. It still may be possible to get
much better results via other means.
Rich
Re: [musl] Re: Compiler support for erasure of sensitive data
On Wed, Sep 09, 2015 at 12:58:36PM -0400, Zack Weinberg wrote: > On 09/09/2015 12:52 PM, paul_kon...@dell.com wrote: > > Then again, suppose all you had is explicit_bzero, and an annotation > > on the data saying it's sensitive. Can static code analyzers take > > care of the rest? If so, this sort of thing doesn't need to be in > > the compiler. > > The thing that absolutely has to be implemented in the compiler (AFAICT) > is register clearing. I'm undecided as to how *necessary* that is. > There certainly can be a lot of sensitive data in registers (e.g. AESNI > puts an entire AES key schedule in xmm registers). I don't know of any > exploits that depended on salvaging such data from registers, but I > don't follow exploit research closely. Conceptually you can "clear all registers" by calling an external asm function that clears all non-call-saved registers internally. Hardened implementations of explicit_bzero could even do this. The caller would (or at least should) not save registers whose old contents it does not intend to use again after the call. And of course all the call-saved registers will get restored when the function returns, preventing any leak via them. I agree it's much better if the compiler can do it, but I think the approach I described is a viable hardening technique that's immediately doable. Rich
Re: Compiler support for erasure of sensitive data
On Wed, Sep 9, 2015 at 12:36 PM, Zack Weinberg wrote: > The ABI dictates basically everything you see. The call to > explicit_bzero has forced the compiler to *create* a second copy of > the variable `k` on the stack, just so it can be erased -- and the > copy in registers survives (at least for a short time), which is not > what the programmer wanted. With or without explicit_bzero, we have > no way of getting rid of the copy in registers. More complicated > scenarios of course exist. > Comments? Please note that I do not have anything like the time > required to implement any of this myself (and I'm ten years out of > practice on GCC and have no experience whatsoever with Clang, > anyway). I'm hoping this catches someone's interest. What level of erasure of sensitive data are you trying to ensure? Assuming that overwriting values in the ISA registers actually completely clears and destroys the values is delusionally naive. Most modern hardware architectures have hardware capabilities to encrypt and protect sensitive data. - David
Re: Compiler support for erasure of sensitive data
> On Sep 9, 2015, at 1:54 PM, David Edelsohn wrote: > > On Wed, Sep 9, 2015 at 12:36 PM, Zack Weinberg wrote: > >> The ABI dictates basically everything you see. The call to >> explicit_bzero has forced the compiler to *create* a second copy of >> the variable `k` on the stack, just so it can be erased -- and the >> copy in registers survives (at least for a short time), which is not >> what the programmer wanted. With or without explicit_bzero, we have >> no way of getting rid of the copy in registers. More complicated >> scenarios of course exist. > >> Comments? Please note that I do not have anything like the time >> required to implement any of this myself (and I'm ten years out of >> practice on GCC and have no experience whatsoever with Clang, >> anyway). I'm hoping this catches someone's interest. > > What level of erasure of sensitive data are you trying to ensure? > Assuming that overwriting values in the ISA registers actually > completely clears and destroys the values is delusionally naive. Could you point to some references about that? > Most modern hardware architectures have hardware capabilities to > encrypt and protect sensitive data. I'm not sure about "most". I haven't worked on any that could do this. I agree it would be good to specify the threat model. Reading between the lines, I believe it is: capture of the software-visible process state after the code is finished with the sensitive data, either via a process dump file, or a debugger. With an explicitly stated list of goals and non-goals we can see whether a proposed solution addresses all, part, or none of the problem space, and whether it is a small solution or one much more powerful than is actually requested. If the threat is indeed delayed process state examination in software, then I think your "dangerously naive" does not apply. If you're talking excavating the chip and doing quantum forensics, that's a different matter. Another threat that I don't believe is covered here is disclosure of copies of the process state held in the OS, like saved context from thread switching, copies of stuff in the page file or in now-freed memory pages, or things like that. paul
Re: Compiler support for erasure of sensitive data
On Wed, Sep 9, 2015 at 2:02 PM, wrote: > >> On Sep 9, 2015, at 1:54 PM, David Edelsohn wrote: >> >> On Wed, Sep 9, 2015 at 12:36 PM, Zack Weinberg wrote: >> >>> The ABI dictates basically everything you see. The call to >>> explicit_bzero has forced the compiler to *create* a second copy of >>> the variable `k` on the stack, just so it can be erased -- and the >>> copy in registers survives (at least for a short time), which is not >>> what the programmer wanted. With or without explicit_bzero, we have >>> no way of getting rid of the copy in registers. More complicated >>> scenarios of course exist. >> >>> Comments? Please note that I do not have anything like the time >>> required to implement any of this myself (and I'm ten years out of >>> practice on GCC and have no experience whatsoever with Clang, >>> anyway). I'm hoping this catches someone's interest. >> >> What level of erasure of sensitive data are you trying to ensure? >> Assuming that overwriting values in the ISA registers actually >> completely clears and destroys the values is delusionally naive. > > Could you point to some references about that? > >> Most modern hardware architectures have hardware capabilities to >> encrypt and protect sensitive data. > > I'm not sure about "most". I haven't worked on any that could do this. Intel, Power, z/Arch, and (probably) SPARC. > > I agree it would be good to specify the threat model. Reading between the > lines, I believe it is: capture of the software-visible process state after > the code is finished with the sensitive data, either via a process dump file, > or a debugger. With an explicitly stated list of goals and non-goals we can > see whether a proposed solution addresses all, part, or none of the problem > space, and whether it is a small solution or one much more powerful than is > actually requested. > > If the threat is indeed delayed process state examination in software, then I > think your "dangerously naive" does not apply. If you're talking excavating > the chip and doing quantum forensics, that's a different matter. If this feature is implying / assuring that all values have been irrecoverably destroyed, and one can find the values in physical register files, then one is being dangerously naive in the assertions / expectations about the feature. One must specify the threat model. - David
Ubsan build of GCC 6.0 fails with: cp/search.c:1011:41: error: 'otype' may be used uninitialized in this function
See:
https://gcc.gnu.org/ml/gcc-testresults/2015-09/msg00699.html
Full error message:
/home/toon/compilers/trunk/gcc/cp/search.c: In function 'int
accessible_p(tree, tree, bool)':
/home/toon/compilers/trunk/gcc/cp/search.c:1011:41: error: 'otype' may
be used uninitialized in this function [-Werror=maybe-uninitialized]
dfs_accessible_data d = { decl, otype };
^
The host compiler is:
toon@moene:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 5.2.1-16'
--with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs
--enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++
--prefix=/usr --program-suffix=-5 --enable-shared
--enable-linker-build-id --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-libstdcxx-time=yes
--with-default-libstdcxx-abi=new --enable-gnu-unique-object
--disable-vtable-verify --enable-libmpx --enable-plugin
--with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
--enable-gtk-cairo
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre
--enable-java-home
--with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64
--with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64
--with-arch-directory=amd64
--with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc
--enable-multiarch --with-arch-32=i586 --with-abi=m64
--with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic
--enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.2.1 20150903 (Debian 5.2.1-16)
Any ideas ?
Kind regards,
--
Toon Moene - e-mail: t...@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG Maartensdijk, The Netherlands
At home: http://moene.org/~toon/; weather: http://moene.org/~hirlam/
Progress of GNU Fortran: http://gcc.gnu.org/wiki/GFortran#news
Re: [musl] Compiler support for erasure of sensitive data
On 09/09/2015 01:13 PM, Rich Felker wrote:
> On Wed, Sep 09, 2015 at 12:47:10PM -0400, Zack Weinberg wrote:
>> On Wed, Sep 9, 2015 at 12:42 PM, Rich Felker wrote:
>>> You're making this harder than it needs to be. The "m" constraint is
>>> the wrong thing to use here. Simply use:
>>>
>>> __asm__(""::"r"(ptr):"memory");
>>
>> Please review my earlier conversation with Adhemerval on exactly this point.
>
> My understanding is that you consider this a "big hammer". Does that
> really matter if the intent is that it only be used in isolated,
> sensitive contexts? Are you just unhappy with the performance cost, or
> concerned that the clobber will cause more spilling of sensitive data?
Please review *all* of my earlier conversation with Adhemerval, in
particular the bit where I compiled libressl three different ways and
analyzed the assembly dumps. I'm sure there's more to be said on the
topic, but *starting* from there.
> the hack with the "m" constraint is wrong and easily fixed
It's not wrong; it is in fact the documented way to express a fixed-size
read access to one block of memory. Look for "ten bytes of a string"
within https://gcc.gnu.org/onlinedocs/gcc-4.8.1/gcc/Extended-Asm.html
(sorry, there don't appear to be anchors).
It merely doesn't work in C++, with Clang, or (maybe) with a block of
memory whose size cannot be determined at compile time.
zw
Re: Compiler support for erasure of sensitive data
On 09/09/2015 02:02 PM, paul_kon...@dell.com wrote: >> On Sep 9, 2015, at 1:54 PM, David Edelsohn >> wrote: >> >> What level of erasure of sensitive data are you trying to ensure? >> Assuming that overwriting values in the ISA registers actually >> completely clears and destroys the values is delusionally naive. > > Could you point to some references about that? I *assume* David is referring to register renaming, which is not architecturally visible... ... > I agree it would be good to specify the threat model. Reading > between the lines, I believe it is: capture of the software-visible > process state after the code is finished with the sensitive data, > either via a process dump file, or a debugger. This is correct. Other avenues for the sensitive data to leak include use-after-free or out-of-bounds memory reads within the program, and malicious code (having gained control via some other bug) scanning memory in bulk. I would consider data leaks via state inaccessible to a program executing at the same privilege level as the code to be hardened to be out of scope. (Which does mean that *when hardening an OS kernel* one would possibly have to worry about special mechanisms that reveal the "true" register file, and the like, but I'd be plenty happy with something that was good enough for user mode.) Techniques involving direct manipulation of the hardware or the microcode, ditto. > Another threat that I don't believe is covered here is disclosure of > copies of the process state held in the OS, like saved context from > thread switching, copies of stuff in the page file or in now-freed > memory pages, or things like that. Some of that might conceivably be in-scope; jmp_buf comes to mind. (I'm not prepared to make an exhaustive list at the moment.) Normally, people programming this kind of code expect to have to lock pages in memory and so on. zw
Re: Ubsan build of GCC 6.0 fails with: cp/search.c:1011:41: error: 'otype' may be used uninitialized in this function
On 09/09/2015 12:36 PM, Toon Moene wrote:
See:
https://gcc.gnu.org/ml/gcc-testresults/2015-09/msg00699.html
Full error message:
/home/toon/compilers/trunk/gcc/cp/search.c: In function 'int
accessible_p(tree, tree, bool)':
/home/toon/compilers/trunk/gcc/cp/search.c:1011:41: error: 'otype' may
be used uninitialized in this function [-Werror=maybe-uninitialized]
dfs_accessible_data d = { decl, otype };
^
Any ideas ?
It looks as though GCC assumes that TYPE can be null even though
it can't (if it was, TYPE_P (type) would then dereference a null
pointer). As a workaround until this is fixed, initializing OTYPE
with type instead of in the else block should get rid of the error.
Here's a small test case that reproduces the bogus warning:
cat t.c && /build/gcc-trunk/gcc/xg++ -B /build/gcc-trunk/gcc
-Wmaybe-uninitialized -O2 -c -fsanitize=undefined t.c
struct S { struct S *next; int i; };
int foo (struct S *s) {
int i;
if (s->i) {
struct S *p;
for (p = s; p; p = p->next)
i = p->i;
}
else
i = 0;
return i;
}
t.c: In function ‘int foo(S*)’:
t.c:14:12: warning: ‘i’ may be used uninitialized in this function
[-Wmaybe-uninitialized]
return i;
^
Martin
Re: Ubsan build of GCC 6.0 fails with: cp/search.c:1011:41: error: 'otype' may be used uninitialized in this function
On 09/09/2015 01:17 PM, Martin Sebor wrote:
On 09/09/2015 12:36 PM, Toon Moene wrote:
See:
https://gcc.gnu.org/ml/gcc-testresults/2015-09/msg00699.html
Full error message:
/home/toon/compilers/trunk/gcc/cp/search.c: In function 'int
accessible_p(tree, tree, bool)':
/home/toon/compilers/trunk/gcc/cp/search.c:1011:41: error: 'otype' may
be used uninitialized in this function [-Werror=maybe-uninitialized]
dfs_accessible_data d = { decl, otype };
^
Any ideas ?
It looks as though GCC assumes that TYPE can be null even though
it can't (if it was, TYPE_P (type) would then dereference a null
pointer). As a workaround until this is fixed, initializing OTYPE
with type instead of in the else block should get rid of the error.
Here's a small test case that reproduces the bogus warning:
cat t.c && /build/gcc-trunk/gcc/xg++ -B /build/gcc-trunk/gcc
-Wmaybe-uninitialized -O2 -c -fsanitize=undefined t.c
struct S { struct S *next; int i; };
int foo (struct S *s) {
int i;
if (s->i) {
struct S *p;
for (p = s; p; p = p->next)
i = p->i;
}
else
i = 0;
return i;
}
t.c: In function ‘int foo(S*)’:
t.c:14:12: warning: ‘i’ may be used uninitialized in this function
[-Wmaybe-uninitialized]
return i;
More likely than not, the sanitization bits get in the way of VRP + jump
threading rotating the loop.
jeff
Re: [musl] Compiler support for erasure of sensitive data
On Wed, Sep 09, 2015 at 02:48:22PM -0400, Zack Weinberg wrote:
> On 09/09/2015 01:13 PM, Rich Felker wrote:
> > On Wed, Sep 09, 2015 at 12:47:10PM -0400, Zack Weinberg wrote:
> >> On Wed, Sep 9, 2015 at 12:42 PM, Rich Felker wrote:
> >>> You're making this harder than it needs to be. The "m" constraint is
> >>> the wrong thing to use here. Simply use:
> >>>
> >>> __asm__(""::"r"(ptr):"memory");
> >>
> >> Please review my earlier conversation with Adhemerval on exactly this
> >> point.
> >
> > My understanding is that you consider this a "big hammer". Does that
> > really matter if the intent is that it only be used in isolated,
> > sensitive contexts? Are you just unhappy with the performance cost, or
> > concerned that the clobber will cause more spilling of sensitive data?
>
> Please review *all* of my earlier conversation with Adhemerval, in
> particular the bit where I compiled libressl three different ways and
> analyzed the assembly dumps. I'm sure there's more to be said on the
> topic, but *starting* from there.
OK, sorry for jumping back in without the full context.
> > the hack with the "m" constraint is wrong and easily fixed
>
> It's not wrong; it is in fact the documented way to express a fixed-size
> read access to one block of memory. Look for "ten bytes of a string"
> within https://gcc.gnu.org/onlinedocs/gcc-4.8.1/gcc/Extended-Asm.html
> (sorry, there don't appear to be anchors).
>
> It merely doesn't work in C++, with Clang, or (maybe) with a block of
> memory whose size cannot be determined at compile time.
It relies on structs containing VLAs which are not standard C nor
supported by any "GNU C" compilers except GCC. And features like this
tend to be really fragile even in GCC because nobody uses them (for
good reason -- they can't be expected to work except on certain GCC
versions). You can disagree if you like, but that's why I called it
"wrong".
Rich
Re: Compiler support for erasure of sensitive data
* Zack Weinberg [2015-09-09 15:03:50 -0400]: > On 09/09/2015 02:02 PM, paul_kon...@dell.com wrote: > >> On Sep 9, 2015, at 1:54 PM, David Edelsohn > >> wrote: > >> > >> What level of erasure of sensitive data are you trying to ensure? > >> Assuming that overwriting values in the ISA registers actually > >> completely clears and destroys the values is delusionally naive. > > > > Could you point to some references about that? > > I *assume* David is referring to register renaming, which is not > architecturally visible... > or async signal handler copying all the register state on sigaltstack or internal counters and debug features making sensitive info observable or timing/cache-effect side channels that let other processes get info or compiling to a highlevel language (js) with different kind of leaks or running under emulator/debugger that can make secrets visible or... > I would consider data leaks via state inaccessible to a program > executing at the same privilege level as the code to be hardened to be > out of scope. (Which does mean that *when hardening an OS kernel* one specifying the info leak at the abstract c machine level is not useful (the memset is not observable there, unless you assign meaning to undefined behaviour which is a can of worms), but you do have to specify the leak on some abstraction level (that is applicable to the targets of a compiler and gives useful security properties in practice) otherwise the attribute is not meaningful. leaks can happen for many reasons that are layers below the control of the compiler, but still observable by high level code.
Re: Using the asm suffix
On 09/07/2015 06:56 PM, David Wohlferd wrote: In order for the doc maintainers to approve this patch, I need to have someone sign off on the technical accuracy. Now that I have included the points we have discussed (attached), hopefully we are there. Original text: https://gcc.gnu.org/onlinedocs/gcc/Asm-Labels.html Proposed text: http://limegreensocks.com/gcc/Asm-Labels.html Still pending is the line I removed about 'static variables in registers' that belongs in the Reg Vars section. I have additional changes I want to make to Reg Vars sections, so once this patch is accepted, I'll post that work. dw AsmLabels4.patch Index: extend.texi === --- extend.texi (revision 226751) +++ extend.texi (working copy) OK. Please install. jeff
gcc-4.9-20150909 is now available
Snapshot gcc-4.9-20150909 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/4.9-20150909/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 4.9 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-4_9-branch revision 227600 You'll find: gcc-4.9-20150909.tar.bz2 Complete GCC MD5=96efa2d72c794c15186239b197809ab0 SHA1=34734f943539a10488761886bafeca36a537cb5b Diffs from 4.9-20150902 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-4.9 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
