Re: Add -fdirect-access-external-data
* Fangrui Song: > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > proposes -fdirect-access-external-data to address some x86-64 > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > it can prevent copy relocations). > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > Since I am going to add this option to Clang and I hope (once GCC decides to > implement this option the two compilers can use the same option name), I bring > it to your attention. One worry I have is that people start building shared objects with direct data access, expecting the main program to be built with indirect access. We already see this today with Qt. It's not really supported well by the toolchain and causes frequent issues. Depending on the ELF ABI in question, the new pair of -f options might not actually be meaningful. It really depends on whether you have reasonably-sized displacements available. I think there are some ABIs where the optimization is theoretically possible, but impractical because the ilimit it imposes on data segment (think AArch64 without adrp).
Re: Add -fdirect-access-external-data
On Sat, Dec 26, 2020 at 7:32 AM Florian Weimer wrote: > > * Fangrui Song: > > > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > > proposes -fdirect-access-external-data to address some x86-64 > > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > > it can prevent copy relocations). > > > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > > > Since I am going to add this option to Clang and I hope (once GCC decides to > > implement this option the two compilers can use the same option name), I > > bring > > it to your attention. > > One worry I have is that people start building shared objects with > direct data access, expecting the main program to be built with > indirect access. We already see this today with Qt. It's not really > supported well by the toolchain and causes frequent issues. It can be solved by ABI extension implemented in linker, ld.so and compiler. > Depending on the ELF ABI in question, the new pair of -f options might > not actually be meaningful. It really depends on whether you have > reasonably-sized displacements available. I think there are some ABIs > where the optimization is theoretically possible, but impractical > because the ilimit it imposes on data segment (think AArch64 without > adrp). -- H.J.
gcc-10-20201226 is now available
Snapshot gcc-10-20201226 is now available on https://gcc.gnu.org/pub/gcc/snapshots/10-20201226/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 10 git branch with the following options: git://gcc.gnu.org/git/gcc.git branch releases/gcc-10 revision e731714eb2b4844771fa6f2a7a80d863490b8e6d You'll find: gcc-10-20201226.tar.xz Complete GCC SHA256=e6343b83651b0e75e346f7a0fbfc683c5e15bad4df20aed4a5df722a5a7bc4b5 SHA1=449a948bb8ad0c7d589d42947c63937a1cd861ab Diffs from 10-20201219 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-10 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
/Ou
King—hawlery
Potential bug in GCC when compiling C to a flat binary
Hi all,
I've encountered a strange issue when compiling C to a flat binary with GCC.
It's questionably a bug, but I hesitate to strongly say that due to my lack of
familiarity with the GCC codebase and the rather obscure nature of what I'm
trying to do. Posting this here in the hopes that someone can either a) confirm
this is in fact a GCC bug (in which case I'll file a bug report) or b) tell me
what I'm doing wrong.
First off, system information:
$ head -1 /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
$ uname -a
Linux desktop 5.9.0-0.bpo.2-amd64 #1 SMP Debian 5.9.6-1~bpo10+1 (2020-11-19)
x86_64 GNU/Linux
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 8.3.0-6'
--with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs
--enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr
--with-gcc-major-version-only --program-suffix=-8
--program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id
--libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
--libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu
--enable-libstdcxx-debug --enable-libstdcxx-time=yes
--with-default-libstdcxx-abi=new --enable-gnu-unique-object
--disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie
--with-system-zlib --with-target-system-zlib --enable-objc-gc=auto
--enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64
--with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic
--enable-offload-targets=nvptx-none --without-cuda-driver
--enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu
--target=x86_64-linux-gnu
Thread model: posix
gcc version 8.3.0 (Debian 8.3.0-6)
I'm currently in the process of writing a packer for ELF binaries (similar to
UPX). To do this, I compile my loader code into a position-independent flat
binary with no dependency on glibc using a custom linker script and then inject
it into an ELF binary, where it as well as the packed binary get loaded on
exec. The loader code is passed initial control, decrypts/decompresses the
packed binary, and hands control to the unpacked binary.
The problem comes when I try to take the address of a function that resides
outside the current translation unit in the loader code. Instead of &func
evaluating to the address of the function func, it evaluates to the first
sizeof(void *) bytes of the _code_ of the function func. I've created a minimal
reproduction here:
$ ls
func.c link.lds Makefile prog.c
$ cat prog.c
extern void func();
void _start()
{
void *ptr = (void *) func;
}
$ cat func.c
void func()
{
}
$ cat link.lds
OUTPUT_FORMAT("binary")
OUTPUT_ARCH(i386:x86-64)
ENTRY(_start)
SECTIONS
{
. = 0x0100;
.text : {
*(.text)
}
.data : {
*(.data)
}
}
rhys@desktop:~/repro$ cat Makefile
CFLAGS = -fpie -nostdlib -nostartfiles -nodefaultlibs -fno-builtin -c -I ..
LDFLAGS = -pie
OBJS = prog.o func.o
BIN = prog
all: $(OBJS)
$(LD) $(LDFLAGS) $(OBJS) -T link.lds -o $(BIN)
%.o: %.c
$(CC) $(CFLAGS) $< -o $@
%.o: %.S
$(AS) $< -o $@
clean:
rm -f $(OBJS) $(BIN)
As can be seen, this setup is pretty simple, _start gets control, and does
nothing other than load the address of func (which is located in another
translation unit) into ptr. Disassembling the output binary with radare2, I see
the following:
┌ 18: fcn. ();
│ ; var int64_t var_8h @ rbp-0x8
│ 0x 55 push rbp
│ 0x0001 4889e5 mov rbp, rsp
│ 0x0004 488b0507. mov rax, qword [0x0012] ;
fcn.0012
│ ;
[0x12:8]=0xc35d90e5894855
│ 0x000b 488945f8 mov qword [var_8h], rax
│ 0x000f 90 nop
│ 0x0010 5d pop rbp
└ 0x0011 c3 ret
┌ 7: fcn.0012 ();
│ 0x0012 55 push rbp
│ 0x0013 4889e5 mov rbp, rsp
│ 0x0016 90 nop
│ 0x0017 5d pop rbp
└ 0x0018 c3 ret
Note how fcn. corresponds to _start and fcn.0012 corresponds to
func. Now note the instructions at address 0x0004 and 0x000b. They
should be loading the _address_ 0x0012 into rax and then loading rax into
var_8h on the stack but instead load the eightbyte value _at_ address
0x0012 int rax (ie. the first eightbytes of func's code).
Removing the OUTPUT_FORMAT("binary") line from link.lds so that the linker
instead creates an ELF binary, the problem is not present:
┌ 18: entry0 ();
│ ; var int64_t va
Re: Potential bug in GCC when compiling C to a flat binary
On Sat, Dec 26, 2020 at 5:41 PM Rhys Rustad-Elliott wrote:
>
> Hi all,
>
> I've encountered a strange issue when compiling C to a flat binary with GCC.
> It's questionably a bug, but I hesitate to strongly say that due to my lack of
> familiarity with the GCC codebase and the rather obscure nature of what I'm
> trying to do. Posting this here in the hopes that someone can either a)
> confirm
> this is in fact a GCC bug (in which case I'll file a bug report) or b) tell me
> what I'm doing wrong.
>
> First off, system information:
>
> $ head -1 /etc/os-release
> PRETTY_NAME="Debian GNU/Linux 10 (buster)"
>
> $ uname -a
> Linux desktop 5.9.0-0.bpo.2-amd64 #1 SMP Debian 5.9.6-1~bpo10+1 (2020-11-19)
> x86_64 GNU/Linux
>
> $ gcc -v
> Using built-in specs.
> COLLECT_GCC=gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
> OFFLOAD_TARGET_NAMES=nvptx-none
> OFFLOAD_TARGET_DEFAULT=1
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Debian 8.3.0-6'
> --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs
> --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr
> --with-gcc-major-version-only --program-suffix=-8
> --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id
> --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
> --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
> --with-default-libstdcxx-abi=new --enable-gnu-unique-object
> --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie
> --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto
> --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64
> --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic
> --enable-offload-targets=nvptx-none --without-cuda-driver
> --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu
> --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 8.3.0 (Debian 8.3.0-6)
>
> I'm currently in the process of writing a packer for ELF binaries (similar to
> UPX). To do this, I compile my loader code into a position-independent flat
> binary with no dependency on glibc using a custom linker script and then
> inject
> it into an ELF binary, where it as well as the packed binary get loaded on
> exec. The loader code is passed initial control, decrypts/decompresses the
> packed binary, and hands control to the unpacked binary.
>
> The problem comes when I try to take the address of a function that resides
> outside the current translation unit in the loader code. Instead of &func
> evaluating to the address of the function func, it evaluates to the first
> sizeof(void *) bytes of the _code_ of the function func. I've created a
> minimal
> reproduction here:
>
> $ ls
> func.c link.lds Makefile prog.c
>
> $ cat prog.c
> extern void func();
>
> void _start()
> {
> void *ptr = (void *) func;
> }
>
> $ cat func.c
> void func()
> {
>
> }
>
> $ cat link.lds
> OUTPUT_FORMAT("binary")
> OUTPUT_ARCH(i386:x86-64)
> ENTRY(_start)
>
> SECTIONS
> {
> . = 0x0100;
> .text : {
> *(.text)
> }
> .data : {
> *(.data)
> }
> }
>
> rhys@desktop:~/repro$ cat Makefile
> CFLAGS = -fpie -nostdlib -nostartfiles -nodefaultlibs -fno-builtin -c -I ..
> LDFLAGS = -pie
>
> OBJS = prog.o func.o
> BIN = prog
>
> all: $(OBJS)
> $(LD) $(LDFLAGS) $(OBJS) -T link.lds -o $(BIN)
>
> %.o: %.c
> $(CC) $(CFLAGS) $< -o $@
>
> %.o: %.S
> $(AS) $< -o $@
>
> clean:
> rm -f $(OBJS) $(BIN)
>
> As can be seen, this setup is pretty simple, _start gets control, and does
> nothing other than load the address of func (which is located in another
> translation unit) into ptr. Disassembling the output binary with radare2, I
> see
> the following:
>
> ┌ 18: fcn. ();
> │ ; var int64_t var_8h @ rbp-0x8
> │ 0x 55 push rbp
> │ 0x0001 4889e5 mov rbp, rsp
> │ 0x0004 488b0507. mov rax, qword [0x0012] ;
> fcn.0012
> │ ;
> [0x12:8]=0xc35d90e5894855
> │ 0x000b 488945f8 mov qword [var_8h], rax
> │ 0x000f 90 nop
> │ 0x0010 5d pop rbp
> └ 0x0011 c3 ret
>
> ┌ 7: fcn.0012 ();
> │ 0x0012 55 push rbp
> │ 0x0013 4889e5 mov rbp, rsp
> │ 0x0016 90 nop
> │ 0x0017 5d pop rbp
> └ 0x0018 c3 ret
>
> Note how fcn. corresponds to _start and fcn.0012 corresponds to
> func. Now note the instructions at address 0x0004 and 0x000b. They
> should be loading the _address_ 0x0012 into rax and then loadin
