Ordering function layout for locality follow-up

[Kyrylo Tkachov via Gcc]

Hello,

Thanks to those that attended the IPA/LTO BoF at GNU Cauldron over the weekend 
and gave us feedback on teaching GCC to optimize for layout locality in the 
callgraph
I’d like to follow-up on the previous work in the area that Honza mentioned to 
see if we can reuse some of it or follow its best practices.
Could you give us some pointers on the previous attempts?
Also, I think richi suggested having a second WPA phase that does the layout 
separately from the partitioning.
Is that something we decided was a good idea?

Thanks again for the great discussion!
Kyrill

x86_64 regression tests arbitrary failing?

[Georg-Johann Lay]

For a trivial change that I'd like to bootstrap + regtest, I am
am getting FAILs like:

$ diff xxx/gcc/testsuite/gcc/gcc.sum  yyy/gcc/testsuite/gcc/gcc.sum
164656c164656
< PASS: c-c++-common/tsan/atomic_stack.c   -O0  output pattern test
---
> FAIL: c-c++-common/tsan/atomic_stack.c   -O0  output pattern test
164659c164659
< FAIL: c-c++-common/tsan/atomic_stack.c   -O2  output pattern test
---
> PASS: c-c++-common/tsan/atomic_stack.c   -O2  output pattern test
164662c164662
< PASS: c-c++-common/tsan/bitfield_race.c   -O0  output pattern test
---
> FAIL: c-c++-common/tsan/bitfield_race.c   -O0  output pattern test
164668c164668
< PASS: c-c++-common/tsan/fd_pipe_race.c   -O0  output pattern test
---
> FAIL: c-c++-common/tsan/fd_pipe_race.c   -O0  output pattern test
164674c164674
< FAIL: c-c++-common/tsan/free_race.c   -O0  output pattern test
---
> PASS: c-c++-common/tsan/free_race.c   -O0  output pattern test

Is there anything special so that these tests don't arbitrary flip
their PASS/FAIL state?

The GCC version that's being tested is current trunk.
The build compiler is GCC v14.2.1.

The two testsuites (one for xxx and one for yyy) are being run
at the same time (in their tmux panes).  For what I know tests
may run at the same time when they are in different build
directories.  The configure command is basically

$ ../../source/gcc/configure

The command to run the testsuites is

$ make check

from the respective top level build directory.

Johann

Re: [CAULDRON] Topics for the Toolchain and Linux kernel BoF

[Jakub Jelinek via Gcc]

On Thu, Sep 12, 2024 at 06:09:53PM +0200, Jose E. Marchesi via Gcc wrote:
> - "noreturn" and jump tables run-time hints
> 
>   It has been expressed on the kernel side the desire of having the C compiler
>   emit run-time hints marking functions that are not supposed to return and
>   also to provide annotations on jump tables.  This is for the benefit of
>   objtool in arm64, see references below.
> 
>   Goal of the discussion:
> 
>   Collect and assess the requirements of these features, discuss their
>   pertinence and the way it could be best implemented.  The outcome of the
>   discussion will then be used to continue the discussion with the clang/llvm
>   and kernel hackers at LPC.
> 
>   References:
>   
> https://lore.kernel.org/linux-arm-kernel/yylmhuxtuanza...@hirez.programming.kicks-ass.net/

What I was suggesting is something like (completely untested, GPL v2.0+ in
case you'd like to use anything from that):
#include "gcc-common.h"

__visible int plugin_is_GPL_compatible;

static struct plugin_info mark_noreturn_plugin_info = {
.version= PLUGIN_VERSION,
.help   = "mark noreturn plugin\n",
};

static unsigned int mark_noreturn_execute(void)
{
if (flags_from_decl_or_type(current_function_decl) & ECF_NORETURN) {
const char *name = 
IDENTIFIER_POINTER(DECL_ASSEMBLER_NAME(current_function_decl));
name = targetm.strip_name_encoding(name);
name = concat("__noreturn_function.", name, NULL);
tree decl = build_decl(UNKNOWN_LOCATION, FUNCTION_DECL,
   get_identifier(name), 
TREE_TYPE(current_function_decl));
DECL_ARTIFICIAL(decl) = 1;
TREE_PUBLIC(decl) = 0;
DECL_WEAK(decl) = DECL_WEAK(current_function_decl);
assemble_alias(current_function_decl, decl);
}
return 0;
}

#define PASS_NAME mark_noreturn

#define NO_GATE

#include "gcc-generate-gimple-pass.h"

__visible int plugin_init(struct plugin_name_args *plugin_info, struct 
plugin_gcc_version *version)
{
int i;
const char * const plugin_name = plugin_info->base_name;
const int argc = plugin_info->argc;
const struct plugin_argument * const argv = plugin_info->argv;
bool enable = true;

PASS_INFO(mark_noreturn, "optimized", 0, PASS_POS_INSERT_BEFORE);

if (!plugin_default_version_check(version, &gcc_version)) {
error(G_("incompatible gcc/plugin versions"));
return 1;
}

for (i = 0; i < argc; ++i) {
if (!strcmp(argv[i].key, "no-mark-noreturn")) {
enable = false;
continue;
}
error(G_("unknown option '-fplugin-arg-%s-%s'"), plugin_name, 
argv[i].key);
}

register_callback(plugin_name, PLUGIN_INFO, NULL, 
&mark_noreturn_plugin_info);

if (!enable)
return 0;

#if BUILDING_GCC_VERSION < 6000
register_callback(plugin_name, PLUGIN_PASS_MANAGER_SETUP, NULL, 
&mark_noreturn_pass_info);
#endif

return 0;
}

Basically just create a __noreturn_function.foobarbaz alias to foobarbaz
function if that function is noreturn.  Haven't investigated if all Linux
kernel arches will be happy with . in the name, other options are $ or
_ or conditionally one of them e.g. based on NO_DOT_IN_LABEL/NO_DOLLAR_IN_LABEL
macros.

> - Struct layout randomization (-frandomize-struct-layout) and debug info
> 
>   The GCC plugin hooks in a way that emitted debug info doesn't match with the
>   resulting randomized structs.  It works in clang because it generates DWARF
>   later in the compilation process.
> 
>   References:
> 
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84052
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116457
> 
>   Goal of the discussion:
> 
>   Determine how to best fix this in the plugin, or by using a different
>   approach.  The outcome of the discussion will then be used to continue the
>   discussion with the clang/llvm and kernel hackers at LPC.
> 
> - Userland stack unwinding from within the Linux kernel
> 
>   There are reasons for wanting to unwind both userland and kernel
>   stacks from within the kernel.  Currently the kernel can unwind kernel
>   stacks based on ORC (which is revese-engineered from kernel compiled
>   objects by objtool) and userland stacks provided stack frame pointers
>   are present.  SFrame is a format similar to ORC, but general enough to
>   be used in userspace, and there is an on-going effort to introduce a
>   SFrame based unwinder in the kernel.  This will require some glibc
>   support as well.
> 
>   References:
> 
>   First prototype (V1) from Josh Poimboeuf:
>   https://lkml.kernel.org/lkml/cover.1699487758.git.jpoim...@kernel.org/

As discussed, either submit a patch to add another plugin callback event
after a new structure is almost ready to be finali

Re: [CAULDRON] Topics for the Toolchain and Linux kernel BoF

[Jose E. Marchesi via Gcc]

[Adding Josh in CC]

> On Thu, Sep 12, 2024 at 06:09:53PM +0200, Jose E. Marchesi via Gcc wrote:
>> - "noreturn" and jump tables run-time hints
>> 
>>   It has been expressed on the kernel side the desire of having the C 
>> compiler
>>   emit run-time hints marking functions that are not supposed to return and
>>   also to provide annotations on jump tables.  This is for the benefit of
>>   objtool in arm64, see references below.
>> 
>>   Goal of the discussion:
>> 
>>   Collect and assess the requirements of these features, discuss their
>>   pertinence and the way it could be best implemented.  The outcome of the
>>   discussion will then be used to continue the discussion with the clang/llvm
>>   and kernel hackers at LPC.
>> 
>>   References:
>>   
>> https://lore.kernel.org/linux-arm-kernel/yylmhuxtuanza...@hirez.programming.kicks-ass.net/
>
> What I was suggesting is something like (completely untested, GPL v2.0+ in
> case you'd like to use anything from that):
> #include "gcc-common.h"
>
> __visible int plugin_is_GPL_compatible;
>
> static struct plugin_info mark_noreturn_plugin_info = {
> .version= PLUGIN_VERSION,
> .help   = "mark noreturn plugin\n",
> };
>
> static unsigned int mark_noreturn_execute(void)
> {
>   if (flags_from_decl_or_type(current_function_decl) & ECF_NORETURN) {
>   const char *name = 
> IDENTIFIER_POINTER(DECL_ASSEMBLER_NAME(current_function_decl));
>   name = targetm.strip_name_encoding(name);
>   name = concat("__noreturn_function.", name, NULL);
>   tree decl = build_decl(UNKNOWN_LOCATION, FUNCTION_DECL,
>  get_identifier(name), 
> TREE_TYPE(current_function_decl));
>   DECL_ARTIFICIAL(decl) = 1;
>   TREE_PUBLIC(decl) = 0;
>   DECL_WEAK(decl) = DECL_WEAK(current_function_decl);
>   assemble_alias(current_function_decl, decl);
>   }
>   return 0;
> }
>
> #define PASS_NAME mark_noreturn
>
> #define NO_GATE
>
> #include "gcc-generate-gimple-pass.h"
>
> __visible int plugin_init(struct plugin_name_args *plugin_info, struct 
> plugin_gcc_version *version)
> {
>   int i;
>   const char * const plugin_name = plugin_info->base_name;
>   const int argc = plugin_info->argc;
>   const struct plugin_argument * const argv = plugin_info->argv;
>   bool enable = true;
>
>   PASS_INFO(mark_noreturn, "optimized", 0, PASS_POS_INSERT_BEFORE);
>
>   if (!plugin_default_version_check(version, &gcc_version)) {
>   error(G_("incompatible gcc/plugin versions"));
>   return 1;
>   }
>
>   for (i = 0; i < argc; ++i) {
>   if (!strcmp(argv[i].key, "no-mark-noreturn")) {
>   enable = false;
>   continue;
>   }
>   error(G_("unknown option '-fplugin-arg-%s-%s'"), plugin_name, 
> argv[i].key);
>   }
>
>   register_callback(plugin_name, PLUGIN_INFO, NULL, 
> &mark_noreturn_plugin_info);
>
>   if (!enable)
>   return 0;
>
> #if BUILDING_GCC_VERSION < 6000
>   register_callback(plugin_name, PLUGIN_PASS_MANAGER_SETUP, NULL, 
> &mark_noreturn_pass_info);
> #endif
>
>   return 0;
> }
>
> Basically just create a __noreturn_function.foobarbaz alias to foobarbaz
> function if that function is noreturn.  Haven't investigated if all Linux
> kernel arches will be happy with . in the name, other options are $ or
> _ or conditionally one of them e.g. based on 
> NO_DOT_IN_LABEL/NO_DOLLAR_IN_LABEL
> macros.
>
>> - Struct layout randomization (-frandomize-struct-layout) and debug info
>> 
>>   The GCC plugin hooks in a way that emitted debug info doesn't match with 
>> the
>>   resulting randomized structs.  It works in clang because it generates DWARF
>>   later in the compilation process.
>> 
>>   References:
>> 
>>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84052
>>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116457
>> 
>>   Goal of the discussion:
>> 
>>   Determine how to best fix this in the plugin, or by using a different
>>   approach.  The outcome of the discussion will then be used to continue the
>>   discussion with the clang/llvm and kernel hackers at LPC.
>> 
>> - Userland stack unwinding from within the Linux kernel
>> 
>>   There are reasons for wanting to unwind both userland and kernel
>>   stacks from within the kernel.  Currently the kernel can unwind kernel
>>   stacks based on ORC (which is revese-engineered from kernel compiled
>>   objects by objtool) and userland stacks provided stack frame pointers
>>   are present.  SFrame is a format similar to ORC, but general enough to
>>   be used in userspace, and there is an on-going effort to introduce a
>>   SFrame based unwinder in the kernel.  This will require some glibc
>>   support as well.
>> 
>>   References:
>> 
>>   First prototype (V1) from Josh Poimboeuf:
>>   https://lkml.kernel.org/lkml/cove

Re: x86_64 regression tests arbitrary failing?

[Arsen Arsenović via Gcc]

Hi,

Georg-Johann Lay  writes:

> For a trivial change that I'd like to bootstrap + regtest, I am
> am getting FAILs like:
>
> $ diff xxx/gcc/testsuite/gcc/gcc.sum  yyy/gcc/testsuite/gcc/gcc.sum
> 164656c164656
> < PASS: c-c++-common/tsan/atomic_stack.c   -O0  output pattern test
> ---
>> FAIL: c-c++-common/tsan/atomic_stack.c   -O0  output pattern test
> 164659c164659
> < FAIL: c-c++-common/tsan/atomic_stack.c   -O2  output pattern test
> ---
>> PASS: c-c++-common/tsan/atomic_stack.c   -O2  output pattern test
> 164662c164662
> < PASS: c-c++-common/tsan/bitfield_race.c   -O0  output pattern test
> ---
>> FAIL: c-c++-common/tsan/bitfield_race.c   -O0  output pattern test
> 164668c164668
> < PASS: c-c++-common/tsan/fd_pipe_race.c   -O0  output pattern test
> ---
>> FAIL: c-c++-common/tsan/fd_pipe_race.c   -O0  output pattern test
> 164674c164674
> < FAIL: c-c++-common/tsan/free_race.c   -O0  output pattern test
> ---
>> PASS: c-c++-common/tsan/free_race.c   -O0  output pattern test
>
> Is there anything special so that these tests don't arbitrary flip
> their PASS/FAIL state?
>
> The GCC version that's being tested is current trunk.
> The build compiler is GCC v14.2.1.
>
> The two testsuites (one for xxx and one for yyy) are being run
> at the same time (in their tmux panes).  For what I know tests
> may run at the same time when they are in different build
> directories.

They can.  That's fine.

> The configure command is basically
>
> $ ../../source/gcc/configure
>
> The command to run the testsuites is
>
> $ make check
>
> from the respective top level build directory.

For TSAN tests, this might happen on some machines, for a reason I am
yet to determine (I've also seen it on cfarm420, but not on my PC).

If you wish to confirm this is spurious, you could rerun that part of
the testsuite on the same source tree and compare the results.  I expect
you'll get differences again (and not the same ones as the ones you
showed above).

I'd love to know why this happens, but haven't looked far into it yet.
-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: Update bootstrap requirement to C++14?

[Jason Merrill via Gcc]

On 9/16/24 11:14 AM, Florian Weimer wrote:

* Jason Merrill via Gcc:


We moved to a bootstrap requirement of C++11 in GCC 11, 8 years after
support was stable in GCC 4.8.


Note that some documentation still says C++03, for example:

| The directories gcc, libcpp and fixincludes may use C++03. They may
| also use the long long type if the host C++ compiler supports
| it. These directories should use reasonably portable parts of C++03,
| so that it is possible to build GCC with C++ compilers other than GCC
| itself. If testing reveals that reasonably recent versions of non-GCC
| C++ compilers cannot compile GCC, then GCC code should be adjusted
| accordingly. (Avoiding unusual language constructs helps immensely.)
| Furthermore, these directories should also be compatible with C++11.



I think this is just out of date.


Indeed.  I had a patch to fix this in my local tree since January, which 
I have now finally pushed.


Jason

Re: x86_64 regression tests arbitrary failing?

[Sam James via Gcc]

Arsen Arsenović via Gcc  writes:

> Hi,
>
> Georg-Johann Lay  writes:
>
>> For a trivial change that I'd like to bootstrap + regtest, I am
>> am getting FAILs like:
>>
>> $ diff xxx/gcc/testsuite/gcc/gcc.sum  yyy/gcc/testsuite/gcc/gcc.sum
>> 164656c164656
>> < PASS: c-c++-common/tsan/atomic_stack.c   -O0  output pattern test
>> ---
>>> FAIL: c-c++-common/tsan/atomic_stack.c   -O0  output pattern test
>> 164659c164659
>> < FAIL: c-c++-common/tsan/atomic_stack.c   -O2  output pattern test
>> ---
>>> PASS: c-c++-common/tsan/atomic_stack.c   -O2  output pattern test
>> 164662c164662
>> < PASS: c-c++-common/tsan/bitfield_race.c   -O0  output pattern test
>> ---
>>> FAIL: c-c++-common/tsan/bitfield_race.c   -O0  output pattern test
>> 164668c164668
>> < PASS: c-c++-common/tsan/fd_pipe_race.c   -O0  output pattern test
>> ---
>>> FAIL: c-c++-common/tsan/fd_pipe_race.c   -O0  output pattern test
>> 164674c164674
>> < FAIL: c-c++-common/tsan/free_race.c   -O0  output pattern test
>> ---
>>> PASS: c-c++-common/tsan/free_race.c   -O0  output pattern test
>>
>> Is there anything special so that these tests don't arbitrary flip
>> their PASS/FAIL state?
>>
>> The GCC version that's being tested is current trunk.
>> The build compiler is GCC v14.2.1.
>>
>> The two testsuites (one for xxx and one for yyy) are being run
>> at the same time (in their tmux panes).  For what I know tests
>> may run at the same time when they are in different build
>> directories.
>
> They can.  That's fine.
>
>> The configure command is basically
>>
>> $ ../../source/gcc/configure
>>
>> The command to run the testsuites is
>>
>> $ make check
>>
>> from the respective top level build directory.
>
> For TSAN tests, this might happen on some machines, for a reason I am
> yet to determine (I've also seen it on cfarm420, but not on my PC).
>
> If you wish to confirm this is spurious, you could rerun that part of
> the testsuite on the same source tree and compare the results.  I expect
> you'll get differences again (and not the same ones as the ones you
> showed above).

In the past, TSAN tests (like the rest of the sanitizers) have been
sensitive to kernel version, config, and sysctl toggles. But I agree
it's a bit odd that these keep flip-flopping (I've seen it too).

I know Jakub plans a libsanitizer sync at some point for trunk, so
probably worth investigating after that happens.