Re: Question about undefined functions' parameters during LTO

2020-03-13 Thread Richard Biener via Gcc
On Thu, Mar 12, 2020 at 5:31 PM Erick Ochoa
 wrote:
>
> Hello,
>
> I am trying to find out the arguments of functions which are undefined
> during LTO.
>
> Basically:
>
> gcc_assert(in_lto_p && !cnode->definition)
> // Do we have arguments?
> gcc_assert(DECL_ARGUMENTS(cnode->decl)) // fails
> // No, we don't.
>
> As I understand it, functions which are not defined are ones which have
> have been declared external.
>
> I believe that, when building an application with -flto, the only
> functions which are not visible during LTO **and** are declared external
> are functions defined in libraries which have not been compiled with
> -flto. An example of this is glibc.
>
> Indeed, I have implemented an analysis pass in gcc which prints out
> undefined functions, and it prints out the following:
>
> undefined function __gcov_merge_add
> undefined function fopen
> undefined function printf
> undefined function __builtin_putchar
> undefined function calloc
> undefined function __gcov_merge_topn
> undefined function strtol
> undefined function free
> ... and more
>
> Now, I am not interested in the bodies of these. I am only interested in
> determining the type of the arguments passed to these functions.
> However, when I call the following function:
>
> ```
> void
> print_parameters_undefined_functions(const cgraph_node *cnode)
> {
>gcc_assert(cnode);
>gcc_assert(in_lto_p);
>gcc_assert(!cnode->definition);
>
>tree function = cnode->decl;
>gcc_assert(function);
>enum tree_code code = TREE_CODE (function);
>bool is_function_decl = FUNCTION_DECL == code;
>gcc_assert (is_function_decl);
>
>log("about to print decl_arguments(%s)\n", cnode->name());
>for (tree parm = DECL_ARGUMENTS (function); parm; parm =
> DECL_CHAIN(parm))
>{
>  log("hello world\n");
>}
> ```
>
> I never see "hello world" but I do see "about to print...".
> Does anyone have any idea on how to obtain the arguments to undefined
> functions?

The argument types or the actual arguments to all calls?  "hello world" sounds
like you want actual arguments.  For those you need to look at the callgraph
edges to the cgraph node of the external functions (node->callers) and there
at the call stmts - which will not be available in WPA mode.

>
> The only way I see to do this, is to walk through the gimple
> instructions, find GIMPLE_CALL statements and look at the argument list
> at that moment. But I was wondering if there's a more efficient way to
> do it.
>
> Thanks!


Re: Question about undefined functions' parameters during LTO

2020-03-13 Thread Erick Ochoa




On 12.03.20 08:48, Jan Hubicka wrote:

Hello,

Hello,


I am trying to find out the arguments of functions which are undefined
during LTO.

Basically:

gcc_assert(in_lto_p && !cnode->definition)
// Do we have arguments?
gcc_assert(DECL_ARGUMENTS(cnode->decl)) // fails
// No, we don't.

As I understand it, functions which are not defined are ones which have have
been declared external.


LTO works in a way that function bodies are loaded from the stream on
demand (using node->get_body or node->get_untransformed_body calls).
Declaration of function parameters are part of function body and that is
why they are missing.

You can get all bodies if you experiment with your pass using the call
above. If you want real LTO pass working at whole propgram eventualy you
will need avoid reading all bodies (or WPA stage will be slow and memory
hungry) and store info you need into summaries.


Thanks. I did tried getting node->get_untransformed_body() and 
node->get_body() and for undefined functions which were not compiled 
with -flto (such as the ones on glibc) an epected exception is triggered.


Take for example the following simple ipa-pass that I have made only for 
this example (can be applied to

54e69cb00da0b50e4fa228a0617e4e8713bbc998 (upstream/master, gcc-master)
Author: GCC Administrator 
Date:   Fri Mar 13 00:16:15 2020 +)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index fa9923bb270..92421fe500e 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1399,6 +1399,7 @@ OBJS = \
incpath.o \
init-regs.o \
internal-fn.o \
+   ipa-hello-world.o \
ipa-cp.o \
ipa-sra.o \
ipa-devirt.o \
diff --git a/gcc/common.opt b/gcc/common.opt
index fa9da505fc2..df807e4e388 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -3392,4 +3392,8 @@ fipa-ra
 Common Report Var(flag_ipa_ra) Optimization
 Use caller save register across calls if possible.

+fipa-hello-world
+Common Report Var(flag_ipa_hello_world) Optimization
+TBD
+
 ; This comment is to ensure we retain the blank line above.
diff --git a/gcc/ipa-hello-world.c b/gcc/ipa-hello-world.c
new file mode 100644
index 000..402c62c6b59
--- /dev/null
+++ b/gcc/ipa-hello-world.c
@@ -0,0 +1,84 @@
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "backend.h"
+#include "tree.h"
+#include "gimple-expr.h"
+#include "predict.h"
+#include "alloc-pool.h"
+#include "tree-pass.h"
+#include "cgraph.h"
+#include "diagnostic.h"
+#include "fold-const.h"
+#include "gimple-fold.h"
+#include "symbol-summary.h"
+#include "tree-vrp.h"
+#include "ipa-prop.h"
+#include "tree-pretty-print.h"
+#include "tree-inline.h"
+#include "ipa-fnsummary.h"
+#include "ipa-utils.h"
+#include "tree-ssa-ccp.h"
+#include "stringpool.h"
+#include "attribs.h"
+
+
+#include 
+
+static unsigned int
+iphw_execute()
+{
+  cgraph_node *node = NULL;
+  std::set functions;
+  FOR_EACH_FUNCTION(node)
+  {
+functions.insert(node);
+  }
+
+  FOR_EACH_DEFINED_FUNCTION(node)
+  {
+functions.erase(node);
+  }
+
+  for (auto it = functions.cbegin(); it != functions.cend(); ++it)
+  {
+cgraph_node *undefined_function = *it;
+if (dump_file) fprintf(dump_file, "getting function body for %s\n", 
undefined_function->name());

+
+//undefined_function->get_untransformed_body();
+undefined_function->get_body();
+  }
+  return 0;
+}
+
+namespace {
+const pass_data pass_data_ipa_hello_world =
+{
+  SIMPLE_IPA_PASS,
+  "hello-world",
+  OPTGROUP_NONE,
+  TV_NONE,
+  (PROP_cfg | PROP_ssa),
+  0,
+  0,
+  0,
+  0,
+};
+
+class pass_ipa_hello_world : public simple_ipa_opt_pass
+{
+public:
+  pass_ipa_hello_world (gcc::context *ctx)
+: simple_ipa_opt_pass(pass_data_ipa_hello_world, ctx)
+  {}
+
+  virtual bool gate(function*) { return flag_ipa_hello_world; }
+  virtual unsigned execute (function*) { return iphw_execute(); }
+};
+} // anon namespace
+
+simple_ipa_opt_pass*
+make_pass_ipa_hello_world (gcc::context *ctx)
+{
+  return new pass_ipa_hello_world (ctx);
+}
diff --git a/gcc/passes.def b/gcc/passes.def
index 2bf2cb78fc5..66f333f81dc 100644
--- a/gcc/passes.def
+++ b/gcc/passes.def
@@ -149,6 +149,7 @@ along with GCC; see the file COPYING3.  If not see
   NEXT_PASS (pass_ipa_profile);
   NEXT_PASS (pass_ipa_icf);
   NEXT_PASS (pass_ipa_devirt);
+  NEXT_PASS (pass_ipa_hello_world);
   NEXT_PASS (pass_ipa_cp);
   NEXT_PASS (pass_ipa_sra);
   NEXT_PASS (pass_ipa_cdtor_merge);
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index a1207a20a3c..377dda689cc 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -501,6 +501,7 @@ extern ipa_opt_pass_d *make_pass_ipa_fn_summary 
(gcc::context *ctxt);

 extern ipa_opt_pass_d *make_pass_ipa_inline (gcc::context *ctxt);
 extern simple_ipa_opt_pass *make_pass_ipa_free_lang_data (gcc::context 
*ctxt);
 extern simple_ipa_opt_pass *make_pass_ipa_free_fn_summary 
(gcc::context *ctxt);

+extern simple_ipa_opt_pass *make_pass_ipa_hello_world (gcc::context *ctxt);
 extern ipa_opt_pass_d *make_

Re: Question about undefined functions' parameters during LTO

2020-03-13 Thread Erick Ochoa




On 13.03.20 00:44, Richard Biener wrote:

On Thu, Mar 12, 2020 at 5:31 PM Erick Ochoa
 wrote:


Hello,

I am trying to find out the arguments of functions which are undefined
during LTO.

Basically:

gcc_assert(in_lto_p && !cnode->definition)
// Do we have arguments?
gcc_assert(DECL_ARGUMENTS(cnode->decl)) // fails
// No, we don't.

As I understand it, functions which are not defined are ones which have
have been declared external.

I believe that, when building an application with -flto, the only
functions which are not visible during LTO **and** are declared external
are functions defined in libraries which have not been compiled with
-flto. An example of this is glibc.

Indeed, I have implemented an analysis pass in gcc which prints out
undefined functions, and it prints out the following:

undefined function __gcov_merge_add
undefined function fopen
undefined function printf
undefined function __builtin_putchar
undefined function calloc
undefined function __gcov_merge_topn
undefined function strtol
undefined function free
... and more

Now, I am not interested in the bodies of these. I am only interested in
determining the type of the arguments passed to these functions.
However, when I call the following function:

```
void
print_parameters_undefined_functions(const cgraph_node *cnode)
{
gcc_assert(cnode);
gcc_assert(in_lto_p);
gcc_assert(!cnode->definition);

tree function = cnode->decl;
gcc_assert(function);
enum tree_code code = TREE_CODE (function);
bool is_function_decl = FUNCTION_DECL == code;
gcc_assert (is_function_decl);

log("about to print decl_arguments(%s)\n", cnode->name());
for (tree parm = DECL_ARGUMENTS (function); parm; parm =
DECL_CHAIN(parm))
{
  log("hello world\n");
}
```

I never see "hello world" but I do see "about to print...".
Does anyone have any idea on how to obtain the arguments to undefined
functions?


The argument types or the actual arguments to all calls?  "hello world" sounds
like you want actual arguments.  For those you need to look at the callgraph
edges to the cgraph node of the external functions (node->callers) and there
at the call stmts - which will not be available in WPA mode.



I'm interested in the argument types to all calls of undefined 
functions. I have better defined what I mean by "undefined functions" in 
a sibling thread: https://gcc.gnu.org/pipermail/gcc/2020-March/231817.html


So, for example, what I'm trying to do is the following:

+static unsigned int
+iphw_execute()
+{
+  cgraph_node *node = NULL;
+  std::set functions;
+  FOR_EACH_FUNCTION(node)
+  {
+functions.insert(node);
+  }
+
+  FOR_EACH_DEFINED_FUNCTION(node)
+  {
+functions.erase(node);
+  }
+
+  for (auto it = functions.cbegin(); it != functions.cend(); ++it)
+  {
+cgraph_node *undefined_function = *it;
+const char *name = undefined_function->name();
+gcc_assert(name);
+if (dump_file) fprintf(dump_file, "getting function arguments for 
%s\n", name);

+
+gcc_assert(undefined_function->decl);
+for (tree parm = DECL_ARGUMENTS (undefined_function->decl); parm; 
parm = DECL_CHAIN (parm))

+ {
+   tree type = TREE_TYPE(parm);
+   if (dump_file) fprintf(dump_file, "I want the type, do I have 
it? %s\n", type ? "true" : "false");

+ }
+  }
+  return 0;
+}

I have added the complete patch below, however the function iphw_execute 
encapsulates the logic I am trying at the moment.


The problem is that while this program runs, DECL_ARGUMENTS returns NULL 
and therefore the loop is never entered. This is true for functions that 
have arguments, such as puts/malloc/... and others in glibc.


I suspect this is because glibc is not compiled with -flto (I don't 
believe it is possible to compile glibc with -flto) and during the 
compilation of a simple hello world the only reference to "puts" is an 
extern declaration in stdio.h


So, do you think there's a way to obtain the argument types without 
having to iterate over gimple code and look for GIMPLE_CALL and look for 
undefined functions, then look at the arguments and their types?


Thanks!

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index fa9923bb270..92421fe500e 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1399,6 +1399,7 @@ OBJS = \
incpath.o \
init-regs.o \
internal-fn.o \
+   ipa-hello-world.o \
ipa-cp.o \
ipa-sra.o \
ipa-devirt.o \
diff --git a/gcc/common.opt b/gcc/common.opt
index fa9da505fc2..df807e4e388 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -3392,4 +3392,8 @@ fipa-ra
 Common Report Var(flag_ipa_ra) Optimization
 Use caller save register across calls if possible.

+fipa-hello-world
+Common Report Var(flag_ipa_hello_world) Optimization
+TBD
+
 ; This comment is to ensure we retain the blank line above.
diff --git a/gcc/ipa-hello-world.c b/gcc/ipa-hello-world.c
new file mode 100644
index 000..5f71f3a411d
--- /dev/null
+++ b/gcc/ipa-hello-world.c

Re: How to extend SLP to support this case

2020-03-13 Thread Richard Biener via Gcc
On Tue, Mar 10, 2020 at 12:32 PM Tamar Christina 
wrote:

>
> > -Original Message-
> > From: Gcc  On Behalf Of Richard Biener
> > Sent: Tuesday, March 10, 2020 11:12 AM
> > To: Kewen.Lin 
> > Cc: GCC Development ; Segher Boessenkool
> > 
> > Subject: Re: How to extend SLP to support this case
> >
> > On Tue, Mar 10, 2020 at 7:52 AM Kewen.Lin  wrote:
> > >
> > > Hi all,
> > >
> > > I'm investigating whether GCC can vectorize the below case on ppc64le.
> > >
> > >   extern void test(unsigned int t[4][4]);
> > >
> > >   void foo(unsigned char *p1, int i1, unsigned char *p2, int i2)
> > >   {
> > > unsigned int tmp[4][4];
> > > unsigned int a0, a1, a2, a3;
> > >
> > > for (int i = 0; i < 4; i++, p1 += i1, p2 += i2) {
> > >   a0 = (p1[0] - p2[0]) + ((p1[4] - p2[4]) << 16);
> > >   a1 = (p1[1] - p2[1]) + ((p1[5] - p2[5]) << 16);
> > >   a2 = (p1[2] - p2[2]) + ((p1[6] - p2[6]) << 16);
> > >   a3 = (p1[3] - p2[3]) + ((p1[7] - p2[7]) << 16);
> > >
> > >   int t0 = a0 + a1;
> > >   int t1 = a0 - a1;
> > >   int t2 = a2 + a3;
> > >   int t3 = a2 - a3;
> > >
> > >   tmp[i][0] = t0 + t2;
> > >   tmp[i][2] = t0 - t2;
> > >   tmp[i][1] = t1 + t3;
> > >   tmp[i][3] = t1 - t3;
> > > }
> > > test(tmp);
> > >   }
> > >
> > > With unlimited costs, I saw loop aware SLP can vectorize it but with
> > > very inefficient codes.  It builds the SLP instance from store group
> > > {tmp[i][0] tmp[i][1] tmp[i][2] tmp[i][3]}, builds nodes {a0, a0, a0,
> > > a0}, {a1, a1, a1, a1}, {a2, a2, a2, a2}, {a3, a3, a3, a3} after
> > > parsing operands for tmp* and t*.  It means it's unable to make the
> > > isomorphic group for a0, a1, a2, a3, although they appears isomorphic
> > > to merge.  Even if it can recognize over_widening pattern and do some
> > > parallel for two a0 from two iterations, but it's still inefficient
> (high cost).
> > >
> > > In this context, it looks better to build  first by
> > > leveraging isomorphic computation trees constructing them, eg:
> > >   w1_0123 = load_word(p1)
> > >   V1_0123 = construct_vec(w1_0123)
> > >   w1_4567 = load_word(p1 + 4)
> > >   V1_4567 = construct_vec(w1_4567)
> > >   w2_0123 = load_word(p2)
> > >   V2_0123 = construct_vec(w2_0123)
> > >   w2_4567 = load_word(p2 + 4)
> > >   V2_4567 = construct_vec(w2_4567)
> > >   V_a0123 = (V1_0123 - V2_0123) + (V1_4567 - V2_4567)<<16
> > >
> > > But how to teach it to be aware of this? Currently the processing
> > > starts from bottom to up (from stores), can we do some analysis on the
> > > SLP instance, detect some pattern and update the whole instance?
> >
> > In theory yes (Tamar had something like that for AARCH64 complex
> rotations
> > IIRC).  And yes, the issue boils down to how we handle SLP discovery.
> I'd like
> > to improve SLP discovery but it's on my list only after I managed to get
> rid of
> > the non-SLP code paths.  I have played with some ideas (even produced
> > hackish patches) to find "seeds" to form SLP groups from using
> multi-level
> > hashing of stmts.
>
> I still have this but missed the stage-1 deadline after doing the
> rewriting to C++ 😊
>
> We've also been looking at this and the approach I'm investigating now is
> trying to get
> the SLP codepath to handle this after it's been fully unrolled. I'm
> looking into whether
> the build-slp can be improved to work for the group size == 16 case that
> it tries but fails
> on.
>
> My intention is to see if doing so would make it simpler to recognize this
> as just 4 linear
> loads and two permutes. I think the loop aware SLP will have a much harder
> time with this
> seeing the load permutations it thinks it needs because of the permutes
> caused by the +/-
> pattern.
>
> One Idea I had before was from your comment on the complex number patch,
> which is to try
> and move up TWO_OPERATORS and undo the permute always when doing +/-. This
> would simplify
> the load permute handling and if a target doesn't have an instruction to
> support this it would just
> fall back to doing an explicit permute after the loads.  But I wasn't sure
> this approach would get me the
> results I wanted.
>
> In the end you don't want a loop here at all. And in order to do the above
> with TWO_OPERATORS I would
> have to let the SLP pattern matcher be able to reduce the group size and
> increase the no# iterations during
> the matching otherwise the matching itself becomes quite difficult in
> certain cases.
>

Just to show where I'm heading I'm attaching current work-in-progress that
introduces
an explicit SLP merge node and implementing SLP_TREE_TWO_OPERATORS that way:

   v1 + v2  v1 - v2
   \  /
  merge

the SLP merge operation is concatenating operands in order and then applies
a lane permutation mask (basically a select from the input lanes).  The
actual patch
depends on earlier cleanups and more preparatory changes are in order to
make
it "nice".  With such SLP merge operation in place it sho

Re: Question about undefined functions' parameters during LTO

2020-03-13 Thread Michael Matz
Hello,

On Fri, 13 Mar 2020, Erick Ochoa wrote:

> +for (tree parm = DECL_ARGUMENTS (undefined_function->decl); parm; parm =
> DECL_CHAIN (parm))
> + {
> +   tree type = TREE_TYPE(parm);
> +   if (dump_file) fprintf(dump_file, "I want the type, do I have it?
> %s\n", type ? "true" : "false");
> + }
> +  }
> +  return 0;
> +}
> 
> I have added the complete patch below, however the function iphw_execute
> encapsulates the logic I am trying at the moment.
> 
> The problem is that while this program runs, DECL_ARGUMENTS returns NULL and
> therefore the loop is never entered. This is true for functions that have
> arguments, such as puts/malloc/... and others in glibc.

As argument (types) conceptually belong to the functions type (not its 
decl), you should look at the function decls type, not at DECL_ARGUMENTS.
See the FOREACH_FUNCTION_ARGS iterator and its helpers.  Note that you 
need to pass it TREE_TYPE(funcdecl).

(DECL_ARGUMENTS is the list of formal parameters viewed from the function 
bodies perspective, so without a body that isn't filled).


Ciao,
Michael.


commits in Bugzilla attributed to others?

2020-03-13 Thread Martin Sebor via Gcc

It looks as though commits with bug fixes appear in Bugzilla comments
made by others(*).  Fox instance, commit r10-7151 for PR 92071 shows
in comment #16 on the bug under Martin Liška's name.

  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92071#c16

Similarly, commits r9-8372, r10-7152, and r8-10121 show up in PR 94119
in comments with Martin's name on them:

  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94119

and likewise here:

  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91913
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94163
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94154
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94063

Is this a known problem?

[*] I haven't seen this for others so maybe Martin's account got messed
up in the migration somehow?

Martin


Re: commits in Bugzilla attributed to others?

2020-03-13 Thread Marek Polacek via Gcc
On Fri, Mar 13, 2020 at 09:46:40AM -0600, Martin Sebor via Gcc wrote:
> It looks as though commits with bug fixes appear in Bugzilla comments
> made by others(*).  Fox instance, commit r10-7151 for PR 92071 shows
> in comment #16 on the bug under Martin Liška's name.
> 
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92071#c16
> 
> Similarly, commits r9-8372, r10-7152, and r8-10121 show up in PR 94119
> in comments with Martin's name on them:
> 
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94119
> 
> and likewise here:
> 
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91913
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94163
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94154
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94063
> 
> Is this a known problem?
> 
> [*] I haven't seen this for others so maybe Martin's account got messed
> up in the migration somehow?

The authors of those commits are correct, it's just that the commit-to-Bugzilla
sync is broken at the moment and Martin L. was doing it manually.

Marek



Re: commits in Bugzilla attributed to others?

2020-03-13 Thread Martin Sebor via Gcc

On 3/13/20 9:50 AM, Marek Polacek wrote:

On Fri, Mar 13, 2020 at 09:46:40AM -0600, Martin Sebor via Gcc wrote:

It looks as though commits with bug fixes appear in Bugzilla comments
made by others(*).  Fox instance, commit r10-7151 for PR 92071 shows
in comment #16 on the bug under Martin Liška's name.

   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92071#c16

Similarly, commits r9-8372, r10-7152, and r8-10121 show up in PR 94119
in comments with Martin's name on them:

   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94119

and likewise here:

   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91913
   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94163
   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94154
   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94063

Is this a known problem?

[*] I haven't seen this for others so maybe Martin's account got messed
up in the migration somehow?


The authors of those commits are correct, it's just that the commit-to-Bugzilla
sync is broken at the moment and Martin L. was doing it manually.


I see.  Thanks for the explanation (and to Martin for all the work
to patch up the problem while it's being solved)!

Martin


Re: commits in Bugzilla attributed to others?

2020-03-13 Thread Marek Polacek via Gcc
On Fri, Mar 13, 2020 at 09:56:58AM -0600, Martin Sebor wrote:
> On 3/13/20 9:50 AM, Marek Polacek wrote:
> > On Fri, Mar 13, 2020 at 09:46:40AM -0600, Martin Sebor via Gcc wrote:
> > > It looks as though commits with bug fixes appear in Bugzilla comments
> > > made by others(*).  Fox instance, commit r10-7151 for PR 92071 shows
> > > in comment #16 on the bug under Martin Liška's name.
> > > 
> > >https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92071#c16
> > > 
> > > Similarly, commits r9-8372, r10-7152, and r8-10121 show up in PR 94119
> > > in comments with Martin's name on them:
> > > 
> > >https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94119
> > > 
> > > and likewise here:
> > > 
> > >https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91913
> > >https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94163
> > >https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94154
> > >https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94063
> > > 
> > > Is this a known problem?
> > > 
> > > [*] I haven't seen this for others so maybe Martin's account got messed
> > > up in the migration somehow?
> > 
> > The authors of those commits are correct, it's just that the 
> > commit-to-Bugzilla
> > sync is broken at the moment and Martin L. was doing it manually.
> 
> I see.  Thanks for the explanation (and to Martin for all the work
> to patch up the problem while it's being solved)!

And I should've said "has been doing it manually" because I don't think it's
been fixed yet :).

And yes, thanks Martin!

Marek



[GSoC 2020] Automatic Detection of Parallel Compilation Viability

2020-03-13 Thread Giuliano Belinassi via Gcc
Hi, all

I want to propose and apply for the following GSoC project: Automatic
Detection of Parallel Compilation Viability.

https://www.ime.usp.br/~belinass/Automatic_Detection_of_Parallel_Compilation_Viability.pdf

Feedback is welcome :)

Here is a markdown version of it:

**Automatic Detection of Parallel Compilation Viability**

[Giuliano Belinassi]{style="color: darkgreen"}\
Timezone: GMT$-$3:00\
University of São Paulo -- Brazil\
IRC: giulianob in \#gcc\
Email: [`giuliano.belina...@usp.br`](mailto:giuliano.belina...@usp.br)\
Github: \
Date:

About Me Computer Science Bachelor (University of São Paulo), currently
pursuing a Masters Degree in Computer Science at the same institution.
I've always been fascinated by topics such as High-Performance Computing
and Code Optimization, having worked with a parallel implementation of a
Boundary Elements Method software in GPU. I am currently conducting
research on compiler parallelization and developing the
[ParallelGcc](https://gcc.gnu.org/wiki/ParallelGcc) project, having
already presented it in [GNU Cauldron
2019](https://www.youtube.com/watch?v=jd6R3IK__1Q).

**Skills**: Strong knowledge in C, Concurrency, Shared Memory
Parallelism, Multithreaded Debugging and other typical programming
tools.

Brief Introduction

In [ParallelGcc](https://gcc.gnu.org/wiki/ParallelGcc), we showed that
parallelizing the Intra Procedural optimizations improves speed when
compiling huge files by a factor of 1.8x in a 4 cores machine, and also
showed that this takes 75% of compilation time.

In this project we plan to use the LTO infrastructure to improve
compilation performance in the non-LTO case, with a tradeoff of
generating a binary as good as if LTO is disabled. Here, we will
automatically detect when a single file will benefit from parallelism,
and proceed with the compilation in parallel if so.

Use of LTO

The Link Time Optimization (LTO) is a compilation technique that allows
the compiler to analyse the program as a whole, instead of analysing and
compiling one file at time. Therefore, LTO is able to collect more
information about the program and generate a better optimization plan.
LTO is divided in three parts:

-   *LGEN (Local Generation)*: Each file is translated to GIMPLE. This
stage runs sequentially in each file and, therefore, in parallel in
the project compilation.

-   *WPA (Whole Program Analysis)*: Run the Inter Procedural Analysis
(IPA) in the entire program. This state runs serially in the
project.

-   *LTRANS (Local Transformation)*: Execute all Intra Procedural
Optimizations in each partition. This stage runs in parallel.

Since WPA can bottleneck the compilation because it runs serially in the
entire project, LTO was designed to produce faster binaries, not to
produce binaries fast.

Here, the proposed use of LTO to address this problem is to run the IPA
for each Translation Unit (TU), instead in the Whole Program, and
automatically detect when to partition the TU into multiple LTRANS to
improve performance. The advantage of this approach is:

-   It can generate binaries as good as when LTO is disabled.

-   It is faster, as we can partition big files into multiple partitions
and compile these partitions in parallel

-   It can interact with GNU Make Jobserver, improving CPU utilization.

Planned Tasks

I plan to use the GSoC time to develop the following topics:

-   Week \[1, 3\] -- April 27 to May 15:\
Update `cc1`, `cc1plus`, `f771`, ..., to partition the data after
IPA analysis directly into multiple LTRANS partitions, instead of
generating a temporary GIMPLE file.

-   Week \[4, 7\] -- May 18 to June 12:\
Update the `gcc` driver to take these multiple LTRANS partitions,
then call the compiler and assembler for each of them, and merge the
results into one object file. Here I will use the LTO LTRANS object
streaming, therefore it should interact with GNU Make Jobserver.

-   Week 8 -- June 15 to 19: **First Evaluation**\
Deliver a non-optimized version of the project. Some programs ought
to be compiled correctly, but probably there will be a huge overhead
because so far there will not be any criteria about when to
partition. Some tests are also planned for this evaluation.

-   Week \[9, 11\] -- June 22 to July 10:\
Implement a criteria about when to partition, and interactively
improve it based on data.

-   Week 12 --- July 13 to 17: **Second Evaluation**\
Deliver a more optimized version of the project. Here we should
filter files that would compile fast from files that would require
partitioning, and therefore we should see some speedup.

-   Week \[13, 15\] --- July 20 to August 10:\
Develop adequate tests coverage and address unexpected issues so
that this feature can be merged to trunk for the next GCC release.

-   Week 16: **Final evaluation**\
Deliver the final product as a series of patches for trun

gcc-8-20200313 is now available

2020-03-13 Thread GCC Administrator
Snapshot gcc-8-20200313 is now available on
  https://gcc.gnu.org/pub/gcc/snapshots/8-20200313/
and on various mirrors, see http://gcc.gnu.org/mirrors.html for details.

This snapshot has been generated from the GCC 8 git branch
with the following options: git://gcc.gnu.org/git/gcc.git branch releases/gcc-8 
revision d72abee890682af7563541d4b174704db627b2a0

You'll find:

 gcc-8-20200313.tar.xzComplete GCC

  SHA256=41557def06c3e742e7a72b6a7feaf92d849f34ff7d22981df6dd25b98b8b74df
  SHA1=bcc0649d6780cdfdb4750ea85b6214dcad7fe95a

Diffs from 8-20200306 are available in the diffs/ subdirectory.

When a particular snapshot is ready for public consumption the LATEST-8
link is updated and a message is sent to the gcc list.  Please do not use
a snapshot before it has been announced that way.


Re: Compiling GCC using an older sysroot

2020-03-13 Thread Ton van den Heuvel via Gcc
On Thu, Mar 12, 2020 at 5:11 PM Paul Smith  wrote:
>
> On Wed, 2020-03-11 at 17:15 +, Jonathan Wakely wrote:
> > My thinking was to build GCC inside the container, then package up the
> > results and install that on your dev machines (outside a container).
> > But actually that won't work because you'd also need to package the
> > glibc from the container, and at that point you're basically back to
> > using a sysroot.
>
> Yes.  It's actually much simpler to use RPMs for this, and far more
> accurate.  I have a vague memory of trying to extract the necessary files
> from a live system, sometime in the past.  Hopefully I only attempted this
> for a few minutes before realizing that getting the RPMs was a far more
> reliable and efficient way to generate a sysroot :).
>
> Thanks!
>

FWIW, the Docker approach is exactly the approach we use at our
company to build recent GCC versions on CentOS5 in our case. What is
needed indeed are the -static-libgcc / -static-libstdc++ flags. Other
than that, it works perfectly. The binaries we ship run on CentOS5
upwards. These are built within a Docker container as well.

For local development, I sync the CentOS5 sysroot that we build in the
container to my local machine running Arch Linux (others at our
company use Ubuntu, Slackware) and I use that to build our software
outside the container. You need to run `mkheaders` that is bundled
with GCC to make this work but other than that it works perfectly and
has so for many years.