[PATCH] Write dependency information (-M*) even if there are errors

[Boris Kolpackov]

Hi,

Currently GCC does not write extracted header dependency information
if there are errors. However, this can be useful when dealing with
outdated generated headers that trigger errors which would have been
resolved if we could update it. A concrete example in our case is a
version check with #error.

The included (trivial) patch changes this behavior. Note also that
this is how Clang already behaves. I've tested the patch in build2
and everything works well (i.e., no invalid dependency output in the
face of various preprocessor errors such as #error, stray #else, etc).

While I don't foresee any backwards-compatibility issues with such
an unconditional change (after all, the compiler still exists with
an error status), if there are concerns, I could re-do it via an
option (e.g., -ME, analogous to -MG).

P.S. I have the paperwork necessary to contribute on file with FSF.

Thanks,
Boris
Index: gcc/c-family/ChangeLog
===
--- gcc/c-family/ChangeLog	(revision 250514)
+++ gcc/c-family/ChangeLog	(working copy)
@@ -1,3 +1,8 @@
+2017-08-06  Boris Kolpackov 
+
+	* c-opts.c (c_common_finish): Write dependency information even if
+	there are errors.
+
 2017-07-14  David Malcolm  
 
 	* c-common.c (try_to_locate_new_include_insertion_point): New
Index: gcc/c-family/c-opts.c
===
--- gcc/c-family/c-opts.c	(revision 250514)
+++ gcc/c-family/c-opts.c	(working copy)
@@ -1152,8 +1152,11 @@
 {
   FILE *deps_stream = NULL;
 
-  /* Don't write the deps file if there are errors.  */
-  if (cpp_opts->deps.style != DEPS_NONE && !seen_error ())
+  /* Note that we write the dependencies even if there are errors. This is
+ useful for handling outdated generated headers that now trigger errors
+ (for example, with #error) that would be resolved by re-generating
+ them. In a sense this complements -MG. */
+  if (cpp_opts->deps.style != DEPS_NONE)
 {
   /* If -M or -MM was seen without -MF, default output to the
 	 output stream.  */

Re: help with PR78809 - inline strcmp for small constant strings

[Martin Sebor]

On 08/04/2017 06:59 AM, Prathamesh Kulkarni wrote:

Hi,
I was having a look at PR78809.
For the test-case:
int t1(const char *s) { return __builtin_strcmp (s, "a"); }

for aarch64, trunk with -O2 generates:
t1:
adrpx1, .LANCHOR0
add x1, x1, :lo12:.LANCHOR0
b   strcmp

For i386, it seems strcmp is expanded inline via cmpstr optab by
expand_builtin_strcmp
if one of the strings is constant. Could we similarly define cmpstr
pattern for AArch64 ?

For constant strings of small length (upto 3?), I was wondering if
it'd be a good idea to
manually unroll strcmp loop, similar to __strcmp_* macros in bits/string.h  ?
For eg in gimple-fold, transform
x = __builtin_strcmp(s, "ab")
to
x = s[0] - 'a';
if (x == 0)
{
  x = s[1] - 'b';
  if (x == 0)
x = s[2];
}


IMO, in general, it's better do this sort of expansion into "low
level" code late, after higher-level optimizations have been done.
Otherwise it can defeat such optimizations in passes that aren't
prepared to handle the low level code.  For instance, in the
following, the call to strcmp could readily be eliminated in
tree-ssa-strlen because the pass knows both of the strings being
compared.  But if the strcmp() call were expanded as you show above
the optimization couldn't be implemented nearly as easily (not
without essentially undoing the work previously done by the folder).

  void f (char *s)
  {
strcpy (s, "a");

if (strcmp (s, "ab") > 0)
  abort ();
  }

Not only that, unless all the arguments are known, the early
expansion can also defeat checks for their validity.  For example,
in strcmp (a, b), if a and b point to the same non-null terminated
or even uninitialized) array the call is folded into zero without
a warning.  If/when tree-ssa-strlen (or some other pass) is enhanced
to detect such invalid strings this bug could be easily detected but
only as long as strcmp call makes it to the pass untransformed.

Martin

PS The examples above are only hypothetical because tree-ssa-
strlen doesn't at present handle strcmp.  Bugs 81703 and
81704 are examples where folding does get in the way today.

gcc-8-20170806 is now available

[gccadmin]

Snapshot gcc-8-20170806 is now available on
  ftp://gcc.gnu.org/pub/gcc/snapshots/8-20170806/
and on various mirrors, see http://gcc.gnu.org/mirrors.html for details.

This snapshot has been generated from the GCC 8 SVN branch
with the following options: svn://gcc.gnu.org/svn/gcc/trunk revision 250904

You'll find:

 gcc-8-20170806.tar.xzComplete GCC

  SHA256=4f6450ab349ae5c7d83837d3cfcd5ac86a3165f5116f221a37117a5656a1cea5
  SHA1=396f42d607fa36593d7daa15dcfdb3bb9f9fab0b

Diffs from 8-20170730 are available in the diffs/ subdirectory.

When a particular snapshot is ready for public consumption the LATEST-8
link is updated and a message is sent to the gcc list.  Please do not use
a snapshot before it has been announced that way.

Re: RFC [testsuite] Obey --load-average

[Daniel Santos]

On 08/03/2017 11:45 AM, Jeff Law wrote:
> On 08/02/2017 11:34 PM, Daniel Santos wrote:
> So does this perform better than make -j X -l X?  I use that with good
> success.
>
> jeff

Sorry for my slow response!

For a short answer, if you have 8 CPU cores and you run make -j8 -l8
check then everything is fine until cron needs to do something that eats
4 CPUs and you end up with a load average of 12.  This is because the
make jobs of the test harness are very long running and make's only
control lever for regulating load is to not start new jobs until the
load falls.  So if all jobs are already launched, make has no way of
reducing the load until all tests on that test set have completed.

Each job that make launches runs an expect script that iterates through
the tests for that .exp file and performs its own parallelization using
lock files written to the directory specified by the
GCC_RUNTEST_PARALLELIZE_DIR environment variable.  It's not a very
pretty picture, but it is a functioning integration.  What this patch
does is to interject a load test and sleep within the expect script's
loop prior to a job grabbing a new lock file.

I'm sure that there are more elegant solutions, but they are likely more
labor intensive.  There are still a few challenges with this approach
because parallelization isn't implemented in a uniform way throughout
the testsuite, so I'm working on disentangling it from other code in
gcc/testsuite/lib/gcc-defs.exp so that libstdc++-v3 can use the same
library.

This will be helpful for me when I need to update a bootstrap and then
re-run tests on both a baseline and a patched branch since I can run two
instances of the testsuite and not end up over-loaded.

Thanks,
Daniel

Re: RFC [testsuite] Obey --load-average

[Daniel Santos]

On 08/03/2017 05:07 PM, Mike Stump wrote:
> On Aug 2, 2017, at 10:34 PM, Daniel Santos  wrote:
>> I'm working on a patch to modify the testsuite to obey the
>> --load-average value if one is passed to make.
> The code seems like a reasonable approach.  Love to see numbers and test 
> scenarios so that others can tell if you've covered their use case.  -j 100 
> is different from -j 4.

Well I've re-tweaked it a little to ensure more even performance with
higher numbers of CPUs, but the only way that I can think of to assure
this given the scheme is to increase the maximum *tolerance* to
something like 1 per 10 CPUs:

set tolerance [expr {(0.0 + $num_jobs - $jobno - 1) / 10}]

Short of adding some type of IPC mechanism, I don't see another way to
do it,  but this doesn't mean that with -j 100 -l 100 we will run a load
average of 110 because only the lowest numbered jobs will use a max load
average of 110 to decide rather or not to run.  Job zero never sleeps,
job #1 will sleep until the load average drops to 109.8 or lower, job #2
until 109.7... job #98 until 100.1, and job #99 until 100.  So if one is
going for a fastest run (with 100 CPUs), something like make -j 100 -l
105 may be a good target, depending upon the statistics for when the
cost of task switching becomes greater than the loss of CPU cycles that
are unutilized due to I/O or whatever.

> People can help chip in numbers, if they have senarios that are less 
> represented.

Yes, that is my hope.

> I don't usually share or use -l, so I don't think I can help test it.  I do 
> wonder if it might be better to use a higher -j (I use -j somewhere between 
> 24 and 50) and use a load limit, even in my situation.

Well that is the theory.  It would be even better if there was a way to
ask the kernel's task accounting for a load average of the last x
seconds (maybe there is and I'm just not aware of it).  Of course, when
you have 24 cores and your load average is 50, then a massive number of
cpu cycles are lost to task switching, data cache (and possibly also
instruction cache) misses, tlb misses, etc.  (Hmm, this actually makes
me wonder if a cgroup can be configured to have a much longer time slice
before being preempted by other processes in its same cgroup.)


> The only concern would be that of portability.  Seems reasonable to let it in 
> and fix up any issues found after the fact.  I like how you ensure low impact 
> when -l isn't used.

Yes and I've already modified the patch to disable on cygwin and mingw
because they implement getloadavg by always returning zero.  I'm sure
you've probably seen the size of Windows' PEB and TEB that it must
initialize for *each* new process -- so the fewer forks or execs on that
OS the better.  I haven't tried to test on Windows in 4-5 months, but I
have this silly hope that a recently fixed race issue will make it
better.  Last time I was only able to run a single test thread at a time. 

Other than that, I hope that it will simply be disabled where it isn't
supported.  I suspect that getloadavg will be more portable than posix
semaphores, which I would eventually like to experiment with as an
alternative to both this patch and the 10 jobs per lock file mechanism.

> Minor nit, tollerance -> tolerance.

Thank you.  Until spell checker, I used to write "paralell" and I blame
dyslexia.  I don't have dyslexia, but it's still dyslexia's fault!

I'm attaching my latest patch, which happens to be wildly broken.  I've
completed a full test run and I'm missing .sum files for libatomic,
libgom, and libitm in addition to a vast number of failed tests, but I
think I'm getting closer!

Thanks,
Daniel

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index efca9169671..f26ff3840b8 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -4039,6 +4039,7 @@ check-parallel-% : site.exp
 	@test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir $(TESTSUITEDIR)/$(check_p_subdir)
 	-$(if $(check_p_subno),@)(rootme=`${PWD_COMMAND}`; export rootme; \
 	srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
+	GCC_RUNTEST_JOBNO=$(check_p_subno) ; export GCC_RUNTEST_JOBNO ; \
 	if [ -n "$(check_p_subno)" ] \
 	   && [ -n "$$GCC_RUNTEST_PARALLELIZE_DIR" ] \
 	   && [ -f $(TESTSUITEDIR)/$(check_p_tool)-parallel/finished ]; then \
diff --git a/gcc/testsuite/lib/gcc-defs.exp b/gcc/testsuite/lib/gcc-defs.exp
index d5fde7ce5e3..bdfd2f0ad65 100644
--- a/gcc/testsuite/lib/gcc-defs.exp
+++ b/gcc/testsuite/lib/gcc-defs.exp
@@ -20,6 +20,8 @@ load_lib wrapper.exp
 
 load_lib target-utils.exp
 
+load_lib parallelize.exp
+
 #
 # ${tool}_check_compile -- Reports and returns pass/fail for a compilation
 #
@@ -148,99 +150,6 @@ proc ${tool}_exit { } {
 }
 }
 
-#
-# runtest_file_p -- Provide a definition for older dejagnu releases
-# 		and assume the old syntax: foo1.exp bar1.c foo2.exp bar2.c.
-# 		(delete after next dejagnu release).
-#
-
-if { [info procs runtest_file_p] == "" } then {
-proc runtest_file_p { runtests testcase } {
-	if 

GCJ wiki page

[Vincent Lefevre]

Hi,

The GCJ wiki page https://gcc.gnu.org/wiki/GCJ appears to be very
obsolete. According to this page, GCJ is part of GCC, but AFAIK,
GCJ has been removed from GCC.

Regards,

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)