GCC 5 snapshots produce broken kernel for powerpc-e500v2-linux-gnuspe?

[Arseny Solokha]

Hello,

I've recently faced an issue I'm afraid I currently unable to debug. When
building an arbitrary version of Linux kernel for powerpc-e500v2-linux-gnuspe
target, it seems gcc prior to 5 produces a good image which boots just fine, and
current gcc 5 snapshots (4.10.0-alpha20140810 for example) produce an image
which hangs just after U-Boot hands over to the kernel.

This behavior is well reproducible on real hardware as well as under qemu. I've
prepared a minimal kernel config which is dysfunctional as is but still enough
to demonstrate the problem in qemu. I believe the exact Linux version number
doesn't actually matter here, but see the attachment for details.

Compare the output produced by u-boot and this minified kernel build using
gcc 4.9.1 and 4.10.0-alpha20140810 snapshot.

% qemu-system-ppc --version
QEMU emulator version 2.1.0, Copyright (c) 2003-2008 Fabrice Bellard

% qemu-system-ppc -cpu e500v2 -M mpc8544ds -bios /usr/share/qemu/u-boot.e500 \
> -kernel arch/powerpc/boot/uImage-gcc4.9.1 -nographic


U-Boot 2014.07-rc1-00079-g2072e72-dirty (May 16 2014 - 13:04:54)

CPU:   Unknown, Version: 0.0, (0x)
Core:  e500, Version: 2.2, (0x80210022)
Clock Configuration:
   CPU0:400  MHz,
   CCB:400  MHz,
   DDR:200  MHz (400 MT/s data rate), LBC: unknown (LCRR[CLKDIV] = 0x00)
L1:D-cache 32 KiB enabled
   I-cache 32 KiB enabled
DRAM:  128 MiB
L2:disabled
Using default environment


PCI: base address e0008000
  00:11.0 - 1af4:1000 - Network controller
PCI1: Bus 00 - 00

In:serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  0
## Booting kernel from Legacy Image at 0200 ...
   Image Name:   Linux-2.6.35+
   Image Type:   PowerPC Linux Kernel Image (gzip compressed)
   Data Size:507635 Bytes = 495.7 KiB
   Load Address: 
   Entry Point:  
   Verifying Checksum ... OK
## Flattened Device Tree blob at e800
   Booting using the fdt blob at 0xe800
   Uncompressing Kernel Image ... OK
   Loading Device Tree to 03fec000, end 03ffefff ... OK
setup_arch: bootmem
mpc85xx_ds_setup_arch()
arch: exit


% qemu-system-ppc -cpu e500v2 -M mpc8544ds -bios /usr/share/qemu/u-boot.e500 \
> -kernel arch/powerpc/boot/uImage-gcc5 -nographic


U-Boot 2014.07-rc1-00079-g2072e72-dirty (May 16 2014 - 13:04:54)



Hit any key to stop autoboot:  0
## Booting kernel from Legacy Image at 0200 ...
   Image Name:   Linux-2.6.35+
   Image Type:   PowerPC Linux Kernel Image (gzip compressed)
   Data Size:505303 Bytes = 493.5 KiB
   Load Address: 
   Entry Point:  
   Verifying Checksum ... OK
## Flattened Device Tree blob at e800
   Booting using the fdt blob at 0xe800
   Uncompressing Kernel Image ... OK
   Loading Device Tree to 03fec000, end 03ffefff ... OK


I also have another seemingly related issue: kernel module built
w/ 4.10.0-alpha20140810 against a kernel configured
for powerpc-e500v2-linux-gnuspe snapshot lacks .gnu.linkonce.this_module and
.rela.gnu.linkonce.this_module sections. Current stable versions of gcc emit
these sections but also drop it (and, consequently, init_module() and
cleanup_module() symbols) when CFLAGS_MODULE is modified in any way, even w/
CFLAGS_MODULE="-mno-isel" for example.

I now have completely no idea what to do next to find a cause of (1) gcc 5
snapshots producing unbootable kernel, and (2) different gcc versions producing
garbled kernel modules when configured for SPE target. As for modules, I've
compared assembler output of gcc configured for powerpc-e300c3-linux-gnu and
powerpc-e500v2-linux-gnuspe and failed to spot real differences, but installed
version of binutils is exactly the same for both configurations, and I'm
completely sure my setup is correct.


Regards.
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.35
# Tue Sep  9 16:29:19 2014
#
# CONFIG_PPC64 is not set

#
# Processor support
#
# CONFIG_PPC_BOOK3S_32 is not set
CONFIG_PPC_85xx=y
# CONFIG_PPC_8xx is not set
# CONFIG_40x is not set
# CONFIG_44x is not set
# CONFIG_E200 is not set
CONFIG_E500=y
# CONFIG_PPC_E500MC is not set
CONFIG_FSL_EMB_PERFMON=y
CONFIG_FSL_EMB_PERF_EVENT=y
CONFIG_FSL_EMB_PERF_EVENT_E500=y
CONFIG_BOOKE=y
CONFIG_FSL_BOOKE=y
# CONFIG_PHYS_64BIT is not set
# CONFIG_SPE is not set
CONFIG_PPC_MMU_NOHASH=y
CONFIG_PPC_MMU_NOHASH_32=y
CONFIG_PPC_BOOK3E_MMU=y
# CONFIG_PPC_MM_SLICES is not set
# CONFIG_SMP is not set
CONFIG_PPC32=y
CONFIG_WORD_SIZE=32
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
CONFIG_MMU=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
# CONFIG_HAVE_SETUP_PER_CPU_AREA is not set
# CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK is not set
CONFIG_IRQ_PER_CPU=y
CONFIG_NR_IRQS=512
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_RWSEM_XCHGADD_ALGO

Re: GCC 5 snapshots produce broken kernel for powerpc-e500v2-linux-gnuspe?

[Markus Trippelsdorf]

On 2014.09.09 at 17:35 +0800, Arseny Solokha wrote:
> Hello,
> 
> I've recently faced an issue I'm afraid I currently unable to debug. When
> building an arbitrary version of Linux kernel for powerpc-e500v2-linux-gnuspe
> target, it seems gcc prior to 5 produces a good image which boots just fine, 
> and
> current gcc 5 snapshots (4.10.0-alpha20140810 for example) produce an image
> which hangs just after U-Boot hands over to the kernel.
> 
> This behavior is well reproducible on real hardware as well as under qemu. 
> I've
> prepared a minimal kernel config which is dysfunctional as is but still enough
> to demonstrate the problem in qemu. I believe the exact Linux version number
> doesn't actually matter here, but see the attachment for details.
> 
> Compare the output produced by u-boot and this minified kernel build using
> gcc 4.9.1 and 4.10.0-alpha20140810 snapshot.
> 
> I now have completely no idea what to do next to find a cause of (1) gcc 5
> snapshots producing unbootable kernel, 

gcc trunk also miscompiles x86_64 kernels currently, but I haven't
looked deeper yet. 
The best way to narrow down the issue is to use git (or svn) bisect to
find out which gcc revision causes the miscompile.  Then you can md5sum
the kernel object files for the bad revision and for the first good
revision and compare the results. After that you can look at the
disassembly of the object files, for which md5sum differs, and try to
figure out the reason why.

-- 
Markus

Re: [PATCH] RE: gcc parallel make check

[Yury Gribov]

On 09/09/2014 10:51 AM, VandeVondele Joost wrote:
> Attached is an extended version of the patch,
> it brings a 100% improvement in make -j32 -k check-gcc

First of all, many thanks for working on this.

+#   ls -1 | ../../../contrib/generate_tcl_patterns.sh 300 
"dg.exp=gfortran.dg/"


How does this work with subdirectories? Can we replace ls with find?

-check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 \
+   21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

$(shell seq 1 40) ?

+  if (_assert_exit) exit 1

Haven't you already exited above?

> A second part of the patch is a new file 
'contrib/generate_tcl_patterns.sh'

> which generates the needed regexp

Can we provide a Makefile target to automatically update Makefile.in?

-Y

Re: GCC 5 snapshots produce broken kernel for powerpc-e500v2-linux-gnuspe?

[pinskia]

> On Sep 9, 2014, at 2:57 AM, Markus Trippelsdorf  
> wrote:
> 
>> On 2014.09.09 at 17:35 +0800, Arseny Solokha wrote:
>> Hello,
>> 
>> I've recently faced an issue I'm afraid I currently unable to debug. When
>> building an arbitrary version of Linux kernel for powerpc-e500v2-linux-gnuspe
>> target, it seems gcc prior to 5 produces a good image which boots just fine, 
>> and
>> current gcc 5 snapshots (4.10.0-alpha20140810 for example) produce an image
>> which hangs just after U-Boot hands over to the kernel.
>> 
>> This behavior is well reproducible on real hardware as well as under qemu. 
>> I've
>> prepared a minimal kernel config which is dysfunctional as is but still 
>> enough
>> to demonstrate the problem in qemu. I believe the exact Linux version number
>> doesn't actually matter here, but see the attachment for details.
>> 
>> Compare the output produced by u-boot and this minified kernel build using
>> gcc 4.9.1 and 4.10.0-alpha20140810 snapshot.
>> 
>> I now have completely no idea what to do next to find a cause of (1) gcc 5
>> snapshots producing unbootable kernel, 
> 
> gcc trunk also miscompiles x86_64 kernels currently, but I haven't
> looked deeper yet. 
> The best way to narrow down the issue is to use git (or svn) bisect to
> find out which gcc revision causes the miscompile.  Then you can md5sum
> the kernel object files for the bad revision and for the first good
> revision and compare the results. After that you can look at the
> disassembly of the object files, for which md5sum differs, and try to
> figure out the reason why.


I have a patch which I need to submit. Maybe by Friday I will do that. It fixes 
the kernel on arm64 but it is generic c front-end patch. 

> 
> -- 
> Markus

Re: [PATCH] RE: gcc parallel make check

[Jakub Jelinek]

On Tue, Sep 09, 2014 at 02:02:18PM +0400, Yury Gribov wrote:
> On 09/09/2014 10:51 AM, VandeVondele Joost wrote:
> > Attached is an extended version of the patch,
> > it brings a 100% improvement in make -j32 -k check-gcc
> 
> First of all, many thanks for working on this.
> 
> +#   ls -1 | ../../../contrib/generate_tcl_patterns.sh 300
> "dg.exp=gfortran.dg/"
> 
> How does this work with subdirectories? Can we replace ls with find?

Generally, if the argument to *.exp doesn't contain a particular
subdirectory, then the wildcard is taken against basenames of the tests.

> -check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
> +check_p_numbers=1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 \
> + 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
> 
> $(shell seq 1 40) ?

Would that be sufficiently portable to weirdo hosts (M$Win, Darwin, ...)?
We require GNU make, so if it can be written in GNU make text functions,
fine, otherwise it is better to keep as is.
> 
> +  if (_assert_exit) exit 1
> 
> Haven't you already exited above?
> 
> > A second part of the patch is a new file
> 'contrib/generate_tcl_patterns.sh'
> > which generates the needed regexp
> 
> Can we provide a Makefile target to automatically update Makefile.in?

No.  As I wrote earlier, splitting on filenames and test counts only is only
very rough split, all the splits really need to be backed out by real timing
data from popular targets.  Also, I'm afraid of some tests being left out
unintentionally (e.g. the wildcards created at some point, then a new test
is added with a weird starting character that hasn't been used before and
suddenly it will not be tested with make -j?).

Jakub

Re: [gomp4] openacc kernels directive support

[Tom de Vries]

On 18-08-14 14:16, Tom de Vries wrote:

On 06-08-14 17:10, Tom de Vries wrote:

We could insert a pass-group here that only deals with functions that have the
kernels directive, and do the auto-par thing in a pass_oacc_kernels (which
should share the majority of the infrastructure with the parloops pass):
...
   NEXT_PASS (pass_build_ealias);
   INSERT_PASSES_AFTER/WITHIN (passes_oacc_kernels)
  NEXT_PASS (pass_ch);
  NEXT_PASS (pass_ccp);
  NEXT_PASS (pass_lim_aux);
  NEXT_PASS (pass_oacc_par);
   POP_INSERT_PASSES ()
...

Any comments, ideas or suggestions ?


I've experimented with implementing this on top of gomp-4_0-branch, and I ran
into PR46032.

PR46032 is about vectorization failure on a function split off by omp
parallelization. The vectorization fails due to aliasing constraints in the
split off function, which are not present in the original code.

In the gomp-4_0-branch, the code marked by the openacc kernels directive is
split off during omp_expand. The generated code has the same additional aliasing
constraints, and in pass_oacc_par the parallelization fails.

The PR46032 contains a tentative patch by Richard Biener, which applies cleanly
on top of 4.6 (I haven't yet reached a level of understanding of
tree-ssa-structalias.c to be able to resolve the conflict in
intra_create_variable_infos when applying on 4.7). The tentative patch involves
running ipa-pta, which is also a pass run after the point where we write out the
lto stream. I'm not sure whether it makes sense to run the pta-ipa pass as part
of the pass_oacc_kernels pass list.

I see three ways of continuing from here:
- take the tentative patch and make it work, including running pta-ipa during
   passes_oacc_kernels
- same, but try somehow to manage without running pta-ipa.
- try to postpone splitting of the function until the end of pass_oacc_par.

Some advice on how to continue from here would be *highly* appreciated. My hunch
atm is to investigate the last option.



Jakub,
Richard,

I've investigated the last option, and published the current state in git-only 
branch vries/oacc-kernels ( 
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vries/oacc-kernels ).


The current state at commit 9255cadc5b6f8f7f4e4506e65a6be7fb3c00cd35 is that:
- a simple loop marked with the oacc kernels directive is analyzed for
   parallelization,
- the loop is then rewritten using oacc parallel and oacc loop directives
- these oacc directives are expanded using omp_expand_local
- this results in the loop being split off into a separate function, while
   the loop is replaced with a GOACC_parallel call
- all this is done before writing out the lto stream
- no support yet for reductions, nested loops, more than one loop nest in
  kernels region

At toplevel, the added pass list looks like this:
...
  NEXT_PASS (pass_build_ealias);
  /* Pass group that runs when there are oacc kernels in the
 function.  */
  NEXT_PASS (pass_oacc_kernels);
  PUSH_INSERT_PASSES_WITHIN (pass_oacc_kernels)
  NEXT_PASS (pass_ch_oacc_kernels);
  NEXT_PASS (pass_tree_loop_init);
  NEXT_PASS (pass_lim);
  NEXT_PASS (pass_ccp);
  NEXT_PASS (pass_parallelize_loops_oacc_kernels);
  NEXT_PASS (pass_tree_loop_done);
  POP_INSERT_PASSES ()
 ...

The main question I'm currently facing is the following: when to do lowering (in 
other words, rewriting of variable access in terms of .omp_data) of the kernels 
region. There are basically 2 passes that contain code to do this:

- pass_lower_omp (on pre-ssa code)
- pass_parallelize_loops (on ssa code)

Atm I'm using pass_lower_omp, and I've added a patch that handles omp-lowered 
code conservatively in ccp and forwprop in order for the lowering to remain 
until arriving at pass_parallelize_loops_oacc_kernels.


But it might turn out to be easier/necessary to handle this in 
pass_parallelize_loops_oacc_kernels instead.


Any advice on this issue, and on the current implementation is welcome.

Thanks,
- Tom

RE: [PATCH] RE: gcc parallel make check

[VandeVondele Joost]

> +#   ls -1 | ../../../contrib/generate_tcl_patterns.sh 300
> "dg.exp=gfortran.dg/"
> 
> How does this work with subdirectories? Can we replace ls with find?

The input to the script is general, you can use this to your advantage. For 
example, I've been using:

 ls -1 g++.*/* | cut -c5- | ../../../contrib/generate_tcl_patterns.sh 700 
old-deja.exp=g++.old-deja/g++.

to split at a deeper level or

find . -name "[0-9A-Za-z]*" -type f -printf "%f\n" | 
../../../../contrib/generate_tcl_patterns.sh 300 dg-torture.exp=torture/

to collect statistics also from subdirs.

> +  if (_assert_exit) exit 1
>
> Haven't you already exited above?

yes, but the END{} block in awk is nevertheless executed, unless protected as 
above.

RE: [PATCH] RE: gcc parallel make check

[VandeVondele Joost]

> No.  As I wrote earlier, splitting on filenames and test counts only is only
> very rough split, all the splits really need to be backed out by real timing
> data from popular targets.  

I'm actually doing quite some testing trying to get a reasonable balance, 
checking 'completed in' in all *.log.sep files. However, it is important that 
the procedure is semi-automatic, otherwise few people will be interested in 
doing so. Furthermore, for parallel performance, it is not so important that 
times are distributed evenly (it is anyway unlikely the number of goals is 
exactly divided by N of -jN), but rather that the goals are ordered (executed) 
from slow to fast (similar to omp schedule guided). Most of the real 
bottlenecks are single letter patterns (e.g. p* since pr is such a common 
filename), and this is ultimately limiting.

In the project (CP2K) I'm working on, we also parallelize testing over 
directories, but we keep a list of approximate runtimes per directory, and keep 
that (global) list sorted. Testing follows that list. As a result, we have near 
perfect parallel speedup, despite (or because) timings per directory ranging 
from a few 100s to 1s. 

> Also, I'm afraid of some tests being left out
> unintentionally (e.g. the wildcards created at some point, then a new test
> is added with a weird starting character that hasn't been used before and
> suddenly it will not be tested with make -j?).

I agree this is an issue, partially addressed by not having to write patterns 
by hand anymore (i.e. a script does this), and by having the script check its 
input. There are something like 10 testnames that do not fall in [0-9A-Za-z], 
as mentioned in a previous email.

Re: [gomp4] openacc kernels directive support

[Richard Biener]

On Tue, 9 Sep 2014, Tom de Vries wrote:

> On 18-08-14 14:16, Tom de Vries wrote:
> > On 06-08-14 17:10, Tom de Vries wrote:
> > > We could insert a pass-group here that only deals with functions that have
> > > the
> > > kernels directive, and do the auto-par thing in a pass_oacc_kernels (which
> > > should share the majority of the infrastructure with the parloops pass):
> > > ...
> > >NEXT_PASS (pass_build_ealias);
> > >INSERT_PASSES_AFTER/WITHIN (passes_oacc_kernels)
> > >   NEXT_PASS (pass_ch);
> > >   NEXT_PASS (pass_ccp);
> > >   NEXT_PASS (pass_lim_aux);
> > >   NEXT_PASS (pass_oacc_par);
> > >POP_INSERT_PASSES ()
> > > ...
> > > 
> > > Any comments, ideas or suggestions ?
> > 
> > I've experimented with implementing this on top of gomp-4_0-branch, and I
> > ran
> > into PR46032.
> > 
> > PR46032 is about vectorization failure on a function split off by omp
> > parallelization. The vectorization fails due to aliasing constraints in the
> > split off function, which are not present in the original code.

Heh.  At least the omp-low.c parts from comment #1 should be pushed
to trunk...

> > In the gomp-4_0-branch, the code marked by the openacc kernels directive is
> > split off during omp_expand. The generated code has the same additional
> > aliasing
> > constraints, and in pass_oacc_par the parallelization fails.
> > 
> > The PR46032 contains a tentative patch by Richard Biener, which applies
> > cleanly
> > on top of 4.6 (I haven't yet reached a level of understanding of
> > tree-ssa-structalias.c to be able to resolve the conflict in
> > intra_create_variable_infos when applying on 4.7). The tentative patch
> > involves
> > running ipa-pta, which is also a pass run after the point where we write out
> > the
> > lto stream. I'm not sure whether it makes sense to run the pta-ipa pass as
> > part
> > of the pass_oacc_kernels pass list.

No, that's not even possible I think.

> > I see three ways of continuing from here:
> > - take the tentative patch and make it work, including running pta-ipa
> > during
> >passes_oacc_kernels
> > - same, but try somehow to manage without running pta-ipa.
> > - try to postpone splitting of the function until the end of pass_oacc_par.

I don't understand the last option?  What is the actual issue you run 
into?  You split oacc kernels off and _then_ run "autopar" on the 
split-off function (and get additional kernels)?

> > Some advice on how to continue from here would be *highly* appreciated. My
> > hunch
> > atm is to investigate the last option.
> > 
> 
> Jakub,
> Richard,
> 
> I've investigated the last option, and published the current state in git-only
> branch vries/oacc-kernels (
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vries/oacc-kernels
> ).
> 
> The current state at commit 9255cadc5b6f8f7f4e4506e65a6be7fb3c00cd35 is that:
> - a simple loop marked with the oacc kernels directive is analyzed for
>parallelization,
> - the loop is then rewritten using oacc parallel and oacc loop directives
> - these oacc directives are expanded using omp_expand_local
> - this results in the loop being split off into a separate function, while
>the loop is replaced with a GOACC_parallel call
> - all this is done before writing out the lto stream
> - no support yet for reductions, nested loops, more than one loop nest in
>   kernels region
> 
> At toplevel, the added pass list looks like this:
> ...
>   NEXT_PASS (pass_build_ealias);
>   /* Pass group that runs when there are oacc kernels in the
>  function.  */

Not sure why pass_oacc_kernels runs before all the other local
cleanups?  I would have put it after pass_cd_dce at least.

>   NEXT_PASS (pass_oacc_kernels);
>   PUSH_INSERT_PASSES_WITHIN (pass_oacc_kernels)
>   NEXT_PASS (pass_ch_oacc_kernels);
>   NEXT_PASS (pass_tree_loop_init);
>   NEXT_PASS (pass_lim);
>   NEXT_PASS (pass_ccp);
>   NEXT_PASS (pass_parallelize_loops_oacc_kernels);
>   NEXT_PASS (pass_tree_loop_done);
>   POP_INSERT_PASSES ()
>  ...
> 
> The main question I'm currently facing is the following: when to do lowering
> (in other words, rewriting of variable access in terms of .omp_data) of the
> kernels region. There are basically 2 passes that contain code to do this:
> - pass_lower_omp (on pre-ssa code)
> - pass_parallelize_loops (on ssa code)

Both use the same utilities.

> Atm I'm using pass_lower_omp, and I've added a patch that handles omp-lowered
> code conservatively in ccp and forwprop in order for the lowering to remain
> until arriving at pass_parallelize_loops_oacc_kernels.

You mean omp-_un_-lowered code?

> But it might turn out to be easier/necessary to handle this in
> pass_parallelize_loops_oacc_kernels instead.

I'd do it similar to how autopar does it (not that autopar is a great
example for a GCC p

Re: ASAN test failures make compare_tests useless

[Thomas Schwinge]

Hi!

On Mon, 18 Aug 2014 17:17:53 +0200, Manuel López-Ibáñez  
wrote:
> On 18 August 2014 16:34, Alexander Potapenko  wrote:
> > On Mon, Aug 18, 2014 at 9:43 AM, Yury Gribov  wrote:
> >> On 08/18/2014 09:42 AM, Yury Gribov wrote:
> >>>
> >>> On 08/16/2014 04:37 AM, Manuel López-Ibáñez wrote:
> 
>  On the compile farm, ASAN tests seem to fail a lot like:
> 
>  FAIL: c-c++-common/asan/global-overflow-1.c   -O0  output pattern
>  test, is ==31166==ERROR: AddressSanitizer failed to allocate
>  0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno:
>  12)
>  ==31166==ReserveShadowMemoryRange failed while trying to map
>  0xdfff0001000 bytes. Perhaps you're using ulimit -v
>  , should match READ of size 1 at 0x[0-9a-f]+ thread T0.*(

I'm also annoyed by this, due to »ulimit -v 4194304« being set.

>  The problem is that those addresses and sizes are very random,
> > The output pattern that must be printed has these addresses masked out
> > (note "0x[0-9a-f]+" in your report).
> > No other lines with varying addresses should be printed.
> 
> For the record, I think the fault lies in the GCC testing
> infrastructure and not in ASAN. It is wrong to print as the test error
> message the output of ASAN. It should print
> 
> FAIL: c-c++-common/asan/global-overflow-1.c   -O0  output pattern
> test, is  ERROR
> 
> This is enough to see that something failed. For details one can go to
> the detailed logs. But I didn't add the asan testing infrastructure
> and I couldn't figure out how to fix this.
> 
> Any suggestions?

Richard Sandiford has already addressed this in DejaGnu upstream,
,
so you now "just" need to wait for the next DejaGnu release to be made
and packaged for your distribution, or you manually patch
/usr/share/dejagnu/dg.exp:dg-test, or add a patched dg-test to a suitable
gcc/testsuite/lib/*.exp file to override the system one.


Grüße,
 Thomas


pgpY_aslZ7qwK.pgp
Description: PGP signature

Re: [PATCH] RE: gcc parallel make check

[Jakub Jelinek]

On Tue, Sep 09, 2014 at 10:57:09AM +, VandeVondele  Joost wrote:
> > No.  As I wrote earlier, splitting on filenames and test counts only is only
> > very rough split, all the splits really need to be backed out by real timing
> > data from popular targets.  
> 
> Furthermore, for parallel performance, it is not
> so important that times are distributed evenly (it is anyway unlikely the
> number of goals is exactly divided by N of -jN), but rather that the goals
> are ordered (executed) from slow to fast (similar to omp schedule guided). 
> Most of the real bottlenecks are single letter patterns (e.g.  p* since
> pr is such a common filename), and this is ultimately limiting.

I disagree.  If e.g. in gcc.dg/ more than a third of testcases are pr*.c,
then running dg.exp=p* in one job and dg.exp=a* in another one etc. is
simply a bad idea, the pr*.c should be split more and some other letters
just be done together.

Even that can be done semi-automatically.

If you get whitespace right, one can provide multiple different wildcards to
a single *.exp file, e.g.
make check-gcc RUNTESTFLAGS="dg.exp='p[0-9A-Za-qs-z]* pr[9A-Za-z]*'" should
cover all tests starting with p other than pr[0-8]*.c (where you could split
say pr[0-2]* into another job, pr[3-5]* into another and pr[6-8]* into
another.

The fact that some check-gcc or check-gfortran test job is early in the list
doesn't mean it will be started early, you need to consider also all other
potentially long jobs like check-g++, check-target-libgomp,
check-target-libstdc++-v3 etc.

Jakub

RE: [PATCH] RE: gcc parallel make check

[VandeVondele Joost]

> If you get whitespace right, one can provide multiple different wildcards to
> a single *.exp file, e.g.
> make check-gcc RUNTESTFLAGS="dg.exp='p[0-9A-Za-qs-z]* pr[9A-Za-z]*'" should
> cover all tests starting with p other than pr[0-8]*.c (where you could split
> say pr[0-2]* into another job, pr[3-5]* into another and pr[6-8]* into
> another.

I think this confirms that it becomes very delicate to try and write these more 
complex patterns. The above would miss p_test.c, p-1.c, etc ? 

For other classes of files the difference is even further down the filename 
(e.g. using dates as in 20020508-3.c going from 2000 to 2014, or avx*), making 
the automatic generation of the patterns more complicated.

I certainly don't want to claim that the patch I have now is perfect, it is 
rather an incremental improvement on the current setup.

Re: [PATCH] RE: gcc parallel make check

[Yury Gribov]

On 09/09/2014 06:14 PM, VandeVondele Joost wrote:

I certainly don't want to claim that the patch I have now is perfect,
it is rather an incremental improvement on the current setup.


I'd second this. Writing patterns manually seems rather inefficient and 
error-prone
(not undoable of course but unnecessarily complicated). And with current 
(crippled)

version Joost already got 100% test time improvement.

-Y

Re: [PATCH] RE: gcc parallel make check

[Jakub Jelinek]

On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote:
> On 09/09/2014 06:14 PM, VandeVondele Joost wrote:
> >I certainly don't want to claim that the patch I have now is perfect,
> >it is rather an incremental improvement on the current setup.
> 
> I'd second this. Writing patterns manually seems rather inefficient and
> error-prone
> (not undoable of course but unnecessarily complicated). And with current
> (crippled)
> version Joost already got 100% test time improvement.

But if there are jobs that just take 1s to complete, then clearly it doesn't
make sense to split them off as separate job.  I think we don't need 100%
even split, but at least roughly is highly desirable.

Jakub

Re: [PATCH] RE: gcc parallel make check

[Yury Gribov]

On 09/09/2014 06:33 PM, Jakub Jelinek wrote:

On Tue, Sep 09, 2014 at 06:27:10PM +0400, Yury Gribov wrote:

On 09/09/2014 06:14 PM, VandeVondele Joost wrote:

I certainly don't want to claim that the patch I have now is perfect,
it is rather an incremental improvement on the current setup.


I'd second this. Writing patterns manually seems rather inefficient and
error-prone
(not undoable of course but unnecessarily complicated). And with current
(crippled)
version Joost already got 100% test time improvement.


But if there are jobs that just take 1s to complete, then clearly it doesn't
make sense to split them off as separate job.  I think we don't need 100%
even split, but at least roughly is highly desirable.


You mean enhancing the script to split across arbitrarily long prefixes?
That would be great.

-Y

RE: [PATCH] RE: gcc parallel make check

[VandeVondele Joost]

Now with gzipped figure.. why do these bounce ?

> But if there are jobs that just take 1s to complete, then clearly it doesn't
> make sense to split them off as separate job.  I think we don't need 100%
> even split, but at least roughly is highly desirable.

Let me add some data, attached is a graph (logscale y) showing the runtime of 
tests before and after my changes (including a new patch for c++). There is 
virtually no change for tests running shorter than 50s, only slowly running 
tests have been split.

Now, there are only very few slow tests remaining:

gcc_trunk/obj.new> find . -name "*.log" | xargs grep " completed in " | sort -n 
-k 5 | tail -n 10
./gcc/testsuite/gcc/gcc.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/gcc.dg/torture/dg-torture.exp 
completed in 521 seconds
./x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
 completed in 530 seconds
./x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
 completed in 553 seconds
./x86_64-unknown-linux-gnu/libgomp/testsuite/libgomp.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/libgomp/testsuite/libgomp.fortran/fortran.exp 
completed in 561 seconds
./gcc/testsuite/gcc/gcc.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/gcc.c-torture/compile/compile.exp 
completed in 625 seconds
./x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
 completed in 683 seconds
./gcc/testsuite/g++/g++.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/g++.dg/dg.exp completed in 702 
seconds
./x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
 completed in 726 seconds
./gcc/testsuite/gcc/gcc.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/gcc/testsuite/gcc.c-torture/execute/execute.exp 
completed in 752 seconds
./x86_64-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.log:testcase 
/data/vjoost/gnu/gcc_trunk/gcc/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp
 completed in 904 seconds

They, of course, limit the ultimate speedup.

timings.png.gz
Description: timings.png.gz

RE: [PATCH] RE: gcc parallel make check

[VandeVondele Joost]

Attached is a further revision of the patch, now dealing with check-c++. 
Roughly 50% speedup here at '-j32' (18m vs 12m). For my setup 
(--enable-languages=c,c++,fortran) I have now improved all targets called in 
'make -j32 -k check'. The latter is now 30% faster (15m vs 20m). Note that 
there are +- 1m fluctuations in these numbers, easily.

I currently have no plans to work on other check targets before this patch is 
committed.

OK for trunk ?

Joost







contrib/ChangeLog

2014-09-09  Joost VandeVondele  

* generate_tcl_patterns.sh: New file.

gcc/fortran/ChangeLog

2014-09-09  Joost VandeVondele  

* Make-lang.in (check_gfortran_parallelize): Improved parallelism.

gcc/Changelog

2014-09-09  Joost VandeVondele  

* Makefile.in (check_gcc_parallelize): Improved parallelism.
(check_p_numbers): Increase maximum value.
(dg_target_exps): Mention targets as separate words only.
(null,space,comma,dg_target_exps_p1,dg_target_exps_p2,
dg_target_exps_p3,dg_target_exps_p4): New variables.

gcc/cp/ChangeLog

2014-09-09  Joost VandeVondele  

* Make-lang.in (check_g++_parallelize): Improved parallelism.

libstdc++-v3/ChangeLog

2014-09-09  Joost VandeVondele  

* testsuite/Makefile.am (check_DEJAGNU_normal_targets): Add
check-DEJAGNUnormal[11-15].
(check-DEJAGNU): Split into 15 jobs for parallel testing.
* testsuite/Makefile.in: Regenerated.
Index: libstdc++-v3/testsuite/Makefile.am
===
--- libstdc++-v3/testsuite/Makefile.am	(revision 215017)
+++ libstdc++-v3/testsuite/Makefile.am	(working copy)
@@ -101,7 +101,7 @@ new-abi-baseline:
 	@test ! -f $*/site.exp || mv $*/site.exp $*/site.bak
 	@mv $*/site.exp.tmp $*/site.exp
 
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10)
+check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15)
 $(check_DEJAGNU_normal_targets): check-DEJAGNUnormal%: normal%/site.exp
 
 # Run the testsuite in normal mode.
@@ -111,7 +111,7 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar
 	if [ -z "$*$(filter-out --target_board=%, $(RUNTESTFLAGS))" ] \
 	&& [ "$(filter -j, $(MFLAGS))" = "-j" ]; then \
 	  $(MAKE) $(AM_MAKEFLAGS) $(check_DEJAGNU_normal_targets); \
-	  for idx in 0 1 2 3 4 5 6 7 8 9 10; do \
+	  for idx in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do \
 	mv -f normal$$idx/libstdc++.sum normal$$idx/libstdc++.sum.sep; \
 	mv -f normal$$idx/libstdc++.log normal$$idx/libstdc++.log.sep; \
 	  done; \
@@ -138,25 +138,35 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar
 	fi; \
 	dirs="`cd $$srcdir; echo [013-9][0-9]_*/*`";; \
 	  normal1) \
-	dirs="`cd $$srcdir; echo [ab]* de* [ep]*/*`";; \
+	dirs="`cd $$srcdir; echo e*/*`";; \
 	  normal2) \
-	dirs="`cd $$srcdir; echo 2[01]_*/*`";; \
+	dirs="`cd $$srcdir; echo 28_*/a*`";; \
 	  normal3) \
-	dirs="`cd $$srcdir; echo 22_*/*`";; \
+	dirs="`cd $$srcdir; echo 23_*/[lu]*`";; \
 	  normal4) \
-	dirs="`cd $$srcdir; echo 23_*/[a-km-tw-z]*`";; \
+	dirs="`cd $$srcdir; echo 2[459]_*/*`";; \
 	  normal5) \
-	dirs="`cd $$srcdir; echo 23_*/[luv]*`";; \
+	dirs="`cd $$srcdir; echo 2[01]_*/*`";; \
 	  normal6) \
-	dirs="`cd $$srcdir; echo 2[459]_*/*`";; \
+	dirs="`cd $$srcdir; echo 23_*/[m-tw-z]*`";; \
 	  normal7) \
-	dirs="`cd $$srcdir; echo 26_*/* 28_*/[c-z]*`";; \
+	dirs="`cd $$srcdir; echo 26_*/*`";; \
 	  normal8) \
 	dirs="`cd $$srcdir; echo 27_*/*`";; \
 	  normal9) \
-	dirs="`cd $$srcdir; echo 28_*/[ab]*`";; \
+	dirs="`cd $$srcdir; echo 22_*/*`";; \
 	  normal10) \
 	dirs="`cd $$srcdir; echo t*/*`";; \
+	  normal11) \
+	dirs="`cd $$srcdir; echo 28_*/b*`";; \
+	  normal12) \
+	dirs="`cd $$srcdir; echo 28_*/[c-z]*`";; \
+	  normal13) \
+	dirs="`cd $$srcdir; echo de* p*/*`";; \
+	  normal14) \
+	dirs="`cd $$srcdir; echo [ab]* 23_*/v*`";; \
+	  normal15) \
+	dirs="`cd $$srcdir; echo 23_*/[a-k]*`";; \
 	esac; \
 	if [ -n "$*" ]; then cd "$*"; fi; \
 	if $(SHELL) -c "$$runtest --version" > /dev/null 2>&1; then \
Index: libstdc++-v3/testsuite/Makefile.in
===
--- libstdc++-v3/testsuite/Makefile.in	(revision 215017)
+++ libstdc++-v3/testsuite/Makefile.in	(working copy)
@@ -301,7 +301,7 @@ lists_of_files = \
 
 extract_symvers = $(glibcxx_builddir)/scripts/extract_symvers
 baseline_subdir := $(shell $(CXX) $(baseline_subdir_switch))
-check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10)
+check_DEJAGNU_normal_targets = $(patsubst %,check-DEJAGNUnormal%,0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15)
 
 # Runs the testsuite, but in compile only mode.
 # Can be used to test sources with non-GNU FE's at various warning
@@ -562,7 +562,7 @@ check-DEJAGNU $(check_DEJAGNU_normal_tar
 	if [ -z "$*$(filter-out --target_board=%, $(RUNTES

Possible violation of the gcc GPL license

[Paolo Inaudi]

Dear GNU developers,
I don't know if this is the right place to signal this, but I believe 
this Android application

https://play.google.com/store/apps/details?id=com.maclab.codepad2
Especially with this plugin
https://play.google.com/store/apps/details?id=com.maclab.codepadgcc
violates the GPL license:  in fact the description states
> This software uses code of GNU Compiler Collection.
but there is no link to any place where you can get source code for the 
application, which seams to be distributed with a non-GPL license. 
Application website is in Chinese and I cannot understand Chinese, so I 
cannot see if source code is distributed there, but anyway, section 6.d 
of the GPL states that you must


d) Convey the object code by offering access from a designated place 
(gratis or for a charge), and offer equivalent access to the 
Corresponding Source in the same way through the same place at no 
further charge. You need not require recipients to copy the 
Corresponding Source along with the object code. If the place to copy 
the object code is a network server, the Corresponding Source may be on 
a different server (operated by you or a third party) that supports 
equivalent copying facilities, provided you maintain clear directions 
next to the object code saying where to find the Corresponding Source. 
Regardless of what server hosts the Corresponding Source, you remain 
obligated to ensure that it is available for as long as needed to 
satisfy these requirements.


And this is clearly not fullfilled, since there are no "clear directions 
next to the object code saying where to find the Corresponding Source".
You may want to contact the author and force him to comply with the 
license terms of the GPL.

Thank you for your work
Paolo

Re: GCC 5 snapshots produce broken kernel for powerpc-e500v2-linux-gnuspe?

[Arseny Solokha]

Makrus, Andrew, thanks for your suggestions.


>> On Sep 9, 2014, at 2:57 AM, Markus Trippelsdorf  
>> wrote:
>> 
>>> On 2014.09.09 at 17:35 +0800, Arseny Solokha wrote:
>>> Hello,
>>> 
>>> I've recently faced an issue I'm afraid I currently unable to debug. When
>>> building an arbitrary version of Linux kernel for 
>>> powerpc-e500v2-linux-gnuspe
>>> target, it seems gcc prior to 5 produces a good image which boots just 
>>> fine, and
>>> current gcc 5 snapshots (4.10.0-alpha20140810 for example) produce an image
>>> which hangs just after U-Boot hands over to the kernel.
>>> 
>>> This behavior is well reproducible on real hardware as well as under qemu. 
>>> I've
>>> prepared a minimal kernel config which is dysfunctional as is but still 
>>> enough
>>> to demonstrate the problem in qemu. I believe the exact Linux version number
>>> doesn't actually matter here, but see the attachment for details.
>>> 
>>> Compare the output produced by u-boot and this minified kernel build using
>>> gcc 4.9.1 and 4.10.0-alpha20140810 snapshot.
>>> 
>>> I now have completely no idea what to do next to find a cause of (1) gcc 5
>>> snapshots producing unbootable kernel, 
>> 
>> gcc trunk also miscompiles x86_64 kernels currently, but I haven't
>> looked deeper yet. 
>> The best way to narrow down the issue is to use git (or svn) bisect to
>> find out which gcc revision causes the miscompile.  Then you can md5sum
>> the kernel object files for the bad revision and for the first good
>> revision and compare the results. After that you can look at the
>> disassembly of the object files, for which md5sum differs, and try to
>> figure out the reason why.

OK, bisect is of course one way to find a culprit revision and then files it
miscompiles. However, one difficulty w/ this approach is that SPE ABI support in
GCC (and probably other places too) is in semi-bitrot state by now, so there is
a preliminary consideration that not every revision could even be buildable. I
don't have any better idea, though.

Regarding the case of missing sections in kernel module built w/ stable GCC
versions after tinkering w/ CFLAGS, what can be the cause? Is it possible that
ld, or as, or kernel linker script is real culprit? Because this started showing
up since 4.8, and the only difference in assembler output I can see between 4.7
and 4.10 is use of isel instructions by the latter even though -misel isn't
appended to kernel CFLAGS by default.


> I have a patch which I need to submit. Maybe by Friday I will do that. It 
> fixes the kernel on arm64 but it is generic c front-end patch. 

So I'll for sure try current trunk next week once again.


Regards.


>> 
>> -- 
>> Markus

Re: Possible violation of the gcc GPL license

[Jeff Law]

On 09/09/14 09:46, Paolo Inaudi wrote:

Dear GNU developers,
I don't know if this is the right place to signal this, but I believe
this Android application

[ ... ]
The right place is license-violat...@gnu.org.  The FSF actually owns the 
copyright on GCC.


You can find further information on how to report a violation here:

http://www.gnu.org/licenses/gpl-violation.html

Thanks,
Jeff

Re: Possible violation of the gcc GPL license

[Paolo Inaudi]

Thank you very much for the pointer, and sorry for my mistake.
Paolo
Il 09/09/2014 18:46, Jeff Law ha scritto:

On 09/09/14 09:46, Paolo Inaudi wrote:

Dear GNU developers,
I don't know if this is the right place to signal this, but I believe
this Android application

[ ... ]
The right place is license-violat...@gnu.org.  The FSF actually owns 
the copyright on GCC.


You can find further information on how to report a violation here:

http://www.gnu.org/licenses/gpl-violation.html

Thanks,
Jeff

Re: Trouble trying to test GCC on a simulator

[Hans-Peter Nilsson]

On Mon, 8 Sep 2014, Pierre-Marie de Rodat wrote:
> # Get newlib and the simulator
> cvs -d :pserver:anon...@sourceware.org:/cvs/src co newlib sim
> # Get binutils
> git clone git://sourceware.org/git/binutils-gdb.git
>
> # Create the combined tree
> rm -rf combined
> mkdir combined
> cd src  && find . -print | cpio -pdlm  ../combined && cd ..
> cd binutils-gdb && find . -print | cpio -pdlmu ../combined && cd ..
> cd gcc  && find . -print | cpio -pdlmu ../combined && cd ..
>
> # Same build/test procedure...
>
> It seems to work fine! (I'm running tests, now...) So thank you very much,
> Tristan.

> I'm going to submit a website patch to update the documentation
> according to this.

On the one hand, I promised to do this some time ago and on the
other you may be faster than me...

I'm using your input above (corrected with Tristan's
observation) and will double-check that it works to build at
least a couple of ports.  Thanks!

Also, I saw some other gotchas.  (We require a recent g++, and
more recent than 2.95 etc.)

brgds, H-P

Re: Some questions about pass web

[Jan Hubicka]

> On 09/03/14 02:35, Steven Bosscher wrote:
> >On Wed, Sep 3, 2014 at 9:17 AM, Bin.Cheng wrote:
> >>Last time I tried, there are several passes after loop_done and before
> >>auto-inc-dec can't handle auto-increment addressing mode, including
> >>fweb.
> >
> >It surprises me that pass_web can't handle AUTOINC. Perhaps I'm off my
> >rocker, but it's always been my understanding that almost all passes
> >handle AUTOINC just fine (or at least conservatively: punt if you see
> >an AUTOINC), and that only CSE really doesn't know about AUTOINC at
> >all.
> In the past autoinc instructions didn't appear until flow (just
> prior to combine) and that was documented behaviour.  So anything
> which was run strictly prior to flow/combine wasn't autoinc aware.
> That may have changed somewhat with the autoinc rewrite.

It is long time since I looked at web, but it should understand read/write refs
and that those must remain unified.  how does DF refs look like for autoinc?

Honza
> 
> jeff

Re: Some questions about pass web

[Carrot Wei]

It is indeed caused by wrong DF information, which is caused by a
wrong fix for bug PR32339.
More discussion is at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63156.

thanks a lot
Guozhi Wei

On Tue, Sep 9, 2014 at 5:31 PM, Jan Hubicka  wrote:
>> On 09/03/14 02:35, Steven Bosscher wrote:
>> >On Wed, Sep 3, 2014 at 9:17 AM, Bin.Cheng wrote:
>> >>Last time I tried, there are several passes after loop_done and before
>> >>auto-inc-dec can't handle auto-increment addressing mode, including
>> >>fweb.
>> >
>> >It surprises me that pass_web can't handle AUTOINC. Perhaps I'm off my
>> >rocker, but it's always been my understanding that almost all passes
>> >handle AUTOINC just fine (or at least conservatively: punt if you see
>> >an AUTOINC), and that only CSE really doesn't know about AUTOINC at
>> >all.
>> In the past autoinc instructions didn't appear until flow (just
>> prior to combine) and that was documented behaviour.  So anything
>> which was run strictly prior to flow/combine wasn't autoinc aware.
>> That may have changed somewhat with the autoinc rewrite.
>
> It is long time since I looked at web, but it should understand read/write 
> refs
> and that those must remain unified.  how does DF refs look like for autoinc?
>
> Honza
>>
>> jeff