Re: Should we remove java from the default bootstrap languages?

[Ralf Wildenhues]

* David Miller wrote on Sat, Jun 21, 2008 at 12:26:03AM CEST:
> 
> I used to be able to bootstrap gcc fully in minutes on average
> hardware 6 or so years ago.  Those days are long gone.  On my largest
> 64 cpu and 128 cpu boxes it takes forever these days.
> 
> The libjava build is notoriously not helped by parallelization because
> certain compiles are extremely expensive, which effectively
> single-threads the build.

Would that be compiles of object files that end up in libgcj (as opposed
to the link, or stuff that depends on libgcj)?  If yes, the lack of
parallelism should be fixable.

IIRC, then objects in libjava were built from lists of source files as a
means to avoid per-object overhead of libtool and some other stuff, and
to produce a bit better code[1].  Now, at least libtool compile mode
overhead should be a fair bit lower than back then (upstream is a bit
better, if that turns out to be significant, GCC could sync again).

I don't know whether the added debug info overhead, or slightly(?) worse
code generation, would be a show-stopper.  There should be no need to go
back to creating one object per source file, a reasonably fine-grained
split should be sufficient.  I don't have such a big machine to test on,
though, last time I looked, on 8-way things looked not too bad.

Have I overlooked something?  Has anybody ever measured the code
generation improvements that [1] brought?

Cheers,
Ralf

[1] 

Re: Should we remove java from the default bootstrap languages?

[Andrew Haley]

Diego Novillo wrote:
> On Fri, Jun 20, 2008 at 16:56, Kaveh R. GHAZI <[EMAIL PROTECTED]> wrote:
> 
>> That aside, our current policy already allows e.g. not testing java if
>> your change is to a part of the compiler that can't possible affect it.
> 
> I didn't make it completely clear, but my suggestion was mostly to
> help us middle/back-end hackers.

Kaveh does have a point, Diego.  The libjava build regularly finds middle-end
problems that are not revealed by bootstrap testing.  So, even though you
may not think that the libjava build is useful to middle/back-end hackers,
it probably is.

Andrew.

Re: Should we remove java from the default bootstrap languages?

[Andrew Haley]

Steven Bosscher wrote:
> On Sat, Jun 21, 2008 at 12:41 AM, Kaveh R. Ghazi <[EMAIL PROTECTED]> wrote:
>> Fundamentally, our philosophy has been to catch errors *before* they get
>> into the repository.  Sure one day of breaking the trunk isn't so bad, but
>> when it breaks it affects hundreds of developers and it adds up.
> 
> But, for languages that are not enabled by default, no-one is directly
> affected except the people who might have caused the breakage, and the
> people who are working on the broken part of the compiler.

They are directly affected by these errors, but they don't know: the middle-end
bugs revealed by the libjava build and testing are real bugs in other languages
too, they're just not detected by bootstrap & test.

Andrew.

Re: gcc-in-cxx branch created

[Ivan Levashew]

Ian Lance Taylor пишет:

As I promised at the summit today, I have created the branch
gcc-in-cxx (I originally said gcc-in-c++, but I decided that it was
better to avoid possible meta-characters).  The goal of this branch is
to develop a version of gcc which is compiled with C++.  Here are my
presentation slides in PDF format: http://airs.com/ian/cxx-slides.pdf .

Well, C's macros are ugly, but C++ isn't a way to go either. The fact 
that C++ is partially implemented in GCC and belongs to the GCC main 
distribution is the only excuse for choosing C++.



--
If you want to get to the top, you have to start at the bottom

Re: C++ warnings vs. errors

[Jonathan Wakely]

2008/6/20 Mark Mitchell:
>
>> Shall I commit this?
>
> Yes, please.

Thanks, Mark, I've  committed it.
Volker, all the problems you noticed should be fixed, if you find any
other cases that seem wrong please let me know.

Cheers,

Jonathan

Re: Should we remove java from the default bootstrap languages?

[Diego Novillo]

On Sat, Jun 21, 2008 at 05:21, Andrew Haley <[EMAIL PROTECTED]> wrote:

> Kaveh does have a point, Diego.  The libjava build regularly finds middle-end
> problems that are not revealed by bootstrap testing.

So does Ada.  This is why I have offered keep building it on my nightly testers.

IME, bugs found during libjava have been also triggered during
libstdc++ and/or C.  Though several folks at the summit mentioned that
they had found bugs triggered only by libjava.

My point remains that libjava has become a serious problem in the
development cycle of GCC.  It takes roughly 3 hours on modern hardware
to finish a GCC bootstrap (with -j2).  A significant chunk of which is
taken by libjava.  If we could at least reduce the overhead by not
building all of it by default, it would be a huge win.


Diego.

Re: gcc-in-cxx branch created

[Ivan Levashew]

Ian Lance Taylor пишет:


The other major TODO is to work out the details of using STL
containers with GC allocated objects.  This means teaching gengtype
how to generate code to traverse STL containers, which would then be
used during GC.  This is not a task for the faint-hearted.

That's one of the many reasons against C++. Even damn Cyclone will beat 
C++ here.


--
If you want to get to the top, you have to start at the bottom

Re: Should we remove java from the default bootstrap languages?

[Richard Kenner]

> IME, bugs found during libjava have been also triggered during
> libstdc++ and/or C.  Though several folks at the summit mentioned that
> they had found bugs triggered only by libjava.

To me, as I said, this is the key issue: how often do we have such bugs?

Re: Should we remove java from the default bootstrap languages?

[NightStrike]

On 6/21/08, Diego Novillo <[EMAIL PROTECTED]> wrote:
> My point remains that libjava has become a serious problem in the
> development cycle of GCC.  It takes roughly 3 hours on modern hardware
> to finish a GCC bootstrap (with -j2).  A significant chunk of which is
> taken by libjava.  If we could at least reduce the overhead by not
> building all of it by default, it would be a huge win.

What do you define as "modern hardware"?

Re: Should we remove java from the default bootstrap languages?

[Diego Novillo]

On Sat, Jun 21, 2008 at 11:27, NightStrike <[EMAIL PROTECTED]> wrote:

> What do you define as "modern hardware"?

Dell Precision 390 Core2 6600 @2.40Ghz.  4Gb RAM.  Fedora 8.


Diego.

Re: Should we remove java from the default bootstrap languages?

[H.J. Lu]

It takes about 50 minutes to bootstrap gcc with -j4 on a Core 2 Quad 2.66GHz
with default language, both 32bit and 64bit enabled. If I use
--enable-checking=assert,
it takes 25 minutes. Given the price of quad core today, there is no
reason no to
use quad core for gcc build.


H.J.
--
On Sat, Jun 21, 2008 at 8:30 AM, Diego Novillo <[EMAIL PROTECTED]> wrote:
> On Sat, Jun 21, 2008 at 11:27, NightStrike <[EMAIL PROTECTED]> wrote:
>
>> What do you define as "modern hardware"?
>
> Dell Precision 390 Core2 6600 @2.40Ghz.  4Gb RAM.  Fedora 8.
>
>
> Diego.
>



-- 
H.J.

Re: Should we remove java from the default bootstrap languages?

[Diego Novillo]

On Sat, Jun 21, 2008 at 11:39, H.J. Lu <[EMAIL PROTECTED]> wrote:
> It takes about 50 minutes to bootstrap gcc with -j4 on a Core 2 Quad 2.66GHz
> with default language, both 32bit and 64bit enabled. If I use
> --enable-checking=assert,
> it takes 25 minutes. Given the price of quad core today, there is no
> reason no to
> use quad core for gcc build.

Irrelevant.


Diego.

Re: Should we remove java from the default bootstrap languages?

[Richard Guenther]

On Sat, Jun 21, 2008 at 12:50 AM, Richard Kenner
<[EMAIL PROTECTED]> wrote:
>> Fundamentally, our philosophy has been to catch errors *before* they get
>> into the repository.  Sure one day of breaking the trunk isn't so bad, but
>> when it breaks it affects hundreds of developers and it adds up.  Everyone
>> separately either stops and waits, or tracks down which patch it was and
>> reverts it so they can continue working.
>
> An interesting question that I see as relevant here and for which I have no
> data is: what percentage of the time does a patch cause an error *only*
> in libjava?   I think you have to weigh the cost of the build of that
> library against the number of bugs that it finds.

Happened to me multiple times.

The thing to tackle is to make libjava build more parallel.

Richard.

Re: Should we remove java from the default bootstrap languages?

[David Miller]

From: Ralf Wildenhues <[EMAIL PROTECTED]>
Date: Sat, 21 Jun 2008 10:58:55 +0200

> Would that be compiles of object files that end up in libgcj (as opposed
> to the link, or stuff that depends on libgcj)?  If yes, the lack of
> parallelism should be fixable.

It's the compilation of the object files, not the linking.

Re: Should we remove java from the default bootstrap languages?

[NightStrike]

On 6/21/08, Diego Novillo <[EMAIL PROTECTED]> wrote:
> On Sat, Jun 21, 2008 at 11:39, H.J. Lu <[EMAIL PROTECTED]> wrote:
> > It takes about 50 minutes to bootstrap gcc with -j4 on a Core 2 Quad 2.66GHz
> > with default language, both 32bit and 64bit enabled. If I use
> > --enable-checking=assert,
> > it takes 25 minutes. Given the price of quad core today, there is no
> > reason no to
> > use quad core for gcc build.
>
> Irrelevant.

How is that irrelevant?  If the argument is that libjava takes too
long to build on modern hardware, and someone else has a different
view of what is modern hardware where the original argument is
invalid... what makes your view "correct" and HJ's view "incorrect"?

For comparison, I build with -j12 on an 8 core machine in 4 minutes.

Re: buildins for vaargs for call abi switch

[NightStrike]

On 6/10/08, Kai Tietz <[EMAIL PROTECTED]> wrote:
> Hi,
>
> While writing some testcases for the call abi switching for x86_64<->w64 I
> came to the point, that it would be may good to add the builtin types and
> functions for the different calling abi, too.
>
> For the target default abi it would be
>
> Index: gcc/gcc/testsuite/gcc.dg/callabi/callabi.h
>

For purposes of the list archive, this is now handled via:
http://gcc.gnu.org/ml/gcc-patches/2008-06/msg00766.html

Re: Should we remove java from the default bootstrap languages?

[Laurent GUERBY]

On Sat, 2008-06-21 at 08:10 -0400, Diego Novillo wrote:
> On Sat, Jun 21, 2008 at 05:21, Andrew Haley <[EMAIL PROTECTED]> wrote:
> 
> > Kaveh does have a point, Diego.  The libjava build regularly finds 
> > middle-end
> > problems that are not revealed by bootstrap testing.
> 
> So does Ada.  This is why I have offered keep building it on my nightly 
> testers.
> 
> IME, bugs found during libjava have been also triggered during
> libstdc++ and/or C.  Though several folks at the summit mentioned that
> they had found bugs triggered only by libjava.
> 
> My point remains that libjava has become a serious problem in the
> development cycle of GCC.  It takes roughly 3 hours on modern hardware
> to finish a GCC bootstrap (with -j2).  A significant chunk of which is
> taken by libjava.  If we could at least reduce the overhead by not
> building all of it by default, it would be a huge win.

Some data: with --enable-languages=all,ada --disable-multilib on 4.3.1
release on a 4 Opteron 2.0 GHz cores machine from the compile farm, I
got for make bootstrap:

-j1 2h18
-j2 1h14
-j4 0h43
-j6 0h42

And for make -k check:

-j1 2h18
-j2 1h11
-j4 0h55
-j6 0h44

Without Ada (=all) bootstrap gets down to 1h54 at -j1 so Ada is 0h24, or
17% of =all, without Ada and Java (=c,c++,fortran,objc) it gets down to
1h17 at -j1 so java is 0h37 or 27% of =all.

On trunk at -j1 with the same configure my autotester for all,ada get
3h22 boostrap (+1h04, +46%) and 2h52 check (+0h34, +25%). It shows that
additional checking on trunk is not cheap (assuming we didn't
get much slower since 4.3).

The compile farm has three 8 cores machines but I didn't try recently
-j8 on them but it should be in the 20-25 minute range with all,ada:

http://gcc.gnu.org/wiki/CompileFarm

Laurent

Re: Should we remove java from the default bootstrap languages?

[Arnaud Charlet]

> Without Ada (=all) bootstrap gets down to 1h54 at -j1 so Ada is 0h24, or
> 17% of =all, without Ada and Java (=c,c++,fortran,objc) it gets down to
> 1h17 at -j1 so java is 0h37 or 27% of =all.

Note that I recently made the ada build parallel on trunk, so these figures
should be very different on trunk, since now building e.g. libada is
much faster with -jxx and will give a significant speed up.

Arno

Re: Should we remove java from the default bootstrap languages?

[Eric Botcazou]

> How is that irrelevant?  If the argument is that libjava takes too
> long to build on modern hardware, and someone else has a different
> view of what is modern hardware where the original argument is
> invalid... what makes your view "correct" and HJ's view "incorrect"?

Quad cores are available for exactly 1 architecture (x86) at affordable 
prices, GCC is supposed to support more than just x86.

-- 
Eric Botcazou

Re: Should we remove java from the default bootstrap languages?

[Laurent GUERBY]

On Sat, 2008-06-21 at 22:05 +0200, Eric Botcazou wrote:
> > How is that irrelevant?  If the argument is that libjava takes too
> > long to build on modern hardware, and someone else has a different
> > view of what is modern hardware where the original argument is
> > invalid... what makes your view "correct" and HJ's view "incorrect"?
> 
> Quad cores are available for exactly 1 architecture (x86) at affordable 
> prices, GCC is supposed to support more than just x86.

I'm curious at how many GCC developpers use non x86/_64 as their
main development machine (and how many non x86/_64 core they use).

Laurent

RE: Should we remove java from the default bootstrap languages?

[Dave Korn]

NightStrike wrote on 21 June 2008 17:08:

> On 6/21/08, Diego Novillo  wrote:
>> On Sat, Jun 21, 2008 at 11:39, H.J. Lu  wrote:
>>> It takes about 50 minutes to bootstrap gcc with -j4 on a Core 2 Quad
>>> 2.66GHz with default language, both 32bit and 64bit enabled. If I use
>>> --enable-checking=assert, it takes 25 minutes. Given the price of quad
>>> core today, there is no 
>>> reason no to
>>> use quad core for gcc build.
>> 
>> Irrelevant.
> 
> How is that irrelevant?  If the argument is that libjava takes too
> long to build on modern hardware, and someone else has a different
> view of what is modern hardware where the original argument is
> invalid... 

  And how exactly does this help non-x86 platform users?  Or people without
the latest hardware?

  I don't think it aids the cause of software freedom to set a high barrier
to entry.  Nor do I think it helps recruit volunteer workers for GCC.  And I
really don't think it's right to just flippantly tell people they have to go
out and spend big money on the latest hardware in order to be able to
participate.  Volunteers and voluntary orgs often don't have budgets or
funding.


cheers,
  DaveK
-- 
Can't think of a witty .sigline today

Re: Should we remove java from the default bootstrap languages?

[Richard Guenther]

On Sat, Jun 21, 2008 at 10:21 PM, Dave Korn <[EMAIL PROTECTED]> wrote:
> NightStrike wrote on 21 June 2008 17:08:
>
>> On 6/21/08, Diego Novillo  wrote:
>>> On Sat, Jun 21, 2008 at 11:39, H.J. Lu  wrote:
 It takes about 50 minutes to bootstrap gcc with -j4 on a Core 2 Quad
 2.66GHz with default language, both 32bit and 64bit enabled. If I use
 --enable-checking=assert, it takes 25 minutes. Given the price of quad
 core today, there is no
 reason no to
 use quad core for gcc build.
>>>
>>> Irrelevant.
>>
>> How is that irrelevant?  If the argument is that libjava takes too
>> long to build on modern hardware, and someone else has a different
>> view of what is modern hardware where the original argument is
>> invalid...
>
>  And how exactly does this help non-x86 platform users?  Or people without
> the latest hardware?
>
>  I don't think it aids the cause of software freedom to set a high barrier
> to entry.  Nor do I think it helps recruit volunteer workers for GCC.  And I
> really don't think it's right to just flippantly tell people they have to go
> out and spend big money on the latest hardware in order to be able to
> participate.  Volunteers and voluntary orgs often don't have budgets or
> funding.

There is the compile-farm that is available to volunteers.

Richard.

Re: gcc-in-cxx branch created

[Ian Lance Taylor]

Ivan Levashew <[EMAIL PROTECTED]> writes:

> Ian Lance Taylor пишет:
>>
>> The other major TODO is to work out the details of using STL
>> containers with GC allocated objects.  This means teaching gengtype
>> how to generate code to traverse STL containers, which would then be
>> used during GC.  This is not a task for the faint-hearted.
>>
> That's one of the many reasons against C++. Even damn Cyclone will
> beat C++ here.

Your comment makes little sense in context.  Nobody could claim that
the existing gengtype code is simple.  Not many people understand how
it works at all.  In order to support STL containers holding GC
objects, it will need to be modified.  The fact that modifying it is
nontrivial is not an argument either for or against C++.

I don't know what you mean by your reference to the Cyclone variant of
C, unless you are trying to say something about gcc's use of garbage
collection.

Ian

Re: Should we remove java from the default bootstrap languages?

[Gerald Pfeifer]

On Sat, 21 Jun 2008, Laurent GUERBY wrote:
> I'm curious at how many GCC developpers use non x86/_64 as their
> main development machine (and how many non x86/_64 core they use).

It's not just backend changes that require testing on non-x86 
architectures.  And it is very worthwhile to get testing for
other changes on such architectures as well.

Gerald

Re: Should we remove java from the default bootstrap languages?

[Laurent GUERBY]

On Sat, 2008-06-21 at 21:03 +0200, Arnaud Charlet wrote:
> > Without Ada (=all) bootstrap gets down to 1h54 at -j1 so Ada is 0h24, or
> > 17% of =all, without Ada and Java (=c,c++,fortran,objc) it gets down to
> > 1h17 at -j1 so java is 0h37 or 27% of =all.
> 
> Note that I recently made the ada build parallel on trunk, so these figures
> should be very different on trunk, since now building e.g. libada is
> much faster with -jxx and will give a significant speed up.

More data on make bootstrap times:

baseline: gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
trunk revision 137004 Sat Jun 21 19:20:06 UTC 2008

-j4 =c
4671.57user 231.40system 22:37.84elapsed 361%CPU

-j4 =c,c++
5432.07user 293.57system 26:43.52elapsed 357%CPU

-j4 =c,ada
6819.21user 304.12system 32:08.14elapsed 369%CPU

-j4 =c,java (,c++ implied)
8909.32user 471.50system 45:33.95elapsed 343%CPU

So about estimated 1h59 for =c,ada at -j1 vs 1h22 for =c
and around 90% of expected speedup with or without Ada (according to
user+system vs elapsed). Ada cost of 0h37, +45% vs =c.

On the two 8 barcelona 2.2 GHz cores machine donated by AMD
to the GCC compile farm I measured:

-j8 =c
4115.01user 177.29system 12:50.72elapsed 556%CPU 

-j8 =c,fortran
4379.00user 205.48system 12:34.10elapsed 607%CPU 
(see below)

-j8 =c,c++
4632.48user 213.45system 14:04.17elapsed 574%CPU

-j8 =c,ada
5805.12user 227.32system 16:57.56elapsed 592%CPU 

-j8 =c,java (,c++ implied)
7845.79user 348.71system 23:45.03elapsed 575%CPU 

I noticed insn-attrtab compilation is about 5mn20s and probably
explain about all of the not so perfect speedup: when I look at top it
takes more than 1 minute per stage finishing alone. I've seen it up to 3
minutes alone. There's a comment in the Makefile about trying to
compile it earlier but it doesn't seem to work all the time
(likely reason behind c,fortran faster than =c)

Is there a plan to cut this one?

Laurent

Re: Should we remove java from the default bootstrap languages?

[Richard Kenner]

> > An interesting question that I see as relevant here and for which I have no
> > data is: what percentage of the time does a patch cause an error *only*
> > in libjava?   I think you have to weigh the cost of the build of that
> > library against the number of bugs that it finds.
> 
> Happened to me multiple times.

Is there a way to characterize the kinds of changes that are more likely
to make this happen?  We see this with Ada, for example: many middle-end
developers by now know which changes are most likely to potentially affect
Ada and do an Ada build too for those cases.  Would this sort of approach
work for Java as well?