Re: Optimize type streaming

2014-07-11 Thread Jan Hubicka
> > Ah, ok. Well, let's hope walk_tree walks all edges the DFS walk walks ;) > A quick look tells me it doesn't walk DECL_VINDEX or > DECL_FUNCTION_PERSONALITY for example. I guess it should - will try patch adding that :) Both seems like just ommisions - these fileds were added quite recently (

Re: Optimize type streaming

2014-07-11 Thread Richard Biener
On Fri, 11 Jul 2014, Jan Hubicka wrote: > > On Fri, 11 Jul 2014, Jan Hubicka wrote: > > > > > > > Hmm, walking everything first and then starting streaming sounds bad > > > > > idea > > > > > for locality. Can't I just re-do the walk since I know what the SCC > > > > > is? > > > > > I will rea

Re: Optimize type streaming

2014-07-11 Thread Jan Hubicka
> On Fri, 11 Jul 2014, Jan Hubicka wrote: > > > > > Hmm, walking everything first and then starting streaming sounds bad > > > > idea > > > > for locality. Can't I just re-do the walk since I know what the SCC is? > > > > I will read the code more after lunch. > > > > > > Might be possible with

Re: Optimize type streaming

2014-07-11 Thread Richard Biener
On Fri, 11 Jul 2014, Jan Hubicka wrote: > > > Hmm, walking everything first and then starting streaming sounds bad idea > > > for locality. Can't I just re-do the walk since I know what the SCC is? > > > I will read the code more after lunch. > > > > Might be possible with a 2nd SCC stack set up

Re: Optimize type streaming

2014-07-11 Thread Jan Hubicka
> > Hmm, walking everything first and then starting streaming sounds bad idea > > for locality. Can't I just re-do the walk since I know what the SCC is? > > I will read the code more after lunch. > > Might be possible with a 2nd SCC stack set up, yes. I set up hash_map to store the hash values

Re: Optimize type streaming

2014-07-11 Thread Richard Biener
On Fri, 11 Jul 2014, Jan Hubicka wrote: > > > > > > We hash only on outgoing SCC edges. You can easily have > > > main variant type T and variants T1,T2 that are same all used by type > > > T again. > > > > Ok, but then the two variants shouldn't be in the same SCC or at > > least not in the sa

Re: Optimize type streaming

2014-07-11 Thread Jan Hubicka
> > > > We hash only on outgoing SCC edges. You can easily have > > main variant type T and variants T1,T2 that are same all used by type > > T again. > > Ok, but then the two variants shouldn't be in the same SCC or at > least not in the same SCC as the main variant. They will because T refers

Re: Optimize type streaming

2014-07-11 Thread Richard Biener
On Fri, 11 Jul 2014, Jan Hubicka wrote: > > > > Well - as we re-use the streamer cache to store the hash value it isn't > > really possible to do better ... (at least I don't have a clever idea) > > OK, no cleverness on my side either. > > > > Yeah (though you wouldn't need the extra hashing -

Re: Optimize type streaming

2014-07-11 Thread Jan Hubicka
> > Well - as we re-use the streamer cache to store the hash value it isn't > really possible to do better ... (at least I don't have a clever idea) OK, no cleverness on my side either. > > Yeah (though you wouldn't need the extra hashing - we only need to know > the hash of the SCC). The curre

Re: Optimize type streaming

2014-07-11 Thread Richard Biener
On Fri, 11 Jul 2014, Jan Hubicka wrote: > > Are we sure it ever stabilizes? But yes, I had something like this in mind > > (just do one iteration always) in case we need to improve hashing. > > Hi, > this is bit quick experiment with strengthening the hash by iteration. I don't > seem to be able

Re: Optimize type streaming

2014-07-11 Thread Jan Hubicka
> Are we sure it ever stabilizes? But yes, I had something like this in mind > (just do one iteration always) in case we need to improve hashing. Hi, this is bit quick experiment with strengthening the hash by iteration. I don't seem to be able to measure WPA time difference for the patch though

Re: Optimize type streaming

2014-07-09 Thread Jan Hubicka
> > Are we sure it ever stabilizes? But yes, I had something like this in mind > (just do one iteration always) in case we need to improve hashing. Not necesarily - imagine two identical type variants, no matter how hard we hash&rehash, they will be same as they are identical. We can just keep

Re: Optimize type streaming

2014-07-09 Thread Richard Biener
On July 9, 2014 2:36:57 PM CEST, Jan Hubicka wrote: >> This part can probably be speed up quite a bit by doing the SCC >unification >> before materializing the SCC, that is, doing the "on-disk" format >compare idea. >> The issue here is that for bigger SCCs that have hash collisions in >their >> e

Re: Optimize type streaming

2014-07-09 Thread Jan Hubicka
> This part can probably be speed up quite a bit by doing the SCC unification > before materializing the SCC, that is, doing the "on-disk" format compare > idea. > The issue here is that for bigger SCCs that have hash collisions in their > entries we need to do the edge walk - but eventually havin

Re: Optimize type streaming

2014-07-09 Thread Richard Biener
On Wed, Jul 9, 2014 at 10:58 AM, Jan Hubicka wrote: > Hello, > perhaps I could write bit more on my longer term plans. At the moment 30% of > firefox WPA is taken > by straming trees and another roughly 30% is taken by inliner. It is bit > anoying but relatively > easy to optimize inliner, but

Re: Optimize type streaming

2014-07-09 Thread Jan Hubicka
Hello, perhaps I could write bit more on my longer term plans. At the moment 30% of firefox WPA is taken by straming trees and another roughly 30% is taken by inliner. It is bit anoying but relatively easy to optimize inliner, but trees represent bigger problem. According to the stats average

Re: Optimize type streaming

2014-07-08 Thread Jan Hubicka
> > We have reverted the patch for now but I note that at least the piece > below is a step backward from doing the compare in the on-disk > format. Why it is step backward from compare in the on-dist format? All the information is still streamed, just not duplicated. Since we explicitly stream

Re: Optimize type streaming

2014-07-07 Thread Richard Biener
On Mon, 30 Jun 2014, Dominique Dhumieres wrote: > > Note that testusite passes with the patch; ... > > Confirmed. There is a typo > s/fileds/fields/ > > I have seen the failures because I now run the gfortran testsuite with > the addition of '-g -flto'. Is it worth pushing in this direction? Y

Re: Optimize type streaming

2014-07-07 Thread Richard Biener
On Mon, 30 Jun 2014, Jan Hubicka wrote: > > > > Please revert the original patch instead which was not tested properly. > > I'll get back to this after I return from vacation. > > OK, I will bootstrap and revert tomorrow morning. > > Note that testusite passes with the patch; the problem appea

Re: Optimize type streaming

2014-07-07 Thread Richard Biener
On Sun, 29 Jun 2014, Jan Hubicka wrote: > > In addition of pr61644 and pr61646, this commit breaks a lot of > > fortran tests with -flto -O0. > Hello, > the problem here is that we have POINTER_TYPE that points to array of variable > length (not sure why it happens only with -O0). The ICE is intro

Re: Optimize type streaming

2014-06-30 Thread Jan Hubicka
> On June 29, 2014 9:53:03 PM CEST, Jan Hubicka wrote: > >> In addition of pr61644 and pr61646, this commit breaks a lot of > >> fortran tests with -flto -O0. > >Hello, > >the problem here is that we have POINTER_TYPE that points to array of > >variable > >length (not sure why it happens only with

Re: Optimize type streaming

2014-06-30 Thread Dominique Dhumieres
> Note that testusite passes with the patch; ... Confirmed. There is a typo s/fileds/fields/ I have seen the failures because I now run the gfortran testsuite with

Re: Optimize type streaming

2014-06-30 Thread Dominique Dhumieres
> Note that testusite passes with the patch; ... Confirmed. There is a typo s/fileds/fields/ I have seen the failures because I now run the gfortran testsuite with the addition of '-g -flto'. Is it worth pushing in this direction? Cheers, Dominique

Re: Optimize type streaming

2014-06-30 Thread Jan Hubicka
> > Please revert the original patch instead which was not tested properly. I'll > get back to this after I return from vacation. OK, I will bootstrap and revert tomorrow morning. Note that testusite passes with the patch; the problem appears only at -O0 -flto and fortran that we apparently do

Re: Optimize type streaming

2014-06-30 Thread Richard Biener
On June 29, 2014 9:53:03 PM CEST, Jan Hubicka wrote: >> In addition of pr61644 and pr61646, this commit breaks a lot of >> fortran tests with -flto -O0. >Hello, >the problem here is that we have POINTER_TYPE that points to array of >variable >length (not sure why it happens only with -O0). The ICE

Re: Optimize type streaming

2014-06-29 Thread Jan Hubicka
> In addition of pr61644 and pr61646, this commit breaks a lot of > fortran tests with -flto -O0. Hello, the problem here is that we have POINTER_TYPE that points to array of variable length (not sure why it happens only with -O0). The ICE is introduced by the fact that we stream the type inside of

Re: Optimize type streaming

2014-06-29 Thread Dominique Dhumieres
In addition of pr61644 and pr61646, this commit breaks a lot of fortran tests with -flto -O0. TIA Dominique FAIL: gfortran.dg/actual_array_constructor_1.f90 -g -flto (internal compiler error) FAIL: gfortran.dg/alloc_comp_assign_6.f90 -g -flto (internal compiler error) FAIL: gfortran.dg/allo

Re: Optimize type streaming

2014-06-28 Thread Jan Hubicka
> Minor comments below, ok with those changes. Thanks! This is version of patch I commited after additional testing on building of some bigger apps with LTO. The main change (in addition to those requested) is that I moved the fixups into lto_copy_fields_not_streamed and added loop before SCC unif

Re: Optimize type streaming

2014-06-27 Thread Richard Biener
On Fri, 27 Jun 2014, Jan Hubicka wrote: > Hi, > the most common types of tree nodes streamed are declarations (5.4M for > Firefox) and types (4.2M for Firefox). This patch makes representation of > types smaller by avoiding saving redundent info about type and its variants. > About 50% of types a