Re: WPA stream_out form & memory consumption

2014-04-14 Thread Jan Hubicka
> On Mon, Apr 7, 2014 at 8:20 PM, Jan Hubicka wrote: > >> AFAIK we settled on a simpler one dropping columns at stream-out time > >> that also helped. > >> > >> As for the correct way to do the optimization we agreed(?) that streaming > >> the locations elsewhere and using references to them is mo

Re: WPA stream_out form & memory consumption

2014-04-08 Thread Richard Biener
On Mon, Apr 7, 2014 at 8:20 PM, Jan Hubicka wrote: >> AFAIK we settled on a simpler one dropping columns at stream-out time >> that also helped. >> >> As for the correct way to do the optimization we agreed(?) that streaming >> the locations elsewhere and using references to them is more appropria

Re: WPA stream_out form & memory consumption

2014-04-08 Thread Martin Liška
On 04/07/2014 07:10 PM, Jan Hubicka wrote: I added new graph for 'xloc.column = 0' hack, just applied this single patch to trunk. Link: https://drive.google.com/file/d/0B0pisUJ80pO1MW11WHdjMk9KQnc/edit?usp=sharing Good, does these two patches combine together well? (they are rater orthogonal,

Re: WPA stream_out form & memory consumption

2014-04-07 Thread Jan Hubicka
> AFAIK we settled on a simpler one dropping columns at stream-out time > that also helped. > > As for the correct way to do the optimization we agreed(?) that streaming > the locations elsewhere and using references to them is more appropriate. > At stream-in (or before stream-out) we can then re

Re: WPA stream_out form & memory consumption

2014-04-07 Thread Jan Hubicka
> > I added new graph for 'xloc.column = 0' hack, just applied this > single patch to trunk. > Link: > https://drive.google.com/file/d/0B0pisUJ80pO1MW11WHdjMk9KQnc/edit?usp=sharing Good, does these two patches combine together well? (they are rater orthogonal, but perhaps with columns disabled l

Re: WPA stream_out form & memory consumption

2014-04-07 Thread Martin Liška
On 04/07/2014 01:49 PM, Richard Biener wrote: On Mon, Apr 7, 2014 at 1:25 PM, Martin Liška wrote: On 04/03/2014 10:40 PM, Jan Hubicka wrote: Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444 cgraph.c:510 (cgraph_all

Re: WPA stream_out form & memory consumption

2014-04-07 Thread Martin Liška
On 04/04/2014 05:10 PM, Martin Liška wrote: On 04/03/2014 03:07 PM, Richard Biener wrote: On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška wrote: On 04/03/2014 11:41 AM, Richard Biener wrote: On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška wrote: On 04/02/2014 04:13 PM, Martin Liška wrote: On

Re: WPA stream_out form & memory consumption

2014-04-07 Thread Richard Biener
On Mon, Apr 7, 2014 at 1:25 PM, Martin Liška wrote: > On 04/03/2014 10:40 PM, Jan Hubicka wrote: Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444 cgraph.c:510 (cgraph_allocate_node)

Re: WPA stream_out form & memory consumption

2014-04-07 Thread Martin Liška
On 04/03/2014 10:40 PM, Jan Hubicka wrote: Firefox: cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% 0: 0.0% 130358176: 6.9% 0: 0.0%1253444 cgraph.c:510 (cgraph_allocate_node) 0: 0.0% 0: 0.0% 182236800: 9.7% 0: 0.0

Re: WPA stream_out form & memory consumption

2014-04-04 Thread Martin Liška
On 04/03/2014 03:07 PM, Richard Biener wrote: On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška wrote: On 04/03/2014 11:41 AM, Richard Biener wrote: On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška wrote: On 04/02/2014 04:13 PM, Martin Liška wrote: On 03/27/2014 10:48 AM, Martin Liška wrote: Prev

Re: WPA stream_out form & memory consumption

2014-04-03 Thread Jan Hubicka
> >Firefox: > >cgraph.c:869 (cgraph_create_edge_1) 0: 0.0% > >0: 0.0% 130358176: 6.9% 0: 0.0%1253444 > >cgraph.c:510 (cgraph_allocate_node) 0: 0.0% > >0: 0.0% 182236800: 9.7% 0: 0.0% 555600 > >toplev.c:960 (

Re: WPA stream_out form & memory consumption

2014-04-03 Thread Jan Hubicka
> On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška wrote: > > > > On 04/03/2014 11:41 AM, Richard Biener wrote: > >> > >> On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška wrote: > >>> > >>> On 04/02/2014 04:13 PM, Martin Liška wrote: > > > On 03/27/2014 10:48 AM, Martin Liška wrote: >

Re: WPA stream_out form & memory consumption

2014-04-03 Thread Richard Biener
On Thu, Apr 3, 2014 at 2:07 PM, Martin Liška wrote: > > On 04/03/2014 11:41 AM, Richard Biener wrote: >> >> On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška wrote: >>> >>> On 04/02/2014 04:13 PM, Martin Liška wrote: On 03/27/2014 10:48 AM, Martin Liška wrote: > > Previous patch

Re: WPA stream_out form & memory consumption

2014-04-03 Thread Richard Biener
On Wed, Apr 2, 2014 at 6:11 PM, Martin Liška wrote: > > On 04/02/2014 04:13 PM, Martin Liška wrote: >> >> >> On 03/27/2014 10:48 AM, Martin Liška wrote: >>> >>> Previous patch is wrong, I did a mistake in name ;) >>> >>> Martin >>> >>> On 03/27/2014 09:52 AM, Martin Liška wrote: On

Re: WPA stream_out form & memory consumption

2014-04-02 Thread Jan Hubicka
> Previous email presents a bit misleading graphs (influenced by > --enable-gather-detailed-mem-stats). > > Firefox: > -flto=9, WPA peak: 8GB, LTRANS peak: 8GB > -flto=4, WPA peak: 5GB, LTRANS peak: 3.5GB > -flto=1, WPA peak: 3.5GB, LTRANS peak: ~1GB > > These data shows that parallel WPA streami

Re: WPA stream_out form & memory consumption

2014-04-02 Thread Jan Hubicka
> > Hello, > taking latest trunk gcc, I built Firefox and Chromium. Both > projects compiled without debugging symbols and -O2 on an 8-core > machine. > > Firefox: > -flto=9, peak memory usage (in LTRANS): 11GB > > Chromium: > -flto=6, peak memory usage (in parallel WPA phase ): 16.5GB I see,

Re: WPA stream_out form & memory consumption

2014-04-02 Thread Martin Liška
On 03/27/2014 10:48 AM, Martin Liška wrote: Previous patch is wrong, I did a mistake in name ;) Martin On 03/27/2014 09:52 AM, Martin Liška wrote: On 03/25/2014 09:50 PM, Jan Hubicka wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do paral

Re: WPA stream_out form & memory consumption

2014-03-27 Thread Martin Liška
Previous patch is wrong, I did a mistake in name ;) Martin On 03/27/2014 09:52 AM, Martin Liška wrote: On 03/25/2014 09:50 PM, Jan Hubicka wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11

Re: WPA stream_out form & memory consumption

2014-03-27 Thread Martin Liška
On 03/25/2014 09:50 PM, Jan Hubicka wrote: Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about 6GB. When WPA start t

Re: WPA stream_out form & memory consumption

2014-03-25 Thread Jan Hubicka
> Hello, >I've been compiling Chromium with LTO and I noticed that WPA > stream_out forks and do parallel: > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. > > I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about > 6GB. When WPA start to fork, memory consumption increa

Re: WPA stream_out form & memory consumption

2014-03-18 Thread Richard Biener
On Tue, Mar 18, 2014 at 4:13 PM, Richard Biener wrote: > On Tue, Mar 18, 2014 at 4:09 PM, Martin Liška wrote: >> Hello, >>I've been compiling Chromium with LTO and I noticed that WPA stream_out >> forks and do parallel: >> http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. >> >> I am un

Re: WPA stream_out form & memory consumption

2014-03-18 Thread Richard Biener
On Tue, Mar 18, 2014 at 4:09 PM, Martin Liška wrote: > Hello, >I've been compiling Chromium with LTO and I noticed that WPA stream_out > forks and do parallel: > http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. > > I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about 6GB

WPA stream_out form & memory consumption

2014-03-18 Thread Martin Liška
Hello, I've been compiling Chromium with LTO and I noticed that WPA stream_out forks and do parallel: http://gcc.gnu.org/ml/gcc-patches/2013-11/msg02621.html. I am unable to fit in 16GB memory: ld uses about 8GB and lto1 about 6GB. When WPA start to fork, memory consumption increases so th