On Wed, 30 Jul 2014, Markus Trippelsdorf wrote:
> On 2014.07.30 at 10:31 +0200, Richard Biener wrote:
> > On Wed, Jul 30, 2014 at 7:51 AM, Markus Trippelsdorf
> > <mar...@trippelsdorf.de> wrote:
> > > On 2014.07.29 at 15:10 +0200, Richard Biener wrote:
> > >> On Tue, 29 Jul 2014, Richard Biener wrote:
> > >>
> > >> >
> > >> > This re-organizes the LTO streamer to do compression transparently
> > >> > in the data-streamer routines (and disables section compression
> > >> > by defaulting to -flto-compression-level=0). This avoids
> > >> > keeping the whole uncompressed sections in memory, only retaining
> > >> > the compressed ones.
> > >> >
> > >> > The downside is that we lose compression of at least the string
> > >> > parts (they are abusing the streaming interface quite awkwardly
> > >> > and doing random-accesses with offsets into the uncompressed
> > >> > section). With a little bit of surgery we can get that back I
> > >> > think (but we'd have to keep the uncompressed piece in memory
> > >> > somewhere which means losing the memory use advantage).
> > >> >
> > >> > Very lightly tested sofar (running lto.exp). I'll try a LTO
> > >> > bootstrap now.
> > >> >
> > >> > I wonder what the change is on WPA memory use for larger
> > >> > projects and what the effect on object file size is.
> > >>
> > >> Updated patch passing LTO bootstrap (one warning fix) and
> > >> with a memory leak fixed.
> > >
> > > Testing with Firefox is impossible at the moment because of PR61885.
> > > One thing I've noticed (before the ICE) is that virtual memory usage is
> > > very high:
> > >
> > > Address Kbytes RSS Dirty Mode Mapping
> > > 0000000000400000 16344 9084 0 r-x-- lto1
> > > 00000000013f6000 36 36 28 rw--- lto1
> > > 00000000013ff000 1072 276 276 rw--- [ anon ]
> > > 00000000034aa000 10154940 1540384 1540384 rw--- [ anon ]
> > > 00002acf04af2000 136 136 0 r-x-- ld-2.19.90.so
> > > 00002acf04b14000 88 88 88 rw--- [ anon ]
> > > ...
> > > ---------------- ------- ------- -------
> > > total kB 12022060 3388396 3377708
> >
> > Maybe there is still a memleak (just checked that LTOing int main() {}
> > doesn't leak).
> >
> > Otherwise I forgot to enable compression at all ;)
>
> He. Also parallel streaming stopped working.
Interesting. Meanwhile after enabling compression the benefit of the
patch is the reduction in LTRANS file size (stage3 cc1) from
460MB to 178MB because we now compress here.
Object file size of stage3-gcc/*.o is increased to 540MB from 506MB
(note LTO bootstrap is with fat LTO objects, so the increase in
LTO IL size is bigger that it looks like). I suppose mainly due
to no longer compressing strings.
I also can observe the appearant memleak - will try to hunt it down.
Richard.
