On Tue, Jun 30, 2026 at 2:38 PM Michal Jireš <[email protected]> wrote:
>
>
> On Tue Jun 30, 2026 at 1:01 PM CEST, Lewis Hyatt <[email protected]> wrote:
> > On Tue, Jun 30, 2026 at 3:52 AM Richard Biener
> > <[email protected]> wrote:
> >>
> >> On Tue, Jun 30, 2026 at 5:03 AM Lewis Hyatt <[email protected]> wrote:
> >> >
> >> > On Wed, Apr 22, 2026 at 04:29:41PM +0200, Richard Biener wrote:
> >> > > On Thu, Jan 1, 2026 at 6:03 PM Lewis Hyatt <[email protected]> wrote:
> >> > > >
> >> > > > After the previous changes in this series, the LTO front end always
> >> > > > has an
> >> > > > appropriate linemap structure for interpreting diagnostic pragmas,
> >> > > > so it is
> >> > > > straightforward to implement them, as is done here.
> >> > > >
> >> > > > The pragmas are streamed out in each linemap section; since all
> >> > > > locations
> >> > > > from a given linemap section will be contiguous in the reconstructed
> >> > > > linemap, they are automatically ordered properly for the existing
> >> > > > diagnostic
> >> > > > pragma infrastructure to work as-is.
> >> > > >
> >> > > > One wrinkle is that a single function may have been streamed out in
> >> > > > multiple
> >> > > > sections. (For example, an inline function will be streamed out in
> >> > > > all
> >> > > > partitions that need it.) In this case, when merging them, LTO keeps
> >> > > > only
> >> > > > one of the sections, as directed by the linker resolution, so the
> >> > > > diagnostic
> >> > > > pragmas that will be in force (in case they were not the same for the
> >> > > > different translation units) will be whichever were applicable to the
> >> > > > section LTO decided to keep.
> >> > >
> >> > > LGTM if the rest of the series is approved.
> >> > >
> >> > > Thanks,
> >> > > Richard.
> >> > >
> >> >
> >> > Hi Richard-
> >> >
> >> > Firstly, thank you again for your time in reviewing these patches. I
> >> > thought
> >> > everything was finally across the finish line, but as I was reviewing the
> >> > patches one more time before pushing them, I realized there is one small
> >> > problem with the new approach. Could I please ask you to look at one more
> >> > patch which addresses that? I attached it here as an incremental change
> >> > to
> >> > the rest of the series, but I would propose to squash it into the other
> >> > patches before pushing.
> >> >
> >> > What I missed was that the LTO front end has a mode of operation for
> >> > incremental linking. I had tested that my new approach works fine with
> >> > "ld -r" (provided that -frandom-seed is not used to remove uniqueness
> >> > from
> >> > the section names), but the LTO front end version of that (which you get
> >> > when using, say, "gcc -r -flto") does more than just copy the sections;
> >> > it
> >> > actually reads all the decls and then re-outputs a new object file with a
> >> > new identifier, which contains a new decls section plus copies of the
> >> > function and constructor sections. This means the linemap sections also
> >> > need
> >> > to get copied into the output file, and also, it means that an input
> >> > file to
> >> > the LTO front end could possibly contain more than one linemap, which was
> >> > not something I had considered. (I had anticipated that inputs contained
> >> > just one linemap, except that in LTRANS mode, there would also be one
> >> > file
> >> > containing all necessary linemaps copied during WPA).
> >>
> >> So I think there's two things now, the older -flto-linker-output=rel and
> >> the newer -flto-incremental.
> >>
> >> That said, I'm not sure about the default behavior of -flto -r and would
> >> suggest to add an explicit -flto-linker-output=rel here to be unambiguous.
> >> Did you try that with -ffat-lto-objects as well?
> >>
> >
> > Thanks, what I have understood is:
> >
> > -flto-incremental is unrelated to incremental linking per se,
> > that's about using a cache directory to store inputs + outputs of
> > WPA+LTRANS, to avoid rerunning the LTRANS step if the partition did
> > not change. I made sure that this still works the same as before my
> > patches, that was one motivation for putting all the linemaps into
> > their own file after WPA, to make sure a change in one partition
> > doesn't needlessly invalidate the cache for a different one.
>
> Do I understand correctly, that LTRANS cache won't notice when location
> changes while its ID remains the same?
>
> In LTRANS, do we use locations purely for diagnostics? = locations
> cannot influnce the binary output?
> And if yes, do we have it documented somewhere that locations cannot be
> used in LTRANS for anything other that diagnostics?
>
A location can't really change without affecting the streamed object
file and invalidating the cache. What ends up streamed out (and
affecting the SHA1) is the map ID and the location_t offset within the
map, plus any attached tree and discriminator. Any change to the line
number or column number will either change the location_t or add a new
map and change all map IDs.
> >
> > -Starting with r6-5271, lto-wrapper provides -flinker-output=rel
> > by default when using -r.
> >
> > -It all works fine with -ffat-lto-objects, yes. The default
> > testsuite options used by most tests (including the new test of
> > dg-lto-do incr-link that I added) include this, and it isn't affected
> > by the linemap changes.
> >
> >> > It wasn't too bad to adapt things to work fine in this case too, so the
> >> > attached patch is not too huge of a change as far as LTO. A lot of the
> >> > patch
> >> > consists of changes to testsuite/lib/lto.exp to enable testing the
> >> > incremental link mode. There was not any test coverage previously for
> >> > this
> >> > sort of operation:
> >> >
> >> > g++ -flto -c t{1,2,3}.cc
> >> > g++ -flto -r -o t12.o t{1,2}.o
> >> > g++ -flto -o t t12.o t3.o
> >> >
> >> > (i.e., partial linking of some files and then a final link of all of them
> >> > together), which is why I didn't see this problem initially. The patch
> >> > adds
> >> > a new directive { dg-lto-do incr-link }, which tests a sequence like the
> >> > above, similar to how { dg-lto-do ar-link } already works. gccint was
> >> > missing documentation of ar-link, so I added something for that as well
> >> > as
> >> > for incr-link.
> >> >
> >> > I tested everything again on x86-64, aarch64 (cfarm425), amd64
> >> > (cfarm420),
> >> > sparc 32-bit (cfarm216), and ppc64be (cfarm121), including bootstrap with
> >> > bootstrap-lto, and it looks good. Please let me know if it looks OK, in
> >> > which case, I am ready to commit the series now. Thanks!
> >>
> >> That said, the patch looks reasonable, but I wouldn't call myself an expert
> >> in the area of incremental LTO, of neither mode.
> >>
> >> So please give Honza and/or Michal the change to chime in.
> >>
> >> Thanks,
> >> Richard.
> >
> > Thanks, sure, I will wait for additional feedback.
> >
> > -Lewis
>
>
>
> >/* Given an input linemap ID, compute how it will be known in the output,
> >given
> > that the output could be the combination of multiple input files. */
> >inline unsigned
> >lto_linemap_output_id (unsigned linemap_id, const lto_file_decl_data
> >*file_data)
> >{
> > return linemap_id + file_data->order;
> >}
>
> What happens when there are two input files both with multiple linemap
> sections? Won't the output IDs overlap?
>
It's arranged that the output ID is unique. file_data->order is
incremented (in lto/lto-common.cc), after reading each file, by the
number of linemaps in that file. So if the input IDs are like (file A:
0,1,2) (file B: 0,1) then on output they will get assigned to
(0,1,2,3,4).
Thanks a lot for taking a look at this! Happy to look into anything further.
-Lewis
