On Tue Jun 30, 2026 at 1:01 PM CEST, Lewis Hyatt <[email protected]> wrote:
> On Tue, Jun 30, 2026 at 3:52 AM Richard Biener
> <[email protected]> wrote:
>>
>> On Tue, Jun 30, 2026 at 5:03 AM Lewis Hyatt <[email protected]> wrote:
>> >
>> > On Wed, Apr 22, 2026 at 04:29:41PM +0200, Richard Biener wrote:
>> > > On Thu, Jan 1, 2026 at 6:03 PM Lewis Hyatt <[email protected]> wrote:
>> > > >
>> > > > After the previous changes in this series, the LTO front end always
>> > > > has an
>> > > > appropriate linemap structure for interpreting diagnostic pragmas, so
>> > > > it is
>> > > > straightforward to implement them, as is done here.
>> > > >
>> > > > The pragmas are streamed out in each linemap section; since all
>> > > > locations
>> > > > from a given linemap section will be contiguous in the reconstructed
>> > > > linemap, they are automatically ordered properly for the existing
>> > > > diagnostic
>> > > > pragma infrastructure to work as-is.
>> > > >
>> > > > One wrinkle is that a single function may have been streamed out in
>> > > > multiple
>> > > > sections. (For example, an inline function will be streamed out in all
>> > > > partitions that need it.) In this case, when merging them, LTO keeps
>> > > > only
>> > > > one of the sections, as directed by the linker resolution, so the
>> > > > diagnostic
>> > > > pragmas that will be in force (in case they were not the same for the
>> > > > different translation units) will be whichever were applicable to the
>> > > > section LTO decided to keep.
>> > >
>> > > LGTM if the rest of the series is approved.
>> > >
>> > > Thanks,
>> > > Richard.
>> > >
>> >
>> > Hi Richard-
>> >
>> > Firstly, thank you again for your time in reviewing these patches. I
>> > thought
>> > everything was finally across the finish line, but as I was reviewing the
>> > patches one more time before pushing them, I realized there is one small
>> > problem with the new approach. Could I please ask you to look at one more
>> > patch which addresses that? I attached it here as an incremental change to
>> > the rest of the series, but I would propose to squash it into the other
>> > patches before pushing.
>> >
>> > What I missed was that the LTO front end has a mode of operation for
>> > incremental linking. I had tested that my new approach works fine with
>> > "ld -r" (provided that -frandom-seed is not used to remove uniqueness from
>> > the section names), but the LTO front end version of that (which you get
>> > when using, say, "gcc -r -flto") does more than just copy the sections; it
>> > actually reads all the decls and then re-outputs a new object file with a
>> > new identifier, which contains a new decls section plus copies of the
>> > function and constructor sections. This means the linemap sections also
>> > need
>> > to get copied into the output file, and also, it means that an input file
>> > to
>> > the LTO front end could possibly contain more than one linemap, which was
>> > not something I had considered. (I had anticipated that inputs contained
>> > just one linemap, except that in LTRANS mode, there would also be one file
>> > containing all necessary linemaps copied during WPA).
>>
>> So I think there's two things now, the older -flto-linker-output=rel and
>> the newer -flto-incremental.
>>
>> That said, I'm not sure about the default behavior of -flto -r and would
>> suggest to add an explicit -flto-linker-output=rel here to be unambiguous.
>> Did you try that with -ffat-lto-objects as well?
>>
>
> Thanks, what I have understood is:
>
> -flto-incremental is unrelated to incremental linking per se,
> that's about using a cache directory to store inputs + outputs of
> WPA+LTRANS, to avoid rerunning the LTRANS step if the partition did
> not change. I made sure that this still works the same as before my
> patches, that was one motivation for putting all the linemaps into
> their own file after WPA, to make sure a change in one partition
> doesn't needlessly invalidate the cache for a different one.
Do I understand correctly, that LTRANS cache won't notice when location
changes while its ID remains the same?
In LTRANS, do we use locations purely for diagnostics? = locations
cannot influnce the binary output?
And if yes, do we have it documented somewhere that locations cannot be
used in LTRANS for anything other that diagnostics?
>
> -Starting with r6-5271, lto-wrapper provides -flinker-output=rel
> by default when using -r.
>
> -It all works fine with -ffat-lto-objects, yes. The default
> testsuite options used by most tests (including the new test of
> dg-lto-do incr-link that I added) include this, and it isn't affected
> by the linemap changes.
>
>> > It wasn't too bad to adapt things to work fine in this case too, so the
>> > attached patch is not too huge of a change as far as LTO. A lot of the
>> > patch
>> > consists of changes to testsuite/lib/lto.exp to enable testing the
>> > incremental link mode. There was not any test coverage previously for this
>> > sort of operation:
>> >
>> > g++ -flto -c t{1,2,3}.cc
>> > g++ -flto -r -o t12.o t{1,2}.o
>> > g++ -flto -o t t12.o t3.o
>> >
>> > (i.e., partial linking of some files and then a final link of all of them
>> > together), which is why I didn't see this problem initially. The patch adds
>> > a new directive { dg-lto-do incr-link }, which tests a sequence like the
>> > above, similar to how { dg-lto-do ar-link } already works. gccint was
>> > missing documentation of ar-link, so I added something for that as well as
>> > for incr-link.
>> >
>> > I tested everything again on x86-64, aarch64 (cfarm425), amd64 (cfarm420),
>> > sparc 32-bit (cfarm216), and ppc64be (cfarm121), including bootstrap with
>> > bootstrap-lto, and it looks good. Please let me know if it looks OK, in
>> > which case, I am ready to commit the series now. Thanks!
>>
>> That said, the patch looks reasonable, but I wouldn't call myself an expert
>> in the area of incremental LTO, of neither mode.
>>
>> So please give Honza and/or Michal the change to chime in.
>>
>> Thanks,
>> Richard.
>
> Thanks, sure, I will wait for additional feedback.
>
> -Lewis
>/* Given an input linemap ID, compute how it will be known in the output, given
> that the output could be the combination of multiple input files. */
>inline unsigned
>lto_linemap_output_id (unsigned linemap_id, const lto_file_decl_data
>*file_data)
>{
> return linemap_id + file_data->order;
>}
What happens when there are two input files both with multiple linemap
sections? Won't the output IDs overlap?
Michal
