Re: How to debug while using LTO?
On Thu, Nov 24, 2022 at 05:53:53PM +0100, Richard Biener wrote: > > > > Am 24.11.2022 um 17:28 schrieb Stefan Schulze Frielinghaus via Gcc > > : > > > > Hi everyone, > > > > Currently I'm looking into a wrong-code bug and would like to understand > > a certain optimization done by combine during local transformation. > > Without LTO I would simply debug cc1 and step through combine. However, > > with LTO enabled AFAIK I have to debug lto1 instead. In order to get > > the lto1 command line of interest according to > > https://gcc.gnu.org/legacy-ml/gcc/2009-11/msg00047.html > > I have to pass -Wl,-debug to gcc in order to get the command for > > collect2 to which itself I have to pass -plugin-opt=-debug in order to > > get the command for lto-wrapper. According to the aforementioned mail I > > should add option -debug to lto-wrapper, however, it appears to me that > > option -debug was removed. I gave options -v and -### a chance without > > luck, i.e., those only print the usual environment variables and > > afterwards a list of object files like > > > > /tmp/ccPEIV35.ltrans0.ltrans.o > > /tmp/ccNmpKfS.debug.temp.o > > /tmp/cceiCIFg.debug.temp.o > > /tmp/ccZ4Qc7E.debug.temp.o > > ... > > > > but no lto1 command. Thus, how do you retrieve the lto1 command? > > > > While desperate I retrieved it manually via strace. However, the lto1 > > command refers to temporary files which have been erased meanwhile. I > > actually didn't expect that because I added -save-temps to all the > > intermediate commands which is also reflected in the environment > > variable COLLECT_GCC_OPTIONS. Thus, how do you keep temporary files? > > Adding -v -save-temps and then running gdb on the lto1 command works and is > what I usually do. Strangely this is not the case for the current project I'm looking at. The link step is done via /usr/local/bin/s390x-linux-gnu-gcc -save-temps \ -O3 -DNDEBUG -flto=auto -fno-fat-lto-objects \ -march=z15 -static -nostdlib \ -Wl,--fatal-warnings -Wl,--no-warn-rwx-segments \ -Werror -Wall -Wundef -T/devel/foo.ld \ obj1.o ... objN.o where -save-temps is given and was also given during compile time of each object obj1.o ... objN.o file. Anyhow, adding -Wl,-debug to the link command above I see that -save-temps is also included in the environment variable COLLECT_GCC_OPTIONS="... -save-temps ..." and I get the linker command /usr/local/lib/gcc/s390x-linux-gnu/13.0.0/../../../../s390x-linux-gnu/bin/ld \ -plugin /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/liblto_plugin.so \ -plugin-opt=/usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto-wrapper \ -plugin-opt=-fresolution=foo.res \ -m elf64_s390 -static -o foo \ -L/usr/local/lib/gcc/s390x-linux-gnu/13.0.0 \ -L/usr/local/lib/gcc/s390x-linux-gnu/13.0.0/../../../../s390x-linux-gnu/lib \ --fatal-warnings --no-warn-rwx-segments \ obj1.o ... objN.o -lgcc -T /devel/foo.ld to which I add -plugin-opt=-debug in order to get the lto-wrapper command /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto-wrapper \ -fresolution=foo.res -flinker-output=exec \ obj1.o ... objN.o to which I add -v -save-temps in order to actually get the lto1 command. From the output printed to stderr I see one lto1 command: /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto1 -quiet -dumpbase ./a.wpa \ -march=z15 -m64 -mzarch -g -O3 -version -fno-openmp -fno-openacc -fno-pie \ -fcf-protection=none -fltrans-output-list=/tmp/ccsuw4Nj.ltrans.out \ -fwpa=32 -fresolution=foo.res -flinker-output=exec @./a.wpa.args.0 However, this lto1 invocation does no local transformation. Having a look at the strace output I see a second lto1 invocation: /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto1 -quiet \ -dumpbase ./a.ltrans0.ltrans -march=z15 -m64 -mzarch -g -O3 \ -fno-openmp -fno-openacc -fno-pie -fcf-protection=none \ -fltrans /tmp/ccsuw4Nj.ltrans0.o -o /tmp/cc69wMp3.s This lto1 invocation seems to originate from Makefile /tmp/cc5piuXT.mk generated by lto-wrapper. To my surprise the Makefile as well as the temporary file /tmp/ccsuw4Nj.ltrans0.o are removed although -save-temps was given all the time. Stepping through lto-wrapper with gdb I see that save_temps = 1; is never executed while walking over decoded_options. For the time being I hard coded save_temps=1 into lto-wrapper.cc and I am now able to debug the lto1 invocation of interest. Not sure whether there is a more elegant way? All this is based upon a rather huge CMake based project and so far I didn't find the time to come up with a small reproducer. Still hoping this might be useful for someone else. Cheers, Stefan
Re: How to debug while using LTO?
On Wed, Nov 30, 2022 at 11:49 AM Stefan Schulze Frielinghaus wrote: > > On Thu, Nov 24, 2022 at 05:53:53PM +0100, Richard Biener wrote: > > > > > > > Am 24.11.2022 um 17:28 schrieb Stefan Schulze Frielinghaus via Gcc > > > : > > > > > > Hi everyone, > > > > > > Currently I'm looking into a wrong-code bug and would like to understand > > > a certain optimization done by combine during local transformation. > > > Without LTO I would simply debug cc1 and step through combine. However, > > > with LTO enabled AFAIK I have to debug lto1 instead. In order to get > > > the lto1 command line of interest according to > > > https://gcc.gnu.org/legacy-ml/gcc/2009-11/msg00047.html > > > I have to pass -Wl,-debug to gcc in order to get the command for > > > collect2 to which itself I have to pass -plugin-opt=-debug in order to > > > get the command for lto-wrapper. According to the aforementioned mail I > > > should add option -debug to lto-wrapper, however, it appears to me that > > > option -debug was removed. I gave options -v and -### a chance without > > > luck, i.e., those only print the usual environment variables and > > > afterwards a list of object files like > > > > > > /tmp/ccPEIV35.ltrans0.ltrans.o > > > /tmp/ccNmpKfS.debug.temp.o > > > /tmp/cceiCIFg.debug.temp.o > > > /tmp/ccZ4Qc7E.debug.temp.o > > > ... > > > > > > but no lto1 command. Thus, how do you retrieve the lto1 command? > > > > > > While desperate I retrieved it manually via strace. However, the lto1 > > > command refers to temporary files which have been erased meanwhile. I > > > actually didn't expect that because I added -save-temps to all the > > > intermediate commands which is also reflected in the environment > > > variable COLLECT_GCC_OPTIONS. Thus, how do you keep temporary files? > > > > Adding -v -save-temps and then running gdb on the lto1 command works and is > > what I usually do. > > Strangely this is not the case for the current project I'm looking at. The > link step is done via > > /usr/local/bin/s390x-linux-gnu-gcc -save-temps \ > -O3 -DNDEBUG -flto=auto -fno-fat-lto-objects \ > -march=z15 -static -nostdlib \ > -Wl,--fatal-warnings -Wl,--no-warn-rwx-segments \ > -Werror -Wall -Wundef -T/devel/foo.ld \ > obj1.o ... objN.o > > where -save-temps is given and was also given during compile time of each > object obj1.o ... objN.o file. Anyhow, adding -Wl,-debug to the link command > above I see that -save-temps is also included in the environment variable > COLLECT_GCC_OPTIONS="... -save-temps ..." and I get the linker command > > /usr/local/lib/gcc/s390x-linux-gnu/13.0.0/../../../../s390x-linux-gnu/bin/ld \ > -plugin /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/liblto_plugin.so \ > -plugin-opt=/usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto-wrapper \ > -plugin-opt=-fresolution=foo.res \ > -m elf64_s390 -static -o foo \ > -L/usr/local/lib/gcc/s390x-linux-gnu/13.0.0 \ > -L/usr/local/lib/gcc/s390x-linux-gnu/13.0.0/../../../../s390x-linux-gnu/lib > \ > --fatal-warnings --no-warn-rwx-segments \ > obj1.o ... objN.o -lgcc -T /devel/foo.ld > > to which I add -plugin-opt=-debug in order to get the lto-wrapper command > > /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto-wrapper \ > -fresolution=foo.res -flinker-output=exec \ > obj1.o ... objN.o > > to which I add -v -save-temps in order to actually get the lto1 command. From > the output printed to stderr I see one lto1 command: > > /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto1 -quiet -dumpbase ./a.wpa \ > -march=z15 -m64 -mzarch -g -O3 -version -fno-openmp -fno-openacc -fno-pie \ > -fcf-protection=none -fltrans-output-list=/tmp/ccsuw4Nj.ltrans.out \ > -fwpa=32 -fresolution=foo.res -flinker-output=exec @./a.wpa.args.0 > > However, this lto1 invocation does no local transformation. Having a look at > the strace output I see a second lto1 invocation: > > /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto1 -quiet \ > -dumpbase ./a.ltrans0.ltrans -march=z15 -m64 -mzarch -g -O3 \ > -fno-openmp -fno-openacc -fno-pie -fcf-protection=none \ > -fltrans /tmp/ccsuw4Nj.ltrans0.o -o /tmp/cc69wMp3.s > > This lto1 invocation seems to originate from Makefile /tmp/cc5piuXT.mk > generated by lto-wrapper. To my surprise the Makefile as well as the > temporary > file /tmp/ccsuw4Nj.ltrans0.o are removed although -save-temps was given all > the > time. Stepping through lto-wrapper with gdb I see that > > save_temps = 1; > > is never executed while walking over decoded_options. > > For the time being I hard coded save_temps=1 into lto-wrapper.cc and I am now > able to debug the lto1 invocation of interest. Not sure whether there > is a more elegant way? > > All this is based upon a rather huge CMake based project and so far I didn't > find the time to come up with a small reproducer. Still hoping this might be > useful for someone else. At some point passing -Wl,-debug -Wl,-v to the gcc driver was necessary but I think we've meanwhile fixed
[RISCV] RISC-V GNU Toolchain Biweekly Sync-up call (Dec 01, 2022)
Hi all, Here is the agenda for tomorrow's RISC-V GNU toolchain meeting. If you have any topics want to discuss or share, please let me know and I will add them to the agenda, thanks. Agenda: - riscv-gnu-toolchain repo's updates - RVV gcc support & RVV c intrinsic api progress - -march option doesn't take ISA strings - New extensions support patches in binutils - ZC* extension support status - Open discuss Wei Wu - PLCT Lab is inviting you to a scheduled Zoom meeting. Topic: RISC-V GNU Toolchain Biweekly Sync-up Time: Dec 01, 2022 11:00 PM Singapore Please download and import the following iCalendar (.ics) files to your calendar system. Weekly: https://calendar.google.com/calendar/ical/lm5bddk2krcmtv5iputjgqvoio%40group.calendar.google.com/public/basic.ics Join Zoom Meeting https://zoom.us/j/89393600951?pwd=ZFpWMkZ6Tm1TbUFXT1hZZjZZMHhRQT09 Meeting ID: 893 9360 0951 Passcode: 899662 BEIJING, China 11:00pThu, Nov 03 2022 12:00aFri, Nov 03 2022 PST/PDT, Pacific Standard Time (US) 7:00aThu, Nov 03 2022 8:00aThu, Nov 03 2022 PHILADELPHIA, United States, Pennsylvania 10:00aThu, Nov 03 2022 11:00aThu, Nov 03 2022 Paris, France 16:00pThu, Nov 03 2022 17:00pThu, Nov 03 2022BEGIN:VCALENDAR PRODID:-//zoom.us//iCalendar Event//EN VERSION:2.0 CALSCALE:GREGORIAN METHOD:PUBLISH CLASS:PUBLIC BEGIN:VTIMEZONE TZID:Asia/Singapore LAST-MODIFIED:20220317T223602Z TZURL:http://tzurl.org/zoneinfo-outlook/Asia/Singapore X-LIC-LOCATION:Asia/Singapore BEGIN:STANDARD TZNAME:+08 TZOFFSETFROM:+0800 TZOFFSETTO:+0800 DTSTART:19700101T00 END:STANDARD END:VTIMEZONE BEGIN:VEVENT DTSTAMP:20220713T134316Z DTSTART;TZID=Asia/Singapore:20220714T23 DTEND;TZID=Asia/Singapore:20220715T00 SUMMARY:RISC-V GNU Toolchain Biweekly Sync-up RRULE:FREQ=WEEKLY;WKST=SU;UNTIL=20230323T16;INTERVAL=2;BYDAY=TH UID:ZOOM89393600951 TZID:Asia/Singapore DESCRIPTION:Wei Wu - PLCT Lab is inviting you to a scheduled Zoom meeting .\n\nJoin Zoom Meeting\nhttps://us02web.zoom.us/j/89393600951?pwd=ZFpWMk Z6Tm1TbUFXT1hZZjZZMHhRQT09\n\nMeeting ID: 893 9360 0951\nPasscode: 89966 2\nOne tap mobile\n+6531651065\,\,89393600951#\,\,\,\,*899662# Singapore \n+6531587288\,\,89393600951#\,\,\,\,*899662# Singapore\n\nDial by your location\n+65 3165 1065 Singapore\n+65 3158 7288 Singapo re\n+1 669 900 9128 US (San Jose)\n+1 669 444 9171 US\n +1 346 248 7799 US (Houston)\n+1 253 215 8782 US (Tacoma) \n+1 312 626 6799 US (Chicago)\n+1 646 558 8656 US (New York)\n+1 646 931 3860 US\n+1 301 715 8592 US (Washingto n DC)\nMeeting ID: 893 9360 0951\nPasscode: 899662\nFind your local numb er: https://us02web.zoom.us/u/kk9cyIPNJ\n\n LOCATION:https://us02web.zoom.us/j/89393600951?pwd=ZFpWMkZ6Tm1TbUFXT1hZZj ZZMHhRQT09 BEGIN:VALARM TRIGGER:-PT10M ACTION:DISPLAY DESCRIPTION:Reminder END:VALARM END:VEVENT END:VCALENDAR
Re: access to include path in front end
Hello,
On Tue, 29 Nov 2022, James K. Lowden wrote:
> I don't understand how to access in a front end the arguments to the -I
> option on the command line.
>
> Cobol has a feature similar to the C preprecessor, known as the
> Compiler Directing Facility (CDF). The CDF has a COPY statement that
> resembles an #include directive in C, and shares the property that COPY
> names a file that is normally found in a "copybook" which, for our
> purposes, is a directory of such files. The name of that directory is
> defined outside the Cobol program.
>
> I would like to use the -I option to pass the names of copybook
> directories to the cobol front end. A bit of exploration yesterday left
> me with the sense that the -I argument, in C at least, is not passed to
> the compiler, but to the preprocessor. Access to -fmax-errors I think
> I've figured out, but -I is a mystery.
>
> I'm a little puzzled by the status quo as I understand it. Unless I
> missed it, it's not discussed in gccint. ISTM ideally there would be
> some kind of getopt(3) processing, and the whole set of command-line
> options captured in an array of structures accessible to any front
> end.
There is, it's just much more complicated than getopt :)
If you're looking at the C frontends for inspiration, then:
c-family/c.opt defines which options are recognized and several specifics
about them, e.g. for -I it has:
I
C ObjC C++ ObjC++ Joined Separate MissingArgError(missing path after %qs)
-I Add to the end of the main include path.
(look at some other examples therein, also in common.opt to get a feel).
Then code in c-family/c-opts.c:c_common_handle_option actually handles the
option:
case OPT_I:
if (strcmp (arg, "-"))
add_path (xstrdup (arg), INC_BRACKET, 0, true);
else
.,.
That function is made a langhook for option processing so that it's
actually called via c/c-objc-common.h:
#define LANG_HOOKS_HANDLE_OPTION c_common_handle_option
If you're also using the model of a compiler driver (i.e. the gcc program,
source in gcc.cc) that actually calls compiler (cc1), assembler and
linker, then you also need to arrange for that program to pass all -I
options to the compiler proper. That is done with the spec language, by
somewhere having '{I*}' in the specs for invoking the cobol compiler.
E.g. look in gcc.cc for '@c' (matching the file extension) how that entry
uses '%(cpp_unique_options)', and how cpp_unique_options is defined for
the specs language:
INIT_STATIC_SPEC ("cpp_unique_options", &cpp_unique_options),
and
static const char *cpp_unique_options =
"%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I\
(the specs language used here is documented in a lengthy comment early in
gcc.cc, "The Specs Language")
The "%@{I*F*}" is the one that makes gcc pass -Iwhatever to cc1 (and
ensures relative order with -F options is retained and puts all these into
an @file if one is given on the cmdline, otherwise leaves it on cmdline).
If you use the compiler driver then using '-v' when invoking it will
quickly tell you if that options passing worked, as it will show the
concrete command it exec's for the compiler proper.
Hope this helps.
Ciao,
Michael.
RE: Code generation: How to define file-scope static variables?
David, for completion I am updating this message with what I've learned. I
do this in the hope that I might assist the next poor souls that find
themselves reverse-engineering front ends because they are creating their
own front end.
I claim only that this worked for me. I can't you tell what assumptions
I've been making, because I don't know what I don't know.
That said: I found that for file-static variables, the trick is to create a
var_decl for the variable with TREE_STATIC(var_decl)=1, and then to call
rest_of_decl_compilation (var_decl, true, false); // top_level is true;
at_end is false
Having done that for an integer_type var_decl with the name
"dubner_at_work", and with an initial value of 123454321, I now see this at
the very beginning of the generated .s file:
##
.file "call-scope-1.cbl"
.text
.Ltext0:
.file 0 "/home/bob/repos/gcc-cobol/gcc/cobol/failures/call-scope-1"
"call-scope-1.cbl"
.data
.align 4
.type dubner_at_work, @object
.size dubner_at_work, 4
dubner_at_work:
.long 123454321
.section.rodata [...]
##
And that's exactly what I wanted.
Thanks for your help. It was your mention of "rest_of_compilation" that
ended this marathon investigation, and I really appreciate it.
Bob Dubner
-Original Message-
From: Gcc On Behalf Of Robert
Dubner
Sent: Monday, November 28, 2022 21:55
To: David Malcolm ; gcc@gcc.gnu.org
Cc: Bob Dubner
Subject: RE: Code generation: How to define file-scope static variables?
David, thank you very much. That looks very much like what I was hoping
for.
I'll dig into it tomorrow.
Heartfelt thanks,
Bob Dubner.
-Original Message-
From: David Malcolm
Sent: Monday, November 28, 2022 18:01
To: Robert Dubner ; gcc@gcc.gnu.org
Cc: 'Bob Dubner'
Subject: Re: Code generation: How to define file-scope static variables?
On Mon, 2022-11-28 at 15:28 -0600, Robert Dubner wrote:
> I am part of a team working on a COBOL front end for GCC.
>
> By reverse engineering other front ends, I learned, some months ago,
> how to create a function_decl GENERIC node that is the root of a
> GENERIC tree describing an entire function.
>
> By calling the routine cgraph_node::finalize_function() with that
> function_decl, the assembly language for that function is created, and
> all is well.
>
> But now I need to be able to create the equivalent of a file-scope
> static variable in C.
>
> This C program file:
>
> //
> static int dubner_at_work = 123454321; int main(int argc, char **argv)
> {
> }
> //
>
> produces, in part, this assembly language:
>
> ###
> .file "ccc.c"
> .text
> .data
> .align 4
> .type dubner_at_work, @object
> .size dubner_at_work, 4
> dubner_at_work:
> .long 123454321
> .text
> .globl main
> .type main, @function
> [...]
> ###
>
> In my own GENERIC generation code, I believe that I am creating a
> proper translation_unit_decl that contains the block and the vars
> nodes for specifying "dubner_at_work".
>
> But I have been unable, after several days of looking, to figure out
> the equivalent of "cgraph_node::finalize_function" for a
> translation_unit_decl. The resulting assembly language doesn't have a
> definition for "dubner_at_work".
>
> Can anybody describe how I can tell the downstream processing that I
> need the translation_unit_decl to actually define storage?
You might find libgccjit's gcc/jit/jit-playback.cc helpful for this, as it
tends to contain minimal code to build trees (generally
simplified/reverse-engineered from the C frontend).
playback::context::global_new_decl makes the VAR_DECL node, and such trees
are added to the jit playback::context's m_globals. In
playback::context::replay, we have:
/* Finalize globals. See how FORTRAN 95 does it in gfc_be_parse_file()
for a simple reference. */
FOR_EACH_VEC_ELT (m_globals, i, global)
rest_of_decl_compilation (global, true, true);
wrapup_global_declarations (m_globals.address(), m_globals.length());
So you'll probably want to do something similar for your globals.
Caveat: this is all reverse-engineered by me/others from the C frontend (and
I haven't touched this code in a while), so I may be missing things here.
Dave
Re: access to include path in front end
On Wed, 30 Nov 2022 08:49:35 +0100 Richard Biener wrote: > > I would like to use the -I option to pass the names of copybook > > directories to the cobol front end. A bit of exploration yesterday > > left me with the sense that the -I argument, in C at least, is not > > passed to the compiler, but to the preprocessor. Access to > > -fmax-errors I think I've figured out, but -I is a mystery. > > The frontends have access to the set of passed options via the > option processing langhooks (and there's also global_options > and global_options_set), -I would be the OPT_I option (currently > only enabled for some frontends, you can add that in your > language specific .opt file). The set of include directories is > not in any way special here. Thanks for clearing that up for me, Richard. I somehow got the impression that the langhooks were needed only for adding options specific to the language. --jkl
Re: access to include path in front end
On Wed, 30 Nov 2022 at 15:59, Michael Matz wrote:
> If you're looking at the C frontends for inspiration, then:
>
> c-family/c.opt defines which options are recognized and several specifics
> about them, e.g. for -I it has:
>
>
> I
> C ObjC C++ ObjC++ Joined Separate MissingArgError(missing path after %qs)
> -I Add to the end of the main include path.
>
>
> (look at some other examples therein, also in common.opt to get a feel).
There is also https://gcc.gnu.org/onlinedocs/gccint/Options.html
describing these files.
> If you're also using the model of a compiler driver (i.e. the gcc program,
> source in gcc.cc) that actually calls compiler (cc1), assembler and
> linker, then you also need to arrange for that program to pass all -I
> options to the compiler proper. That is done with the spec language, by
> somewhere having '{I*}' in the specs for invoking the cobol compiler.
> E.g. look in gcc.cc for '@c' (matching the file extension) how that entry
> uses '%(cpp_unique_options)', and how cpp_unique_options is defined for
> the specs language:
>
> INIT_STATIC_SPEC ("cpp_unique_options", &cpp_unique_options),
>
> and
>
> static const char *cpp_unique_options =
> "%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I\
>
> (the specs language used here is documented in a lengthy comment early in
> gcc.cc, "The Specs Language")
Also documented at https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html
