Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc wrote: > > Hi, the create_gcov tool was probably removed with the assumption that it > was only used with Google GCC branch, but it is actually used with GCC > trunk as well. > > Given that, the tool will be restored in the github repo. It seems to build > and work fine with the regression test. > > The tool may ust work as it is right now, but there is no guarantee it > won't break in the future unless someone in the GCC community tries to > maintain it. I think if we want to keep the feature it makes sense to provide create_gcov functionality either directly from perf (input data producer) or from gcc (data consumer). Of course I have no idea about its complexity, license or implementation language ... Having the tool third-party makes keeping the whole chain working more difficult. Richard. > Thanks, > > David > > On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > > GCC documentation for AutoFDO points to create_gcov tool that converts > > perf.data file into gcov format that can be consumed by gcc with > > -fauto-profile (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > > > > > I noticed that the source code for create_gcov has been deleted from > > https://github.com/google/autofdo on April 7. I asked about that change > > in that repo and got the following reply: > > > > > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > > > > > "Actually we didn't use create_gcov and havn't updated create_gcov for > > years, and we also didn't have enough tests to guarantee it works (It was > > gcc-4.8 when we used and verified create_gcov). If you need it, it is > > welcomed to update create_gcov and add it to the respository." > > > > > > > > Does this mean that AutoFDO is currently dead in gcc? > > > > > > Hello. > > > > > > Yes. I know that even basic test cases have been broken for years in the > > GCC. > > > It's new to me that create_gcov was removed. > > > > > > I tend to send patch to GCC that will remove AutoFDO from GCC. > > > I known Bin spent some time working on AutoFDO, has he came up to > > something? > > > > The GCC side of auto-FDO is not that hard. We have most of > > infrastructure in place, but stopping point for me was always difficulty > > to get gcov-tool working. If some maintainer steps up, I think I can > > fix GCC side. > > > > I am bit unsure how important feature it is - we have FDO that works > > quite well for most users but I know there are some users of the LLVM > > implementation and there is potential to tie this with other hardware > > events to asist i.e. if conversion (where one wants to know how well CPU > > predicts the jump rather than just the jump probability) which I always > > found potentially interesting. > > > > Honza > > > > > > Martin > > > > > > > > > > > Thanks, > > > > > > > > Eugene > > > > > > > > >
Re: State of AutoFDO in GCC
On 4/23/21 9:00 AM, Richard Biener via Gcc wrote: > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > wrote: >> >> Hi, the create_gcov tool was probably removed with the assumption that it >> was only used with Google GCC branch, but it is actually used with GCC >> trunk as well. >> >> Given that, the tool will be restored in the github repo. It seems to build >> and work fine with the regression test. >> >> The tool may ust work as it is right now, but there is no guarantee it >> won't break in the future unless someone in the GCC community tries to >> maintain it. Hi. The current situation is that AutoFDO doesn't work with pretty simple test-cases we have in testsuite: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379 These are ~5 years old and nothing has happened. I'm pretty sure the current autofdo can't emit a .gcda file format that I've changed in the recent years. > > I think if we want to keep the feature it makes sense to provide create_gcov > functionality either directly from perf (input data producer) or from gcc > (data consumer). Of course I have no idea about its complexity, license > or implementation language ... For me, it's just an i386 feature (maybe aarch64 has perf counters too?), supported only by vendor (Intel) and I'm not planning working on that. I don't like having a feature that is obviously broken and potential GCC users get bad experience every time they try to use it. Can we at least deprecate the feature for GCC 11? If these is enough interest, we can fix it, if not, I would remove it in GCC 13 timeframe. Thoughts? Martin > > Having the tool third-party makes keeping the whole chain working more > difficult. > > Richard. > >> Thanks, >> >> David >> >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: >> On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > GCC documentation for AutoFDO points to create_gcov tool that converts >>> perf.data file into gcov format that can be consumed by gcc with >>> -fauto-profile (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > I noticed that the source code for create_gcov has been deleted from >>> https://github.com/google/autofdo on April 7. I asked about that change >>> in that repo and got the following reply: > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > "Actually we didn't use create_gcov and havn't updated create_gcov for >>> years, and we also didn't have enough tests to guarantee it works (It was >>> gcc-4.8 when we used and verified create_gcov). If you need it, it is >>> welcomed to update create_gcov and add it to the respository." > > Does this mean that AutoFDO is currently dead in gcc? Hello. Yes. I know that even basic test cases have been broken for years in the >>> GCC. It's new to me that create_gcov was removed. I tend to send patch to GCC that will remove AutoFDO from GCC. I known Bin spent some time working on AutoFDO, has he came up to >>> something? >>> >>> The GCC side of auto-FDO is not that hard. We have most of >>> infrastructure in place, but stopping point for me was always difficulty >>> to get gcov-tool working. If some maintainer steps up, I think I can >>> fix GCC side. >>> >>> I am bit unsure how important feature it is - we have FDO that works >>> quite well for most users but I know there are some users of the LLVM >>> implementation and there is potential to tie this with other hardware >>> events to asist i.e. if conversion (where one wants to know how well CPU >>> predicts the jump rather than just the jump probability) which I always >>> found potentially interesting. >>> >>> Honza Martin > > Thanks, > > Eugene > >>>
origin/trunk branch - who added it?
Hello. Few weeks ago, I noticed we have a new remote branch that follows origin/master: https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/trunk Do we know who added it? And what was the motivation? Thanks, Martin
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 9:18 AM Martin Liška wrote: > > On 4/23/21 9:00 AM, Richard Biener via Gcc wrote: > > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > > wrote: > >> > >> Hi, the create_gcov tool was probably removed with the assumption that it > >> was only used with Google GCC branch, but it is actually used with GCC > >> trunk as well. > >> > >> Given that, the tool will be restored in the github repo. It seems to build > >> and work fine with the regression test. > >> > >> The tool may ust work as it is right now, but there is no guarantee it > >> won't break in the future unless someone in the GCC community tries to > >> maintain it. > > Hi. > > The current situation is that AutoFDO doesn't work with pretty simple > test-cases > we have in testsuite: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379 > > These are ~5 years old and nothing has happened. > > I'm pretty sure the current autofdo can't emit a .gcda file format that > I've changed in the recent years. > > > > > I think if we want to keep the feature it makes sense to provide create_gcov > > functionality either directly from perf (input data producer) or from gcc > > (data consumer). Of course I have no idea about its complexity, license > > or implementation language ... > > For me, it's just an i386 feature (maybe aarch64 has perf counters too?), > supported > only by vendor (Intel) and I'm not planning working on that. > I don't like having a feature that is obviously broken and potential GCC > users get > bad experience every time they try to use it. > > Can we at least deprecate the feature for GCC 11? If these is enough interest, > we can fix it, if not, I would remove it in GCC 13 timeframe. I have no opinion here and I'm not opposed to ripping out the feature from GCC 12. Since it doesn't work anyway there's no need for any deprecation. Richard. > Thoughts? > Martin > > > > > Having the tool third-party makes keeping the whole chain working more > > difficult. > > > > Richard. > > > >> Thanks, > >> > >> David > >> > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > >> > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > GCC documentation for AutoFDO points to create_gcov tool that converts > >>> perf.data file into gcov format that can be consumed by gcc with > >>> -fauto-profile (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > I noticed that the source code for create_gcov has been deleted from > >>> https://github.com/google/autofdo on April 7. I asked about that change > >>> in that repo and got the following reply: > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > "Actually we didn't use create_gcov and havn't updated create_gcov for > >>> years, and we also didn't have enough tests to guarantee it works (It was > >>> gcc-4.8 when we used and verified create_gcov). If you need it, it is > >>> welcomed to update create_gcov and add it to the respository." > > > > Does this mean that AutoFDO is currently dead in gcc? > > Hello. > > Yes. I know that even basic test cases have been broken for years in the > >>> GCC. > It's new to me that create_gcov was removed. > > I tend to send patch to GCC that will remove AutoFDO from GCC. > I known Bin spent some time working on AutoFDO, has he came up to > >>> something? > >>> > >>> The GCC side of auto-FDO is not that hard. We have most of > >>> infrastructure in place, but stopping point for me was always difficulty > >>> to get gcov-tool working. If some maintainer steps up, I think I can > >>> fix GCC side. > >>> > >>> I am bit unsure how important feature it is - we have FDO that works > >>> quite well for most users but I know there are some users of the LLVM > >>> implementation and there is potential to tie this with other hardware > >>> events to asist i.e. if conversion (where one wants to know how well CPU > >>> predicts the jump rather than just the jump probability) which I always > >>> found potentially interesting. > >>> > >>> Honza > > Martin > > > > > Thanks, > > > > Eugene > > > > >>> >
GCC 8.5 Status Report (2021-04-23)
Status == GCC 8.5 release and closing of the 8 branch is several months overdue, we don't have enough time to maintain trunk and 4 supported release branches. Therefore, I'd like to do 8.5-rc1 on 7th of May and release 8.5 and close the branch a week after that. We have one P1 bug though - P98358 - it is unclear if just changing the testcase is all that is needed or if some C++ FE changes are needed. If anyone has anything to backport to 8.5, please do so in the upcoming 2 weeks. Thanks. Quality Data Priority # Change from last report --- --- P11 P2 287 + 28 P3 32 - 2 P4 152 - 3 P5 21 - 1 --- --- Total P1-P3 320 + 26 Total 493 + 22 Previous Report === https://gcc.gnu.org/legacy-ml/gcc/2020-03/msg00043.html
Second GCC 11.1 Release Candidate available from gcc.gnu.org
Some blocker bugs were reported against the first release candidate of GCC 11.1, so there is a second release candidate for GCC 11.1 available from https://gcc.gnu.org/pub/gcc/snapshots/11.1.0-RC-20210423/ ftp://gcc.gnu.org/pub/gcc/snapshots/11.1.0-RC-20210423 and shortly its mirrors. It has been generated from git revision r11-8287-g7a7fc01b9d20afb1a2b805d93cb838a38e656420. I have so far bootstrapped and tested the release candidate on x86_64-linux and i686-linux. Please test it and report any issues to bugzilla. If all goes well, I'd still like to release 11.1 on Tuesday, April 27th.
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 12:00 AM Richard Biener wrote: > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > wrote: > > > > Hi, the create_gcov tool was probably removed with the assumption that it > > was only used with Google GCC branch, but it is actually used with GCC > > trunk as well. > > > > Given that, the tool will be restored in the github repo. It seems to > build > > and work fine with the regression test. > > > > The tool may ust work as it is right now, but there is no guarantee it > > won't break in the future unless someone in the GCC community tries to > > maintain it. > > I think if we want to keep the feature it makes sense to provide > create_gcov > functionality either directly from perf (input data producer) or from gcc > (data consumer). Of course I have no idea about its complexity, license > or implementation language ... > > Right. What it takes is a perf data (can be text format) parser to produce the format GCC needs, but someone in the community needs to take the lead. It should not involve too much effort. David Having the tool third-party makes keeping the whole chain working more > difficult. > > Richard. > > > Thanks, > > > > David > > > > On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > > > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > > > GCC documentation for AutoFDO points to create_gcov tool that > converts > > > perf.data file into gcov format that can be consumed by gcc with > > > -fauto-profile ( > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > > https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > > > > > > > I noticed that the source code for create_gcov has been deleted > from > > > https://github.com/google/autofdo on April 7. I asked about that > change > > > in that repo and got the following reply: > > > > > > > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > > > > > > > "Actually we didn't use create_gcov and havn't updated create_gcov > for > > > years, and we also didn't have enough tests to guarantee it works (It > was > > > gcc-4.8 when we used and verified create_gcov). If you need it, it is > > > welcomed to update create_gcov and add it to the respository." > > > > > > > > > > Does this mean that AutoFDO is currently dead in gcc? > > > > > > > > Hello. > > > > > > > > Yes. I know that even basic test cases have been broken for years in > the > > > GCC. > > > > It's new to me that create_gcov was removed. > > > > > > > > I tend to send patch to GCC that will remove AutoFDO from GCC. > > > > I known Bin spent some time working on AutoFDO, has he came up to > > > something? > > > > > > The GCC side of auto-FDO is not that hard. We have most of > > > infrastructure in place, but stopping point for me was always > difficulty > > > to get gcov-tool working. If some maintainer steps up, I think I can > > > fix GCC side. > > > > > > I am bit unsure how important feature it is - we have FDO that works > > > quite well for most users but I know there are some users of the LLVM > > > implementation and there is potential to tie this with other hardware > > > events to asist i.e. if conversion (where one wants to know how well > CPU > > > predicts the jump rather than just the jump probability) which I always > > > found potentially interesting. > > > > > > Honza > > > > > > > > Martin > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Eugene > > > > > > > > > > > > >
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 12:18 AM Martin Liška wrote: > On 4/23/21 9:00 AM, Richard Biener via Gcc wrote: > > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > > wrote: > >> > >> Hi, the create_gcov tool was probably removed with the assumption that > it > >> was only used with Google GCC branch, but it is actually used with GCC > >> trunk as well. > >> > >> Given that, the tool will be restored in the github repo. It seems to > build > >> and work fine with the regression test. > >> > >> The tool may ust work as it is right now, but there is no guarantee it > >> won't break in the future unless someone in the GCC community tries to > >> maintain it. > > Hi. > > The current situation is that AutoFDO doesn't work with pretty simple > test-cases > we have in testsuite: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379 > > These are ~5 years old and nothing has happened. > > I'm pretty sure the current autofdo can't emit a .gcda file format that > I've changed in the recent years. > I have not not looked into the details, but is it possible the problem is at GCC side (e.g, debug info quality etc)? > > > > > I think if we want to keep the feature it makes sense to provide > create_gcov > > functionality either directly from perf (input data producer) or from gcc > > (data consumer). Of course I have no idea about its complexity, license > > or implementation language ... > > For me, it's just an i386 feature (maybe aarch64 has perf counters too?), > supported > only by vendor (Intel) and I'm not planning working on that. > I don't like having a feature that is obviously broken and potential GCC > users get > bad experience every time they try to use it. > It must be working at sometime, so perhaps a bisect of GCC revisions can reveal when it regressed. > Can we at least deprecate the feature for GCC 11? If these is enough > interest, > we can fix it, if not, I would remove it in GCC 13 timeframe. > This is a major feature in Clang. Deprecating it in GCC will be unfortunate. David > > Thoughts? > Martin > > > > > Having the tool third-party makes keeping the whole chain working more > > difficult. > > > > Richard. > > > >> Thanks, > >> > >> David > >> > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > >> > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > GCC documentation for AutoFDO points to create_gcov tool that > converts > >>> perf.data file into gcov format that can be consumed by gcc with > >>> -fauto-profile ( > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > I noticed that the source code for create_gcov has been deleted from > >>> https://github.com/google/autofdo on April 7. I asked about that > change > >>> in that repo and got the following reply: > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > "Actually we didn't use create_gcov and havn't updated create_gcov > for > >>> years, and we also didn't have enough tests to guarantee it works (It > was > >>> gcc-4.8 when we used and verified create_gcov). If you need it, it is > >>> welcomed to update create_gcov and add it to the respository." > > > > Does this mean that AutoFDO is currently dead in gcc? > > Hello. > > Yes. I know that even basic test cases have been broken for years in > the > >>> GCC. > It's new to me that create_gcov was removed. > > I tend to send patch to GCC that will remove AutoFDO from GCC. > I known Bin spent some time working on AutoFDO, has he came up to > >>> something? > >>> > >>> The GCC side of auto-FDO is not that hard. We have most of > >>> infrastructure in place, but stopping point for me was always > difficulty > >>> to get gcov-tool working. If some maintainer steps up, I think I can > >>> fix GCC side. > >>> > >>> I am bit unsure how important feature it is - we have FDO that works > >>> quite well for most users but I know there are some users of the LLVM > >>> implementation and there is potential to tie this with other hardware > >>> events to asist i.e. if conversion (where one wants to know how well > CPU > >>> predicts the jump rather than just the jump probability) which I always > >>> found potentially interesting. > >>> > >>> Honza > > Martin > > > > > Thanks, > > > > Eugene > > > > >>> > >
Re: State of AutoFDO in GCC
> On Fri, Apr 23, 2021 at 12:18 AM Martin Liška wrote: > > > On 4/23/21 9:00 AM, Richard Biener via Gcc wrote: > > > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > > > wrote: > > >> > > >> Hi, the create_gcov tool was probably removed with the assumption that > > it > > >> was only used with Google GCC branch, but it is actually used with GCC > > >> trunk as well. > > >> > > >> Given that, the tool will be restored in the github repo. It seems to > > build > > >> and work fine with the regression test. > > >> > > >> The tool may ust work as it is right now, but there is no guarantee it > > >> won't break in the future unless someone in the GCC community tries to > > >> maintain it. > > > > Hi. > > > > The current situation is that AutoFDO doesn't work with pretty simple > > test-cases > > we have in testsuite: > > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379 > > > > These are ~5 years old and nothing has happened. Basic functionality is to identify hot parts of programs and give basic idea of branch probabilities. Bugs above are about other optimizations (indirect inlining, hot-cold partitioning and peeling). First one ought to be working but still indirect call profiling accounts for relatively minor portion of FDO benefits, so auto-FDO should be usefable even w/o these working if tooling was fixed. Expecting hot-cold partitioning and peeling to work reliably for testcases designed for normalFDO is IMO bit unrealistic. Perf is not limited to x86, so we should get basic usability for all major architectures. David, what CPUs are supported with auto-FDO on LLVM side? > > This is a major feature in Clang. Deprecating it in GCC will be unfortunate. I also like overall idea of auto-FDO just never had much time to get it into shape. Every stage1 I am trying to fix something that was broken for a while (like ICF last stage1) and fix it, so option would be to focus on autoFDO this stage1 even though I have quite few other things in TODO list. I never looked into the tools translating perf data to gcc compatible format and how easy would be to keep this alive. David, how does perf->LLVM path work these days? Honza > > David > > > > > Thoughts? > > Martin > > > > > > > > Having the tool third-party makes keeping the whole chain working more > > > difficult. > > > > > > Richard. > > > > > >> Thanks, > > >> > > >> David > > >> > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > > >> > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > GCC documentation for AutoFDO points to create_gcov tool that > > converts > > >>> perf.data file into gcov format that can be consumed by gcc with > > >>> -fauto-profile ( > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > > > I noticed that the source code for create_gcov has been deleted from > > >>> https://github.com/google/autofdo on April 7. I asked about that > > change > > >>> in that repo and got the following reply: > > > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > > > "Actually we didn't use create_gcov and havn't updated create_gcov > > for > > >>> years, and we also didn't have enough tests to guarantee it works (It > > was > > >>> gcc-4.8 when we used and verified create_gcov). If you need it, it is > > >>> welcomed to update create_gcov and add it to the respository." > > > > > > Does this mean that AutoFDO is currently dead in gcc? > > > > Hello. > > > > Yes. I know that even basic test cases have been broken for years in > > the > > >>> GCC. > > It's new to me that create_gcov was removed. > > > > I tend to send patch to GCC that will remove AutoFDO from GCC. > > I known Bin spent some time working on AutoFDO, has he came up to > > >>> something? > > >>> > > >>> The GCC side of auto-FDO is not that hard. We have most of > > >>> infrastructure in place, but stopping point for me was always > > difficulty > > >>> to get gcov-tool working. If some maintainer steps up, I think I can > > >>> fix GCC side. > > >>> > > >>> I am bit unsure how important feature it is - we have FDO that works > > >>> quite well for most users but I know there are some users of the LLVM > > >>> implementation and there is potential to tie this with other hardware > > >>> events to asist i.e. if conversion (where one wants to know how well > > CPU > > >>> predicts the jump rather than just the jump probability) which I always > > >>> found potentially interesting. > > >>> > > >>> Honza > > > > Martin > > > > > > > > Thanks, > > > > > > Eugene > > > > > > > >>> > > > >
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 9:54 AM Jan Hubicka wrote: > > On Fri, Apr 23, 2021 at 12:18 AM Martin Liška wrote: > > > > > On 4/23/21 9:00 AM, Richard Biener via Gcc wrote: > > > > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > > > > wrote: > > > >> > > > >> Hi, the create_gcov tool was probably removed with the assumption > that > > > it > > > >> was only used with Google GCC branch, but it is actually used with > GCC > > > >> trunk as well. > > > >> > > > >> Given that, the tool will be restored in the github repo. It seems > to > > > build > > > >> and work fine with the regression test. > > > >> > > > >> The tool may ust work as it is right now, but there is no guarantee > it > > > >> won't break in the future unless someone in the GCC community tries > to > > > >> maintain it. > > > > > > Hi. > > > > > > The current situation is that AutoFDO doesn't work with pretty simple > > > test-cases > > > we have in testsuite: > > > > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379 > > > > > > These are ~5 years old and nothing has happened. > > Basic functionality is to identify hot parts of programs and give basic > idea of branch probabilities. Bugs above are about other optimizations > (indirect inlining, hot-cold partitioning and peeling). First one ought > to be working but still indirect call profiling accounts for relatively > minor portion of FDO benefits, so auto-FDO should be usefable even w/o > these working if tooling was fixed. Expecting hot-cold partitioning and > peeling to work reliably for testcases designed for normalFDO is IMO bit > unrealistic. > > Perf is not limited to x86, so we should get basic usability for all > major architectures. David, what CPUs are supported with auto-FDO on > LLVM side? > > > > This is a major feature in Clang. Deprecating it in GCC will be > unfortunate. > > I also like overall idea of auto-FDO just never had much time to get it > into shape. Every stage1 I am trying to fix something that was broken > for a while (like ICF last stage1) and fix it, so option would be to > focus on autoFDO this stage1 even though I have quite few other things > in TODO list. I never looked into the tools translating perf data to > gcc compatible format and how easy would be to keep this alive. > > David, how does perf->LLVM path work these days? > It uses create_llvm_prof tool which is in the same git repo. The data parsing part is shared with create_gcov, but the writer is obviously different for the two tools. David > Honza > > > > David > > > > > > > > Thoughts? > > > Martin > > > > > > > > > > > Having the tool third-party makes keeping the whole chain working > more > > > > difficult. > > > > > > > > Richard. > > > > > > > >> Thanks, > > > >> > > > >> David > > > >> > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > > > >> > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > > GCC documentation for AutoFDO points to create_gcov tool that > > > converts > > > >>> perf.data file into gcov format that can be consumed by gcc with > > > >>> -fauto-profile ( > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > > > > > I noticed that the source code for create_gcov has been deleted > from > > > >>> https://github.com/google/autofdo on April 7. I asked about that > > > change > > > >>> in that repo and got the following reply: > > > > > > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > > > > > "Actually we didn't use create_gcov and havn't updated > create_gcov > > > for > > > >>> years, and we also didn't have enough tests to guarantee it works > (It > > > was > > > >>> gcc-4.8 when we used and verified create_gcov). If you need it, it > is > > > >>> welcomed to update create_gcov and add it to the respository." > > > > > > > > Does this mean that AutoFDO is currently dead in gcc? > > > > > > Hello. > > > > > > Yes. I know that even basic test cases have been broken for years > in > > > the > > > >>> GCC. > > > It's new to me that create_gcov was removed. > > > > > > I tend to send patch to GCC that will remove AutoFDO from GCC. > > > I known Bin spent some time working on AutoFDO, has he came up to > > > >>> something? > > > >>> > > > >>> The GCC side of auto-FDO is not that hard. We have most of > > > >>> infrastructure in place, but stopping point for me was always > > > difficulty > > > >>> to get gcov-tool working. If some maintainer steps up, I think I > can > > > >>> fix GCC side. > > > >>> > > > >>> I am bit unsure how important feature it is - we have FDO that > works > > > >>> quite well for most users but I know there are some users of the > LLVM > > > >>> implementation and there is potential to tie this with other > hardware > > > >>> events to asist i.e. if conversion (where on
Re: State of AutoFDO in GCC
> > It uses create_llvm_prof tool which is in the same git repo. The data > parsing part is shared with create_gcov, but the writer is obviously > different for the two tools. OK and what are the main differences between llvmand gcc format? Honza > > David > > > > Honza > > > > > > David > > > > > > > > > > > Thoughts? > > > > Martin > > > > > > > > > > > > > > Having the tool third-party makes keeping the whole chain working > > more > > > > > difficult. > > > > > > > > > > Richard. > > > > > > > > > >> Thanks, > > > > >> > > > > >> David > > > > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > > > > >> > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > > > GCC documentation for AutoFDO points to create_gcov tool that > > > > converts > > > > >>> perf.data file into gcov format that can be consumed by gcc with > > > > >>> -fauto-profile ( > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > > > > > > > I noticed that the source code for create_gcov has been deleted > > from > > > > >>> https://github.com/google/autofdo on April 7. I asked about that > > > > change > > > > >>> in that repo and got the following reply: > > > > > > > > > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > > > > > > > "Actually we didn't use create_gcov and havn't updated > > create_gcov > > > > for > > > > >>> years, and we also didn't have enough tests to guarantee it works > > (It > > > > was > > > > >>> gcc-4.8 when we used and verified create_gcov). If you need it, it > > is > > > > >>> welcomed to update create_gcov and add it to the respository." > > > > > > > > > > Does this mean that AutoFDO is currently dead in gcc? > > > > > > > > Hello. > > > > > > > > Yes. I know that even basic test cases have been broken for years > > in > > > > the > > > > >>> GCC. > > > > It's new to me that create_gcov was removed. > > > > > > > > I tend to send patch to GCC that will remove AutoFDO from GCC. > > > > I known Bin spent some time working on AutoFDO, has he came up to > > > > >>> something? > > > > >>> > > > > >>> The GCC side of auto-FDO is not that hard. We have most of > > > > >>> infrastructure in place, but stopping point for me was always > > > > difficulty > > > > >>> to get gcov-tool working. If some maintainer steps up, I think I > > can > > > > >>> fix GCC side. > > > > >>> > > > > >>> I am bit unsure how important feature it is - we have FDO that > > works > > > > >>> quite well for most users but I know there are some users of the > > LLVM > > > > >>> implementation and there is potential to tie this with other > > hardware > > > > >>> events to asist i.e. if conversion (where one wants to know how > > well > > > > CPU > > > > >>> predicts the jump rather than just the jump probability) which I > > always > > > > >>> found potentially interesting. > > > > >>> > > > > >>> Honza > > > > > > > > Martin > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Eugene > > > > > > > > > > > > > >>> > > > > > > > > > >
Re: State of AutoFDO in GCC
> Hi. > > The current situation is that AutoFDO doesn't work with pretty simple > test-cases > we have in testsuite: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379 > > These are ~5 years old and nothing has happened. > > I'm pretty sure the current autofdo can't emit a .gcda file format that > I've changed in the recent years. I believe that the gcda files consumed by auto-FDO are not really sharing the normal gcov file format, just the low level i/o. See read_profile in auto-profile.c. Honza > > > > > I think if we want to keep the feature it makes sense to provide create_gcov > > functionality either directly from perf (input data producer) or from gcc > > (data consumer). Of course I have no idea about its complexity, license > > or implementation language ... > > For me, it's just an i386 feature (maybe aarch64 has perf counters too?), > supported > only by vendor (Intel) and I'm not planning working on that. > I don't like having a feature that is obviously broken and potential GCC > users get > bad experience every time they try to use it. > > Can we at least deprecate the feature for GCC 11? If these is enough interest, > we can fix it, if not, I would remove it in GCC 13 timeframe. > > Thoughts? > Martin > > > > > Having the tool third-party makes keeping the whole chain working more > > difficult. > > > > Richard. > > > >> Thanks, > >> > >> David > >> > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote: > >> > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > GCC documentation for AutoFDO points to create_gcov tool that converts > >>> perf.data file into gcov format that can be consumed by gcc with > >>> -fauto-profile (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > I noticed that the source code for create_gcov has been deleted from > >>> https://github.com/google/autofdo on April 7. I asked about that change > >>> in that repo and got the following reply: > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > "Actually we didn't use create_gcov and havn't updated create_gcov for > >>> years, and we also didn't have enough tests to guarantee it works (It was > >>> gcc-4.8 when we used and verified create_gcov). If you need it, it is > >>> welcomed to update create_gcov and add it to the respository." > > > > Does this mean that AutoFDO is currently dead in gcc? > > Hello. > > Yes. I know that even basic test cases have been broken for years in the > >>> GCC. > It's new to me that create_gcov was removed. > > I tend to send patch to GCC that will remove AutoFDO from GCC. > I known Bin spent some time working on AutoFDO, has he came up to > >>> something? > >>> > >>> The GCC side of auto-FDO is not that hard. We have most of > >>> infrastructure in place, but stopping point for me was always difficulty > >>> to get gcov-tool working. If some maintainer steps up, I think I can > >>> fix GCC side. > >>> > >>> I am bit unsure how important feature it is - we have FDO that works > >>> quite well for most users but I know there are some users of the LLVM > >>> implementation and there is potential to tie this with other hardware > >>> events to asist i.e. if conversion (where one wants to know how well CPU > >>> predicts the jump rather than just the jump probability) which I always > >>> found potentially interesting. > >>> > >>> Honza > > Martin > > > > > Thanks, > > > > Eugene > > > > >>> >
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 10:16 AM Jan Hubicka wrote: > > > > It uses create_llvm_prof tool which is in the same git repo. The data > > parsing part is shared with create_gcov, but the writer is obviously > > different for the two tools. > > OK and what are the main differences between llvmand gcc format? > > GCC expects GCOV format, I think while LLVM uses a newly designed binary format. David > Honza > > > > David > > > > > > > Honza > > > > > > > > David > > > > > > > > > > > > > > Thoughts? > > > > > Martin > > > > > > > > > > > > > > > > > Having the tool third-party makes keeping the whole chain working > > > more > > > > > > difficult. > > > > > > > > > > > > Richard. > > > > > > > > > > > >> Thanks, > > > > > >> > > > > > >> David > > > > > >> > > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka > wrote: > > > > > >> > > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > > > > GCC documentation for AutoFDO points to create_gcov tool that > > > > > converts > > > > > >>> perf.data file into gcov format that can be consumed by gcc > with > > > > > >>> -fauto-profile ( > > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > > > > > > > > > I noticed that the source code for create_gcov has been > deleted > > > from > > > > > >>> https://github.com/google/autofdo on April 7. I asked about > that > > > > > change > > > > > >>> in that repo and got the following reply: > > > > > > > > > > > > > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > > > > > > > > > "Actually we didn't use create_gcov and havn't updated > > > create_gcov > > > > > for > > > > > >>> years, and we also didn't have enough tests to guarantee it > works > > > (It > > > > > was > > > > > >>> gcc-4.8 when we used and verified create_gcov). If you need > it, it > > > is > > > > > >>> welcomed to update create_gcov and add it to the respository." > > > > > > > > > > > > Does this mean that AutoFDO is currently dead in gcc? > > > > > > > > > > Hello. > > > > > > > > > > Yes. I know that even basic test cases have been broken for > years > > > in > > > > > the > > > > > >>> GCC. > > > > > It's new to me that create_gcov was removed. > > > > > > > > > > I tend to send patch to GCC that will remove AutoFDO from GCC. > > > > > I known Bin spent some time working on AutoFDO, has he came > up to > > > > > >>> something? > > > > > >>> > > > > > >>> The GCC side of auto-FDO is not that hard. We have most of > > > > > >>> infrastructure in place, but stopping point for me was always > > > > > difficulty > > > > > >>> to get gcov-tool working. If some maintainer steps up, I > think I > > > can > > > > > >>> fix GCC side. > > > > > >>> > > > > > >>> I am bit unsure how important feature it is - we have FDO that > > > works > > > > > >>> quite well for most users but I know there are some users of > the > > > LLVM > > > > > >>> implementation and there is potential to tie this with other > > > hardware > > > > > >>> events to asist i.e. if conversion (where one wants to know how > > > well > > > > > CPU > > > > > >>> predicts the jump rather than just the jump probability) which > I > > > always > > > > > >>> found potentially interesting. > > > > > >>> > > > > > >>> Honza > > > > > > > > > > Martin > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Eugene > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 10:27 AM Xinliang David Li wrote: > > > On Fri, Apr 23, 2021 at 10:16 AM Jan Hubicka wrote: > >> > >> > It uses create_llvm_prof tool which is in the same git repo. The data >> > parsing part is shared with create_gcov, but the writer is obviously >> > different for the two tools. >> >> OK and what are the main differences between llvmand gcc format? >> >> GCC expects GCOV format, I think while LLVM uses a newly designed binary > format. > > Sorry I missed your next reply. I forgot about the details of GCC' format. David > > >> Honza >> > >> > David >> > >> > >> > > Honza >> > > > >> > > > David >> > > > >> > > > > >> > > > > Thoughts? >> > > > > Martin >> > > > > >> > > > > > >> > > > > > Having the tool third-party makes keeping the whole chain >> working >> > > more >> > > > > > difficult. >> > > > > > >> > > > > > Richard. >> > > > > > >> > > > > >> Thanks, >> > > > > >> >> > > > > >> David >> > > > > >> >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka >> wrote: >> > > > > >> >> > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: >> > > > > > GCC documentation for AutoFDO points to create_gcov tool >> that >> > > > > converts >> > > > > >>> perf.data file into gcov format that can be consumed by gcc >> with >> > > > > >>> -fauto-profile ( >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). >> > > > > > >> > > > > > I noticed that the source code for create_gcov has been >> deleted >> > > from >> > > > > >>> https://github.com/google/autofdo on April 7. I asked about >> that >> > > > > change >> > > > > >>> in that repo and got the following reply: >> > > > > > >> > > > > > >> > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 >> > > > > > >> > > > > > "Actually we didn't use create_gcov and havn't updated >> > > create_gcov >> > > > > for >> > > > > >>> years, and we also didn't have enough tests to guarantee it >> works >> > > (It >> > > > > was >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If you need >> it, it >> > > is >> > > > > >>> welcomed to update create_gcov and add it to the respository." >> > > > > > >> > > > > > Does this mean that AutoFDO is currently dead in gcc? >> > > > > >> > > > > Hello. >> > > > > >> > > > > Yes. I know that even basic test cases have been broken for >> years >> > > in >> > > > > the >> > > > > >>> GCC. >> > > > > It's new to me that create_gcov was removed. >> > > > > >> > > > > I tend to send patch to GCC that will remove AutoFDO from >> GCC. >> > > > > I known Bin spent some time working on AutoFDO, has he came >> up to >> > > > > >>> something? >> > > > > >>> >> > > > > >>> The GCC side of auto-FDO is not that hard. We have most of >> > > > > >>> infrastructure in place, but stopping point for me was always >> > > > > difficulty >> > > > > >>> to get gcov-tool working. If some maintainer steps up, I >> think I >> > > can >> > > > > >>> fix GCC side. >> > > > > >>> >> > > > > >>> I am bit unsure how important feature it is - we have FDO that >> > > works >> > > > > >>> quite well for most users but I know there are some users of >> the >> > > LLVM >> > > > > >>> implementation and there is potential to tie this with other >> > > hardware >> > > > > >>> events to asist i.e. if conversion (where one wants to know >> how >> > > well >> > > > > CPU >> > > > > >>> predicts the jump rather than just the jump probability) >> which I >> > > always >> > > > > >>> found potentially interesting. >> > > > > >>> >> > > > > >>> Honza >> > > > > >> > > > > Martin >> > > > > >> > > > > > >> > > > > > Thanks, >> > > > > > >> > > > > > Eugene >> > > > > > >> > > > > >> > > > > >>> >> > > > > >> > > > > >> > > >> >
Re: State of AutoFDO in GCC
> On Fri, Apr 23, 2021 at 10:27 AM Xinliang David Li > wrote: > > > > > > > On Fri, Apr 23, 2021 at 10:16 AM Jan Hubicka wrote: > > > >> > > >> > It uses create_llvm_prof tool which is in the same git repo. The data > >> > parsing part is shared with create_gcov, but the writer is obviously > >> > different for the two tools. > >> > >> OK and what are the main differences between llvmand gcc format? > >> > >> GCC expects GCOV format, I think while LLVM uses a newly designed binary > > format. > > > > > Sorry I missed your next reply. I forgot about the details of GCC' format. Thanks for explanation. How hard would be to make GCC consume this format? Is is reasonably stable and documented somewhere? Does LLVM's auto-FDO support non-Intel CPUs these days? Honza > > David > > > > > >> Honza > >> > > >> > David > >> > > >> > > >> > > Honza > >> > > > > >> > > > David > >> > > > > >> > > > > > >> > > > > Thoughts? > >> > > > > Martin > >> > > > > > >> > > > > > > >> > > > > > Having the tool third-party makes keeping the whole chain > >> working > >> > > more > >> > > > > > difficult. > >> > > > > > > >> > > > > > Richard. > >> > > > > > > >> > > > > >> Thanks, > >> > > > > >> > >> > > > > >> David > >> > > > > >> > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka > >> wrote: > >> > > > > >> > >> > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > >> > > > > > GCC documentation for AutoFDO points to create_gcov tool > >> that > >> > > > > converts > >> > > > > >>> perf.data file into gcov format that can be consumed by gcc > >> with > >> > > > > >>> -fauto-profile ( > >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > >> > > > > > > >> > > > > > I noticed that the source code for create_gcov has been > >> deleted > >> > > from > >> > > > > >>> https://github.com/google/autofdo on April 7. I asked about > >> that > >> > > > > change > >> > > > > >>> in that repo and got the following reply: > >> > > > > > > >> > > > > > > >> > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > >> > > > > > > >> > > > > > "Actually we didn't use create_gcov and havn't updated > >> > > create_gcov > >> > > > > for > >> > > > > >>> years, and we also didn't have enough tests to guarantee it > >> works > >> > > (It > >> > > > > was > >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If you need > >> it, it > >> > > is > >> > > > > >>> welcomed to update create_gcov and add it to the respository." > >> > > > > > > >> > > > > > Does this mean that AutoFDO is currently dead in gcc? > >> > > > > > >> > > > > Hello. > >> > > > > > >> > > > > Yes. I know that even basic test cases have been broken for > >> years > >> > > in > >> > > > > the > >> > > > > >>> GCC. > >> > > > > It's new to me that create_gcov was removed. > >> > > > > > >> > > > > I tend to send patch to GCC that will remove AutoFDO from > >> GCC. > >> > > > > I known Bin spent some time working on AutoFDO, has he came > >> up to > >> > > > > >>> something? > >> > > > > >>> > >> > > > > >>> The GCC side of auto-FDO is not that hard. We have most of > >> > > > > >>> infrastructure in place, but stopping point for me was always > >> > > > > difficulty > >> > > > > >>> to get gcov-tool working. If some maintainer steps up, I > >> think I > >> > > can > >> > > > > >>> fix GCC side. > >> > > > > >>> > >> > > > > >>> I am bit unsure how important feature it is - we have FDO that > >> > > works > >> > > > > >>> quite well for most users but I know there are some users of > >> the > >> > > LLVM > >> > > > > >>> implementation and there is potential to tie this with other > >> > > hardware > >> > > > > >>> events to asist i.e. if conversion (where one wants to know > >> how > >> > > well > >> > > > > CPU > >> > > > > >>> predicts the jump rather than just the jump probability) > >> which I > >> > > always > >> > > > > >>> found potentially interesting. > >> > > > > >>> > >> > > > > >>> Honza > >> > > > > > >> > > > > Martin > >> > > > > > >> > > > > > > >> > > > > > Thanks, > >> > > > > > > >> > > > > > Eugene > >> > > > > > > >> > > > > > >> > > > > >>> > >> > > > > > >> > > > > > >> > > > >> > >
"musttail" statement attribute for GCC?
Would it be feasible to implement a "musttail" statement attribute in GCC to get a guarantee that tail call optimization will be performed? Such an attribute has just landed in the trunk of Clang (https://reviews.llvm.org/D99517). It makes it possible to write algorithms that use arbitrarily long chains of tail calls without risk of blowing the stack. I would love to see something like this land in GCC also (ultimately I'd love to see it standardized). I wrote more about some motivation for guaranteed tail calls here: https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html GCC successfully optimizes tail calls in many cases already. What would it take to provide an actual guarantee, and make it apply to non-optimized builds also? The Clang implementation enforces several rules that must hold for the attribute to be correct, including: - It must be on a function call that is tail position. - Caller and callee must have compatible function signatures (including the implicit "this", if any), differing only in cv qualifiers. - Caller and callee must use the same calling convention. - Caller and callee may not be constructors or destructors. - All arguments, the return type, and any temporaries created must be trivially destructible. - All variables currently in scope must be trivially destructible. - The caller cannot be in a try block, an Objective-C block, or a statement expression. Some of these restrictions may be overly strict for some calling conventions, but they are useful as the "least common denominator" that should be safe in all cases. When implementing this in Clang, we found that we were able to reuse some of the checks around goto and asm goto, since a tail call is sort of like a goto out of the function's scope. Thanks, Josh
Re: origin/trunk branch - who added it?
On Fri, Apr 23, 2021 at 5:13 AM Martin Liška wrote: > > Few weeks ago, I noticed we have a new remote branch that follows > origin/master: > https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/trunk > > Do we know who added it? And what was the motivation? I did; I've always preferred that name (for metaphorical consistency with 'branch'), and there's been a wider move away from 'master' in git repositories. I figured we could support both easily enough, but the hooks still need updating to treat 'trunk' the same way they treat 'master'. Jason
Re: State of AutoFDO in GCC
On Fri, Apr 23, 2021 at 12:28 PM Jan Hubicka wrote: > > On Fri, Apr 23, 2021 at 10:27 AM Xinliang David Li > > wrote: > > > > > > > > > > > On Fri, Apr 23, 2021 at 10:16 AM Jan Hubicka wrote: > > > > > >> > > > >> > It uses create_llvm_prof tool which is in the same git repo. The > data > > >> > parsing part is shared with create_gcov, but the writer is obviously > > >> > different for the two tools. > > >> > > >> OK and what are the main differences between llvmand gcc format? > > >> > > >> GCC expects GCOV format, I think while LLVM uses a newly designed > binary > > > format. > > > > > > > > Sorry I missed your next reply. I forgot about the details of GCC' > format. > > Thanks for explanation. How hard would be to make GCC consume this > format? Is is reasonably stable and documented somewhere? > The text format is documented here: https://clang.llvm.org/docs/UsersManual.html The binary format is not documented. The binary format is not guaranteed to be backward compatible, so sharing the same format may not be the best way as changes for clang may break GCC. Since linux perf format does not change, the tool should be relatively stable with low maintenance cost. Changes are needed only when some new AutoFDO features are added to the compiler side. Does LLVM's auto-FDO support non-Intel CPUs these days? > It supports LBR like events, so it is CPU vendor dependent. For ARM, using ETM can achieve the goal, but I don't have detailed knowledge of it. David > > Honza > > > > David > > > > > > > > >> Honza > > >> > > > >> > David > > >> > > > >> > > > >> > > Honza > > >> > > > > > >> > > > David > > >> > > > > > >> > > > > > > >> > > > > Thoughts? > > >> > > > > Martin > > >> > > > > > > >> > > > > > > > >> > > > > > Having the tool third-party makes keeping the whole chain > > >> working > > >> > > more > > >> > > > > > difficult. > > >> > > > > > > > >> > > > > > Richard. > > >> > > > > > > > >> > > > > >> Thanks, > > >> > > > > >> > > >> > > > > >> David > > >> > > > > >> > > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka < > hubi...@ucw.cz> > > >> wrote: > > >> > > > > >> > > >> > > > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > >> > > > > > GCC documentation for AutoFDO points to create_gcov tool > > >> that > > >> > > > > converts > > >> > > > > >>> perf.data file into gcov format that can be consumed by > gcc > > >> with > > >> > > > > >>> -fauto-profile ( > > >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > >> > > > > > > > >> > > > > > I noticed that the source code for create_gcov has been > > >> deleted > > >> > > from > > >> > > > > >>> https://github.com/google/autofdo on April 7. I asked > about > > >> that > > >> > > > > change > > >> > > > > >>> in that repo and got the following reply: > > >> > > > > > > > >> > > > > > > > >> > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > >> > > > > > > > >> > > > > > "Actually we didn't use create_gcov and havn't updated > > >> > > create_gcov > > >> > > > > for > > >> > > > > >>> years, and we also didn't have enough tests to guarantee > it > > >> works > > >> > > (It > > >> > > > > was > > >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If you > need > > >> it, it > > >> > > is > > >> > > > > >>> welcomed to update create_gcov and add it to the > respository." > > >> > > > > > > > >> > > > > > Does this mean that AutoFDO is currently dead in gcc? > > >> > > > > > > >> > > > > Hello. > > >> > > > > > > >> > > > > Yes. I know that even basic test cases have been broken > for > > >> years > > >> > > in > > >> > > > > the > > >> > > > > >>> GCC. > > >> > > > > It's new to me that create_gcov was removed. > > >> > > > > > > >> > > > > I tend to send patch to GCC that will remove AutoFDO from > > >> GCC. > > >> > > > > I known Bin spent some time working on AutoFDO, has he > came > > >> up to > > >> > > > > >>> something? > > >> > > > > >>> > > >> > > > > >>> The GCC side of auto-FDO is not that hard. We have most > of > > >> > > > > >>> infrastructure in place, but stopping point for me was > always > > >> > > > > difficulty > > >> > > > > >>> to get gcov-tool working. If some maintainer steps up, I > > >> think I > > >> > > can > > >> > > > > >>> fix GCC side. > > >> > > > > >>> > > >> > > > > >>> I am bit unsure how important feature it is - we have FDO > that > > >> > > works > > >> > > > > >>> quite well for most users but I know there are some users > of > > >> the > > >> > > LLVM > > >> > > > > >>> implementation and there is potential to tie this with > other > > >> > > hardware > > >> > > > > >>> events to asist i.e. if conversion (where one wants to > know > > >> how > > >> > > well > > >> > > > > CPU > > >> > > > > >>> predicts the jump rather than just the jump probability) > > >> which I > > >> > > always
Re: "musttail" statement attribute for GCC?
On Fri, 2021-04-23 at 12:44 -0700, Josh Haberman via Gcc wrote: > Would it be feasible to implement a "musttail" statement attribute in > GCC to get a guarantee that tail call optimization will be performed? > > Such an attribute has just landed in the trunk of Clang > (https://reviews.llvm.org/D99517). It makes it possible to write > algorithms that use arbitrarily long chains of tail calls without risk > of blowing the stack. I would love to see something like this land in > GCC also (ultimately I'd love to see it standardized). FWIW I implemented something like this in GCC's middle-end here: https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=9a385c2d3d74ffed78f2ed9ad47b844d2f294105 exposing it in API form for libgccjit: https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=15c671a79ca66df5b1de70dd1a0b78414fe003ef https://gcc.gnu.org/onlinedocs/jit/topics/expressions.html#c.gcc_jit_rvalue_set_bool_require_tail_call but IIRC it's not yet exposed to the regular GCC frontends. Dave > > I wrote more about some motivation for guaranteed tail calls here: > > https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html > > GCC successfully optimizes tail calls in many cases already. What > would it take to provide an actual guarantee, and make it apply to > non-optimized builds also? > The Clang implementation enforces several rules that must hold for the > attribute to be correct, including: > > - It must be on a function call that is tail position. > - Caller and callee must have compatible function signatures > (including the implicit "this", if any), differing only in cv > qualifiers. > - Caller and callee must use the same calling convention. > - Caller and callee may not be constructors or destructors. > - All arguments, the return type, and any temporaries created must be > trivially destructible. > - All variables currently in scope must be trivially destructible. > - The caller cannot be in a try block, an Objective-C block, or a > statement expression. > > Some of these restrictions may be overly strict for some calling > conventions, but they are useful as the "least common denominator" > that should be safe in all cases. When implementing this in Clang, we > found that we were able to reuse some of the checks around goto and > asm goto, since a tail call is sort of like a goto out of the > function's scope. > > Thanks, > Josh
Re: "musttail" statement attribute for GCC?
David Malcolm via Gcc wrote: On Fri, 2021-04-23 at 12:44 -0700, Josh Haberman via Gcc wrote: Would it be feasible to implement a "musttail" statement attribute in GCC to get a guarantee that tail call optimization will be performed? Such an attribute has just landed in the trunk of Clang (https://reviews.llvm.org/D99517). It makes it possible to write algorithms that use arbitrarily long chains of tail calls without risk of blowing the stack. I would love to see something like this land in GCC also (ultimately I'd love to see it standardized). FWIW I implemented something like this in GCC's middle-end here: https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=9a385c2d3d74ffed78f2ed9ad47b844d2f294105 exposing it in API form for libgccjit: https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=15c671a79ca66df5b1de70dd1a0b78414fe003ef https://gcc.gnu.org/onlinedocs/jit/topics/expressions.html#c.gcc_jit_rvalue_set_bool_require_tail_call but IIRC it's not yet exposed to the regular GCC frontends. I did try to use it this ^ for GCC coroutines (where such a guarantee is pretty important) However, the issue there is that not all targets support indirect tailcalls. What about tailcalls between DSOs? Are those also excluded in the clang impl? Iain Dave I wrote more about some motivation for guaranteed tail calls here: https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html GCC successfully optimizes tail calls in many cases already. What would it take to provide an actual guarantee, and make it apply to non-optimized builds also? The Clang implementation enforces several rules that must hold for the attribute to be correct, including: - It must be on a function call that is tail position. - Caller and callee must have compatible function signatures (including the implicit "this", if any), differing only in cv qualifiers. - Caller and callee must use the same calling convention. - Caller and callee may not be constructors or destructors. - All arguments, the return type, and any temporaries created must be trivially destructible. - All variables currently in scope must be trivially destructible. - The caller cannot be in a try block, an Objective-C block, or a statement expression. Some of these restrictions may be overly strict for some calling conventions, but they are useful as the "least common denominator" that should be safe in all cases. When implementing this in Clang, we found that we were able to reuse some of the checks around goto and asm goto, since a tail call is sort of like a goto out of the function's scope. Thanks, Josh
Re: "musttail" statement attribute for GCC?
David Malcolm via Gcc wrote: > FWIW I implemented something like this in GCC's middle-end here: > https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=9a385c2d3d74ffed78f2ed9ad47b844d2f294105 > exposing it in API form for libgccjit: > https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=15c671a79ca66df5b1de70dd1a0b78414fe003ef > https://gcc.gnu.org/onlinedocs/jit/topics/expressions.html#c.gcc_jit_rvalue_set_bool_require_tail_call > but IIRC it's not yet exposed to the regular GCC frontends. That's great to hear that there is some existing infrastructure around this. Your code includes some checks that we should consider adding to Clang (especially the check against "returns twice"). Why the prohibition against noreturn though? I've found that tail calls to noreturn functions are useful for longjmp()-based error handling, and musttail could be a useful workaround for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=10837#c2. On Fri, Apr 23, 2021 at 1:10 PM Iain Sandoe wrote: > I did try to use it this ^ for GCC coroutines (where such a guarantee is > pretty important) > > However, the issue there is that not all targets support indirect tailcalls. I'm not familiar with this limitation. What targets do not support indirect tail calls? > What about tailcalls between DSOs? What issues are there around DSOs? Clang/LLVM don't treat these specially AFAIK, and it seems that tail calls through a PLT should work okay? Thanks, Josh
Re: GCC 8.5 Status Report (2021-04-23)
On Fri, Apr 23, 2021 at 7:55 AM Jakub Jelinek wrote: > > Status > == > > GCC 8.5 release and closing of the 8 branch is several months overdue, > we don't have enough time to maintain trunk and 4 supported release branches. > Therefore, I'd like to do 8.5-rc1 on 7th of May and release 8.5 and close > the branch a week after that. > We have one P1 bug though - P98358 - it is unclear if just changing the > testcase is all that is needed or if some C++ FE changes are needed. The error message quoted in the PR looks correct, so I think changing the testcase is sufficient. > If anyone has anything to backport to 8.5, please do so in the upcoming 2 > weeks. Thanks. > > > Quality Data > > > Priority # Change from last report > --- --- > P11 > P2 287 + 28 > P3 32 - 2 > P4 152 - 3 > P5 21 - 1 > --- --- > Total P1-P3 320 + 26 > Total 493 + 22 > > > Previous Report > === > > https://gcc.gnu.org/legacy-ml/gcc/2020-03/msg00043.html >
gcc-9-20210423 is now available
Snapshot gcc-9-20210423 is now available on https://gcc.gnu.org/pub/gcc/snapshots/9-20210423/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 9 git branch with the following options: git://gcc.gnu.org/git/gcc.git branch releases/gcc-9 revision bbceb17103e965c1fc3f7fdbfc5182696acd9ea9 You'll find: gcc-9-20210423.tar.xzComplete GCC SHA256=e364ff2b12ee82f139ee014fe2f9099c6c2d51f4848f0af96845ede4d667b475 SHA1=e8fc787b832cb5f95a4e26beb0456794710f7763 Diffs from 9-20210416 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-9 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
Re: "musttail" statement attribute for GCC?
On Fri, Apr 23, 2021 at 2:45 PM Josh Haberman via Gcc wrote: > > Would it be feasible to implement a "musttail" statement attribute in > GCC to get a guarantee that tail call optimization will be performed? > > Such an attribute has just landed in the trunk of Clang > (https://reviews.llvm.org/D99517). It makes it possible to write > algorithms that use arbitrarily long chains of tail calls without risk > of blowing the stack. I would love to see something like this land in > GCC also (ultimately I'd love to see it standardized). > > I wrote more about some motivation for guaranteed tail calls here: > https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html So I was looking into the use case presented in that web page, it is the same reason why GCC has the computed gotos extension already. Is there a reason why you can't just use that instead? Thanks, Andrew Pinski > > GCC successfully optimizes tail calls in many cases already. What > would it take to provide an actual guarantee, and make it apply to > non-optimized builds also? > > The Clang implementation enforces several rules that must hold for the > attribute to be correct, including: > > - It must be on a function call that is tail position. > - Caller and callee must have compatible function signatures > (including the implicit "this", if any), differing only in cv > qualifiers. > - Caller and callee must use the same calling convention. > - Caller and callee may not be constructors or destructors. > - All arguments, the return type, and any temporaries created must be > trivially destructible. > - All variables currently in scope must be trivially destructible. > - The caller cannot be in a try block, an Objective-C block, or a > statement expression. > > Some of these restrictions may be overly strict for some calling > conventions, but they are useful as the "least common denominator" > that should be safe in all cases. When implementing this in Clang, we > found that we were able to reuse some of the checks around goto and > asm goto, since a tail call is sort of like a goto out of the > function's scope. > > Thanks, > Josh
Re: "musttail" statement attribute for GCC?
On Fri, Apr 23, 2021 at 4:34 PM Andrew Pinski wrote: > > On Fri, Apr 23, 2021 at 2:45 PM Josh Haberman via Gcc wrote: > > > > I wrote more about some motivation for guaranteed tail calls here: > > https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html > > So I was looking into the use case presented in that web page, it is > the same reason why GCC has the computed gotos extension already. > Is there a reason why you can't just use that instead? Computed gotos solve some of the problems associated with interpreter main loops, but not others. Computed gotos help avoid a bounds check on the switch(). They can sometimes help ensure that the dispatch jmp is replicated at the end of each case, thus giving each case a unique branch prediction address, though this effect is somewhat fragile since the compiler will sometimes decide to merge these paths into one: https://godbolt.org/z/T9a6173WP Computed gotos cannot help with the two main issues mentioned in the article: 1. The larger a function is, and the more complex and connected its control flow, the harder it is for the compiler’s register allocator to keep the most important data in registers. 2. When fast paths and slow paths are intermixed in the same function, the presence of the slow paths compromises the code quality of the fast paths. Tail calls can help solve both of these problems: 1. If we break down the big interpreter function into many small functions, one per operation, we get control over register allocation at each function boundary. As long as each function is simple enough not to spill, this data will be kept in registers continuously through all the fast paths. 2. If we put fast paths and slow paths in separate functions entirely, then the slow paths cannot negatively affect the code generation of the fast paths. We spent a lot of time trying to work within the framework of computed gotos but were unable to get the results we wanted. Tail calls generated substantially better code than anything else we tried. There are also some practical reasons why tail calls are better for us, particularly that they make it easier to reuse the code across different "instantiations" of the interpreter. For the case of protobuf parsing, each protobuf message has a distinct set of fields with different field numbers and types. With tail calls we can create a distinct dispatch table for each message type, but the function pointers in these tables can point to a set of functions that are shared across all message types. Thanks, Josh
Re: Second GCC 11.1 Release Candidate available from gcc.gnu.org
On 4/23/21 8:31 AM, Jakub Jelinek via Gcc wrote: Some blocker bugs were reported against the first release candidate of GCC 11.1, so there is a second release candidate for GCC 11.1 available from https://gcc.gnu.org/pub/gcc/snapshots/11.1.0-RC-20210423/ ftp://gcc.gnu.org/pub/gcc/snapshots/11.1.0-RC-20210423 and shortly its mirrors. It has been generated from git revision r11-8287-g7a7fc01b9d20afb1a2b805d93cb838a38e656420. I have so far bootstrapped and tested the release candidate on x86_64-linux and i686-linux. Please test it and report any issues to bugzilla. If all goes well, I'd still like to release 11.1 on Tuesday, April 27th. I bootstrapped and tested this on powerpc64 on power 7, 8, 9, and 10 systems and LE and BE as appropriate. I saw nothing unexpected.
