History of GCC

[Will Hawkins]

Hello everyone!

My name is Will Hawkins and I am a longtime user of gcc and admirer of
the project. I hope that this is the proper forum for the question I
am going to ask. If it isn't, please accept my apology and ignore me.

I am a real geek and I love the history behind open source projects.
I've found several good resources about the history of "famous" open
source projects and organizations (including, but definitely not
limited to, the very interesting Free as in Freedom 2.0).

Unfortunately there does not appear to be a good history of the
awesome and fundamental GCC project. I know that there is a page on
the wiki (https://gcc.gnu.org/wiki/History) but that is really the
best that I can find.

Am I missing something? Are there good anecdotes about the history of
the development of GCC that you think I might find interesting? Any
pointers would be really great!

Thanks for taking the time to read my questions. Thanks in advance for
any information that you have to offer. I really appreciate everyone's
effort to make such a great compiler suite. It's only with such a
great compiler that all our other open source projects are able to
succeed!

Thank you!
Will

Re: History of GCC

[Will Hawkins]

On Wed, Oct 26, 2016 at 9:07 AM, Ian Lance Taylor  wrote:
> On Tue, Oct 25, 2016 at 10:53 PM, Will Hawkins  wrote:
>>
>> My name is Will Hawkins and I am a longtime user of gcc and admirer of
>> the project. I hope that this is the proper forum for the question I
>> am going to ask. If it isn't, please accept my apology and ignore me.
>>
>> I am a real geek and I love the history behind open source projects.
>> I've found several good resources about the history of "famous" open
>> source projects and organizations (including, but definitely not
>> limited to, the very interesting Free as in Freedom 2.0).
>>
>> Unfortunately there does not appear to be a good history of the
>> awesome and fundamental GCC project. I know that there is a page on
>> the wiki (https://gcc.gnu.org/wiki/History) but that is really the
>> best that I can find.
>>
>> Am I missing something? Are there good anecdotes about the history of
>> the development of GCC that you think I might find interesting? Any
>> pointers would be really great!
>>
>> Thanks for taking the time to read my questions. Thanks in advance for
>> any information that you have to offer. I really appreciate everyone's
>> effort to make such a great compiler suite. It's only with such a
>> great compiler that all our other open source projects are able to
>> succeed!
>
> There is some history and links at
> https://en.wikipedia.org/wiki/GNU_Compiler_Collection .
>
> In my opinion, the history of GCC is not really one of drama or even
> anecdotes, except for the EGCS split.  There are plenty of people who
> work on GCC out of personal interest, but for decades now the majority
> of work on GCC has been by people paid to work on it.  I expect that
> the result is less interesting as history and more interesting as
> software.
>
> Ian

Ian,

Thank you for your response! I don't think that there has to be
controversy to be interesting. Obviously that split/reunification was
important, but I think that there might even be some value in
documenting the minutia of the project's growth. In other words, what
was the process for incorporating each new version of the C++
standard? Who and why did GCC start a frontend for X language? Things
like that.

Thanks again for your response!

Will

Re: History of GCC

[Will Hawkins]

On Wed, Oct 26, 2016 at 11:55 AM, Jeff Law  wrote:
> On 10/26/2016 07:07 AM, Ian Lance Taylor wrote:
>>
>> On Tue, Oct 25, 2016 at 10:53 PM, Will Hawkins  wrote:
>>>
>>>
>>> My name is Will Hawkins and I am a longtime user of gcc and admirer of
>>> the project. I hope that this is the proper forum for the question I
>>> am going to ask. If it isn't, please accept my apology and ignore me.
>>>
>>> I am a real geek and I love the history behind open source projects.
>>> I've found several good resources about the history of "famous" open
>>> source projects and organizations (including, but definitely not
>>> limited to, the very interesting Free as in Freedom 2.0).
>>>
>>> Unfortunately there does not appear to be a good history of the
>>> awesome and fundamental GCC project. I know that there is a page on
>>> the wiki (https://gcc.gnu.org/wiki/History) but that is really the
>>> best that I can find.
>>>
>>> Am I missing something? Are there good anecdotes about the history of
>>> the development of GCC that you think I might find interesting? Any
>>> pointers would be really great!
>>>
>>> Thanks for taking the time to read my questions. Thanks in advance for
>>> any information that you have to offer. I really appreciate everyone's
>>> effort to make such a great compiler suite. It's only with such a
>>> great compiler that all our other open source projects are able to
>>> succeed!
>>
>>
>> There is some history and links at
>> https://en.wikipedia.org/wiki/GNU_Compiler_Collection .
>>
>> In my opinion, the history of GCC is not really one of drama or even
>> anecdotes, except for the EGCS split.  There are plenty of people who
>> work on GCC out of personal interest, but for decades now the majority
>> of work on GCC has been by people paid to work on it.  I expect that
>> the result is less interesting as history and more interesting as
>> software.
>
> Agreed.  Speaking for myself, I got interested in GCC to solve a problem,
> then another, then another...  Hacking GCC made for an interesting hobby for
> a few years.  I never imagined it would turn into a career, but 20 years
> later, here I am.  I wouldn't be surprised if others have followed a similar
> path.
>
> jeff

Thank you Ian, Joel and Jeff for your responses!

I really appreciate it. As I said in my first message, I am a real
geek and I certainly did not mean to imply that I thought many people
would find the history interesting. That said, I think that there
might be some people who do find it so.

As an example of the type of software history that I find interesting,
consider these:

https://www.youtube.com/watch?v=TMjgShRuYbg
or
https://www.youtube.com/watch?v=2kEJoWfobpA
or
https://www.usenix.org/system/files/login/articles/03_lu_010-017_final.pdf
or as a final example
https://www.youtube.com/watch?v=69edOm889V4

To answer the question you are probably asking, "No, I have no idea
why I enjoy this type of stuff as much as I do!"

Can any of you recall a turning point where development went from
being driven by hobbyists to being driven by career developers? As a
result of that shift, has there been a change in the project's
priorities? Have there been conflicts between the employer's interests
and those of the project (in terms of project goals, licensing issues,
code quality, etc)?

In any event, I really appreciate your answers. If you have any
information that you think I might find interesting, please feel free
to pass it along. Thanks again!

Will

Re: History of GCC

[Will Hawkins]

On Wed, Oct 26, 2016 at 1:06 PM, Jakub Jelinek  wrote:
> On Wed, Oct 26, 2016 at 06:57:31PM +0200, Marek Polacek wrote:
>> I think you can learn a lot if you follow the Changes pages, so e.g.
>> , and go back down the history until
>> you reach the ancient .
>
> Even older releases, while they don't have changes.html, have changes/news
> etc. written in the pages referenced from
> https://gcc.gnu.org/releases.html#timeline
> Also see https://gcc.gnu.org/develop.html#timeline
> For questions like who has added feature XYZ, the best source is just the
> source repository's history or ChangeLog files.
>
> Jakub

Jakub, Marek,

Great suggestions! In fact, I had just thought of the same thing!

Thanks for your response!
Will

Re: History of GCC

[Will Hawkins]

On Wed, Oct 26, 2016 at 1:15 PM, Ian Lance Taylor  wrote:
> On Wed, Oct 26, 2016 at 9:31 AM, Will Hawkins  wrote:
>>
>> Thank you for your response! I don't think that there has to be
>> controversy to be interesting. Obviously that split/reunification was
>> important, but I think that there might even be some value in
>> documenting the minutia of the project's growth. In other words, what
>> was the process for incorporating each new version of the C++
>> standard? Who and why did GCC start a frontend for X language? Things
>> like that.
>
> It is easier to answer specific questions.
>
> There have always been GCC developers that have tracked the evolution
> of C++.  The first C++ standard was of course in 1998, at which point
> the language was over 10 years old, so there were a lot of C++
> language changes before then.  GCC has generally acquired new language
> features as they were being adopted into the standard, usually
> controlled by options like the current -std=c++1z.  This of course
> means that the new features have shifted as the standard has shifted,
> but as far as I know that hasn't happened too often.
>
> GCC started as a C compiler.  The C++ frontend was started by Michael
> Tiemann around 1987 or so.  It started as a patch and was later
> incorporated into the mainline.
>
> The Objective C frontend was started at NeXT.  They originally
> intended to keep it proprietary, but when they understood that the GPL
> made that impossible they contributed it back.  I forget when the
> Objective C++ frontend came in.
>
> Cygnus Support developed the Chill and, later, Java frontends.  The
> Chill frontend was removed later, and in fact the Java frontend was
> removed just recently.
>
> As I recall Fortran was a hobbyist project that eventually made it in.
> There were two competing forks, I think.  I don't remember too much
> about that off the top of my head.
>
> The Ada frontend was developed at AdaCore.
>
> The Go frontend was written by me, mostly because I like Go and I've
> been working on GCC for a long time.  I work at Google, and Go was
> developed at Google, but there wouldn't be a GCC Go frontend if I
> hadn't decided to write one.
>
> There is a Modula frontend that is always close to getting in.  I
> think there is a Pascal frontend out there too, somewhere.  And a D
> frontend.
>
> Ian

Wow, thanks Ian! This is awesome stuff! As I read through it, I may
have some additional questions. If I do, would you mind if I emailed
you directly? Thanks again for taking the time to write all this down!
Fascinating!

Will

Re: History of GCC

[Will Hawkins]

On Wed, Oct 26, 2016 at 1:28 PM, Richard Kenner
 wrote:
>> The Ada frontend was developed at AdaCore.
>
> The Ada frontend was developed at NYU, as an Air Force-funded project
> to show that Ada95 (then called Ada9X) was implementable.  AdaCore was
> later formed once that was complete to provide commercial support for
> the Ada compiler.  The members of that NYU project were the initial
> team at AdaCore.

Such great information, Richard! Thanks so much!
Will

Re: History of GCC

[Will Hawkins]

On Wed, Oct 26, 2016 at 2:23 PM, Eric Gallager  wrote:
> On 10/26/16, Ian Lance Taylor  wrote:
>> On Wed, Oct 26, 2016 at 9:31 AM, Will Hawkins  wrote:
>>>
>>> Thank you for your response! I don't think that there has to be
>>> controversy to be interesting. Obviously that split/reunification was
>>> important, but I think that there might even be some value in
>>> documenting the minutia of the project's growth. In other words, what
>>> was the process for incorporating each new version of the C++
>>> standard? Who and why did GCC start a frontend for X language? Things
>>> like that.
>>
>> It is easier to answer specific questions.
>>
>> There have always been GCC developers that have tracked the evolution
>> of C++.  The first C++ standard was of course in 1998, at which point
>> the language was over 10 years old, so there were a lot of C++
>> language changes before then.  GCC has generally acquired new language
>> features as they were being adopted into the standard, usually
>> controlled by options like the current -std=c++1z.  This of course
>> means that the new features have shifted as the standard has shifted,
>> but as far as I know that hasn't happened too often.
>>
>> GCC started as a C compiler.  The C++ frontend was started by Michael
>> Tiemann around 1987 or so.  It started as a patch and was later
>> incorporated into the mainline.
>>
>> The Objective C frontend was started at NeXT.  They originally
>> intended to keep it proprietary, but when they understood that the GPL
>> made that impossible they contributed it back.  I forget when the
>> Objective C++ frontend came in.
>
>
> The Objective C++ frontend was contribute by Apple. The earliest
> proposal I can find for adding it was in 2001 for GCC 3.x:
> https://gcc.gnu.org/ml/gcc/2001-11/msg00609.html
> However, it didn't actually make it in to the FSF version until 4.1:
> https://gcc.gnu.org/ml/gcc-patches/2005-05/msg01781.html
> https://gcc.gnu.org/ml/gcc-patches/2005-12/msg01812.html
> Personally, I think one of the interesting stories of GCC history is
> how Apple used to be really involved in GCC development until 2007, at
> which point the GPL3 and iPhone came out, and Apple abandoned GCC for
> llvm/clang. If you read through the mailing list archives on
> gcc.gnu.org, you can find all sorts of emails from people with "at
> apple dot com" email addresses in the early 2000s, until they just
> sort of stopped later that decade. Even llvm/clang was originally just
> another branch of gcc, and Chris Lattner was even going to contribute
> it and keep it part of gcc, but then he never got around to getting
> his copyright assignment paperwork filed, and then Apple turned it
> into a separate project:
> https://gcc.gnu.org/ml/gcc/2005-11/msg00888.html
> https://gcc.gnu.org/ml/gcc/2006-03/msg00706.html
>
>
>>
>> Cygnus Support developed the Chill and, later, Java frontends.  The
>> Chill frontend was removed later, and in fact the Java frontend was
>> removed just recently.
>>
>> As I recall Fortran was a hobbyist project that eventually made it in.
>> There were two competing forks, I think.  I don't remember too much
>> about that off the top of my head.
>>
>> The Ada frontend was developed at AdaCore.
>>
>> The Go frontend was written by me, mostly because I like Go and I've
>> been working on GCC for a long time.  I work at Google, and Go was
>> developed at Google, but there wouldn't be a GCC Go frontend if I
>> hadn't decided to write one.
>>
>> There is a Modula frontend that is always close to getting in.  I
>> think there is a Pascal frontend out there too, somewhere.  And a D
>> frontend.
>>
>> Ian
>>

I want to thank each individual for his/her reply, but I don't want to
SPAM the list. So, I will do it in one email!

Thanks! This is so much more information than I expected to get and
it's just amazing. Thanks again!

Will

Basic Block Statistics

[Will Hawkins]

Hello everyone!

I apologize if this is not the right venue to ask this question and/or
this is a waste of your time.

I was just wondering if there are statistics that gcc can emit that
includes either a) the average number of instructions per basic block
and/or b) the average size (in bytes) per basic block in a compilation
unit.

If nothing like this exists, I am more than happy to code something up
if people besides me think that it might be interesting.

I promise that I googled for information before asking, but I can't
guarantee that I didn't miss anything. Again, I apologize if I just
needed to RTFM better.

Thanks in advance for any responses!
Will

Re: Basic Block Statistics

[Will Hawkins]

On Tue, May 16, 2017 at 2:33 PM, Jeff Law  wrote:
> On 05/16/2017 12:24 PM, Will Hawkins wrote:
>> Hello everyone!
>>
>> I apologize if this is not the right venue to ask this question and/or
>> this is a waste of your time.
>>
>> I was just wondering if there are statistics that gcc can emit that
>> includes either a) the average number of instructions per basic block
>> and/or b) the average size (in bytes) per basic block in a compilation
>> unit.
>>
>> If nothing like this exists, I am more than happy to code something up
>> if people besides me think that it might be interesting.
>>
>> I promise that I googled for information before asking, but I can't
>> guarantee that I didn't miss anything. Again, I apologize if I just
>> needed to RTFM better.
> I don't think we have anything which inherently will give you this
> information.
>
> It'd be a useful thing to have though.  Implementation may be made more
> difficult by insns that generate > 1 instruction.
>
> Jeff

Thank you, Mr. Law. I think that this is something I'd really like to
work on. As I start to take a peak into how hard/easy this is to
implement, I may circle back and ask some additional technical
questions.

Thanks for your quick response!
Will

Re: Basic Block Statistics

[Will Hawkins]

On Tue, May 16, 2017 at 2:45 PM, David Malcolm  wrote:
> On Tue, 2017-05-16 at 14:24 -0400, Will Hawkins wrote:
>> Hello everyone!
>>
>> I apologize if this is not the right venue to ask this question
>> and/or
>> this is a waste of your time.
>>
>> I was just wondering if there are statistics that gcc can emit that
>> includes either a) the average number of instructions per basic block
>> and/or b) the average size (in bytes) per basic block in a
>> compilation
>> unit.
>>
>> If nothing like this exists, I am more than happy to code something
>> up
>> if people besides me think that it might be interesting.
>>
>> I promise that I googled for information before asking, but I can't
>> guarantee that I didn't miss anything. Again, I apologize if I just
>> needed to RTFM better.
>>
>> Thanks in advance for any responses!
>> Will
>
> I don't think anything like this currently exists, but it's probably
> doable via a plugin, e.g. by hooking up a new RTL pass somewhere
> towards the end of the pass pipeline.
>
> That said, IIRC basic blocks aren't used in the final passes;
> presumably they're not meaningful after the "free_cfg" pass.
>
> Hope this is helpful

Very helpful, thank you Mr. Malcolm!

Will

> Dave

Re: Basic Block Statistics

[Will Hawkins]

As I started looking into this, it seems like PLUGIN_FINISH is where
my plugin will go. Everything is great so far. However, when plugins
at that event are invoked, they get no data. That means I will have to
look into global structures for information regarding the compilation.
Are there pointers to the documentation that describe the relevant
global data structures that are accessible at this point?

I am looking through the source code and documentation and can't find
what I am looking for. I am happy to continue working, but thought I'd
ask just in case I was missing something silly.

Thanks again for all your help getting me started on this!
Will

On Tue, May 16, 2017 at 2:54 PM, Jeff Law  wrote:
> On 05/16/2017 12:37 PM, Will Hawkins wrote:
>> On Tue, May 16, 2017 at 2:33 PM, Jeff Law  wrote:
>>> On 05/16/2017 12:24 PM, Will Hawkins wrote:
>>>> Hello everyone!
>>>>
>>>> I apologize if this is not the right venue to ask this question and/or
>>>> this is a waste of your time.
>>>>
>>>> I was just wondering if there are statistics that gcc can emit that
>>>> includes either a) the average number of instructions per basic block
>>>> and/or b) the average size (in bytes) per basic block in a compilation
>>>> unit.
>>>>
>>>> If nothing like this exists, I am more than happy to code something up
>>>> if people besides me think that it might be interesting.
>>>>
>>>> I promise that I googled for information before asking, but I can't
>>>> guarantee that I didn't miss anything. Again, I apologize if I just
>>>> needed to RTFM better.
>>> I don't think we have anything which inherently will give you this
>>> information.
>>>
>>> It'd be a useful thing to have though.  Implementation may be made more
>>> difficult by insns that generate > 1 instruction.
>>>
>>> Jeff
>>
>> Thank you, Mr. Law. I think that this is something I'd really like to
>> work on. As I start to take a peak into how hard/easy this is to
>> implement, I may circle back and ask some additional technical
>> questions.
> Sure.  On-list is best.
> Jeff

Re: Basic Block Statistics

[Will Hawkins]

On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>> As I started looking into this, it seems like PLUGIN_FINISH is where
>> my plugin will go. Everything is great so far. However, when plugins
>> at that event are invoked, they get no data. That means I will have to
>> look into global structures for information regarding the compilation.
>> Are there pointers to the documentation that describe the relevant
>> global data structures that are accessible at this point?
>>
>> I am looking through the source code and documentation and can't find
>> what I am looking for. I am happy to continue working, but thought I'd
>> ask just in case I was missing something silly.
>>
>> Thanks again for all your help getting me started on this!
> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
> basic blocks.

Thank you so much for your response!

I just found this as soon as you sent it. Sorry for wasting your time!


>
> Assuming you're running late, you'll then want to walk each insn within
> the bb.  So something like this
>
> basic_block bb
> FOR_EACH_BB (bb)
>   {
> rtx_insn *insn;
> FOR_BB_INSNS (bb, insn)
>   {
> /* Do something with INSN.  */
>   }
>   }
>
>
> Note that if you're running too late the CFG may have been released, in
> which case this code wouldn't do anything.

I will just have to experiment to see exactly when the right time to
invoke this plugin to get the best data.

Thanks again!
Will

>
> jeff

Re: Basic Block Statistics

[Will Hawkins]

On Wed, May 17, 2017 at 1:04 PM, Will Hawkins  wrote:
> On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
>> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>>> As I started looking into this, it seems like PLUGIN_FINISH is where
>>> my plugin will go. Everything is great so far. However, when plugins
>>> at that event are invoked, they get no data. That means I will have to
>>> look into global structures for information regarding the compilation.
>>> Are there pointers to the documentation that describe the relevant
>>> global data structures that are accessible at this point?
>>>
>>> I am looking through the source code and documentation and can't find
>>> what I am looking for. I am happy to continue working, but thought I'd
>>> ask just in case I was missing something silly.
>>>
>>> Thanks again for all your help getting me started on this!
>> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
>> basic blocks.
>
> Thank you so much for your response!
>
> I just found this as soon as you sent it. Sorry for wasting your time!
>
>
>>
>> Assuming you're running late, you'll then want to walk each insn within
>> the bb.  So something like this
>>
>> basic_block bb
>> FOR_EACH_BB (bb)
>>   {
>> rtx_insn *insn;
>> FOR_BB_INSNS (bb, insn)
>>   {
>> /* Do something with INSN.  */
>>   }
>>   }
>>
>>
>> Note that if you're running too late the CFG may have been released, in
>> which case this code wouldn't do anything.

This macro seems to require that there be a valid cfun. This seems to
imply that the macro will work only where the plugin callback is
invoked before/after a pass that does some optimization for a
particular function. In particular, at PLUGIN_FINISH, cfun is NULL.
This makes perfect sense.

Since PLUGIN_FINISH is the place where diagnostics are supposed to be
printed, I was wondering if there was an equivalent iterator for all
translation units (from which I could derive functions, from which I
could derive basic blocks) that just "FINISH"ed compiling?

The other way to approach the problem, I suppose, is to just
accumulate those stats at the end of each pass execution phase and
then simply print them when PLUGIN_FINISH is invoked.

I'm sorry to make this so difficult. I am just wondering about the way
that the community expects the plugins to be written in the most
modular fashion.

Thanks again for walking me through all this!
Will

>
> I will just have to experiment to see exactly when the right time to
> invoke this plugin to get the best data.
>
> Thanks again!
> Will
>
>>
>> jeff

Re: Basic Block Statistics

[Will Hawkins]

On Wed, May 17, 2017 at 2:41 PM, Will Hawkins  wrote:
> On Wed, May 17, 2017 at 1:04 PM, Will Hawkins  wrote:
>> On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
>>> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>>>> As I started looking into this, it seems like PLUGIN_FINISH is where
>>>> my plugin will go. Everything is great so far. However, when plugins
>>>> at that event are invoked, they get no data. That means I will have to
>>>> look into global structures for information regarding the compilation.
>>>> Are there pointers to the documentation that describe the relevant
>>>> global data structures that are accessible at this point?
>>>>
>>>> I am looking through the source code and documentation and can't find
>>>> what I am looking for. I am happy to continue working, but thought I'd
>>>> ask just in case I was missing something silly.
>>>>
>>>> Thanks again for all your help getting me started on this!
>>> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
>>> basic blocks.
>>
>> Thank you so much for your response!
>>
>> I just found this as soon as you sent it. Sorry for wasting your time!
>>
>>
>>>
>>> Assuming you're running late, you'll then want to walk each insn within
>>> the bb.  So something like this
>>>
>>> basic_block bb
>>> FOR_EACH_BB (bb)
>>>   {
>>> rtx_insn *insn;
>>> FOR_BB_INSNS (bb, insn)
>>>   {
>>> /* Do something with INSN.  */
>>>   }
>>>   }
>>>
>>>
>>> Note that if you're running too late the CFG may have been released, in
>>> which case this code wouldn't do anything.
>
> This macro seems to require that there be a valid cfun. This seems to
> imply that the macro will work only where the plugin callback is
> invoked before/after a pass that does some optimization for a
> particular function. In particular, at PLUGIN_FINISH, cfun is NULL.
> This makes perfect sense.
>
> Since PLUGIN_FINISH is the place where diagnostics are supposed to be
> printed, I was wondering if there was an equivalent iterator for all
> translation units (from which I could derive functions, from which I
> could derive basic blocks) that just "FINISH"ed compiling?


Answering my own question for historical purposes and anyone else who
might need this:

  FOR_EACH_VEC_ELT(*all_translation_units, i, t)

is exactly what I was looking for!

Sorry for the earlier spam and thank you for your patience!
Will

>
> The other way to approach the problem, I suppose, is to just
> accumulate those stats at the end of each pass execution phase and
> then simply print them when PLUGIN_FINISH is invoked.
>
> I'm sorry to make this so difficult. I am just wondering about the way
> that the community expects the plugins to be written in the most
> modular fashion.
>
> Thanks again for walking me through all this!
> Will
>
>>
>> I will just have to experiment to see exactly when the right time to
>> invoke this plugin to get the best data.
>>
>> Thanks again!
>> Will
>>
>>>
>>> jeff

Re: Basic Block Statistics

[Will Hawkins]

On Wed, May 17, 2017 at 2:59 PM, Will Hawkins  wrote:
> On Wed, May 17, 2017 at 2:41 PM, Will Hawkins  wrote:
>> On Wed, May 17, 2017 at 1:04 PM, Will Hawkins  wrote:
>>> On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
>>>> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>>>>> As I started looking into this, it seems like PLUGIN_FINISH is where
>>>>> my plugin will go. Everything is great so far. However, when plugins
>>>>> at that event are invoked, they get no data. That means I will have to
>>>>> look into global structures for information regarding the compilation.
>>>>> Are there pointers to the documentation that describe the relevant
>>>>> global data structures that are accessible at this point?
>>>>>
>>>>> I am looking through the source code and documentation and can't find
>>>>> what I am looking for. I am happy to continue working, but thought I'd
>>>>> ask just in case I was missing something silly.
>>>>>
>>>>> Thanks again for all your help getting me started on this!
>>>> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
>>>> basic blocks.
>>>
>>> Thank you so much for your response!
>>>
>>> I just found this as soon as you sent it. Sorry for wasting your time!
>>>
>>>
>>>>
>>>> Assuming you're running late, you'll then want to walk each insn within
>>>> the bb.  So something like this
>>>>
>>>> basic_block bb
>>>> FOR_EACH_BB (bb)
>>>>   {
>>>> rtx_insn *insn;
>>>> FOR_BB_INSNS (bb, insn)
>>>>   {
>>>> /* Do something with INSN.  */
>>>>   }
>>>>   }
>>>>
>>>>
>>>> Note that if you're running too late the CFG may have been released, in
>>>> which case this code wouldn't do anything.
>>
>> This macro seems to require that there be a valid cfun. This seems to
>> imply that the macro will work only where the plugin callback is
>> invoked before/after a pass that does some optimization for a
>> particular function. In particular, at PLUGIN_FINISH, cfun is NULL.
>> This makes perfect sense.
>>
>> Since PLUGIN_FINISH is the place where diagnostics are supposed to be
>> printed, I was wondering if there was an equivalent iterator for all
>> translation units (from which I could derive functions, from which I
>> could derive basic blocks) that just "FINISH"ed compiling?
>
>
> Answering my own question for historical purposes and anyone else who
> might need this:
>
>   FOR_EACH_VEC_ELT(*all_translation_units, i, t)
>
> is exactly what I was looking for!
>
> Sorry for the earlier spam and thank you for your patience!
> Will


Well, I thought that this was what I wanted, but it turns out perhaps
I was wrong. So, I am turning back for some help. Again, i apologize
for the incessant emails.

I would have thought that a translation unit tree node's chain would
point to all the nested tree nodes. This does not seem to be the case,
however. Am I missing something? Or is this the intended behavior?

Again, thank you for your patience!
Will

>
>>
>> The other way to approach the problem, I suppose, is to just
>> accumulate those stats at the end of each pass execution phase and
>> then simply print them when PLUGIN_FINISH is invoked.
>>
>> I'm sorry to make this so difficult. I am just wondering about the way
>> that the community expects the plugins to be written in the most
>> modular fashion.
>>
>> Thanks again for walking me through all this!
>> Will
>>
>>>
>>> I will just have to experiment to see exactly when the right time to
>>> invoke this plugin to get the best data.
>>>
>>> Thanks again!
>>> Will
>>>
>>>>
>>>> jeff

Re: Basic Block Statistics

[Will Hawkins]

On Fri, May 19, 2017 at 4:40 PM, Jeff Law  wrote:
> On 05/17/2017 08:22 PM, Will Hawkins wrote:
>> On Wed, May 17, 2017 at 2:59 PM, Will Hawkins  wrote:
>>> On Wed, May 17, 2017 at 2:41 PM, Will Hawkins  wrote:
>>>> On Wed, May 17, 2017 at 1:04 PM, Will Hawkins  wrote:
>>>>> On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
>>>>>> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>>>>>>> As I started looking into this, it seems like PLUGIN_FINISH is where
>>>>>>> my plugin will go. Everything is great so far. However, when plugins
>>>>>>> at that event are invoked, they get no data. That means I will have to
>>>>>>> look into global structures for information regarding the compilation.
>>>>>>> Are there pointers to the documentation that describe the relevant
>>>>>>> global data structures that are accessible at this point?
>>>>>>>
>>>>>>> I am looking through the source code and documentation and can't find
>>>>>>> what I am looking for. I am happy to continue working, but thought I'd
>>>>>>> ask just in case I was missing something silly.
>>>>>>>
>>>>>>> Thanks again for all your help getting me started on this!
>>>>>> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
>>>>>> basic blocks.
>>>>>
>>>>> Thank you so much for your response!
>>>>>
>>>>> I just found this as soon as you sent it. Sorry for wasting your time!
>>>>>
>>>>>
>>>>>>
>>>>>> Assuming you're running late, you'll then want to walk each insn within
>>>>>> the bb.  So something like this
>>>>>>
>>>>>> basic_block bb
>>>>>> FOR_EACH_BB (bb)
>>>>>>   {
>>>>>> rtx_insn *insn;
>>>>>> FOR_BB_INSNS (bb, insn)
>>>>>>   {
>>>>>> /* Do something with INSN.  */
>>>>>>   }
>>>>>>   }
>>>>>>
>>>>>>
>>>>>> Note that if you're running too late the CFG may have been released, in
>>>>>> which case this code wouldn't do anything.
>>>>
>>>> This macro seems to require that there be a valid cfun. This seems to
>>>> imply that the macro will work only where the plugin callback is
>>>> invoked before/after a pass that does some optimization for a
>>>> particular function. In particular, at PLUGIN_FINISH, cfun is NULL.
>>>> This makes perfect sense.
>>>>
>>>> Since PLUGIN_FINISH is the place where diagnostics are supposed to be
>>>> printed, I was wondering if there was an equivalent iterator for all
>>>> translation units (from which I could derive functions, from which I
>>>> could derive basic blocks) that just "FINISH"ed compiling?
>>>
>>>
>>> Answering my own question for historical purposes and anyone else who
>>> might need this:
>>>
>>>   FOR_EACH_VEC_ELT(*all_translation_units, i, t)
>>>
>>> is exactly what I was looking for!
>>>
>>> Sorry for the earlier spam and thank you for your patience!
>>> Will
>>
>>
>> Well, I thought that this was what I wanted, but it turns out perhaps
>> I was wrong. So, I am turning back for some help. Again, i apologize
>> for the incessant emails.
>>
>> I would have thought that a translation unit tree node's chain would
>> point to all the nested tree nodes. This does not seem to be the case,
>> however. Am I missing something? Or is this the intended behavior?
> I think there's a fundamental misunderstanding.

You are right, Mr. Law. I'm really sorry for the confusion. I got
things straightened out in my head and now I am making great progress.
>
> We don't hold the RTL IR for all the functions in a translation unit in
> memory at the same time.  You have to look at the RTL IR for each as its
> generated.

Thank you, as ever, for your continued input. I am going to continue
to work and I will keep everyone on the list posted and let you know
when it is complete.

Thanks again and have a great rest of the weekend!

Will
>
> jeff

Re: Basic Block Statistics

[Will Hawkins]

I just wanted to send a quick follow up.

Thanks to the incredible support on this list from Mr. Law and support
in IRC from segher, djgpp and dmalcolm, I was able to put together a
serviceable little plugin that does some very basic statistic
generation on basic blocks.

Here is a link to the source with information about how to build/run:
https://github.com/whh8b/bb_stats

If you are interested in more information, just send me an email.

Thanks again for everyone's help!
Will

On Sat, May 20, 2017 at 11:29 PM, Will Hawkins  wrote:
> On Fri, May 19, 2017 at 4:40 PM, Jeff Law  wrote:
>> On 05/17/2017 08:22 PM, Will Hawkins wrote:
>>> On Wed, May 17, 2017 at 2:59 PM, Will Hawkins  wrote:
>>>> On Wed, May 17, 2017 at 2:41 PM, Will Hawkins  wrote:
>>>>> On Wed, May 17, 2017 at 1:04 PM, Will Hawkins  wrote:
>>>>>> On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
>>>>>>> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>>>>>>>> As I started looking into this, it seems like PLUGIN_FINISH is where
>>>>>>>> my plugin will go. Everything is great so far. However, when plugins
>>>>>>>> at that event are invoked, they get no data. That means I will have to
>>>>>>>> look into global structures for information regarding the compilation.
>>>>>>>> Are there pointers to the documentation that describe the relevant
>>>>>>>> global data structures that are accessible at this point?
>>>>>>>>
>>>>>>>> I am looking through the source code and documentation and can't find
>>>>>>>> what I am looking for. I am happy to continue working, but thought I'd
>>>>>>>> ask just in case I was missing something silly.
>>>>>>>>
>>>>>>>> Thanks again for all your help getting me started on this!
>>>>>>> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
>>>>>>> basic blocks.
>>>>>>
>>>>>> Thank you so much for your response!
>>>>>>
>>>>>> I just found this as soon as you sent it. Sorry for wasting your time!
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Assuming you're running late, you'll then want to walk each insn within
>>>>>>> the bb.  So something like this
>>>>>>>
>>>>>>> basic_block bb
>>>>>>> FOR_EACH_BB (bb)
>>>>>>>   {
>>>>>>> rtx_insn *insn;
>>>>>>> FOR_BB_INSNS (bb, insn)
>>>>>>>   {
>>>>>>> /* Do something with INSN.  */
>>>>>>>   }
>>>>>>>   }
>>>>>>>
>>>>>>>
>>>>>>> Note that if you're running too late the CFG may have been released, in
>>>>>>> which case this code wouldn't do anything.
>>>>>
>>>>> This macro seems to require that there be a valid cfun. This seems to
>>>>> imply that the macro will work only where the plugin callback is
>>>>> invoked before/after a pass that does some optimization for a
>>>>> particular function. In particular, at PLUGIN_FINISH, cfun is NULL.
>>>>> This makes perfect sense.
>>>>>
>>>>> Since PLUGIN_FINISH is the place where diagnostics are supposed to be
>>>>> printed, I was wondering if there was an equivalent iterator for all
>>>>> translation units (from which I could derive functions, from which I
>>>>> could derive basic blocks) that just "FINISH"ed compiling?
>>>>
>>>>
>>>> Answering my own question for historical purposes and anyone else who
>>>> might need this:
>>>>
>>>>   FOR_EACH_VEC_ELT(*all_translation_units, i, t)
>>>>
>>>> is exactly what I was looking for!
>>>>
>>>> Sorry for the earlier spam and thank you for your patience!
>>>> Will
>>>
>>>
>>> Well, I thought that this was what I wanted, but it turns out perhaps
>>> I was wrong. So, I am turning back for some help. Again, i apologize
>>> for the incessant emails.
>>>
>>> I would have thought that a translation unit tree node's chain would
>>> point to all the nested tree nodes. This does not seem to be the case,
>>> however. Am I missing something? Or is this the intended behavior?
>> I think there's a fundamental misunderstanding.
>
> You are right, Mr. Law. I'm really sorry for the confusion. I got
> things straightened out in my head and now I am making great progress.
>>
>> We don't hold the RTL IR for all the functions in a translation unit in
>> memory at the same time.  You have to look at the RTL IR for each as its
>> generated.
>
> Thank you, as ever, for your continued input. I am going to continue
> to work and I will keep everyone on the list posted and let you know
> when it is complete.
>
> Thanks again and have a great rest of the weekend!
>
> Will
>>
>> jeff

Re: GCC and Meltdown and Spectre vulnerabilities

[Will Hawkins]

On Thu, Jan 4, 2018 at 10:10 PM, Eric Gallager  wrote:
> Is there anything GCC could be doing at the compiler level to mitigate
> the recently-announced Meltdown and Spectre vulnerabilities? From
> reading about them, it seems like they involve speculative execution
> and indirect branch prediction, and those are the domain of things the
> compiler deals with, right? (For reference, Meltdown is CVE-2017-5754,
> and Spectre is CVE-2017-5753 and CVE-2017-5715)
>
> Just wondering,
> Eric

Check out

https://support.google.com/faqs/answer/7625886

and especially

http://git.infradead.org/users/dwmw2/gcc-retpoline.git/shortlog/refs/heads/gcc-7_2_0-retpoline-20171219

I'd love to hear what other people have heard!

Will

Re: About Bug 52485

[Will Hawkins]

Thanks to your brand new Bugzilla account, you may now comment! :-)


You will receive instructions on how to reset your default default
password and access your account. Please let me know if you have any
questions or trouble gaining access.

I'd be happy to help in any way that I can!

Thanks for contributing to GCC!
Will


On Wed, May 9, 2018 at 4:08 AM, SHIH YEN-TE  wrote:
> Want to comment on "Bug 52485 - [c++11] add an option to disable c++11 
> user-defined literals"
>
>
> It's a pity GCC doesn't support this, which forces me to give up introducing 
> newer C++ standard into my project. I know it is ridiculous, but we must know 
> the real world is somehow ridiculous as well as nothing is perfect.
>
>