-ffunction-sections and -fdata-sections documentation
Hello, I would like to update the documentation of these compiler flags and have some questions. The -ffunction-sections and -fdata-sections documentation is currently: "Place each function or data item into its own section in the output file if the target supports arbitrary sections. The name of the function or the name of the data item determines the section’s name in the output file. Use these options on systems where the linker can perform optimizations to improve locality of reference in the instruction space. Most systems using the ELF object format and SPARC processors running Solaris 2 have linkers with such optimizations. AIX may have these optimizations in the future. Only use these options when there are significant benefits from doing so. When you specify these options, the assembler and linker create larger object and executable files and are also slower. You cannot use |gprof| on all systems if you specify this option, and you may have problems with debugging if you specify both this option and -g." The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? This "AIX may have these optimizations in the future." is there since at least 1996. What is the current AIX status? Is the "Only use these options when there are significant benefits from doing so. When you specify these options, the assembler and linker create larger object and executable files and are also slower. You cannot use |gprof| on all systems if you specify this option, and you may have problems with debugging if you specify both this option and -g." still correct on the systems of today? Do these options affect the code generation? -- Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.hu...@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
Re: -ffunction-sections and -fdata-sections documentation
On Fri, Oct 13, 2017 at 3:06 AM, Sebastian Huber wrote: > Hello, > > I would like to update the documentation of these compiler flags and have > some questions. The -ffunction-sections and -fdata-sections documentation > is currently: > > "Place each function or data item into its own section in the output file if > the target supports arbitrary sections. The name of the function or the name > of the data item determines the section’s name in the output file. > > Use these options on systems where the linker can perform optimizations to > improve locality of reference in the instruction space. Most systems using > the ELF object format and SPARC processors running Solaris 2 have linkers > with such optimizations. AIX may have these optimizations in the future. > > Only use these options when there are significant benefits from doing so. > When you specify these options, the assembler and linker create larger > object and executable files and are also slower. You cannot use |gprof| on > all systems if you specify this option, and you may have problems with > debugging if you specify both this option and -g." > > The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? > > This "AIX may have these optimizations in the future." is there since at > least 1996. What is the current AIX status? > > Is the "Only use these options when there are significant benefits from > doing so. When you specify these options, the assembler and linker create > larger object and executable files and are also slower. You cannot use > |gprof| on all systems if you specify this option, and you may have problems > with debugging if you specify both this option and -g." still correct on the > systems of today? > > Do these options affect the code generation? Out of date documentation in GCC? What a surprise! -ffunction-sections and -fdata-sections are available on AIX. The AIX linker rearranges sections (CSECTs) based on the call graph. The performance impact varies. I'm not certain what the central question of the email message is. Thanks, David
Re: -ffunction-sections and -fdata-sections documentation
On 10/13/2017 01:06 AM, Sebastian Huber wrote: > Hello, > > I would like to update the documentation of these compiler flags and > have some questions. The -ffunction-sections and -fdata-sections > documentation is currently: > > "Place each function or data item into its own section in the output > file if the target supports arbitrary sections. The name of the function > or the name of the data item determines the section’s name in the output > file. > > Use these options on systems where the linker can perform optimizations > to improve locality of reference in the instruction space. Most systems > using the ELF object format and SPARC processors running Solaris 2 have > linkers with such optimizations. AIX may have these optimizations in the > future. > > Only use these options when there are significant benefits from doing > so. When you specify these options, the assembler and linker create > larger object and executable files and are also slower. You cannot use > |gprof| on all systems if you specify this option, and you may have > problems with debugging if you specify both this option and -g." > > The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? > > This "AIX may have these optimizations in the future." is there since at > least 1996. What is the current AIX status? > > Is the "Only use these options when there are significant benefits from > doing so. When you specify these options, the assembler and linker > create larger object and executable files and are also slower. You > cannot use |gprof| on all systems if you specify this option, and you > may have problems with debugging if you specify both this option and > -g." still correct on the systems of today? > > Do these options affect the code generation? They can affect code generation. By placing each object into its own section it's no longer viable to use one object to refer to another because the relative addresses are unknown until link time. Without those options the assembler can compute the offset between two objects within a given section when they come from the same translation unit. While such tricks aren't valid C code, the compiler and assembler can work together under the hood to perform these kinds of optimizations. The documentation is clearly dated. A patch to update it would be greatly appreciated :-) Jeff
Re: Feature request: -Wno-unknown-warnings to silently ignore unknown warning control flags.
On 10/12/2017 10:49 PM, Oren Ben-Kiki wrote: On Fri, Oct 13, 2017 at 4:47 AM, Martin Sebor mailto:mse...@gmail.com>> wrote: My organization was particularly focused on warnings so I'm quite familiar with the challenges you're hoping to overcome. I figured I wasn't the only one :-) But it is good to hear from others. My best advice is to either write a script to determine the options the target compiler supports and use only those, or write a compiler driver that handles this for you (it doesn't relieve you of the tedious task of researching all the options and hard wiring them into the driver(*)). Yes, this is probably the best one can do right now. This isn't to say that GCC shouldn't make it easier, but as you note, even if and when it does, it will take years before the solution is universally available. Making the world a better place for the next generation is a worthy goal ;-) [*] We wrote a script scrape those off the online HTML manual and create a "database" mapping options to GCC versions they were introduced in (or first documented in, as not every option always gets documented as it gets added). Sounds interesting! I don't suppose this driver is available as a separate standalone tool? Ours was a proprietary script but I'm pretty sure sure I've seen similar compiler wrappers in open source. If I'm not mistaken projects like DistCC and the no longer maintained (AFAIK) Trolltech TeamBuilder come with drivers that do similar things. There are bound to be others. Good luck! Martin
Re: -ffunction-sections and -fdata-sections documentation
On 13 October 2017 at 17:02, David Edelsohn wrote: > On Fri, Oct 13, 2017 at 3:06 AM, Sebastian Huber > wrote: >> Hello, >> >> I would like to update the documentation of these compiler flags and have >> some questions. The -ffunction-sections and -fdata-sections documentation >> is currently: >> >> "Place each function or data item into its own section in the output file if >> the target supports arbitrary sections. The name of the function or the name >> of the data item determines the section’s name in the output file. >> >> Use these options on systems where the linker can perform optimizations to >> improve locality of reference in the instruction space. Most systems using >> the ELF object format and SPARC processors running Solaris 2 have linkers >> with such optimizations. AIX may have these optimizations in the future. >> >> Only use these options when there are significant benefits from doing so. >> When you specify these options, the assembler and linker create larger >> object and executable files and are also slower. You cannot use |gprof| on >> all systems if you specify this option, and you may have problems with >> debugging if you specify both this option and -g." >> >> The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? >> >> This "AIX may have these optimizations in the future." is there since at >> least 1996. What is the current AIX status? >> >> Is the "Only use these options when there are significant benefits from >> doing so. When you specify these options, the assembler and linker create >> larger object and executable files and are also slower. You cannot use >> |gprof| on all systems if you specify this option, and you may have problems >> with debugging if you specify both this option and -g." still correct on the >> systems of today? >> >> Do these options affect the code generation? > > Out of date documentation in GCC? What a surprise! > > -ffunction-sections and -fdata-sections are available on AIX. The AIX > linker rearranges sections (CSECTs) based on the call graph. The > performance impact varies. > > I'm not certain what the central question of the email message is. > > Thanks, David I think they're asking: 1. Is it worth removing the note about Solaris, given it get EOLed in 2006. 2. Does AIX's linker perform optimization "to improve locality of reference in the instruction space" 3. Does options still cause problems with debugging(when used with -g) and with gprof?
Global analysis of RTL
My team and I are working on a procedural abstraction (PA) optimization for GCC based on the ARM Embedded Toolchain distribution that uses GCC 6.2.1. Our initial attempt to get something functional has been successful, but it is a very messy solution because we don't see a reasonable way to do an analysis of a compilation unit or whole program at the level or RTL. From what we have been able to gleen (note: we're all relative newcomers to the GCC code base), all RTL transformation and analysis is done at the level of functions only. Once pass_final runs, the RTL has been output to assembler, meaning no more changes to RTL is possible. We have not been able to find a reasonable way to collect all the RTL for a compilation unit to do an analysis across functions. Any solution we have come up with involves some serious changes to cgraphunit.c and final.c, among others. While we're willing to put in the effort to make these changes, we're wondering if anyone knows of something that would allow global RTL analysis without making large-scale changes. It seems as though the structure of the pass manager does not allow for this. The code structure suggests that we should either do an IPA pass on GIMPLE or we work as an assembler pass, thus forgoing work on RTL at all. Is this is what we should be doing? If anyone has some guidance here, it would be appreciated. -- Geoff
Re: -ffunction-sections and -fdata-sections documentation
- Am 13. Okt 2017 um 16:02 schrieb David Edelsohn dje@gmail.com: > On Fri, Oct 13, 2017 at 3:06 AM, Sebastian Huber > wrote: >> Hello, >> >> I would like to update the documentation of these compiler flags and have >> some questions. The -ffunction-sections and -fdata-sections documentation >> is currently: >> >> "Place each function or data item into its own section in the output file if >> the target supports arbitrary sections. The name of the function or the name >> of the data item determines the section’s name in the output file. >> >> Use these options on systems where the linker can perform optimizations to >> improve locality of reference in the instruction space. Most systems using >> the ELF object format and SPARC processors running Solaris 2 have linkers >> with such optimizations. AIX may have these optimizations in the future. >> >> Only use these options when there are significant benefits from doing so. >> When you specify these options, the assembler and linker create larger >> object and executable files and are also slower. You cannot use |gprof| on >> all systems if you specify this option, and you may have problems with >> debugging if you specify both this option and -g." >> >> The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? >> >> This "AIX may have these optimizations in the future." is there since at >> least 1996. What is the current AIX status? >> >> Is the "Only use these options when there are significant benefits from >> doing so. When you specify these options, the assembler and linker create >> larger object and executable files and are also slower. You cannot use >> |gprof| on all systems if you specify this option, and you may have problems >> with debugging if you specify both this option and -g." still correct on the >> systems of today? >> >> Do these options affect the code generation? > > Out of date documentation in GCC? What a surprise! > > -ffunction-sections and -fdata-sections are available on AIX. The AIX > linker rearranges sections (CSECTs) based on the call graph. The > performance impact varies. > > I'm not certain what the central question of the email message is. The central question for me is the effect on code generation.
Re: Global analysis of RTL
On 10/13/2017 12:18 PM, Geoff Wozniak wrote: > My team and I are working on a procedural abstraction (PA) optimization > for GCC based on the ARM Embedded Toolchain distribution that uses GCC > 6.2.1. Our initial attempt to get something functional has been > successful, but it is a very messy solution because we don't see a > reasonable way to do an analysis of a compilation unit or whole program > at the level or RTL. From what we have been able to gleen (note: we're > all relative newcomers to the GCC code base), all RTL transformation and > analysis is done at the level of functions only. Once pass_final runs, > the RTL has been output to assembler, meaning no more changes to RTL is > possible. We have not been able to find a reasonable way to collect > all the RTL for a compilation unit to do an analysis across functions. > Any solution we have come up with involves some serious changes to > cgraphunit.c and final.c, among others. While we're willing to put in > the effort to make these changes, we're wondering if anyone knows of > something that would allow global RTL analysis without making > large-scale changes. It seems as though the structure of the pass > manager does not allow for this. The code structure suggests that we > should either do an IPA pass on GIMPLE or we work as an assembler pass, > thus forgoing work on RTL at all. Is this is what we should be doing? > If anyone has some guidance here, it would be appreciated. -- Geoff There are no real mechanisms for doing IPA optimizations or analysis on RTL. If you could reformulate to work as an IPA pass on gimple that would be my recommendation. You could probably use pieces of the IPA-ICF infrastructure to help identify semantically equivalent blocks to stitch together and outline. Jeff
Re: -ffunction-sections and -fdata-sections documentation
- Am 13. Okt 2017 um 16:11 schrieb Jeff Law l...@redhat.com: > On 10/13/2017 01:06 AM, Sebastian Huber wrote: >> Hello, >> >> I would like to update the documentation of these compiler flags and >> have some questions. The -ffunction-sections and -fdata-sections >> documentation is currently: >> >> "Place each function or data item into its own section in the output >> file if the target supports arbitrary sections. The name of the function >> or the name of the data item determines the section’s name in the output >> file. >> >> Use these options on systems where the linker can perform optimizations >> to improve locality of reference in the instruction space. Most systems >> using the ELF object format and SPARC processors running Solaris 2 have >> linkers with such optimizations. AIX may have these optimizations in the >> future. >> >> Only use these options when there are significant benefits from doing >> so. When you specify these options, the assembler and linker create >> larger object and executable files and are also slower. You cannot use >> |gprof| on all systems if you specify this option, and you may have >> problems with debugging if you specify both this option and -g." >> >> The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? >> >> This "AIX may have these optimizations in the future." is there since at >> least 1996. What is the current AIX status? >> >> Is the "Only use these options when there are significant benefits from >> doing so. When you specify these options, the assembler and linker >> create larger object and executable files and are also slower. You >> cannot use |gprof| on all systems if you specify this option, and you >> may have problems with debugging if you specify both this option and >> -g." still correct on the systems of today? >> >> Do these options affect the code generation? > They can affect code generation. By placing each object into its own > section it's no longer viable to use one object to refer to another > because the relative addresses are unknown until link time. Without > those options the assembler can compute the offset between two objects > within a given section when they come from the same translation unit. Is this similar to moving all distinct objects into one compiler generated structure? > > While such tricks aren't valid C code, the compiler and assembler can > work together under the hood to perform these kinds of optimizations. > > The documentation is clearly dated. A patch to update it would be > greatly appreciated :-) I will try to update it.
Re: -ffunction-sections and -fdata-sections documentation
On Fri, Oct 13, 2017 at 2:34 PM, Sebastian Huber wrote: >>> Do these options affect the code generation? >> They can affect code generation. By placing each object into its own >> section it's no longer viable to use one object to refer to another >> because the relative addresses are unknown until link time. Without >> those options the assembler can compute the offset between two objects >> within a given section when they come from the same translation unit. > > Is this similar to moving all distinct objects into one compiler generated > structure? It places each function and each datum into a separate section, which can be placed or removed independently. It is not combining data or altering the order of structures. It allows the linker to position functions and data items as individual components instead of a single object file "text" or "data" blobs. Thanks, David
Re: -ffunction-sections and -fdata-sections documentation
- Am 13. Okt 2017 um 20:39 schrieb David Edelsohn dje@gmail.com: > On Fri, Oct 13, 2017 at 2:34 PM, Sebastian Huber > wrote: > Do these options affect the code generation? >>> They can affect code generation. By placing each object into its own >>> section it's no longer viable to use one object to refer to another >>> because the relative addresses are unknown until link time. Without >>> those options the assembler can compute the offset between two objects >>> within a given section when they come from the same translation unit. >> >> Is this similar to moving all distinct objects into one compiler generated >> structure? > > It places each function and each datum into a separate section, which > can be placed or removed independently. It is not combining data or > altering the order of structures. It allows the linker to position > functions and data items as individual components instead of a single > object file "text" or "data" blobs. Is the optimization opportunity described by Jeff (which is prevented by the use of function and data sections) similar to moving all distinct objects into one compiler generated structure?
Re: -ffunction-sections and -fdata-sections documentation
>> It places each function and each datum into a separate section, which >> can be placed or removed independently. It is not combining data or >> altering the order of structures. It allows the linker to position >> functions and data items as individual components instead of a single >> object file "text" or "data" blobs. > > Is the optimization opportunity described by Jeff (which is prevented by the > use of function and data sections) similar to moving all distinct objects > into one compiler generated structure? No. - David
Re: -ffunction-sections and -fdata-sections documentation
On 10/13/2017 12:45 PM, Sebastian Huber wrote: > > > - Am 13. Okt 2017 um 20:39 schrieb David Edelsohn dje@gmail.com: > >> On Fri, Oct 13, 2017 at 2:34 PM, Sebastian Huber >> wrote: >> > Do these options affect the code generation? They can affect code generation. By placing each object into its own section it's no longer viable to use one object to refer to another because the relative addresses are unknown until link time. Without those options the assembler can compute the offset between two objects within a given section when they come from the same translation unit. >>> >>> Is this similar to moving all distinct objects into one compiler generated >>> structure? >> >> It places each function and each datum into a separate section, which >> can be placed or removed independently. It is not combining data or >> altering the order of structures. It allows the linker to position >> functions and data items as individual components instead of a single >> object file "text" or "data" blobs. > > Is the optimization opportunity described by Jeff (which is prevented by the > use of function and data sections) similar to moving all distinct objects > into one compiler generated structure? Yes. And it's something that is inhibited by -fdata-sections because with -fdata-sections the relative locations of any two objects is unknown until link time. -ffunction-sections will do things like inhibit call->short-call relaxations that are sometimes done by the assembler. jeff
Re: Feature request: -Wno-unknown-warnings to silently ignore unknown warning control flags.
On Thu, Oct 12, 2017 at 4:46 AM, Oren Ben-Kiki wrote: > Motivation/Use case: > > * Since gcc/g++ intentionally does not have `-Weverything`, there is a > number of explicit `-W...` flags one might wish to specify explicitly. Fair > enough. > > * Additional `-W...` flags are introduced in new gcc/g++ versions, which > check for new potential code smells, possibly related to later language > standards. That's great (thanks!). > > * Not all platforms run the latest gcc/g++ compiler version. So build tools > need to deal with a range of versions. > > Taking these three points together, we have a problem. The build tool > wishes to specify `-Wshiny-new-warning` but may only do so if the gcc/g++ > compiler version is above some version X. > > This is difficult for several reasons: > > * The documentation does not clearly specify the 1st compiler version that > supports each error/warning command line flag. One has to manually > backtrack through the release notes for all the compiler versions. > > In fact the documentation also doesn't make it easy to see which flags are > covered by other flags - one has to read through the documentation, one > flag at a time, and extract the information from the English text. Some > form of a more structured table would have been awesome: 1st compiler > version supporting the flag, the list of other flags such as `-Wall` that > imply the flag, maybe a list of related flags such as `-f...`. But that > aside... > > * Even if we know which compiler version to check for, this quickly becomes > a PITA to maintain in the build tool, given the large number of compiler > versions in use in the wild. Having many versions means the compiler has > been actively progressing at a fast rate, which is great! But it makes it > nasty to deal with providing the correct warning flags for the correct > compiler version. > > This is (mostly) solved with the new proposed flag: the build tool would > simply list all the desired warning flags, which would be silently ignored > by older compiler versions. > > Typos are a potential issue. One would have to run a "recent" version > without the `-Wno-unknown-warnings` to catch such typos; that would be > rare, and is simple enough to do (and automate in the build tool if > desired). > > Another obvious issue is that the new `-Wno-unknown-warnings` flag would > only be available starting in a future compiler version. So build tools > would have to deal with the problem for a long time before the flag would > provide its full benefits. Barring having a working time machine ;-) > > However, it is much simpler for a build tool to do a single test of whether > or not the compiler version supports the proposed new flag, than to deal > with the full problem of multiple versions. If the new flag is not > supported, the build tool could fall back to using a conservative set of > warning flags. > > Also, as time goes by, older compiler versions would eventually be phased > out, so even this test would be (eventually) removed. Sure, it would take a > decade or more, but: as the saying goes, "The best time to plant a tree is > 20 years ago; the second best time is today". > > Thanks, > > Oren Ben-Kiki If you use autoconf to generate a configure script for your project, I recommend using the macros from gnulib's manywarnings.m4 and related files: https://www.gnu.org/software/gnulib/manual/html_node/warnings.html#warnings https://www.gnu.org/software/gnulib/manual/html_node/manywarnings.html#manywarnings
Re: -ffunction-sections and -fdata-sections documentation
On 10/13/2017 12:06 AM, Sebastian Huber wrote: The end-of-life of Solaris 2.6 was 2006. Is it worth to mention this here? The reference to Solaris 2.6 is no longer useful. Just mention ELF here. This "AIX may have these optimizations in the future." is there since at least 1996. What is the current AIX status? David answered this. Is the "Only use these options when there are significant benefits from doing so. When you specify these options, the assembler and linker create larger object and executable files and are also slower. You cannot use |gprof| on all systems if you specify this option, and you may have problems with debugging if you specify both this option and -g." still correct on the systems of today? You can get larger objects, because as Jeff mentioned, some compile-time/assembly-time optimizations get disabled. That should probably be clarified. The assembler/linker will be slower because they will have more work to do, more relocations, more sections, larger object files. Some old systems could not support both -ffunction-sections and -pg together, this used to give a warning, which was removed in 2012. I believe this is obsolete. The likely explanation for this doc is https://gcc.gnu.org/ml/gcc-help/2008-11/msg00139.html which mentions that it was already long ago fixed at that time. Using -g should not be a problem on an ELF/DWARF system, which is what most systems use nowadays. There could be issues with other object files/debug info formats, but this is unclear. I suspect this comment is obsolete and can be removed. The doc should probably refer to the linker --gc-sections option, as this is what makes -ffunction-sections useful for most people, by reducing the code size by eliminating unused functions. Do these options affect the code generation? Jeff answered this. Jim
Re: GCC Buildbot Update - Definition of regression
On Wed, 2017-10-11 at 16:17 +0200, Marc Glisse wrote:
> On Wed, 11 Oct 2017, David Malcolm wrote:
>
> > On Wed, 2017-10-11 at 11:18 +0200, Paulo Matos wrote:
> > >
> > > On 11/10/17 11:15, Christophe Lyon wrote:
> > > >
> > > > You can have a look at
> > > > https://git.linaro.org/toolchain/gcc-compare-results.git/
> > > > where compare_tests is a patched version of the contrib/
> > > > script,
> > > > it calls the main perl script (which is not the prettiest thing
> > > > :-)
> > > >
> > >
> > > Thanks, that's useful. I will take a look.
> >
> > You may also want to look at this script I wrote:
> >
> > https://github.com/davidmalcolm/jamais-vu
> >
> > (it has Python classes for working with DejaGnu output)
>
> By the way, David, how do you handle comparisons for the jit
> testsuite? jv
> gives
>
> Tests that went away in build/gcc/testsuite/jit/jit.sum: 81
> ---
>
> PASS: t
> PASS: test-
> PASS: test-arith-overflow.c
> PASS: test-arith-overflow.c.exe iteration 1 of 5: verify_uint_over
> PASS: test-arith-overflow.c.exe iteration 2 of 5: verify_uint_o
> PASS: test-arith-overflow.c.exe iteration 3 of 5: verify
> [...]
>
> Tests appeared in build/gcc/testsuite/jit/jit.sum: 78
> -
>
> PASS: test-arith-overflow.c.exe iteration 1
> PASS: test-arith-overflow.c.exe iteration 2 of
> PASS: test-arith-overflow.c.exe iteration 4 of 5: verify_u
> PASS: test-combination.
> PASS: test-combination.c.exe it
> [...]
>
> The issue is more likely in the testsuite, but I assume you have a
> workflow that allows working around the issue?
I believe the issue here is PR jit/69435 ("Truncated lines in
jit.log").
I suspect that the attachment in comment #2 there ought to fix it
(sorry that this issue stalled; in the meantime I've been simplying
verifying the absence of FAILs and checking the number of PASSes in
jit.sum).
Dave
