Re: Linemap and pph
Gabriel Charette a écrit:
> Alright, so after looking even more at the linemap code, here is what
> I'm thinking now:
>
> So the main problem is:
> with the linemap logic is that linemap_line_start adds a new table
> entry if line_delta < 0 (I don't understand why this is needed, my
> assumption is that it is because the rest of the logic depends on
> set->highest_location, thus we want to make sure we are higher than
> the highest so far? can someone clarify why we need this?)
Here is how I understand this. Sorry if it's too obvious.
The linemap_line_start model follows that of a parser that parses tokens
from left to right, and from top to bottom. Let's imagine a main CU
called M which has this layout:
< start, locus #0
[... tokens ...]
#include "A" <--- locus #1
[... tokens ...]
There are going to be at least three line maps (instances of struct
line_map) created and added to the line map set (struct line_maps) of M:
one for loci from #0 to right before #1, at least one for loci coming
from the included header A (there can be several line maps here, if A
itself includes other headers) and another one from loci coming right
after #1.
A hard invariant here is that source_locations yielded by a given line
map must be less than the struct line_map::start_location of the next
line map. This is (partly) so that line map lookup (when we want to get
the line map that corresponds to a given source_location) can be done
using a binary search, which is faster than just doing a linear search.
A side effect of this is that source_locations handed out by
linemap_line_start increase monotonically.
With these assumptions in mind, when the parser code calls
linemap_line_start to get the source_location corresponding to a new
line at column 0, the only case where the new line is less than the
preceding line (so that line_delta < 0) is if the parser is entering a
new file. E.g, it is entering the header A, coming from M. Or, it is
getting back to M, leaving A. In that case, it seems required for
linemap_line_start to create the new line map for A or for M.
> My solution:
> I will add a boolean flag to linemap_line_start's parameters,
> allowEarlierStartLineStart, which when true, will not create a new
> entry in the line_table even if the line_delta < 0.
If I understand correctly, you are acting like if you were parsing from
right to left and from bottom to top, for the content of A.
In that context, you need to keep the hard invariant I talked about
above, so that line map lookup keeps working, at least.
In other words, when you generate a source_location for a token T0 that
comes topologically /before/ the token T1 you generated source_location
for at the previous step, you need to make sure that source_location of
T0 is less than source_location of T1, and that the struct
line_map::start_location of the current map is less than T0. If T0
comes from a different header than T1 (suppose both T1 and T0 comes from
headers that are included by A) then a new line map (different from the
line map for T1) needs to be created for T0. Now I am not sure if in
your design, there is going to be only one pph for A, or if there is
going to be one pph for each header included by A as well. I would have
imagined the later case to be more appropriate.
It seems to me that for you approach to work, you should at least make
sure to add a new line map when T0 comes a different header than T1,
while processing the pph of A.
> Instead of relying on highest_location to find the current line's
> source location, it will use the start_location and add to it the
> delta between to_line and map->to_line).
If the map->start_location was allocated correctly, I guess this could
work assuming the constraints I raised above are respected.
Note however that there are corner cases like having so long lines in
the source code being parsed that we fill the column-space of the line
map so quickly that we need to allocate more line maps for a given file.
These are corner cases taken care of by linemap_line_start today that
you could miss if you are trying to walk in reverse order like this.
Those would have been addressed by serializing the original line map set
of A that we knew was correct, barring the value of their
map->start_location and map->to_line.
> I will also add a new linemap function,
> linemap_position_for_line_and_column, which will be just like
> linemap_position_for_column, except that it won't assume the
> highest_line in the set is the one we are currently on (this function
> will thus allow us to get a source_location without depending on the
> current assumptions on the order these functions are called in).
Heh. I have added a function with that exact name in my patch set to
track locations across macro expansions. Look at the source code at
http://tinyurl.com/3egxp47, search for
linemap_position_for_line_and_column (obviously). The implicit
assumption made by that function is that the {line,column} p
Romain Geissler copyright assignment
Hi, Romain is doing an internship at STMicroelectronics on GCC plugins, and as his mentor, I confirm and/or inform you that he is covered by the copyright assignement RT 211150 between ST and FSF. Regards. -- Yvan ROUX STMicroelectronics
Re: Linemap and pph
On Tue, Jul 26, 2011 at 4:25 AM, Dodji Seketeli wrote: > Gabriel Charette a écrit: > >> Alright, so after looking even more at the linemap code, here is what >> I'm thinking now: >> >> So the main problem is: >> with the linemap logic is that linemap_line_start adds a new table >> entry if line_delta < 0 (I don't understand why this is needed, my >> assumption is that it is because the rest of the logic depends on >> set->highest_location, thus we want to make sure we are higher than >> the highest so far? can someone clarify why we need this?) > > Here is how I understand this. Sorry if it's too obvious. > > The linemap_line_start model follows that of a parser that parses tokens > from left to right, and from top to bottom. Let's imagine a main CU > called M which has this layout: > > < start, locus #0 > > [... tokens ...] > > #include "A" <--- locus #1 > > [... tokens ...] > > There are going to be at least three line maps (instances of struct > line_map) created and added to the line map set (struct line_maps) of M: > one for loci from #0 to right before #1, at least one for loci coming > from the included header A (there can be several line maps here, if A > itself includes other headers) and another one from loci coming right > after #1. > > A hard invariant here is that source_locations yielded by a given line > map must be less than the struct line_map::start_location of the next > line map. This is (partly) so that line map lookup (when we want to get > the line map that corresponds to a given source_location) can be done > using a binary search, which is faster than just doing a linear search. > A side effect of this is that source_locations handed out by > linemap_line_start increase monotonically. > > With these assumptions in mind, when the parser code calls > linemap_line_start to get the source_location corresponding to a new > line at column 0, the only case where the new line is less than the > preceding line (so that line_delta < 0) is if the parser is entering a > new file. E.g, it is entering the header A, coming from M. Or, it is > getting back to M, leaving A. In that case, it seems required for > linemap_line_start to create the new line map for A or for M. > Thanks for this explanation, that's pretty much how I had understood the linemap code, but you clarified some of the things I wasn't clear about. >> My solution: >> I will add a boolean flag to linemap_line_start's parameters, >> allowEarlierStartLineStart, which when true, will not create a new >> entry in the line_table even if the line_delta < 0. > > If I understand correctly, you are acting like if you were parsing from > right to left and from bottom to top, for the content of A. > Right. > In that context, you need to keep the hard invariant I talked about > above, so that line map lookup keeps working, at least. > > In other words, when you generate a source_location for a token T0 that > comes topologically /before/ the token T1 you generated source_location > for at the previous step, you need to make sure that source_location of > T0 is less than source_location of T1, and that the struct > line_map::start_location of the current map is less than T0. If T0 > comes from a different header than T1 (suppose both T1 and T0 comes from > headers that are included by A) then a new line map (different from the > line map for T1) needs to be created for T0. Now I am not sure if in > your design, there is going to be only one pph for A, or if there is > going to be one pph for each header included by A as well. I would have > imagined the later case to be more appropriate. > Correct we are using the multiple pph approach (one for each included header in the header). This actually makes the problem harder than I had originally pictured it yesterday. I'm now thinking using serialization is going to be a better solution because of that. >> Instead of relying on highest_location to find the current line's >> source location, it will use the start_location and add to it the >> delta between to_line and map->to_line). > > If the map->start_location was allocated correctly, I guess this could > work assuming the constraints I raised above are respected. > > Note however that there are corner cases like having so long lines in > the source code being parsed that we fill the column-space of the line > map so quickly that we need to allocate more line maps for a given file. > These are corner cases taken care of by linemap_line_start today that > you could miss if you are trying to walk in reverse order like this. > Those would have been addressed by serializing the original line map set > of A that we knew was correct, barring the value of their > map->start_location and map->to_line. > One more reason for serialization! >> I will also add a new linemap function, >> linemap_position_for_line_and_column, which will be just like >> linemap_position_for_column, except that it won't assume the >> highest_line in the
Re: Please apply toplevel patches to both gcc and src
"Joseph S. Myers" writes: > I'd like to remind people that when they commit a toplevel build system > patch to GCC they should commit it to the src repository as well, or ask a > build system maintainer to do so if they don't have write access to src. > It looks like at least the following recent patches have not been > committed to src. I have merged these changes to the binutils repository. Ian > > 2011-07-20 David Edelsohn > > * Makefile.tpl (POSTSTAGE1_CONFIGURE_FLAGS): Add libsupc++ to > link directories. > * Makefile.in: Rebuild. > > 2011-07-20 Ian Lance Taylor > > PR bootstrap/49787 > * configure.ac: Move --enable-bootstrap handling earlier in file. > If --enable-bootstrap and either --enable-build-with-cxx or > --enable-build-poststage1-with-cxx, enable C++ automatically. > * configure: Rebuild. > > 2011-07-19 Ian Lance Taylor > > * configure.ac: Add --enable-build-poststage1-with-cxx. If set, > make C++ a boot_language. Set and substitute > POSTSTAGE1_CONFIGURE_FLAGS. > * Makefile.tpl (POSTSTAGE1_CONFIGURE_FLAGS): New variable. > (STAGE[+id+]_CONFIGURE_FLAGS): Add $(POSTSTAGE1_CONFIGURE_FLAGS). > * configure, Makefile.in: Rebuild. > > 2011-07-16 Jason Merrill > > * Makefile.def (language=c++): Add check-c++0x and > check-target-libmudflap-c++. > * Makefile.tpl (check-target-libmudflap-c++): New. > * Makefile.in: Regenerate. > > 2011-07-16 Matthias Klose > > * Makefile.tpl (EXTRA_CONFIGARGS_LIBJAVA): Define. > * Makefile.def (target_modules/libjava): Pass > $(EXTRA_CONFIGARGS_LIBJAVA). > * configure.ac: Pass --disable-static in EXTRA_CONFIGARGS_LIBJAVA, > if not configured with --enable-static-libjava. > * Makefile.in: Regenerate. > * configure: Likewise. > > 2011-07-15 Jason Merrill > > * Makefile.in (check-c++): Move check-gcc-c++0x after > check-target-libstdc++-v3. > > 2011-07-13 Jason Merrill > > * Makefile.in (check-gcc-c++0x): New. > (check-c++): Depend on it. > > 2011-06-22 Hans-Peter Nilsson > > PR regression/47836 > PR bootstrap/23656 > PR other/47733 > PR bootstrap/49247 > PR c/48825 > * configure.ac (target_libraries): Remove target-libiberty. > Remove case-statement setting skipdirs=target-libiberty for > multiple targets. Remove checking target_configdirs and > removing target-libiberty but keeping target-libgcc if > otherwise empty. > * Makefile.def (target_modules): Don't add libiberty. > (dependencies): Remove all traces of target-libiberty. > * configure, Makefile.in: Regenerate.
Re: Linemap and pph
As mentionned in my previous email, I'm now thinking of serializing the linemap from the header by pph'ed, here is how I think this could work: From what I understand, the source_locations allocated for everything in a given set of headers (from the LC_ENTER for the header in the line_table up to, and including everything in between, the corresponding LC_LEAVE) is dependent on only one thing; the value of line_table->highest_location when the header was inserted (i.e. if in two different contexts we happen to have the same line_table->highest_location when inserting header A.h, all of the tokens for A.h in each context will have the same source_location). If the previous statement is true, then we calculate an offset between the line_table->highest_location as it was in the serialized version of the header and as it is in the C file in which we are about to de-serialize the line table. We then need to update some things based on that offset: for each line_map entry in the line_table: start_location for each source_location field on tokens coming in from the header Some other things need to be updated with simple logic: for each line_map entry in the line_table: included_from for the line_table itself: highest_location, highest_line, max_column_hint Some things just stay as is when de-serialized: for each line_map entry in the line_table: reason, sysp, to_file, to_line, column_bits After doing this we should have the same line_map we would have had when doing a regular compile (non-pph), and the tokens coming from the header file should have the correct source_location, simply by adding the calculated offset to the one they had previously (this also allows us to only serialize the source_location for each token as opposed to serializing it's expanded location, saving space in the pph). Let me know what you think, Gabriel On Tue, Jul 26, 2011 at 11:10 AM, Gabriel Charette wrote: > On Tue, Jul 26, 2011 at 4:25 AM, Dodji Seketeli wrote: >> Gabriel Charette a écrit: >> >>> Alright, so after looking even more at the linemap code, here is what >>> I'm thinking now: >>> >>> So the main problem is: >>> with the linemap logic is that linemap_line_start adds a new table >>> entry if line_delta < 0 (I don't understand why this is needed, my >>> assumption is that it is because the rest of the logic depends on >>> set->highest_location, thus we want to make sure we are higher than >>> the highest so far? can someone clarify why we need this?) >> >> Here is how I understand this. Sorry if it's too obvious. >> >> The linemap_line_start model follows that of a parser that parses tokens >> from left to right, and from top to bottom. Let's imagine a main CU >> called M which has this layout: >> >> < start, locus #0 >> >> [... tokens ...] >> >> #include "A" <--- locus #1 >> >> [... tokens ...] >> >> There are going to be at least three line maps (instances of struct >> line_map) created and added to the line map set (struct line_maps) of M: >> one for loci from #0 to right before #1, at least one for loci coming >> from the included header A (there can be several line maps here, if A >> itself includes other headers) and another one from loci coming right >> after #1. >> >> A hard invariant here is that source_locations yielded by a given line >> map must be less than the struct line_map::start_location of the next >> line map. This is (partly) so that line map lookup (when we want to get >> the line map that corresponds to a given source_location) can be done >> using a binary search, which is faster than just doing a linear search. >> A side effect of this is that source_locations handed out by >> linemap_line_start increase monotonically. >> >> With these assumptions in mind, when the parser code calls >> linemap_line_start to get the source_location corresponding to a new >> line at column 0, the only case where the new line is less than the >> preceding line (so that line_delta < 0) is if the parser is entering a >> new file. E.g, it is entering the header A, coming from M. Or, it is >> getting back to M, leaving A. In that case, it seems required for >> linemap_line_start to create the new line map for A or for M. >> > > Thanks for this explanation, that's pretty much how I had understood > the linemap code, but you clarified some of the things I wasn't clear > about. > >>> My solution: >>> I will add a boolean flag to linemap_line_start's parameters, >>> allowEarlierStartLineStart, which when true, will not create a new >>> entry in the line_table even if the line_delta < 0. >> >> If I understand correctly, you are acting like if you were parsing from >> right to left and from bottom to top, for the content of A. >> > > Right. > >> In that context, you need to keep the hard invariant I talked about >> above, so that line map lookup keeps working, at least. >> >> In other words, when you generate a source_location for a token T0 that >> comes topologically /before/ the token T1 you gener
gcc-4.4-20110726 is now available
Snapshot gcc-4.4-20110726 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/4.4-20110726/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 4.4 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-4_4-branch revision 176812 You'll find: gcc-4.4-20110726.tar.bz2 Complete GCC MD5=71dad52ca924be32c02dd1cdf75f9401 SHA1=18dcb39c013406544ea2feb52a931f908a48 Diffs from 4.4-20110719 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-4.4 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
Re: [Bug driver/49858] lost ability to use driver for external language support?
On Tue, Jul 26, 2011 at 6:31 PM, joseph at codesourcery dot com wrote: > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49858 > > --- Comment #1 from joseph at codesourcery dot com dot com> 2011-07-26 23:31:27 UTC --- > On Tue, 26 Jul 2011, bigotp at acm dot org wrote: > > > Is there a mechanism by which the driver can be informed of options that it > > should allow in this situation, given that the list of these options is not > > known at the time the driver is compiled? > > No. By design there is now a structured notion of options shared by the > driver and cc1 and a single mechanism for option parsing, that > consistently rejects unknown options rather than having them sometimes > only inconsistently rejected depending on which phases of compilation are > run, and that is intended in future to support multilib selection based on > logical option state rather than option text, and maybe eventually > disallowing options without --help text. > > It was a mistake that specs were ever documented in the user manual; they > should be considered a purely internal interface between different parts > of GCC. It is not expected that you can drop in support for new languages > without at least part of that support being present when GCC is built > (GNU/Linux distributors may include the minimal set of files defining > specs and options for languages such as D and Pascal when they build their > main GCC packages, for example, so that the driver then supports those > languages even though the language compilers themselves are built > separately). On one hand, I think a regression like this warrants discussion, as this removes a feature that has been supported by gcc (and arguably documented as supported) for many years. Worst, the people who depend on it are folks who are unlikely to be involved in gcc development and will not be aware of the change until after 4.7.0 comes out and their systems have to be rearchitected to work with new vendor-provided gcc installations. I only found it now because I'm updating an out-of-tree back-end (TI msp430) and most of my real-world applications are in TinyOS and depend on nesC. On the other hand, I'm sympathetic to the position that specs are a matter of internal implementation feature and that the change is reasonable. If that's going to be the resolution, though, the spec files material should be moved to the internals documentation. Peter > (I do not rule out that the plugin mechanism could be extended in future > to allow driver plugins, that might modify the unknown option handling, > but the change away from allowing arbitrary options that just happen to be > matched by some spec, to having a structured notion allowing a meaningful > enumeration of all supported options and their associated properties, is > deliberate. It would also be reasonable to provide a way to export the > properties of GCC's options for use in external wrappers to know what > options take arguments. But processing options unknown to GCC is outside > the intended uses of the driver at present, although you may be able to > find a hack where your wrapper hides them within options known to GCC that > take an argument but where the driver doesn't validate that argument in > any way; -fplugin-arg-* might be the most sensible option to use.) > > -- > Configure bugmail: http://gcc.gnu.org/bugzilla/userprefs.cgi?tab=email > --- You are receiving this mail because: --- > You reported the bug.
Re: [RFC] Remove -freorder-blocks-and-partition
On Mon, Jul 25, 2011 at 6:30 PM, Joern Rennecke wrote: > Quoting Xinliang David Li : > >> In xalancbmk, with the partition option, most of object files have >> nonzero size cold sections generated. The text size of the binary is >> increased to 3572728 bytes from 3466790 bytes. Profiling the program >> using the training input shows the following differences. With >> partitioning, number of executed branch instructions slightly >> increases, but itlb misses and icache load misses are significantly >> lower compared with the binary without partitioning. > > It is nice to have confirmation that for this benchmark, the optimization > causes a speedup because it works as intended, however... > dealII and bzip2 degrades about 1.4%. > > ... I think the question was more directed at what causes the > performance degradation for these two benchmarks. > > If we could retain most of the speedups when the optimization works well > but avoid most of the slowdown in the benchmarks that are currently hurt, > we could improve the overall SPEC06 score. And hopefully, this would > also be beneficial to other code. Agree. There are certainly problems in the partition pass, as for bzip2 the icache misses actually go up with partition, which is not expected. It needs further analysis. David >
Re: [RFC] Remove -freorder-blocks-and-partition
On 07/27/2011 06:51 AM, Xinliang David Li wrote: > If we could retain most of the speedups when the optimization works well > but avoid most of the slowdown in the benchmarks that are currently hurt, > we could improve the overall SPEC06 score. And hopefully, this would > also be beneficial to other code. Agree. There are certainly problems in the partition pass, as for bzip2 the icache misses actually go up with partition, which is not expected. It needs further analysis. It's probably too aggressive. Icache misses go up because a) the overall size of the executable grows; b) cold parts are probably not cold enough in the case of bzip2.f Paolo
