Re: AutoFDO profile toolchain is open-sourced
Hi,
One more question.
On 10.04.2015 23:39, Jan Hubicka wrote:
I must say I did not even try running AutoFDO myself (so I am happy to hear
it works).
I tried to use executable create_gcov built from AutoFDO repository at
github.
The problem is that the data generated by this program has size 1600
bytes not depending on the profile data given to it.
Steps to reproduce the issue:
1. Build AutoFDO under x86_64
2. Build, for example, the benchmark ytest.c (see attachment):
g++ -O2 -o ytest ytest.c -g2
(I used g++ that was built just now from gcc-5-branch branch from
git://gcc.gnu.org/git/gcc.git)
3. Run it under perf to collect the profile data:
sudo perf record ./ytest
The perf reports no error and says that
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.125 MB perf.data (~5442 samples) ]
Perf generates perf.data.
4. Run create_gcov on the obtained data:
create_gcov --binary ytest --profile perf.data --gcov ytest.gcov
--debug_dump
It creates 2 files:
* ytest.gcov which is 1600 bytes of size
* ytest.gcov.imports which is empty
Also there is no debug output from the program.
If I run create_llvm_prof on the data
create_llvm_prof --binary ytest --profile perf.data --out ytest.out
--debug_dump
It reports the following log:
Length of symbol map: 1
Number of functions: 0
and creates an empty file ytest.out.
Which is not true: all functions in the benchmark are marked with
__attribute__((noinline)) and readelf says that they stay in the binary:
readelf -s ytest | grep px_cycle
56: 00400640 111 FUNCGLOBAL DEFAULT 12 _Z8px_cyclei
readelf -s ytest | grep py_cycle
60: 004006b036 FUNCGLOBAL DEFAULT 12 _Z8py_cyclev
The size of resulting gcov data is the same (1600 bytes) for different
levels of debug information (-g0, -g1, -g2) and for different input
sources files.
What am I doing wrong?
--
Best regards,
Ilya Palachev
#define DX (480*4)
#define DY (640*4)
int* src = new int[DX*DY];
int* dst = new int[DX*DY];
int pxm = DX;
int pym = DY;
void px_cycle(int py) __attribute__((noinline));
void px_cycle(int py) {
int *p1 = dst + (py*pxm);
int *p2 = src + (pym - py - 1);
for (int px = 0; px < pxm; px++) {
if (px < pym && py < pxm) {
*p1 = *p2;
}
p1++;
p2 += pym;
}
}
void py_cycle() __attribute__((noinline));
void py_cycle() {
for (int py = 0; py < pym; py++) {
px_cycle(py);
}
}
int main() {
int i;
for (i = 0; i < 100; i++) {
py_cycle();
}
return 0;
}
Re: is it time to mass change from .c to .cc in gcc/ ?
On Wed, Apr 15, 2015 at 2:09 AM, Trevor Saunders wrote: > On Tue, Apr 14, 2015 at 10:46:19AM +0200, Richard Biener wrote: >> On Tue, Apr 14, 2015 at 7:20 AM, Trevor Saunders >> wrote: >> > Hi! >> > >> > To be clear I only want to talk about gcc/**/*.c but *not* testsuite/ >> > >> > The Question of changing from .c to a more standard C++ file extension >> > has come up a couple times. I believe its reasonable accurate to say >> > the consensus is moderately in favor of doing this at some point. The >> > biggest concern was of course being able to access pre rename history >> > easily. I know git will either handle this by default or with an option >> > depending on the command, and svn claims it can handle renames so we >> > should be good on that front. The other question was if we should wait >> > to do this at the same time as a reorganization of directory structure. >> > That was back in august 2013, about a year and a half ago, and we >> > haven't done it or really moved forward with a plan to do it. It seems >> > to me that if we do this part now we can then deal with moving files >> > into directories later piece by piece and not need to move everything at >> > once. If we want to go ahead with renaming we should pick a time, I >> > think some people have advanced the idea of doing it just after a >> > branch, on the other hand last year we held off on the big gimple >> > refactoring until after the branch had released a .1. >> > >> > thoughts? >> >> I see no value in doing this but making branch maintainance awkward. > > I think its mostly valuable to cause less confusion of new people, and > though it is a simpler thing every little thing can be the thing that > breaks the cammel's back. I don't buy this kind of argument given that the switch to C++ has complicated things instead of simplifying them. > Yes its not all that hard to configure > editors and what not to handle it properly, but every new person needs > to do it, and looking up configuration options takes time that can more > profitably be spent. > > That said keeping backports as easy as possible is also certainly > important. I'm curious why renames hurt doing backports, I'm pretty > confident git cherry-pick will handle it for you, and if you like patch > files for some reason I'd think its easy to fix up with sed though > running that for each backport by hand would get a little old. So if git can simplify the issues then the appropriate time to do this mass rename is when we switch to git. Richard. > > Trev > >> >> Richard. >> >> > Trev
Re: is it time to mass change from .c to .cc in gcc/ ?
On 4/15/2015 10:09, Richard Biener wrote: > On Wed, Apr 15, 2015 at 2:09 AM, Trevor Saunders > wrote: > I don't buy this kind of argument given that the switch to C++ has > complicated things instead of simplifying them. I've written before about how problematic having c++ files with .c extensions was for importing gcc into DragonFly's base. The system .mk files have targets based on extension (e.g. .c=.o) and it chooses cc or c++ based on those targets. We had to install difficult hacks to work around it. Honestly, mandatory renaming of the files should have been a condition of allowing c++ in the code base. e.g. "guys, if we do this, we have to rename all files requiring compilation by c++ compiler". I know you can wave this off as "we provide a build system for gcc, this isn't a valid reason" but the fact is this is counter-intuitive and confusing. I also think this should be fixed properly, and ripping off the band-aid seems reasonable to me. Regards, John
broken link for Programming Languages Software Award on gcc homepage
Hi, the link for 'ACM SIGPLAN Programming Languages Software Award' in the news list on gcc.gnu.org is http://www.sigplan.org/node/231, as discussed here ( https://gcc.gnu.org/ml/gcc/2014-06/msg00136.html ). Following the link gives me: ... Page Not Found The page you were looking for was not found. Sorry. You were looking for http://www.sigplan.org/node#231 Please email the SIGPLAN Information Director, if you think the file should be here, and we will fix it. ... A generic link mentioning the award is : http://www.sigplan.org/Awards/Software/ . Thanks, - Tom
Re: is it time to mass change from .c to .cc in gcc/ ?
On 15.04.2015 3:09, Trevor Saunders wrote: > On Tue, Apr 14, 2015 at 10:46:19AM +0200, Richard Biener wrote: >> I see no value in doing this but making branch maintainance awkward. > > I think its mostly valuable to cause less confusion of new people, and > though it is a simpler thing every little thing can be the thing that > breaks the cammel's back. Yes its not all that hard to configure > editors and what not to handle it properly, but every new person needs > to do it, and looking up configuration options takes time that can more > profitably be spent. Well, for me, as a newcomer, configuring the editor and getting used to non-standard file extensions was, perhaps, one of the easiest parts (but yes, that's a bit confusing). You mentioned that you are planning to do reorganization of the directory structure. That would be really helpful. LLVM has two separate directories for utility classes, functions and custom datatypes (llvm/include/llvm/ADT and llvm/include/llvm/Support). In GCC all those useful things are scattered among 771 files in gcc directory, some are in libiberty. There is no need to wait for reorganization to at least create a list of such support data structures and functions (something like this: http://llvm.org/docs/ProgrammersManual.html). The good thing is that we already have excellent documentation for some of them in source code (for example vec.h and is-a.h), but we are missing the list. > > That said keeping backports as easy as possible is also certainly > important. I'm curious why renames hurt doing backports, I'm pretty > confident git cherry-pick will handle it for you, and if you like patch > files for some reason I'd think its easy to fix up with sed though > running that for each backport by hand would get a little old. > IIRC, SVN can dump the whole repository and import it again. So, if it's really desired, it's possible to rename the files "in the past" without loosing the history by filtering the dump (but git mirror must be rebuilt in this case). -- Regards, Mikhail Maltsev
Re: is it time to mass change from .c to .cc in gcc/ ?
On Wed, Apr 15, 2015 at 10:09:14AM +0200, Richard Biener wrote: > On Wed, Apr 15, 2015 at 2:09 AM, Trevor Saunders > wrote: > > On Tue, Apr 14, 2015 at 10:46:19AM +0200, Richard Biener wrote: > >> On Tue, Apr 14, 2015 at 7:20 AM, Trevor Saunders > >> wrote: > >> > Hi! > >> > > >> > To be clear I only want to talk about gcc/**/*.c but *not* testsuite/ > >> > > >> > The Question of changing from .c to a more standard C++ file extension > >> > has come up a couple times. I believe its reasonable accurate to say > >> > the consensus is moderately in favor of doing this at some point. The > >> > biggest concern was of course being able to access pre rename history > >> > easily. I know git will either handle this by default or with an option > >> > depending on the command, and svn claims it can handle renames so we > >> > should be good on that front. The other question was if we should wait > >> > to do this at the same time as a reorganization of directory structure. > >> > That was back in august 2013, about a year and a half ago, and we > >> > haven't done it or really moved forward with a plan to do it. It seems > >> > to me that if we do this part now we can then deal with moving files > >> > into directories later piece by piece and not need to move everything at > >> > once. If we want to go ahead with renaming we should pick a time, I > >> > think some people have advanced the idea of doing it just after a > >> > branch, on the other hand last year we held off on the big gimple > >> > refactoring until after the branch had released a .1. > >> > > >> > thoughts? > >> > >> I see no value in doing this but making branch maintainance awkward. > > > > I think its mostly valuable to cause less confusion of new people, and > > though it is a simpler thing every little thing can be the thing that > > breaks the cammel's back. > > I don't buy this kind of argument given that the switch to C++ has > complicated things instead of simplifying them. personally I think C++ has made gcc substantially easier to understand, but that oppinion could be biased by all sorts of things. IN any case I'd be interested to know what's more complicated so I might try and fix them, but that's pretty off topic. > > Yes its not all that hard to configure > > editors and what not to handle it properly, but every new person needs > > to do it, and looking up configuration options takes time that can more > > profitably be spent. > > > > That said keeping backports as easy as possible is also certainly > > important. I'm curious why renames hurt doing backports, I'm pretty > > confident git cherry-pick will handle it for you, and if you like patch > > files for some reason I'd think its easy to fix up with sed though > > running that for each backport by hand would get a little old. > > So if git can simplify the issues then the appropriate time to do this > mass rename is when we switch to git. fair enough, that's an issue I personally care about much more. However I was reminded of this one, and it would be easy enough to do so figured I'd ask. Trev this. > > Richard. > > > > > Trev > > > >> > >> Richard. > >> > >> > Trev
Re: Re: is it time to mass change from .c to .cc in gcc/ ?
On 04/15/2015 11:53 AM, Mikhail Maltsev wrote: You mentioned that you are planning to do reorganization of the directory structure. That would be really helpful. LLVM has two separate directories for utility classes, functions and custom datatypes (llvm/include/llvm/ADT and llvm/include/llvm/Support). In GCC all those useful things are scattered among 771 files in gcc directory, some are in libiberty. There are many intermediate projects that would help this reorganization. But someone needs to do them. Anybody can come up with things to do and grand plans (see https://gcc.gnu.org/wiki/ImprovementProjects and https://gcc.gnu.org/wiki/ModularGCC and https://gcc.gnu.org/wiki/rearch and ). Somebody needs to flesh out the details and start implementing them. The "right" steps to see progress: 1. You think that some concrete change may improve GCC, for example, to move vec.h and is-a.h to some gcc/include/core. Ask if the key people (Global Reviewers in this case) will accept this change. Be concrete, say that you will do it (not 'we could do it'). Perhaps even present a proof-of-concept to see how difficult it will be. 2.1 If not, then goto 1 and think about something else (or perhaps people misunderstood what you meant? See point 7 at https://gcc.gnu.org/wiki/Community). It is not worth it fixating on something. Moreover, your ideas will be judged according to your experience and past successes. Start with something small and no contentious. Then, revisit the more contentious things. 2.2 If yes, incorporate feedback (not include/core but include/support, or not is-a.h but hash-map.h). 3. Sent a patch. The smaller, the better. 4. Repeat and rinse. There is no need to wait for reorganization to at least create a list of such support data structures and functions (something like this: http://llvm.org/docs/ProgrammersManual.html). The good thing is that we already have excellent documentation for some of them in source code (for example vec.h and is-a.h), but we are missing the list. No "maybe we", please :) (Point 11 here: https://gcc.gnu.org/wiki/Community) Just do it. Sent a patch for gccint.texi or create a webpage in the wiki to draft it or just draft it in plain-text and send it to the mailing list. Follow the four steps above. Cheers from someone that has said "maybe we" a lot, but got a few things implemented in the end, Manuel.
Undefined Local Symbol on PowerPC
Hi I think this is a bug in GCC 5 on the PowerPC. The code has compiled and linked fine for years with various versions of GCC and friends. powerpc-rtems4.11-gcc (GCC) 5.0.0 20150412 (experimental) bootloader.o:(.debug_loc+0x3ce2): undefined reference to `.LCL2' This looks like a symbol generation problem with GCC $ powerpc-rtems4.11-gcc --pipe -DHAVE_CONFIG_H -D__BOOT__ -B../../../../../../../lib/ -B../../../../../../../mvme2100/lib/ -specs bsp_specs -qrtems -mrelocatable -msoft-float -mstrict-align -fno-builtin -Wall -mmultiple -mstring -O2 -fomit-frame-pointer -ffixed-r13 -mno-sdata -mcpu=603e -Dppc603e -O2 -g -mmultiple -mstring -mstrict-align -Wall -Wmissing-prototypes -Wimplicit-function-declaration -Wstrict-prototypes -Wnested-externs -S `test -f '../../../powerpc/shared/bootloader/zlib.c' || echo '../../../../../../../../../rtems/c/src/lib/libbsp/powerpc/motorola_powerpc/bootloader/'`../../../powerpc/shared/bootloader/zlib.c $ grep LCL2 *.s .LCL22: lwz 0,.LCL22-.LCF22(30) .LCL23: lwz 0,.LCL23-.LCF23(30) .LCL24: lwz 0,.LCL24-.LCF24(30) .LCL25: lwz 0,.LCL25-.LCF25(30) .4byte.LCL2 .4byte.LCL2 Based on the grep, the .4byte directives are referencing a bogus symbol. Does this look like a GCC bug? If so, I will file a PR with the preprocessed source. Thanks. -- Joel Sherrill, Ph.D. Director of Research & Development joel.sherr...@oarcorp.comOn-Line Applications Research Ask me about RTEMS: a free RTOS Huntsville AL 35805 Support Available(256) 722-9985
gcc-4.9-20150415 is now available
Snapshot gcc-4.9-20150415 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/4.9-20150415/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 4.9 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-4_9-branch revision 222136 You'll find: gcc-4.9-20150415.tar.bz2 Complete GCC MD5=73ed7d3c410c8031d663834595c055d7 SHA1=cf0c82ccc0e8bf947a393fc83b7069db2b27eb30 Diffs from 4.9-20150408 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-4.9 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
Re: Undefined Local Symbol on PowerPC
On Wed, Apr 15, 2015 at 04:10:33PM -0500, Joel Sherrill wrote: > Based on the grep, the .4byte directives are referencing a bogus symbol. > > Does this look like a GCC bug? Yes, unless you have some horrible asm there referencing the symbol. -- Alan Modra Australia Development Lab, IBM
[RFC] Documenting support functions and data structures
Hi, all! I think GCC documentation is missing a description of commonly used data structures and support functions. For example, "GCC internals" document contains information about, say, GIMPLE IR and functions for manipulating it. But it does not describe general data structures, such as hash maps or vectors. In fact, these structures are documented rather well in the source code, but I could not find a list of them. I mean something like this: http://llvm.org/docs/ProgrammersManual.html#important-and-useful-llvm-apis So, I want to create a similar page in GCC's internal docs, but I don't know what should be included (i.e. did I miss something important, or did I include something obsolete), so I ask for some assistance. Here is a draft (a list of relevant files from ./gcc, ./include and ./libiberty directories). This list does not yet include functions for interacting with host OS (it's a separate issue) and does not include compiler-specific structures (like, say, control flow graph or varpool) and algorithms. Ordering is somewhat random. === Memory management === gcc/alloc-pool.c - pool allocator gcc/ggc* - mark-and-sweep garbage collector (includes sources of gengtype, a program which generates traversal routines from annotated C++ code) gcc/ggc-page.c - size-class based allocator (used by garbage collector). (?? isn't it the same as "slab allocator") gcc/hash-table.h - contains some simple allocator templates libiberty/xmalloc.c - wrappers for malloc and friends with checking === Data structures === gcc/bitmap.c - sparse integer set (based on linked lists of chunks) gcc/sbitmap.c - "simple bitmap" - vector-based bitmap gcc/et-forest.c - ET-forest data structure (used for efficient solution of dynamic least common ancestor problem) gcc/fibonacci_heap.h (header-only) - mergeable priority queues gcc/graphds.c - graph representation and manipulation functions gcc/hash-table.c - hash table implementation (base for hash-set and hash-map) gcc/hash-map.h (header-only) - hash map gcc/hash-set.h (header-only) - hash set gcc/sparse-set.c - another sparse set implementation (I think, I'll make a table, with comparison of different implementations w.r.t. size, available operations and their run time) gcc/vec.c - vector template, implements several different memory allocation strategies include/obstack.h - object stack libiberty/splay-tree.c - splay tree === RTTI, type manipulation === gcc/is-a.h (header-only) - customizable low-overhead RTTI and dynamic casts gcc/system.h (header-only) - several implementations of const_cast === Algorithms === libiberty/md5.c, sha1.c - corresponding hash functions libiberty/crc32.c - CRC32 checksum (BTW, why is bitwise implementation from tree.c used everywhere instead of table algorithm?) gcc/inchash.c - incremental hashing libiberty/bsearch.c - binary search (and other stuff reimplementing libc functions - is this a workaround for broken system libc?) === Multiprecision arithmetics === gcc/hwint.c - "host wide int" - macros for representing host system's int64_t/uint64_t in a portable way and some related functions gcc/wide-int.cc, namespace hw - generic finite precision integers (including variable-precision and target-dependent precision integers + traits) gcc/double-int.c - 128-bit fixed precision integer type gcc/fixed-value.c - fixed precision real arithmetics, uses target-dependent representation gcc/real.c - floating-point arithmetics, including target-dependent representation gcc/mpfr.c - conversion routines from GCC internal float representation to MPFR gcc/sreal.c - simple software floating-point arithmetics === Debugging, statistics === gcc/dbgcnt.c - debug counters gcc/errors.c - diagnostics for internal use (in generator programs) gcc/statistics.c - arbitrary statistics counters and histograms gcc/timevar.c - timing variables and timing stacks gcc/system.h (header) - assertions, static_assert === Text manipulation === gcc/pretty-print.c - text formatting and output libiberty/concat.c - multiple string concatenation libiberty/regex.c - regular expressions === Serialization === gcc/data-streamer*.c - general marshalling routines gcc/ggc* - marshalling routines, which use code generated by gengtype; used for precompiled headers I'm asking for advice on correct categorization, missing stuff, etc. -- Regards, Mikhail Maltsev
Re: is it time to mass change from .c to .cc in gcc/ ?
On 15.04.2015 19:21, Manuel López-Ibáñez wrote: > The "right" steps to see progress: > > 1. You think that some concrete change may improve GCC, for example, to > move vec.h and is-a.h to some gcc/include/core. Ask if the key people > (Global Reviewers in this case) will accept this change. <...> > 2.1 If not, then goto 1 and think about something else (or perhaps > people misunderstood what you meant? Well, sort of :). I did not mean that I want to try to design the structure for GCC source code tree. I was talking about documentation issues only. > Just do it. Hmm, indeed... > Sent a patch for gccint.texi or create a webpage in the wiki > to draft it or just draft it in plain-text and send it to the mailing > list. Started with the latter. By the way, what is the policy concerning getting write access to the wiki? Regards, Mikhail Maltsev
