MetaC++ announcement
I've written a library which isn't exactly on-topic of gcc development,
but is gcc related and there have been multiple requests for a gcc
feature like this on the list(-fdump-translation-unit...), so here is
some information about it:
What?
-
MetaC++
It is a library which is able to read and write C++ source code and
makes the tree available to its clients by API and by a XML format it
can read and write.
Parsing is done by a patched GCC.
The tree is a representation of language constructs, not a syntax tree.
Why?
-
It's primarily aimed at source-to-source code translators, but can also
be used by source code analysis tools, stub generators, source-to-UML
converters, etc.
By using GCC as parser it has good C++ language support.
Where?
-
http://www-user.uni-bremen.de/~strasser/metacpp/
Language Support? Status?
-
"Alpha" status.
Full C++ should work, including templates. It has been tested with C++
STL and parts of boost.
Statements, i.e. function bodies, are ignored in this version, so no
full translation yet, but it's already usable for header translation and
source code analysis which don't need bodies.
The most common expressions for initializers and template specialization
arguments are supported. Others are still ignored, and will be supported
in the next version along with function bodies.
Rewriting of boost is not yet working completely because of advanced
initializer expressions to constants which are then used as
instantiation arguments, reading works.
(Library is still leaking a few hundred kilobytes each gcc invocation,
I'd appreciate help with this, see previous mail on the list)
API Documentation?
-
UML Diagrams:
http://www-user.uni-bremen.de/~strasser/metacpp/uml.pdf
XML Example?
-
C++ std string header as gzipped XML:
http://www-user.uni-bremen.de/~strasser/metacpp/string.xml.gz
There are many other things included from this header, one of the last
TemplateClassDefinitions inside NamespaceDefinition "std" is
"std::basic_string". (Id 10339)
Generated output:
http://www-user.uni-bremen.de/~strasser/metacpp/string.cpp
Code Example?
-
Virtually inherit all classes from class "base":
object base;
//find or create "base"
object tu;
Code::InputFile input("in.cpp");
input >> tu;
TranslationUnit::tree_iterator
it=tu->find_recursive();
for(;it != tu->end();++it){
object baseSpec=BaseSpecifier::New();
baseSpec->SetClass(base);
baseSpec->SetVirtual();
it->GetBases().push_back(baseSpec,*it);
}
Code::OutputFile output("out.cpp");
output << tu;
You never want to change all elements, including STL etc.!
-
Yes. There will be some mechanism to mark elements with attributes when
it is source-to-source ready(see "Language Support"), something like:
persistent class MyDatabaseObject;
distribute void function();
For now you can decide this only by language constructs(its name, what
namespace it is in, etc.)
XML format?
-
You can use the library for reading and writing. However, the xml format
is automatically derived from the object model,data field "isVirtual"
in class "BaseSpecifier" is called "BaseSpecifier.IsVirtual" in xml
files. Normal fields and pointer<>'s are attributes, element<>'s and
list<>'s are subnodes.
--
Stefan Strasser
Re: MetaC++ announcement
Florian Weimer schrieb: The tree is a representation of language constructs, not a syntax tree. Does it include representation information, e.g. offsets of class members? Unfortunately, the XML sample is too unwieldy to tell. 8-/ it does preserve order of members, so processed source is compatible with unprocessed source. it does not read any variable alignment information(except c++ bitfield size) and I don't see a reason to do that. but in case you need that it can be added quite easily, search for "Parse(object -- Stefan Strasser
Re: GCC3 to GCC4 performance regression. Bug?
Steve Ellcey schrieb:
Test Case ---
I think is the same bug(which was not considered one back then) as
benjamin redelings described in the thread "C++ math optimization
problem...".
there are again unnecessary memory accesses as if the memory were
volatile, which could be moved out of the inner loop.
and gcc 3.4 does it in this case(there are other cases where both fail,
see the math optimization thread)
and once again changes which shouldn't have any effect enormously affect
the inner loop.
#define L_CONST 500
void *malloc(long size);
struct plan7_s {
int M;
int **tsc; /* transition scores [0.6][1.M-1]*/
};
struct dpmatrix_s {
int **mmx;
};
struct dpmatrix_s *mx;
void
AllocPlan7Body(struct plan7_s *hmm, int M)
{
int i;
hmm->tsc= malloc (7 * sizeof(int *));
hmm->tsc[0] = malloc ((M+16) * sizeof(int));
mx->mmx = (int **) malloc(sizeof(int *) * (L_CONST+1));
for (i = 0; i <= L_CONST; i++) {
mx->mmx[i] = malloc (M+2+16);
}
return;
}
void
P7Viterbi(int L, int M, struct plan7_s *hmm, int **mmx)
{
int i,k;
for (i = 1; i <= L; i++) {
for (k = 1; k <= M; k++) {
mmx[i][k] = mmx[i-1][k-1] + hmm->tsc[0][k-1];
}
}
}
main ()
{
struct plan7_s *hmm;
char dsq[L_CONST];
int i;
hmm = (struct plan7_s *) malloc (sizeof (struct plan7_s));
mx = (struct dpmatrix_s *) malloc (sizeof (struct dpmatrix_s));
AllocPlan7Body(hmm, 10);
for (i = 0; i < 60; i++) {
P7Viterbi(500, 10, hmm, mx->mmx);
}
}
--
Stefan Strasser
inefficient code output?
is there a reason for code output like the following or is this a bug? if it is I can try to provide a simple example. movl %ebx, -200(%ebp) movl %ebx, -196(%ebp) movl %eax, 4(%esp) movl -200(%ebp), %edx movl -196(%ebp), %ecx gcc 3.4.3, -O3 -march=pentium4 -mtune=pentium4 -mfpmatch=sse -msse2 no "volatile" involved. -- Stefan Strasser
Re: i want to connect gcc's front-end to my'back-end
하태준 schrieb: > i want to connect gcc's front-end to my'back-end > > how to connect it? > > first i want to connect gcc's tree IR and my back-end's IR > > but i can't > > then i try to connect gcc's RTL that was made -dr option > > but that is impossible > > so, i try to use gcc 4.0 > > i want to use gcc's gimple tree (that is gcc's IR) > > how to i connect gcc's gimple to my IR > > i want to print gimple tree! > > -fdump-tree-gimple option's output is not a Intermediate Representation > > that is not a tree! > > help me... why don't you explain further what exactly you'd like to do. if you want to use gcc as parser only to your compiler look here: http://gcc.gnu.org/onlinedocs/gccint/Trees.html (and note that your code will have to be GPL conforming.) if you need to get an external representation of your source code look here: http://www-user.zfn.uni-bremen.de/~strasser/metacpp/ (function bodies and thus complete source code is almost done and will be on the website soon) another project with different design decisions is: http://www.gccxml.org if you want to port gcc to another CPU or similar I'm sure there is some backend-howto on gcc website, but gimple trees are not the right way. there are more than one IRs in gcc so you need to be more clear. and note that gcc debugging outputs are not designed to be processed. -- Stefan Strasser
Re: Use Bohem's GC for compiler proper in 4.1?
Andrew Haley schrieb: Sam Lauber writes: > I know that Bohem's GC is used in the Java runtime for GCC. > However, the compiler proper itself can _really_ cramp people's > avalible RAM (for those who don't belive me and have Windows w/ > DJGPP, change all the memory controls from `auto' to the highest > value and just try to compile libiberty/regex.c), so my suggestion > is usage of Bohem's GC in the compiler proper itself. Do you have any reason to believe that such a change would reduce memory consumption? I have reason to believe that it would increase performance on low memory systems/large translation units: it seems that current gcc gc performs a full memory scan on each collection, right? at least if gcc uses more memory than physically available it spends a _very_ long time swapping during collections. boehm gc could help here since it is a generational collector. -- Stefan Strasser
Re: i want to connect gcc's front-end to my'back-end
a) what makes your backend different from gcc's backend so it is worth porting it? 하태준 schrieb: > sorry, my english is not good, > > Umm... > > my project is that Connect to Gcc's front-end and My back-end > > first gcc parse sorce code there is an IR created while parsing which is simply called "Trees" and is documented here: http://gcc.gnu.org/onlinedocs/gccint/Trees.html#Trees this is probably what you want. start gdb debugger, break at function cp_finish_file and write "display debug_tree(global_namespace)". you should see a dump of the global namespace. > my back-end has own IR that's based on C++. > > and, my back-end has connect EDG's Front-end > > EDG's Front-end parse sorce program and make EDG's IR > > and EDG's IR translate our IR and our IR make a assemble code > > my project is change EDG's front-end to GCC's front-end because GPL expect some work to do. gcc does some transformations while parsing already, e.g. when calling virtual functions -- Stefan Strasser
Re: Use Bohem's GC for compiler proper in 4.1?
Mike Stump schrieb: On Friday, April 1, 2005, at 08:48 AM, Stefan Strasser wrote: if gcc uses more memory than physically available it spends a _very_ long time swapping Swapping, what's that? Here's $20, go buy a gigabyte. expect memory to become a problem again with the advent of multicore and people wanting to run more than one gcc at a time. I do have swapping on a 1 GB machine with 2 CPUs(-> 2 GCCs) but that wasn't my argument, swapping was just what made me assume that gcc gc does a full memory scan on each collection. non-generational gc also performs bad without swapping on large heaps. but I understand the objection that you want to keep exact GC(although I'm not enough into gc to decide if it's worth it). -- Stefan Strasser
Re: Use Bohem's GC for compiler proper in 4.1?
Kaveh R. Ghazi schrieb: > I do have swapping on a 1 GB machine with 2 CPUs(-> 2 GCCs) If you can demonstrate that say GCC-4.0 uses much more memory than 3.4 or 3.3 to compile some code, then file a PR with preprocessed source and someone will eventually look at it. I haven't thought about a regression, I was only using 3.4, but look at this: mem peakuser sys gcc 3.4 -S -O0 476 MB1m39s 2s gcc 4.0 -S -O0 655 MB2m23s 3s icc -S -O0 264 MB1m24s 15s the file makes quite heavy use of virtual inheritance so there are a lot of virtual tables involved. are there any known performance bugs in this area or should I file a PR? any suggestions on how to simplify the testcase? (preprocessed is ~60k lines) -- Stefan Strasser
Re: Use Bohem's GC for compiler proper in 4.1?
Kaveh R. Ghazi schrieb: I'm curious what the 3.3 numbers are, 3.3 => 4.0 is a small improvement cpu-wise(not mem-wise). 3.4 is much better than both: gcc-Version 3.3.5 (Debian 1:3.3.5-8) mem 426 mb user2m10.870s sys 0m2.250s gcc-Version 3.4.4 20041218 (prerelease) (Debian 3.4.3-6) mem 387 mb user1m4.920s sys 0m1.210s gcc-Version 4.0.0 20041218 (experimental) mem 581 mb user2m0.800s sys 0m2.720s (it's another file than yesterday, because on this one 3.4 -> 4.0 is more extreme) The regression in 4.0 is pretty bad, definitely file a PR. http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20733 -- Stefan Strasser
Re: Use Bohem's GC for compiler proper in 4.1?
Andrew Pinski schrieb: On Apr 2, 2005, at 6:12 PM, Stefan Strasser wrote: gcc-Version 4.0.0 20041218 (experimental) this 4.0.0 is almost 4 months old. That is not a far comparison as there was speedups after that and other bug fixes. you're right, I wasn't prepared to do performance tests: gcc-Version 3.4.4 20041218 (prerelease) (Debian 3.4.3-6) user0m57.250s sys 0m1.050s gcc-Version 4.0.0 20041218 (experimental) user1m53.810s sys 0m2.850s gcc-Version 4.0.0 20050402 (prerelease) user0m40.290s sys 0m1.070s impressing. thank you very much to whoever fixed this. mem usage 3.4/current 4.0 is about equal. intel: user0m42.080s sys 0m9.040s (slightly different file due to PR20734) -- Stefan Strasser
Re: GCC 4.0 Status Report (2005-04-05)
Mark Mitchell schrieb: Andrew Pinski wrote: On Apr 4, 2005, at 7:26 PM, Mark Mitchell wrote: 20734 rejects valid pointer to member Not yet assigned. How is this less Critical? This would breaks lots of code, it is template related too as it is not rejected when not in templates. Clearly this is a judgement call. However, I think it's less critical because pointers-to-members and, in particular, pointers-to-members in templates are relatively less common. And, it's rejects-valid, so at least you know that you have a problem. And there are workarounds. In contrast, the top bug on the list is known to cause silent miscompilation of Qt, a very major package, and it's not easy to know where the problem is. what is the workaround to this? (in the case the class the pointer target is a member of is a template parameter, unlike in the PR test case) -- Stefan Strasser
Re: Questions about a constant array reference in the C++ front end
Kazu Hirata schrieb: I am trying to fold array[7] into 2. It turns out that if T is an ARRAY_REF, TREE_READONLY (TREE_OPERAND (T, 0)) is 0. Why? I don't know anything about fold but in general a c++ array in the frontend is cv-qualified, not its elements. Another question. How is a RANGE_EXPR used in a C++'s array constructor? The CONSTRUCTOR section of tree.def says I created an array with more than one thousand elements. I still did not see a RANGE_EXPR in the array's CONSTRUCTOR. How do I get a RANGE_EXPR in a CONSTRUCTOR? I have been processing large source codes including STL, boost and custom code including function bodies and I have never seen a RANGE_EXPR. I suppose it's only used at later stages or only in other language's frontends. Regards, -- Stefan Strasser
Re: Questions about a constant array reference in the C++ front end
Nathan Sidwell schrieb:
Stefan Strasser wrote:
I don't know anything about fold but in general a c++ array in the
frontend is cv-qualified, not its elements.
this is untrue. the elements hold the qualification.
right
I have been processing large source codes including STL, boost and
custom code including function bodies and I have never seen a
RANGE_EXPR. I suppose it's only used at later stages or only in other
language's frontends.
Incorrect. RANGE_EXPRs get generated during processing of an array's
initializer -- very early on in the C++ FE.
as said there have not been one case in large sources.
I can't recall exactly. It might be for default initialization.
Something like
ptr = new int[100] ();
...does not initialize anything
int ar[]={0,0,0,0}; does, but not with RANGE_EXPR
--
Stefan Strasser
Re: C++ math optimization problem...
I think it is a bug, or a "missing feature".
I tried to simpify the testcase below and ended up with a comlete
different testcase, but it causes the same problem:
it seems to be about FPU registers, if anything causes the compiler to
store the value to memory, it treats it as it would be volatile.
void otherfunc();
void test(){
double result=0.0; //stored in fpu register
otherfunc();//"result" saved to memory
for (int j = 1; j < 1; ++j) result += 1.0; //"result" read
and written to/from memory each cycle
std::cerr << result << std::endl;
}
on i386 the inner loops look like this:
without otherfunc():
.L7:
decl %eax
fadd %st, %st(1)
jns .L7
with otherfunc():
.L7:
fldl -8(%ebp)
decl %eax
fadd %st(1), %st //FPU stack ordering problem?
fstl -8(%ebp)
js .L11
fstp %st(0)
jmp .L7
.L11:
I don't really know what keywords to search for in bugzilla, could
anyone please look up if this is a known bug?
Mfg,
Benjamin Redelings I schrieb:
Hi,
I have a C++ program that runs slower under 4.0 CVS than 3.4. So, I
am trying to make some test-cases that might help deduce the reason.
However, when I reduced this testcase sufficiently, it began behaving
badly under BOTH 3.4 and 4.0 but I guess I should start with the
most reduced case first.
Basically, the code just does a lot of multiplies and adds.
However, if I take the main loop outside of an if-block, it goes 5x
faster. Also, if I implement an array as 'double*' instead of
'vector' it also goes 5x faster. Using valarray instead
of vector does not give any improvement.
MATH INSIDE IF-BLOCK
% time ./2h 1
double addition
result = 83283300.006041
real0m0.995s
user0m1.000s
sys 0m0.000s
MATH OUTSIDE IF-BLOCK
% time ./2i 1
result = 83283299.98
real0m0.218s
user0m0.220s
sys 0m0.000s
Should I submit a PR? Any help would be appreciated...
-BenRI
begin testcase -
#include
const int OUTER = 10;
const int INNER = 1000;
using namespace std;
int main(int argn, char *argv[])
{
int s = atoi(argv[1]);
double result;
if (s == 1) { //remove this condition to get a 5x speedup
// initialize d
vector d(INNER); //change to double* to get 5x speedup
for (int i = 0; i < INNER; i++)
d[i] = double(1+i) / INNER;
// calc result
result=0;
for (int i = 0; i < OUTER; ++i)
for (int j = 1; j < INNER; ++j)
result += d[j]*d[j-1] + d[j-1];
}
else
exit(-1);
printf("result = %f\n",result);
return 0;
}
--- end testcase --
--
Stefan Strasser
Re: C++ math optimization problem...
Andrew Pinski schrieb: This is a target bug as it does not effect any reasonable processor. With -mfpmath=sse -msse2 I get: .L2: decl%eax addsd %xmm1, %xmm0 jne .L2 my example was about version 3.4.4, which still has this problem with sse options: .L5: movsd -8(%ebp), %xmm1 decl %eax addsd %xmm0, %xmm1 movsd %xmm1, -8(%ebp) jns .L5 you're right with 4.0 about my example. but the testcase by benjamin still has this problem, with 4.0, with sse: the inner loop: .L126: incl %eax movsd -8(%edx), %xmm0 movsd (%edx), %xmm1 addl $8, %edx cmpl $1000, %eax mulsd %xmm0, %xmm1 addsd %xmm1, %xmm0 addsd -48(%ebp), %xmm0 movsd %xmm0, -48(%ebp) jne .L126 inner loop with one of the changes benjamin suggested, which shouldn't have any effect: .L124: incl %eax movsd -8(%edx), %xmm0 movsd (%edx), %xmm1 addl $8, %edx cmpl $1000, %eax mulsd %xmm0, %xmm1 addsd %xmm1, %xmm0 addsd %xmm0, %xmm2 jne .L124 -- Stefan Strasser
gcc leaking?
are there any allocation schemes besides garbage collection in gcc which preserve some memory for reuse which could cause memory leaks if not cleaned up, or are these bugs? (which don#t matter in the normal compilation process of course) I'm using gcc as a library and experiencing memory leaks. I need a shared address space with gcc so invoking gcc is not an option. the leaks add up, because I need to reload gcc shared library since it's not safe to call gcc twice. I loose about 500kb a compilation. is there anything besides garbage collection I can free before unloading? (gc pages are released). Thanks, -- Stefan Strasser
Re: gcc leaking?
Tommy Vercetti schrieb: I don't know what's "refrubish rate" of gc, but I would say that any garbage collector is a pretty much cause of solid leak of memory (unless it frees memory when not used anymore, but I doubt they do). gcc gc does free memory when it has not been used in the last 2 collections. on a normal termination there are still gc roots so there are still pages allocated, but I've done a collection with no roots and GC says 0k allocated, and there's still a leak. it must come from another part of gcc. there is a pool allocator, but it is not used at all(at least when compiling c++). would it help to do leak checking on libiberty alloc functions or is than done regularily anyway? -- Stefan Strasser
