On May 12, 2009, at 11:05 AM, Vladimir Makarov wrote:
Chris Lattner wrote:
On May 12, 2009, at 6:56 AM, Vladimir Makarov wrote:
A few people asked me to do a new comparison of GCC releases and
LLVM as the new GCC release and LLVM were out recently.
You can find the comparison on http://vmakarov.fedorapeople.org/
spec/
The comparison for x86 (32-bit mode) was done on Pentium4 and for
x86_64 (64-bit mode) on Core I7.
Some changes in the performance were big since GCC 3.2 and it is
sometimes hard to see small changes on the posted graphs.
Therefore I put original tables used to generate the graphs.
Looking at the llvm 2.5 vs gcc 4.4 comparison is very interesting,
thank you for putting this together Vladimir! I find these numbers
particularly interesting because you're comparing simple options
like -O2 and -O3 instead of the crazy spec tuning mix :). This is
much more likely to be representative of what real users will get
on their apps.
Some random thoughts:
1. I have a hard time understanding the code size numbers. Does
10% mean that GCC is generating 10% bigger or 10% smaller code than
llvm?
The change is reported relative to LLVM. So 10% means that GCC
generates 10% bigger code than LLVM and -10% means that GCC
generates 10% less code.
Ok! It is interesting that GCC seems to generate consistently larger
code at both -O2 and -O3 in x86-64 mode (over 20% larger in -O3).
Perhaps that also is impacting the compile time numbers as well.
2. You change two variables in your configurations: micro
architecture and pointer size. Would you be willing to run x86-32
Core i7 numbers as well? LLVM in particular is completely untuned
for the (really old and quirky) "netburst" architecture, but I'm
interested to see how it runs for you on more modern Core i7 or
Core2 processors in 32-bit mode.
I used the same processor (P4) and options for x86 as for the GCC
release comparison. I did not know that LLVM is badly tuned for P4,
sorry. I could do the same comparison for x86 on Core i7 without
specific tuning (there is no tuning for i7 yet) but it takes a lot
of time. May be it will be ready on next week.
No problem at all, I appreciate you running the numbers!
It would also be very interesting to include LLVM's LTO support, which
gives a pretty dramatic win on SPEC. However, I don't know how
difficult it is to use on linux (on the mac, you just pass -O4 at
compile time, and everything works). I've heard that Gold has a new
plugin to make LTO transparent on linux as well, but I have no
experience with it, and it is probably more trouble than you want to
take. Does gcc 4.4 include the LTO branch yet?
-Chris
3. Your SPEC FP benchmarks tell me two things: GCC 4.4's fortran
support is dramatically better than 4.2's (which llvm 2.5 uses),
and your art/mgrid hacks apparently do great stuff :).
4. Your SPEC INT numbers are more interesting to me. It looks like
you guys have some significant wins in 175.vpr, 197.crafty, and
other benchmarks. At some point, I'll have to see what you guys
are doing :)
Thanks for the info, great stuff!
-Chris
