Andrew Hutchinson wrote:
Here my input:
For starters gcc has testsuite that can be used. It's not perfect but
its quite demanding - even if we cant do all the tests.
Yes, we could probably pull a subset of meaningful test from that which
would give us a great start. The hard/tedious part will be generating
our expected results.
After than I suggest some "benchmark" that would produce more normal
code and also give qualitative indications of performance (size is easy,
speed would be nice).
Finally, regression tests using testcases and bug reports.
Performance regressions are particularly nasty to test. First you have
to have code that can sensitize an optimization that you want. Then,
you need to have some expected results. The compare of actual to
expected is difficult -- what are you measuring? And how close is good
enough? Exactly clock count? Clock count within a guard band? A
specific assembler sequence that must happen (but actual registers used
don't matter)? Some registers matter?
Then you have the test matrix... what is the expected result with -Os
versus -O3? etc, etc, etc.
All this presumes a test framework that can capture all the required
outputs to check against expected results: .S, clock count, code size,
exit code,.... Oh, and getting the right numerical (or whatever)
answer... and are we getting it for the right reason? The compiler
might show a great speed and code size improvement by mistakenly
ignoring the volatile keyword in some optimization, and a simulator
might never show a wrong answer :(
Having been purely a gcc and avr-gcc user, I don't have any idea if any
parts of this framework already exist in a form that addresses these
problems.
Anyway, the test matrix is huge, and a simulator is going to limit the
ultimate performance of the test rig. So we won't be able to do
everything we can think of.
On the plus side of the ledger: Our audience, being embedded developers,
have very specific needs, so that can help prioritize.
OK, this e-mail was a shotgun blast of issues completely devoid of
specifics :) :)
Andy
-dave
Dave N6NZ wrote:
Well, after spamming the wrong list with this, I hope I've got the
right place now :(
-dave
------------------------------------------------------------------------
Subject:
Re: [avr-gcc-list] GCC-AVR Register optimisations
From:
Dave N6NZ <[EMAIL PROTECTED]>
Date:
Thu, 10 Jan 2008 12:35:10 -0800
To:
gEDA user mailing list <[EMAIL PROTECTED]>
To:
gEDA user mailing list <[EMAIL PROTECTED]>
Weddington, Eric wrote:
- I, and others, are very interested in what you are using to test your
proposed changes. I have plans to put together an AVR Benchmark Suite,
consisting of a variety of publicly available programs that can be used
to test the compiler performance over time. It definitely needs to have
different types of programs, and publicly available programs so there
are no issues with distributing such a Benchmark Suite. I welcome any
collaboration on this.
Hi,
Over the course of years I have worked in and managed various
different hardware and software validation teams. This is probably a
good way for me to contribute, since I'm not a compiler back-end guru,
but I have relevant experience writing and reviewing test plans. Some
of those test plans even targeted C compilers :)
-dave
------------------------------------------------------------------------
_______________________________________________
AVR-GCC-list mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list
_______________________________________________
AVR-GCC-list mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list
