Hi This is a general question to all you working with GCC benchmarking.
I have been working with code benchmarks like CSiBE for ARM. >From time to time unpredicted results appears where numbers gets worse by no >reason. When looking into what could cause this unpredictable behaviour, I found that there are (at least for CSiBE), tons of code warnings that could cause unpredictable code generation like -Wuninitialized and -Wmaybe-uninitialized. Alot of these warning indicates real bugs. I added this issue in bugzilla, #Bug 85880 - "Different code generation for uninitialized variables" Though it got (correctly) rejected, since its not a bug. But still I'm thinking how this apply to benchmarking code, and to how to approach and address this fact. So my question is how to approach this problems when doing benchmarking, ofcourse we want the benchmark to mirror as near as 'real life' code as possible. But if code contains real bugs, and issues that could cause unpredictable code generation, should such code be banned from benchmarking, since results might be misleading? On the other hand, the compiler should generate best code for any input? What do you think, should benchmarking code not being allowed to have eg warnings like -Wuninitialized and maybe -Wmaybe-uninitialized? Are there more warnings that indicate unpredictable code generations due to bad code, or are the root cause that these are 'bugs', and we should not allow real bugs at all in benchmarking code? I've read some about uninitialized variable issues also at this link which was interesting, its a wider discussion, and ofcourse a very hard problem to solve. https://gcc.gnu.org/wiki/Better_Uninitialized_Warnings Thanks, Best Regards, Fredrik
