On July 3, 2014 10:47:52 PM CEST, Mark Wielaard <m...@redhat.com> wrote: >On Thu, 2014-07-03 at 22:14 +0200, Jakub Jelinek wrote: >> On Thu, Jul 03, 2014 at 10:04:35PM +0200, Mark Wielaard wrote: >> > On Thu, 2014-07-03 at 21:52 +0200, Richard Biener wrote: >> > > On July 3, 2014 8:38:14 PM CEST, Jakub Jelinek <ja...@redhat.com> >wrote: >> > > >On Thu, Jul 03, 2014 at 08:37:07PM +0200, Richard Biener wrote: >> > > >> Well, simply removing the regression testing for LTO is a >> > > >maintainance nightmare as well. >> > > >> >> > > >> The guality testsuite is very noisy anyway with all the xfail >and >> > > >xpass. >> > > > >> > > >Let's keep it as is then? >> > > >> > > That works for me. >> > >> > I don't find that very satisfactory. I want to add more guality >tests, >> > but the fact that they are unreliable and by default introduce even >more >> > FAILs when lto is enabled makes that not very attractive. I do like >> > Jakub's suggestion to disable the guality tests be run with lto by >> > default, but provide an environment variable to enable them for >those >> > that want to try them anyway. Shall I implement that? >> >> They aren't that unrealiable (at least, if people committing patches >don't >> ignore regressions in there). Just one should diff >contrib/test_summary >> output from earlier builds to the latest, that way it is clear what >is a >> regression and what is not. > >The are much more unreliable than any other test. With guality.exp >disabled one can just eyeball the results and investigate new FAILS. >There are only a handful. When you include guality.exp you can easily >get the impression the gcc testsuite is really bad (and it isn't!) And >the problem is that it makes adding new tests a pain. See my new tests, >they introduce new FAILs because LTO is enabled by default for >guality.exp at the moment. It just results in a slow increase of FAILs >that people have to ignore. And I am afraid that will just result in >people missing real regressions. > >I don't mind if there is active work to fix LTO DWARF debuginfo >generation issues and the guality.exp LTO failures will soon disappear, >but if there is no active work on reducing the amount of failures and >introducing new guality.exp testcases will keep adding more FAILs I >think we are much better off disabling them for now.
Can't you simply add proper xfails for lto when you add new tests that fail with lto? (Please quickly check why - usually the tests are just optimized in an unexpected way - compare with -fwhole-program behavior for example). BTW, reducing the number of lto variants checked to -O2 -flto would be fine with me. Richard. >Thanks, > >Mark
