On Fri, 2016-09-16 at 14:00 -0600, Jeff Law wrote: > On 09/14/2016 02:37 AM, Richard Biener wrote: > > > > Note that while the "snippets" may largely work (not sure how many > > you tried to come up with) I see the issue that a lot of RTL "unit > > tests" would need some trees set up, like to properly form > > MEM_EXPRs > > or REG_DECL or even SYMBOL_REFs. So I fear that the scope of > > unit-tests we can implement with the proposed scheme is very > > limited > > (you may also need other stuff setup, like alias analysis or parts > > of > > IRA or cost analysis parts). So I agree a separate testing backend > > is a good way to make unit-testing more stable on the target side > > we > > also need a way to provide input on some of the global state that > > is > > currently set up by frontends. > Agreed across the board. I think we're still early in the learning > phase on this stuff. I shudder when I think about the amount of > state > that we depend on, but which is not represented in the RTL dumps. > > But I do think there are some things we can test for in an RTL self > testing framework and that having one would be a significant step > forward. > > So I think we have two big questions. > > First, does David's work have value as a way to directly test pieces > of > the RTL pipeline without having to generate all the RTL bits by hand. > I > think the answer is yes. > > Second, will David's work help identify internal state that is not > expressed in the RTL dumps or poor modularity (ie, cases like trees > embedded within the RTL structures). I think the answer to this is > yes > as well. > > Third, is a framework like Bernd's useful as well and can it be mated > with David's work. And I think the answer is yes & yes as well with > the > result being more useful than either Bernd or David's work in > isolation.
As far as I understand Bernd's suggestion there seem to be two parts to it: (a) a kind of virtual target, which can act in different ways depending on the needs of a test case. e.g. dynamically select 32 bit vs 64 bit, does it have a CCmode, how many stages is the pipeline etc etc. (b) a separate build of the "gcc" subdir, configured to use (a). These seem like laudable goal, but I see it as orthogonal to my patch kit. It'd also be a massive expansion of scope. Also, re (a) any given test is likely to be tested against a specific target. That could be a real target, or a particular setting of a virtual target. The tests in my patch kit have largely gated on the specific targets I was testing with. So is Bernd's suggestion a prerequisite for my work, or can my work stand alone from it? > > > > But my biggest worry is with putting unit-tests into cc1 itself -- > > even more so with RTL unit tests of this kind than with all the > > other > > ones we have. We'll quickly have 99% of a source file comprised of > > RTL unit tests rather than source (and cc1 object size as well). > > Hardly something we want to have (not even mentioning bootstrap > > time > > issues). > I can live revisiting this -- I always expected we would after we > started building out the framework. There are some interrelated questions here: (a) where do the dumps live? (string fragments embedded in the source file vs external files) (b) -fself-test vs DejaGnu tests that use a real frontend. In the latter case, is the frontend "rtl1", or an extension of "cc1" with an "__RTL" marker? For (a), I'd like to do support both (in that it's clear we need support for external files, but it seems trivial to support embedding). I've worked with both approaches in the past, and both are useful (a two-liner may make sense to live "inline", as they get larger you'd want them in a separate file). For (b), I'd like to do both: if nothing else, the loader itself needs selftests. Plus selftests allow for tests that are more fine-grained than just the level of one optimization pass. But I'd anticipate most of them being external. For selftests that load external files, there's a slight wart - how are they accessed? There needs to be some way to tell it which directory to look in. Also, what happens when build != host? I have some followup patches that both build out an actual frontend on top of the loader, and extend cc1, but there's some ugly diagnostic and linemap issues to resolve. > > > > Yes, putting the unit-tests in source files makes us not require > > exporting an interface to the parts we are testing. But that's > > about > > the only advantage I can see. You didn't show that it isn't > > possible > > to put the small test you were writing into a RTL-frontendish test > > which starts compiling the function with the test with the pass you > > are about to unit-test. > The other advantage is tests which use the internal APIs are easy to > identify/fix when an internal API is changed. > > Jeff >
