On Wednesday 20 July 2011, Ralf Wildenhues wrote: > * Stefano Lattarini wrote on Wed, Jul 20, 2011 at 10:05:14PM CEST: > > Hi Ralf, thanks for the tips. > > Rather, 5 minutes of searching the web. But sure, anytime. ;-) > > > On Tuesday 19 July 2011, Ralf Wildenhues wrote: > > > Test suite environments: > > > any that use the above > > > > > I see now that prove(1), the default command line interface to TAP::* > > modules, has a `--state' option that allow it to store the results of > > the previous testsuite run. But the exact way this is done seems to > > be an internal detail; should I look into this nonetheless? > > Why do you need to ask? Wasn't one of the first parts of this project > looking into how other test environments work? > Yes, but had overlooked and/or glissed over this feature of prove because it didn't seem relevant at the time. And similarly for other features too BTW, such as `--archive', `--shuffle', `--recurse', passing arguments to tests using `::' ... And similarly for other programs and testing environments.
> > > dejagnu > > > > > After having read the blog post you link below, I'd rather not look into > > this at all ... > > Learning from real-world software, including mistakes, can be a lot more > enlightening than from shiny but unused software. > > > > qmtest > > > > > Just as an aside: the documentation of this tool seem to employ the same > > terminology we are using in automake w.r.t. test results (in particular, > > it uses "FAIL" and "ERROR" in the same way we do, making clear distinction > > between the two); so we've probably managed to avoid NIH in this respect. > > No. Avoiding NIH means not doing your own thing in the first place, > but doing the research first, and then acting intelligently afterwards. > I think I've already pointed out that, for the distinction between FAILs and ERRORs, I took inspiration from the Python and xUnitw worlds (and to a lesser degree from TAP). > > > http://en.wikipedia.org/wiki/Portal:Software_Testing lists several > > There are probably a dozen more in this web page alone. I don't think > it is asked too much to at least look around in them in search for > valuable data for all kinds of things (future features you don't want to > prevent being possible by wrong design decisions now). Even much more > generally than just for the result format. > > Please take this as an opportunity, not as a burden. > I'll take it as both ;-) > Sometimes copying > ideas is just the right, the best thing. And learning from others > almost always is. > The tone of you reply until here suggests that I've failed to make one important thing about my previous answer clear: it was *absolutely not* meant to be a definitive answer! Of course I'll look at the pointers you've provided (or at least some of them), and act accordingly to the findings. But I still think my patch should be applied, and then improved or reworked later to better integrate with such "research findings". > > > Autotest > > > > > Hmmm... the recheck target of autotest appears to work through some > > sort of "screen scraping" applied to the testsuite logs (not an example > > I'm eager to follow I must say), and the format of the generated > > `testsuite.log' itself seems quite ad-hoc, "grown" rather than designed. > > Sure, hacked that in a few hours, and I'm not proud of it, but it > works reliably enough and was fast enough. > > Cheers, > Ralf > Thanks, Stefano
