On Thursday 04 November 2010, Ralf Wildenhues wrote: > * Stefano Lattarini wrote on Thu, Nov 04, 2010 at 01:03:17PM CET: > > On Wednesday 03 November 2010, Ralf Wildenhues wrote: > > > * Stefano Lattarini wrote on Wed, Nov 03, 2010 at 02:27:47PM CET: > > > > Just one question: what about the already-existing "tests-init" branch? > > > > Should I try to bring it to a point where it can be easily merged into > > > > master, and then forget about it, or should it continue to live parallel > > > > to master? > > > > > > Right now the branch looks like its changes would be fairly safe to > > > merge to master, but also, it would mean that sharing testsuite patches > > > between branch-1.11 and master may become a bit more cumbersome. So I'd > > > say it depends a bit on where you want to go with this in the long run. > > Well, I'd like to: > > > > 1. reorganize the code in "tests/defs.in" in a more rational way > > (basically just a reordering of the code); > > Seems like a good idea. > > > 2. (re)introduce the `run_command' function and use it throughout; > > I like it but before it can be merged it needs test exposure on > Tru64/OSF, IRIX, and Solaris at least, ideally also AIX and HP-UX. > You should be able to test on these systems in the near future, > hopefully. Hmm... more details about this off-list.
> > 3. make it easier for the user to override the aclocal "search path" > > in the testsuite (a new pending patch from Paolo Bonzini could > > help with this); > > Why would the user need to mess with our testsuite? It shouldn't; the feature proposed by Paolo (with a patch) is for "production use", not for the testsuite -- but we could also use it to improve our testsuite, without additiona hassle. > Oh, is this so Libtool testing works with distcheck and with unusual > setups? Basically, yes. But as I said, once the Paolo's patch is in place, implementing point (3) should be (almost) trivial. > > 4. improve requirements' declaration in Automake test scripts, and > > make it easier to run the Automake testsuite using different C, > > C++ and Fortran compilers, without leaning too much towards the > > GNU compilers as it does now. > > This is a noble goal; I support it in a way that tests which are aimed > at showing off Automake API which is supposed to be portable to > different compilers, should also work with different compilers. Tests > which are aimed at verifying more internal things do not need to be made > artificially portable to external tools. Agreed. > Also, we should strive to not > make ourselves too dependent on bugs in Autoconf or Libtool; while that > can be helpful to uncover bugs in those tools, it also means that we > make ourselves liable to timely fixes, or version tests and all. OK; but I'd prefer some spurious failures here and there to lurking dormant bugs and/or a false sense of security due to lack of testsuite exposure to a larger array of configurations. > > These major changes would be interspesed with minor code cleanups > > and refactoring, such as (to list a few) avoiding to leak temporary > > variables from `tests/defs' into the test scripts, renaming `$curdir' > > to (say) `$testbuilddir' for better clarity, and improve the messages > > from skipped tests. > > The minor cleanups sound ok, with the caveat that they have good chances > of breaking merges, or test suite additions coming from the stable > branch. It is thus a good idea not to change things gratuitously here, > and it may be prudent to do defs API changes in a branch off of maint, > so it can be merged into other active branches before they are merged > into master. Good idea: taking note of it. BTW, I think that for potentially problematic changes I could just forget the 72-hours-gracetime, and wait for a thorough review from you (or someone else, if anyone steps ahead ;-). > > > We can easily move to a strategy of having new testsuite stuff added to > > > master only by default, and only care about branch-1.11 for fixing > > > existing regressions and serious bugs, which would probably make the > > > situation easier. > > Yes, I'd like to go this way. > > OK then. Good; I'll try to do the merge of "tests-init" later (or tomorrow), and push to master if the testsuite still passes. > > > Of course there are other active branches too, that shouldn't have a > > > harder-than-necessary time. > > > Ideally, all the tests that work with current `tests/defs' should > > continue to work with the new one; the only problem might be that > > some new tests committed to another branch might cause *maintcheck* > > failures when merged to master, but that is not a big deal IMHO, > > and easily fixed anyway. > OK. And as you noticed, commits that change the API of `tests/defs' can be done in a branch off of maint, so that they can be merged into other active branches before they are merged into master. > > > We could treat merges similarly to patches in announcing the intention > > > to merge 72h before the fact, in those cases where there is doubt over > > > the merge (or the precise point at which a merge should be done). > > This won't be necessary if we stick to master for tests extensions and > > refactoring. > > Well, don't let the above paragraph of mine be incentive to use less > branches. That's not what I intended. Doing things in separate > branches *when appropriate* is good! True, but most of the changes I'm planning should be small, self-contained, and not distruptive at all, so that thay could be easily done directly in master, or even in maint. > > > I must admit that I don't have all that much experience with merges of > > > longer-term branches done by more than one person; one alternative would > > > be that you suggest the merge and I do it (if that works out). Testing > > > such merges usually requires a bit of additional work, in that both > > > branches need to be checked for global changes that need adjusting in > > > the respective other branch. Let's see how this works out. > > > > In the end, are you OK with having me to merge "test-init" to master right > > away, and do future testsuite work on master only? > > Yes. Good. Thanks, Stefano
