Arsen Arsenović wrote:
On Monday, 26 September 2022 03:11:43 CEST Jacob Bachmeyer wrote:
The tests that it fixed probably need some attention in the form of
moving some "preclean" steps to GCC's ${tool}-dg-test.
I'll add giving them a look to my TODO.
Please keep me informed about the results of looking into those; if this
has actually exposed bugs in the GCC testsuite, I may add (as a
testsuite debugging aid) an option to run ${tool}-dg-prune in an extra
pass before analyzing the output. I am tentatively considering a
variable DG_PRUNE_EARLY_AND_OFTEN that could be set on the runtest
command line to arrange for this.
Keeping the current interface while fixing the {PASS,FAIL}+UNSUPPORTED
issue is also possible: we can build up a list of results from
scanning the messages, but not actually report them to the framework
until after checking the return value of ${tool}-dg-prune for an
"abandon test" flag. Note that ${tool}-dg-test could produce this
flag in lieu of actual output and ${tool}-dg-prune simply pass it
along. (I suspect that the current Tcl list facilities that would
enable this were not yet available when dg.exp was written more than
20 years ago.)
The main reason I am reluctant to simply accept the "abandon test"
flags directly from ${tool}-dg-test is that that would completely
break backwards compatibility: testsuites depending on this would
absolutely need DejaGnu 1.6.4, and I think we may be able to get most
of the way there (at least getting an UNSUPPORTED result reliably
into the summary) with existing code.
Right, I didn't think of that. Indeed, aggregating the results to
dispatch them after pruning (but before excess error) should cover all
of these cases.
This is still the plan, although I must ask if you know of a reason
(other than inertia) not to continue aggregating results until the very
end of the dg-test procedure.
Excellent! It is now tracked as PR58065. I expect to have a
regression test and a preliminary patch in a few days.
That's great. Thank you very much!
My apologies for the delay: developing unit tests for dg.exp has proven
to be more involved than initially expected. (Starting with writing a
pure-Tcl VFS shim for the internal unit test support infrastructure
because a getdirs test in utils.test will fail if any more subtrees are
added to testsuite/runtest.libs. Why yes, I now plan to eventually also
make the other tests independent of the actual filesystem now that I
have had to write this shim.)
-- Jacob
