On Wed, 2020-07-22 at 17:05 -0500, Jacob Bachmeyer wrote:
> I had been planning to look at this later, but fixing PR42399
> brought
> this front-and-center. The DejaGnu unit testing protocol is not
> (yet)
> documented (will be fixed as part of PR42399) and there is a strange
> quirk in its handler.
>
> What produces lines with prefixes matching {[0-9][0-9]:..:..:}
> and/or
> why is host_execute looking for that pattern? These lines have the
> effect only of resetting a timeout counter and printing the line if
> verbose is 3 or higher but are otherwise ignored.
>
>
> The DejaGnu unit testing protocol is very simple, and I think I can
> work
> with this fairly well for the long-term. DejaGnu spawns a unit test
> program and reads that program's output. Arguments for the unit
> test
> program can be specified in the testsuite. The unit test program
> can
> produce any output, but DejaGnu is sensitive to lines beginning with
> a
> tab character, followed by uppercase letters, followed by a colon.
> These lines (currently) indicate test results and (in the future)
> provide additional information to effect proper testing. (Examples
> of
> future extensions now planned include support for known failures and
> framework-assessed expected failures. Currently, the dejagnu.h
> header
> provides xfail/xpass functions to allow unit tests to declare
> expected
> failure, but host_execute does not recognize these results; this
> second
> bug is now slated to be fixed for 1.6.3.)
>
> Unit test programs may also produce other output for the benefit of
> a
> developer directly running them or reading the DejaGnu log, but the
> sequence {\n\t[][[:upper:]]*:} should be considered reserved for the
> DejaGnu unit testing support.
>
>
> Anyone know why that pattern was included in host_execute or have
> other
> comments?
In terms of "other comments":
FWIW within gcc, the jit.dg testsuite for libgccjit has a copy of
host_execute ("fixed_host_execute") to workaround issues I've run into:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/testsuite/jit.dg/jit.exp
It has drifted somewhat from the DejaGnu original; for example it
gained the ability to parse valgrind output and convert leaks into
pass/fail results.
(I also ran into the issue that dejagnu.h's pass/fail C functions
aren't thread-safe, which I hack around in my testsuite, replacing them
in multi-threaded tests with ones guarded by a mutex).
Hope this is constructive
Dave
