On 12/30/24 14:17, Eduardo Costa wrote:
Hi!,
I'm new to dejagnu and would like to know, as I see the `debug.exp'
file under `/usr/share/dejagnu', what is the 'tcl debugger' mentioned
in this file that we can use to debug the scripts? Is it referring to
`tkcon', `tcl-pro-debug', or other I don't know about?
There is some kind of ancient debugging support in Expect that I do not
yet fully understand that `debugger.exp' extends.
I have so far always used "printf debugging" with DejaGnu, but I do have
updating lib/debugger.exp on my TODO list, as it uses an old API for
watchpoints that was deprecated in Tcl 8.4 and removed in Tcl 8.5.(Tcl
still has watchpoints, just with a slightly different interface now.)
Also, if I want to test a C program, I understand I can use `expect'
functions (maybe together with utility functions declared in, say,
`dg.exp') to compile and examine it's output, but what about testing
it at API level (the internal functions it's made of)?
The `dg.exp' module is intended for testing compilers and interpreters.
(DejaGnu was originally written to test the GNU toolchain: GCC, GDB,
GNU binutils, etc.) That module is useful for testing programs that
take input and produce output or additional programs.
I've had some success including `dejagnu.h' in a test file, moving all
my code into a convenience library I now link this test file against,
and using `dejagnu.h' functions from within such test-file to somewhat
get to see the expected output when I run `make check'.
This is how unit testing with DejaGnu is supposed to work: you write
unit test programs that emit DejaGnu's unit test protocol and use the
"host_execute" procedure in lib/dejagnu.exp in your testsuite code to
run the unit test programs and collect the results.
The `dejagnu.h' header (the entirety of "libdejagnu" currently) provides
an interface for C and C++ programs to produce DejaGnu unit test
protocol. Note that `dejagnu.h' is GPL, but the protocol is simple and
documented if you cannot use the GPL for your program and thus need to
write your own support module.
The GNU Coding Standards recommend providing library interfaces when
practical.
(<URL:https://www.gnu.org/prep/standards/standards.html#Standards-for-Graphical-Interfaces>)
Organizing your code as a frontend to a library also allows you to
easily add other frontends in the future.
You have already taken the first steps on this path: consider that you
now have one "regular use" frontend and one "unit test" frontend. You
can use the Expect facilities in DejaGnu for testing a CLI "regular use"
frontend (atop the backend; this is known as "system testing") in
addition to using the "unit test" frontend to directly test the backend
(known as (of course) "unit testing"). Well-engineered software has both
kinds of tests.
I did setup a `Makefile.am' under `testsuite' to compile this test
program and called `host-execute' on it, but I'm now guessing the
"normal" workflow is to compile whatever the test files via
tcl/expect/dejagnu, maybe use `dg.exp' functions to check the result,
and also `framework.exp' functions (i.e: `setup_xfail',
`check-conditional-xfail', etc) to have it automatically responding to
the output? Is this more or less how it works?
Nope! Look at DejaGnu's own Makefile.am for the unit tests for
`dejagnu.h' itself (search for "check_PROGRAMS") for a worked example.
(It is slightly complicated because DejaGnu does not use recursive make,
but should give you the basic idea.)
The `dg.exp' functions are only intended for compiling test programs
when testing compilers or other toolchain components. Normal unit test
programs should be built using make, as dependencies of the "check" target.
There is a sound software engineering basis for this: tests should
have, as much as possible, the same environment as the "serious" code to
ensure that bugs cannot be missed in testing due to a different environment.
Note that for now I'm not interested in cross-compilation or remote
debugging, but simply using dejagnu to test my program as thoroughly
as possible, so I'd really appreciate any comments or insights,
because nor I've had much previous experience with tcl/expect, neither
I still fully understand dejagnu's architecture. Also, a comment or
two regarding `config/gdb-comm.exp' and `config/gdb-stub.exp' would be
really appreciated, if only to figure out how these are supposed to be
used.
Those appear to be interface modules for using GDB to load and run
programs on embedded targets. For native unit tests, you will want the
"unix" target, which simply runs the test programs on the same host as
DejaGnu itself.
If it would be the case that some small c project is known to exist
doing just what I'm trying to, a pointer to it would be really
helpful. I know of gcc, coreutils, and gdb, gnu poke (mostly tests
interactively), and `ctlseqs' found in sourceware's git repository
(doesn't test the API but program's output), but the formers are way
too optimized for a newcomer to understand anyways. Even I get back to
these every now and then as I learn new things about
dejagnu/tcl/expect on their respective manuals/books, I suspect I'm
far from making full sense of it.
The best sources in my experience have been the "n" section manpages for
the commands in Tcl and the expect(1) manpage for Expect itself. The
DejaGnu reference manual is still incomplete; improving it remains on my
TODO list.
DejaGnu's own testsuite, while not always ideal examples, is also a good
place to look.
For `dejagnu.h', the C and C++ programs in testsuite/libdejagnu give
simple cases of using the API, although they do not actually perform any
tests, instead simply emitting the results requested on their command lines.
However, the `harness.exp' file in that same directory is not a good
example for running unit tests. Instead, look at the tests for the
framework side of the unit test protocol to find a better example (in
testsuite/runtest.main/nested/testsuite/stat.test/unit-sub.exp) of using
host_execute with a fairly complex command.
-- Jacob
