Dear Bruno, dear Simon,
thank you for your replies.
I understand that valgrind-tests and the proposed "timeout-tests" solution
are not completely equivalent. Nevertheless, I still think that some
timeout functionality provided by Gnulib would be useful.
Bruno's solution
**
#if HAVE_DECL_ALARM
/* Declare failure if test takes too long, by using default abort
caused by SIGALRM. */
int alarm_value = 600;
signal (SIGALRM, SIG_DFL);
alarm (alarm_value);
#endif
**
works for unit tests that have been written specifically for the project.
It doesn't help, though, if I want to test (production) binaries that are
to be installed because I generally don't want testing code inside them.
Moreover, use cases for a baked-in timeout are not restricted to tests. For
example, I may want to restrict the build time of certain components in
situations where a logical error may lead to infinite build times (a simple
example is that of a Scheme compiler used as a build tool; thanks to
Turing-completeness of Scheme macros, such a build may not terminate).
Marc
Am Fr., 30. Apr. 2021 um 11:24 Uhr schrieb Simon Josefsson <
si...@josefsson.org>:
> Bruno Haible <br...@clisp.org> writes:
>
> > So, I don't think the "let's treat timeout like valgrind" approach is
> going
> > to work. Instead, you need to design a way to deal with timeouts,
> independently.
>
> Hi! I think Marc's request for functionality to introduce timeouts for
> self-tests is a good one. However I reach the same conclusion as Bruno,
> that having a module like valgrind-tests is probably not the best way to
> solve it. To me, having a timeout seems like an essential feature of a
> self-test framework. I know automake isn't primarily a self-test
> framework, but it has concepts for it and the test framework has been
> improved significantly over the years, so I think adding a timeout
> functionality to automake makes sense. What do bug-automake people
> think?
>
> The functionality could be conditioned on the coreutils 'timeout' tool,
> and if that tool exists, and appears to work, running all self-tests
> under that tool could be done automatically. The default self-test
> timeout be quite generous (say 17 hours?) but it should be easy to
> modify both by end-user and project developer. If we want to be
> conservative, the functionality could be opt-in initially, and then
> after a few years become the default behaviour.
>
> Thoughts?
>
> /Simon
>
