This refactoring should cause no API of functionality change,
and is meant only to simplify the future implementation of TAP
and SubUnit testsuite drivers. More precisely, our roadmap is
to move most of the "testsuite driving" features out of the
Automake-generated Makefiles, and into external scripts with
well-defined interfaces. This will allow the user to define
its own personalized testsuite drivers, and will also offer us
a framework upon which to implement our new TAP and SubUnit
drivers, all in a very unobtrusive way and retaining an high
degree of code reuse and backward-compatibility.
* lib/pt-driver: New auxiliary script.
* lib/Makefile.am (dist_SCRIPT_DATA): Add it.
* automake.in (handle_tests): Require the new auxiliary script
`pt-driver', and define new makefile variable `$(am__pt_driver)',
used to call it. Perform new substitution on `DRIVER' when
processing the `check2.am' file.
* lib/check.am (am__tty_colors): Define new shell variable
`$am__color_tests'.
(am__rst_section): Removed, its role taken over by the new
`pt-driver' script.
(am__test_driver_flags): New variable, contains the command
line options passed to `pt-driver'.
(am__check_pre): Do not deal with temporary files and exit
traps anymore, as the `pt-driver' script takes care of that now.
Define shell variable `$am__enable_hard_errors', used by
`$(am__test_driver_flags)'. Reorder so that we don't need to
save and restore the value of the `TERM' environment variable
anymore.
Other related adjustments.
(am__check_post): Remove, as its role has been completely taken
over by the `pt-driver' script.
* am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
?!GENERIC?%OBJ%): Call the test script through the Automake
substituted `%DRIVER%', and honor the command-line options
in `$(am__test_driver_flags)'. Do not call the obsoleted
`$(am__check_post)' anymore.
* tests/check.test: Adjust.
* tests/check2.test : Likewise.
* tests/check3.test : Likewise.
* tests/check4.test : Likewise.
* tests/check10.test: Likewise.
* tests/color.test: Likewise.
* tests/color2.test: Likewise.
* tests/comment9.test: Likewise.
* tests/dejagnu.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/maken3.test: Likewise.
* tests/maken4.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/posixsubst-tests.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/check-no-pt-driver.test: New test.
* tests/parallel-tests-pt-driver.test: Likewise.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
---
ChangeLog | 59 ++++++++++++++++
NEWS | 6 ++
automake.in | 8 ++-
lib/Automake/tests/Makefile.in | 69 ++++++++-----------
lib/Makefile.am | 2 +-
lib/Makefile.in | 2 +-
lib/am/check.am | 73 ++++++++------------
lib/am/check2.am | 10 ++-
lib/pt-driver | 129 +++++++++++++++++++++++++++++++++++
tests/Makefile.am | 2 +
tests/Makefile.in | 71 ++++++++-----------
tests/check-no-pt-driver.test | 33 +++++++++
tests/check.test | 2 +
tests/check10.test | 4 +
tests/check2.test | 4 +
tests/check3.test | 2 +-
tests/check4.test | 9 ++-
tests/color.test | 10 +++-
tests/color2.test | 4 +-
tests/comment9.test | 2 +
tests/dejagnu.test | 2 +
tests/exeext4.test | 2 +-
tests/maken3.test | 2 +-
tests/maken4.test | 2 +-
tests/parallel-tests-interrupt.test | 16 ++++-
tests/parallel-tests-pt-driver.test | 93 +++++++++++++++++++++++++
tests/posixsubst-tests.test | 2 +-
tests/repeated-options.test | 2 +-
28 files changed, 481 insertions(+), 141 deletions(-)
create mode 100755 lib/pt-driver
create mode 100755 tests/check-no-pt-driver.test
create mode 100755 tests/parallel-tests-pt-driver.test
diff --git a/ChangeLog b/ChangeLog
index aad41c7..0771b24 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,62 @@
+2011-06-15 Stefano Lattarini <stefano.lattar...@gmail.com>
+
+ parallel-tests: add auxiliary script 'pt-driver', refactor
+ This refactoring should cause no API of functionality change,
+ and is meant only to simplify the future implementation of TAP
+ and SubUnit testsuite drivers. More precisely, our roadmap is
+ to move most of the "testsuite driving" features out of the
+ Automake-generated Makefiles, and into external scripts with
+ well-defined interfaces. This will allow the user to define
+ its own personalized testsuite drivers, and will also offer us
+ a framework upon which to implement our new TAP and SubUnit
+ drivers, all in a very unobtrusive way and retaining an high
+ degree of code reuse and backward-compatibility.
+ * lib/pt-driver: New auxiliary script.
+ * lib/Makefile.am (dist_SCRIPT_DATA): Add it.
+ * automake.in (handle_tests): Require the new auxiliary script
+ `pt-driver', and define new makefile variable `$(am__pt_driver)',
+ used to call it. Perform new substitution on `DRIVER' when
+ processing the `check2.am' file.
+ * lib/check.am (am__tty_colors): Define new shell variable
+ `$am__color_tests'.
+ (am__rst_section): Removed, its role taken over by the new
+ `pt-driver' script.
+ (am__test_driver_flags): New variable, contains the command
+ line options passed to `pt-driver'.
+ (am__check_pre): Do not deal with temporary files and exit
+ traps anymore, as the `pt-driver' script takes care of that now.
+ Define shell variable `$am__enable_hard_errors', used by
+ `$(am__test_driver_flags)'. Reorder so that we don't need to
+ save and restore the value of the `TERM' environment variable
+ anymore.
+ Other related adjustments.
+ (am__check_post): Remove, as its role has been completely taken
+ over by the `pt-driver' script.
+ * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+ ?!GENERIC?%OBJ%): Call the test script through the Automake
+ substituted `%DRIVER%', and honor the command-line options
+ in `$(am__test_driver_flags)'. Do not call the obsoleted
+ `$(am__check_post)' anymore.
+ * tests/check.test: Adjust.
+ * tests/check2.test : Likewise.
+ * tests/check3.test : Likewise.
+ * tests/check4.test : Likewise.
+ * tests/check10.test: Likewise.
+ * tests/color.test: Likewise.
+ * tests/color2.test: Likewise.
+ * tests/comment9.test: Likewise.
+ * tests/dejagnu.test: Likewise.
+ * tests/exeext4.test: Likewise.
+ * tests/maken3.test: Likewise.
+ * tests/maken4.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/posixsubst-tests.test: Likewise.
+ * tests/repeated-options.test: Likewise.
+ * tests/check-no-pt-driver.test: New test.
+ * tests/parallel-tests-pt-driver.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+ * NEWS: Update.
+
2011-06-08 Stefano Lattarini <stefano.lattar...@gmail.com>
test defs: new function 'fatal_', for hard errors
diff --git a/NEWS b/NEWS
index 3c8f8ce..6db2fe0 100644
--- a/NEWS
+++ b/NEWS
@@ -11,6 +11,12 @@ New in 1.11a:
- New `cscope' target to build a cscope database for the source tree.
+* Changes to Automake-generated testsuite harnesses:
+
+ - The parallel-tests driver is now implemented (partly at least) with
+ the help of automake-provided auxiliary scripts (e.g., `pt-driver'),
+ instead of relying entirely on code in the generated Makefile.in.
+
* Miscellaneous changes:
- The `dist' and `dist-all' targets now can run compressors in parallel.
diff --git a/automake.in b/automake.in
index 3d6993a..7eb5464 100644
--- a/automake.in
+++ b/automake.in
@@ -4989,8 +4989,12 @@ sub handle_tests
append_exeext { exists $known_programs{$_[0]} } 'XFAIL_TESTS'
if (var ('XFAIL_TESTS'));
- if (option 'parallel-tests')
+ if (my $parallel_tests = option 'parallel-tests')
{
+ require_conf_file ($parallel_tests->{position}, FOREIGN,
'pt-driver');
+ define_variable ('am__pt_driver',
+ "\$(SHELL) $am_config_aux_dir/pt-driver",
+ INTERNAL);
define_variable ('TEST_SUITE_LOG', 'test-suite.log', INTERNAL);
define_variable ('TEST_SUITE_HTML', '$(TEST_SUITE_LOG:.log=.html)',
INTERNAL);
my $suff = '.test';
@@ -5041,6 +5045,7 @@ sub handle_tests
GENERIC => 0,
OBJ => $obj,
SOURCE => $val,
+ DRIVER => '$(am__pt_driver)',
COMPILE =>'$(' . $compile . ')',
EXT => '',
am__EXEEXT => 'FALSE');
@@ -5080,6 +5085,7 @@ sub handle_tests
GENERIC => 1,
OBJ => '',
SOURCE => '$<',
+ DRIVER => '$(am__pt_driver)',
COMPILE => '$(' . $compile .
')',
EXT => $test_suffix,
am__EXEEXT => $am_exeext);
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index 3274122..d3ed3dd 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -75,11 +75,13 @@ DIST_SOURCES =
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='[0;31m'; \
grn='[0;32m'; \
lgn='[1;32m'; \
@@ -107,9 +109,8 @@ am__nobase_list = $(am__nobase_strip_setup); \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title and section.
+# Restructured Text title.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by ". ") in a box.
am__text_box = $(AWK) '{ \
n = split($$0, lines, "\\. "); max = 0; \
@@ -125,60 +126,46 @@ am__text_box = $(AWK) '{ \
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_driver_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
+am__pt_driver = $(SHELL) $(top_srcdir)/lib/pt-driver
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
am__test_logs1 = $(TESTS:=.log)
@@ -474,7 +461,9 @@ recheck recheck-html:
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS)
TEST_LOGS="'"$$list"'"'
.pl.log:
- @p='$<'; $(am__check_pre) $(PL_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) \
+ $(am__pt_driver) $(am__test_driver_flags) -- \
+ $(PL_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 5bdc02e..2354af4 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -29,7 +29,7 @@ dist_pkgvdata_DATA = COPYING INSTALL texinfo.tex ansi2knr.c
ansi2knr.1 \
scriptdir = $(pkgvdatadir)
dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+ symlink-tree ar-lib pt-driver
EXTRA_DIST = gnupload
diff --git a/lib/Makefile.in b/lib/Makefile.in
index 91e9bae..8147a3d 100644
--- a/lib/Makefile.in
+++ b/lib/Makefile.in
@@ -242,7 +242,7 @@ dist_pkgvdata_DATA = COPYING INSTALL texinfo.tex ansi2knr.c
ansi2knr.1 \
scriptdir = $(pkgvdatadir)
dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+ symlink-tree ar-lib pt-driver
EXTRA_DIST = gnupload
all: all-recursive
diff --git a/lib/am/check.am b/lib/am/check.am
index 97ecb68..7774de8 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -20,11 +20,13 @@ if %?COLOR%
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='[0;31m'; \
grn='[0;32m'; \
lgn='[1;32m'; \
@@ -32,8 +34,7 @@ test "X$(AM_COLOR_TESTS)" != Xno \
std='[m'; \
}
else !%?COLOR%
-am__tty_colors = \
-red=; grn=; lgn=; blu=; std=
+am__tty_colors = red= grn= lgn= blu= std= am__color_tests=no
endif !%?COLOR%
.PHONY: check-TESTS
@@ -70,9 +71,8 @@ include inst-vars.am
## test is XFAIL or not. You can disable this feature by setting the
## variable DISABLE_HARD_ERRORS to a nonempty value.
-# Restructured Text title and section.
-am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
+# Restructured Text title.
+am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
# Put stdin (possibly several lines separated by ". ") in a box.
am__text_box = $(AWK) '{ \
@@ -91,59 +91,46 @@ am__text_box = $(AWK) '{ \
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_driver_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
+
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
+## The use of $dir below is required to account for VPATH
+## rewriting done by Sun make.
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
$(TEST_SUITE_LOG): $(TEST_LOGS)
@$(am__sh_e_setup); \
diff --git a/lib/am/check2.am b/lib/am/check2.am
index 054c62d..f3116c8 100644
--- a/lib/am/check2.am
+++ b/lib/am/check2.am
@@ -1,5 +1,5 @@
## automake - create Makefile.in from Makefile.am
-## Copyright (C) 2008, 2009 Free Software Foundation, Inc.
+## Copyright (C) 2008, 2009, 2011 Free Software Foundation, Inc.
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
@@ -17,7 +17,9 @@
## From a test file to a log file.
?GENERIC?%EXT%.log:
?!GENERIC?%OBJ%: %SOURCE%
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %DRIVER% $(am__test_driver_flags) -- \
+ %COMPILE% "$$tst"
## If no programs are built in this package, then this rule is removed
## at automake time. Otherwise, %am__EXEEXT% expands to a configure time
@@ -25,5 +27,7 @@
## conflict with the previous one.
if %am__EXEEXT%
?GENERIC?%EXT%$(EXEEXT).log:
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) \
+ %DRIVER% $(am__test_driver_flags) -- \
+ %COMPILE% "$$tst"
endif %am__EXEEXT%
diff --git a/lib/pt-driver b/lib/pt-driver
new file mode 100755
index 0000000..78b6d18
--- /dev/null
+++ b/lib/pt-driver
@@ -0,0 +1,129 @@
+#! /bin/sh
+# pt-driver - basic driver script for the `parallel-tests' mode.
+
+scriptversion=2011-06-14.16; # UTC
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <bug-autom...@gnu.org> or send patches to
+# <automake-patches@gnu.org>.
+
+# Make unconditional expansion of undefined variables an error. This
+# helps a lot in preventing typo-related bugs.
+set -u
+
+fatal ()
+{
+ echo "$0: fatal: $*" >&2
+ exit 1
+}
+
+usage_error ()
+{
+ echo "$0: $*" >&2
+ print_usage >&2
+ exit 2
+}
+
+print_usage ()
+{
+ cat <<END
+Usage:
+ pt-driver [--help|--version] --test-name=NAME --log-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--] TEST-SCRIPT
+The \`--test-name' and \`--log-file' options are mandatory.
+END
+}
+
+# Restructured Text section.
+rst_section () { sed 'p;s/./=/g;p;g'; }
+
+# TODO: better error handling in option parsing (in particular, ensure
+# TODO: $logfile and $test_name are defined).
+test_name= # Used for reporting.
+logfile= # Where to save the result and output of the test script.
+expect_failure=no
+color_tests=no
+enable_hard_errors=yes
+while test $# -gt 0; do
+ case $1 in
+ --help) print_usage; exit $?;;
+ --version) echo "pt-driver $scriptversion"; exit $?;;
+ --test-name) test_name=$2; shift;;
+ --log-file) logfile=$2; shift;;
+ --color-tests) color_tests=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --enable-hard-errors) enable_hard_errors=$2; shift;;
+ --) shift; break;;
+ -*) usage_error "invalid option: '$1'";;
+ esac
+ shift
+done
+
+if test $color_tests = yes; then
+ red='[0;31m' # Red.
+ grn='[0;32m' # Green.
+ lgn='[1;32m' # Light green.
+ blu='[1;34m' # Blue.
+ std='[m' # No color.
+else
+ red= grn= lgn= blu= std=
+fi
+
+tmpfile=$logfile-t
+do_exit='rm -f $tmpfile; (exit $st); exit $st'
+trap "st=129; $do_exit" 1
+trap "st=130; $do_exit" 2
+trap "st=141; $do_exit" 13
+trap "st=143; $do_exit" 15
+rm -f $tmpfile
+
+# Test script is run here.
+"$@" >$tmpfile 2>&1
+estatus=$?
+if test $enable_hard_errors = no && test $estatus -eq 99; then
+ estatus=1
+fi
+
+case $estatus:$expect_failure in
+ 0:yes) col=$red; res=XPASS;;
+ 0:*) col=$grn; res=PASS ;;
+ 77:*) col=$blu; res=SKIP ;;
+ 99:*) col=$red; res=FAIL ;;
+ *:yes) col=$lgn; res=XFAIL;;
+ *:*) col=$red; res=FAIL ;;
+esac
+echo "${col}${res}${std}: $test_name"
+echo "$res: $test_name (exit: $estatus)" | rst_section > $logfile
+cat $tmpfile >> $logfile
+rm -f $tmpfile
+
+# Local Variables:
+# mode: shell-script
+# sh-indentation: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "scriptversion="
+# time-stamp-format: "%:y-%02m-%02d.%02H"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e68f6d7..8efefab 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -234,6 +234,7 @@ check12.test \
check-exported-srcdir.test \
check-tests-in-builddir.test \
check-tests_environment.test \
+check-no-pt-driver.test \
checkall.test \
clean.test \
clean2.test \
@@ -717,6 +718,7 @@ parallel-tests-unreadable-log.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
+parallel-tests-pt-driver.test \
parse.test \
percent.test \
percent2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 7e5fd09..14fcf7b 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -80,11 +80,13 @@ DIST_SOURCES =
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
+am__color_tests=no; \
red=; grn=; lgn=; blu=; std=; \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='[0;31m'; \
grn='[0;32m'; \
lgn='[1;32m'; \
@@ -112,9 +114,8 @@ am__nobase_list = $(am__nobase_strip_setup); \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title and section.
+# Restructured Text title.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
-am__rst_section = sed 'p;s/./=/g;p;g'
# Put stdin (possibly several lines separated by ". ") in a box.
am__text_box = $(AWK) '{ \
n = split($$0, lines, "\\. "); max = 0; \
@@ -130,60 +131,46 @@ am__text_box = $(AWK) '{ \
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to all log compiler wrappers.
+am__test_driver_flags = \
+ --test-name "$$f" \
+ --log-file '$@' \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
+am__pt_driver = $(SHELL) $(top_srcdir)/lib/pt-driver
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
TEST_EXTENSIONS = .test
@@ -501,6 +488,7 @@ check12.test \
check-exported-srcdir.test \
check-tests-in-builddir.test \
check-tests_environment.test \
+check-no-pt-driver.test \
checkall.test \
clean.test \
clean2.test \
@@ -984,6 +972,7 @@ parallel-tests-unreadable-log.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
+parallel-tests-pt-driver.test \
parse.test \
percent.test \
percent2.test \
@@ -1491,7 +1480,9 @@ recheck recheck-html:
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS)
TEST_LOGS="'"$$list"'"'
.test.log:
- @p='$<'; $(am__check_pre) $(TEST_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) \
+ $(am__pt_driver) $(am__test_driver_flags) -- \
+ $(TEST_LOG_COMPILE) "$$tst"
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
diff --git a/tests/check-no-pt-driver.test b/tests/check-no-pt-driver.test
new file mode 100755
index 0000000..57408e0
--- /dev/null
+++ b/tests/check-no-pt-driver.test
@@ -0,0 +1,33 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'pt-wrap' doesn't get needlessly installed
+# or referenced when the 'parallel-tests' option is not used.
+
+parallel_tests=no
+. ./defs || Exit 1
+
+echo 'TESTS = foo.test' > Makefile.am
+
+$ACLOCAL
+
+for opts in '' '-a' '--add-missing --copy'; do
+ $AUTOMAKE $opts
+ $FGREP 'pt-driver' Makefile.in && Exit 1
+ find . | $FGREP 'pt-driver' && Exit 1
+done
+
+:
diff --git a/tests/check.test b/tests/check.test
index 54432ec..da30d5e 100755
--- a/tests/check.test
+++ b/tests/check.test
@@ -22,6 +22,8 @@ cat > Makefile.am << 'END'
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > pt-driver
+
: > frob.test
$ACLOCAL
diff --git a/tests/check10.test b/tests/check10.test
index 7025b98..4705c74 100755
--- a/tests/check10.test
+++ b/tests/check10.test
@@ -48,6 +48,10 @@ cp fail fail2
cp xfail xfail2
cp skip skip2
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir"/lib/pt-driver .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
diff --git a/tests/check2.test b/tests/check2.test
index 9b8bfb1..ffccf10 100755
--- a/tests/check2.test
+++ b/tests/check2.test
@@ -43,6 +43,10 @@ echo.sh:
CLEANFILES = echo.sh
END
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir"/lib/pt-driver .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
diff --git a/tests/check3.test b/tests/check3.test
index be40e3f..ccfb923 100755
--- a/tests/check3.test
+++ b/tests/check3.test
@@ -54,7 +54,7 @@ END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure --prefix "`pwd`/inst"
$MAKE check >stdout
cat stdout
diff --git a/tests/check4.test b/tests/check4.test
index 99adb23..1391033 100755
--- a/tests/check4.test
+++ b/tests/check4.test
@@ -46,8 +46,15 @@ chmod +x ok.sh dir/fail.sh
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+
+if test x"$parallel_tests" = x"yes"; then
+ $AUTOMAKE --add-missing
+else
+ $AUTOMAKE
+fi
+
./configure --prefix "`pwd`/inst"
+
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
grep 'FAIL: fail.sh' stdout
diff --git a/tests/color.test b/tests/color.test
index 570b21d..2bf5b6d 100755
--- a/tests/color.test
+++ b/tests/color.test
@@ -67,8 +67,14 @@ cp pass xpass
chmod +x pass fail skip xpass xfail
$ACLOCAL
-$AUTOMAKE
$AUTOCONF
+
+if test x"$parallel_tests" = x"yes"; then
+ $AUTOMAKE --add-missing
+else
+ $AUTOMAKE
+fi
+
./configure
test_color ()
@@ -95,3 +101,5 @@ test_no_color ()
AM_COLOR_TESTS=always $MAKE -e check >stdout && { cat stdout; Exit 1; }
cat stdout
test_color
+
+:
diff --git a/tests/color2.test b/tests/color2.test
index eb9c527..4f8862e 100755
--- a/tests/color2.test
+++ b/tests/color2.test
@@ -80,7 +80,7 @@ cp pass xpass
chmod +x pass fail skip xpass xfail
$ACLOCAL
-$AUTOMAKE
+$AUTOMAKE -a
$AUTOCONF
./configure
@@ -120,3 +120,5 @@ AM_COLOR_TESTS=no MAKE=$MAKE expect -f expect-make >stdout \
|| { cat stdout; Exit 1; }
cat stdout
test_no_color
+
+:
diff --git a/tests/comment9.test b/tests/comment9.test
index e95d99b..cdb0758 100755
--- a/tests/comment9.test
+++ b/tests/comment9.test
@@ -30,6 +30,8 @@ TESTS = \
7.test
EOF
+: > pt-driver
+
$ACLOCAL
$AUTOMAKE
diff --git a/tests/dejagnu.test b/tests/dejagnu.test
index df385b0..105e53e 100755
--- a/tests/dejagnu.test
+++ b/tests/dejagnu.test
@@ -25,6 +25,8 @@ AUTOMAKE_OPTIONS = dejagnu
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > pt-driver
+
$ACLOCAL
$AUTOMAKE
diff --git a/tests/exeext4.test b/tests/exeext4.test
index c3f968c..2ec3bca 100755
--- a/tests/exeext4.test
+++ b/tests/exeext4.test
@@ -50,7 +50,7 @@ END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing --copy
./configure
$MAKE print-bin > output
cat output
diff --git a/tests/maken3.test b/tests/maken3.test
index 5a77f1a..eb54d35 100755
--- a/tests/maken3.test
+++ b/tests/maken3.test
@@ -149,7 +149,7 @@ check_targets ()
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
diff --git a/tests/maken4.test b/tests/maken4.test
index 5c21943..4a11910 100755
--- a/tests/maken4.test
+++ b/tests/maken4.test
@@ -154,7 +154,7 @@ check_targets ()
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
diff --git a/tests/parallel-tests-interrupt.test
b/tests/parallel-tests-interrupt.test
index 45e55d4..116a205 100755
--- a/tests/parallel-tests-interrupt.test
+++ b/tests/parallel-tests-interrupt.test
@@ -28,12 +28,20 @@ END
cat > Makefile.am << 'END'
TESTS = foo.test
-## Ugly, but required by foo.test. See below.
-TEST_LOG_COMPILER = echo $$$$ > pid && exec 9>&2 && $(SHELL) -x
+## Provide more debugging info.
+TEST_LOG_COMPILER = $(SHELL) -x
+## Rut required by foo.test; see below.
+AM_TESTS_ENVIRONMENT = 9>&2
END
# This is hacky and ugly, but has the great advantage of avoiding us a lot
# of pain with background processes and related synchronization issues.
+
+cat - "$top_testsrcdir"/lib/pt-driver > pt-driver <<'END'
+#!/bin/sh
+echo $$ > pid
+END
+
cat > foo.test << 'END'
#!/bin/sh
exec 2>&9
@@ -48,7 +56,7 @@ chmod a+x foo.test
$ACLOCAL
$AUTOCONF
-$AUTOMAKE -a
+$AUTOMAKE
./configure
@@ -60,6 +68,8 @@ for signum in $trapped_signals; do
rm -f pid fail *.log *.log-t
env signum=$signum $MAKE check && { ls -l; Exit 1; }
ls -l
+ # These files shouldn't exist, but in case they do, their content might
+ # provide helpful information about the causes of the failure(s).
cat foo.log-t || :
cat foo.log || :
cat test-suite.log || :
diff --git a/tests/parallel-tests-pt-driver.test
b/tests/parallel-tests-pt-driver.test
new file mode 100755
index 0000000..cc71303
--- /dev/null
+++ b/tests/parallel-tests-pt-driver.test
@@ -0,0 +1,93 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'pt-driver' gets automatically installed
+# in the correct directory by 'parallel-tests' option.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+: Try first with parallel-tests defined in AM_INIT_AUTOMAKE.
+
+mkdir am-init-automake
+cd am-init-automake
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([my_aux_dir])
+AM_INIT_AUTOMAKE([parallel-tests])
+AC_CONFIG_FILES([Makefile sub/Makefile])
+AC_OUTPUT
+END
+
+mkdir sub my_aux_dir
+
+cat > Makefile.am <<END
+SUBDIRS = sub
+TESTS = foo.test
+END
+
+cat > sub/Makefile.am <<END
+TESTS = bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE -a 2>stderr || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . sub my_aux_dir # For debugging.
+test -f my_aux_dir/pt-driver
+test ! -r pt-driver
+test ! -r sub/pt-driver
+
+grep '^configure\.in:3:.*installing.*my_aux_dir/pt-driver' stderr
+
+cd ..
+
+: Now try with parallel-tests defined in AUTOMAKE_OPTIONS.
+
+mkdir automake-options
+cd automake-options
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([build-aux])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([dir/GNUmakefile])
+AC_OUTPUT
+END
+
+mkdir build-aux dir
+
+cat > dir/GNUmakefile.am <<END
+TESTS = foo.test
+AUTOMAKE_OPTIONS = parallel-tests
+TESTS += bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE --add-missing --copy dir/GNUmakefile 2>stderr \
+ || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . dir build-aux # For debugging.
+test -f build-aux/pt-driver
+test ! -r pt-driver
+test ! -r dir/pt-driver
+
+grep '^dir/GNUmakefile\.am:2:.*installing.*build-aux/pt-driver' stderr
+
+:
diff --git a/tests/posixsubst-tests.test b/tests/posixsubst-tests.test
index 26ae69f..3f45c72 100755
--- a/tests/posixsubst-tests.test
+++ b/tests/posixsubst-tests.test
@@ -50,7 +50,7 @@ chmod +x foo1.test bary
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
$MAKE check >out 2>&1 && { cat out; Exit 1; }
diff --git a/tests/repeated-options.test b/tests/repeated-options.test
index c3c2e29..5591ffe 100755
--- a/tests/repeated-options.test
+++ b/tests/repeated-options.test
@@ -58,7 +58,7 @@ int main (void)
}
END
-cp $testsrcdir/../lib/compile .
+cp "$top_testsrcdir"/lib/compile "$top_testsrcdir"/lib/pt-driver .
$ACLOCAL
$AUTOMAKE --foreign --foreign -Wall 2>stderr || { cat stderr >&2; Exit 1; }
--
1.7.2.3
