Hello automakers.
I have rebased the patch on latest master, since the recent(ish) work
on the tests-init branch had introduced some conflicts (easy to solve,
luckily). I've also fixed an oversight in the previous patch: the
presence of the exit trap was checked for /bin/sh, not $SHELL, despite
the test cases were being run with $SHELL.
This patch clearly requires a thorough review, so the 72-hour rule
doesn't apply here. I'll thus wait for a proper review before pushing.
OK for the `tests-init' branch?
Regards,
Stefano
From aed50b1583875689620caa190068a753b10a80ec Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <stefano.lattar...@gmail.com>
Date: Fri, 3 Dec 2010 15:24:14 +0100
Subject: [PATCH] Testsuite: use $SHELL to run tests which are shell scripts.
* tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
configure-time $SHELL is used to run the tests.
(TEST_EXTENSIONS): Defined to `.test', for clarity.
* tests/defs: Add code to re-execute by default the running
test script with configure-time $SHELL. Updated comments.
* configure.ac: Check for a working `set -e' using $SHELL,
not /bin/sh.
* tests/defs-static.in: Update comments.
* tests/README (Supported shells): Updated.
(Getting details from failures): Don't tell that tests are
run by /bin/sh by default.
---
ChangeLog | 15 ++++++++++++++
configure | 6 ++--
configure.ac | 4 +-
tests/Makefile.am | 6 +++++
tests/Makefile.in | 6 ++++-
tests/README | 20 +++++++++++++-----
tests/defs | 53 +++++++++++++++++++++++++++++++++++++++++++++++--
tests/defs-static.in | 9 ++++++-
8 files changed, 102 insertions(+), 17 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 232469c..1da120d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2010-12-03 Stefano Lattarini <stefano.lattar...@gmail.com>
+
+ Testsuite: use $SHELL to run tests which are shell scripts.
+ * tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
+ configure-time $SHELL is used to run the tests.
+ (TEST_EXTENSIONS): Defined to `.test', for clarity.
+ * tests/defs: Add code to re-execute by default the running
+ test script with configure-time $SHELL. Updated comments.
+ * configure.ac: Check for a working `set -e' using $SHELL,
+ not /bin/sh.
+ * tests/defs-static.in: Update comments.
+ * tests/README (Supported shells): Updated.
+ (Getting details from failures): Don't tell that tests are
+ run by /bin/sh by default.
+
2010-12-02 Stefano Lattarini <stefano.lattar...@gmail.com>
* tests/silent8.test: Use "|| Exit 1" after ". ./defs".
diff --git a/configure b/configure
index 089b32a..6b39e49 100755
--- a/configure
+++ b/configure
@@ -2757,12 +2757,12 @@ $as_echo "$ac_cv_path_FGREP" >&6; }
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether /bin/sh has working 'set -e' with exit trap" >&5
-$as_echo_n "checking whether /bin/sh has working 'set -e' with exit trap... " >&6; }
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $SHELL has working 'set -e' with exit trap" >&5
+$as_echo_n "checking whether $SHELL has working 'set -e' with exit trap... " >&6; }
if ${am_cv_sh_errexit_works+:} false; then :
$as_echo_n "(cached) " >&6
else
- if /bin/sh -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
+ if $SHELL -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
then
am_cv_sh_errexit_works=yes
else
diff --git a/configure.ac b/configure.ac
index 577541c..8af0b60 100644
--- a/configure.ac
+++ b/configure.ac
@@ -177,9 +177,9 @@ AC_SUBST([MODIFICATION_DELAY])
AC_PROG_EGREP
AC_PROG_FGREP
-AC_CACHE_CHECK([whether /bin/sh has working 'set -e' with exit trap],
+AC_CACHE_CHECK([whether $SHELL has working 'set -e' with exit trap],
[am_cv_sh_errexit_works],
-[if /bin/sh -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
+[if $SHELL -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
then
am_cv_sh_errexit_works=yes
else
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 83dcdf0..06390c7 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -19,6 +19,12 @@
MAINTAINERCLEANFILES =
EXTRA_DIST = ChangeLog-old
+TEST_EXTENSIONS = .test
+# Run the tests with the shell detected at configure time. Also, tell
+# them not to needlessly re-execute themselves with that shell, since
+# we know they are already running under it in our setup.
+TEST_LOG_COMPILER = AM_TESTS_REEXEC=no $(SHELL)
+
XFAIL_TESTS = \
all.test \
auxdir2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 87dff85..c9e9183 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -183,7 +183,6 @@ RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
-TEST_EXTENSIONS = .test
am__test_logs1 = $(TESTS:=.log)
TEST_LOGS = $(am__test_logs1:.test.log=.log)
TEST_LOG_COMPILE = $(TEST_LOG_COMPILER) $(AM_TEST_LOG_FLAGS) \
@@ -284,6 +283,11 @@ top_srcdir = @top_srcdir@
MAINTAINERCLEANFILES = $(parallel_tests) $(instspc_tests)
EXTRA_DIST = ChangeLog-old gen-parallel-tests instspc-tests.sh \
$(TESTS)
+TEST_EXTENSIONS = .test
+# Run the tests with the shell detected at configure time. Also, tell
+# them not to needlessly re-execute themselves with that shell, since
+# we know they are already running under it in our setup.
+TEST_LOG_COMPILER = AM_TESTS_REEXEC=no $(SHELL)
XFAIL_TESTS = all.test auxdir2.test cond17.test gcj6.test txinfo5.test \
$(instspc_xfail_tests)
parallel_tests = backcompat5-p.test check-p.test check10-p.test \
diff --git a/tests/README b/tests/README
index 88da783..03d4cff 100644
--- a/tests/README
+++ b/tests/README
@@ -51,10 +51,10 @@ Interpretation
Getting details from failures
-----------------------------
- Each test is a shell script, and by default is run by /bin/sh.
- In a non-VPATH build you can run them directly, they will be verbose.
- By default, verbose output of a test foo.test is retained in the log
- file foo.log. A summary log is created in the file test-suite.log.
+ Each test is a shell script. In a non-VPATH build you can run the
+ tests directly, they will be verbose. By default, verbose output of
+ a test foo.test is retained in the log file foo.log. A summary log
+ is created in the file test-suite.log.
You can limit the set of files using the TESTS variable, and enable
detailed test output at the end of the test run with the VERBOSE
@@ -66,6 +66,15 @@ Getting details from failures
Supported shells
----------------
+ By default, the tests are run by the $SHELL detected at configure
+ time. They also take care to re-execute themselves with that shell,
+ unless told not to. So, to run the tests with a different shell, say
+ `/path/to/sh', the user must resort to something like:
+ $ AM_TESTS_REEXEC=no /path/to/sh ./foo.test
+ to run a test directly, and something like:
+ $ AM_TESTS_REEXEC=no make check TEST_LOG_COMPILER=/path/to/sh
+ to run the test(s) through the makefile test driver.
+
The test scripts are written with portability in mind, so that they
should run with any decent Bourne-compatible shell.
@@ -75,8 +84,7 @@ Supported shells
no easy workaround. Thus, if you want to run a test script, say
foo.test, with Zsh, you *can't* simply do `zsh foo.test', but you
*must* resort to:
- zsh -o no_function_argzero foo.test
-
+ AM_TESTS_REEXEC=no zsh -o no_function_argzero foo.test
Note that this problem does not occur if zsh is executed through a
symlink with a basename of `sh', since in that case it starts
in Bourne compatibility mode. So you should be perfectly safe when
diff --git a/tests/defs b/tests/defs
index 8f748b1..79b80cf 100644
--- a/tests/defs
+++ b/tests/defs
@@ -16,6 +16,12 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# IMPORTANT NOTE: This file should execute correctly with any system's
+# /bin/sh shell, and not only with configure-time detected $CONFIG_SHELL,
+# until differently and explicitly specified.
+
+
## -------------------------------------------------------- ##
## Source static setup and definitions for the testsuite. ##
## -------------------------------------------------------- ##
@@ -33,9 +39,9 @@ test -f ./defs-static || {
me=`echo "$0" | sed -e 's,.*[\\/],,;s/\.test$//'`
-## ---------------------------------------- ##
-## Sanity checks and environment cleanup. ##
-## ---------------------------------------- ##
+## ---------------------- ##
+## Early sanity checks. ##
+## ---------------------- ##
# Ensure $testsrcdir is set correctly.
test -f "$testsrcdir/defs-static.in" || {
@@ -49,6 +55,47 @@ test -f "$testbuilddir/defs-static" || {
exit 1
}
+# Ensure we can find ourselves.
+test -f "$0" || {
+ echo "$me: unable to find myself: $0" >&2
+ exit 1
+}
+
+
+## ------------------------------------ ##
+## Ensure we run with a proper shell. ##
+## ------------------------------------ ##
+
+# Make sure we run with the shell detected at configure time (unless
+# the user forbids it).
+case ${AM_TESTS_REEXEC-yes} in
+ 1|[yY]|[yY]es)
+ AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+ # Cannot simply do `opts=$-', since the content of $- is not
+ # portable among different shells. So try to propagate only
+ # the portable and interesting options.
+ case $- in
+ *x*v*|*v*x) opts=-vx;;
+ *v*) opts=-v;;
+ *x*) opts=-x;;
+ *) opts=;;
+ esac
+ echo $me: exec $SHELL $opts "$0" "$*"
+ exec $SHELL $opts "$0" ${1+"$@"}
+ # NOTREACHED
+ echo "$me: failed to re-execute with $SHELL" >&2
+ exit 99
+ ;;
+esac
+
+# NOTE: From this point on, we can assume this file is being executed
+# by the configure-time detected $CONFIG_SHELL.
+
+
+## ---------------------- ##
+## Environment cleanup. ##
+## ---------------------- ##
+
# Unset some MAKE... variables that may cause $MAKE to act like a
# recursively invoked sub-make. Any $MAKE invocation in a test is
# conceptually an independent invocation, not part of the main
diff --git a/tests/defs-static.in b/tests/defs-static.in
index 2ca4b57..449e05d 100644
--- a/tests/defs-static.in
+++ b/tests/defs-static.in
@@ -16,9 +16,14 @@
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
# Defines and minimal setup for Automake testing environment.
-# IMPORTANT: Multiple inclusions of this file should be idempotent!
+
+# IMPORTANT NOTES AND REQUIREMENTS
+# - Multiple inclusions of this file should be idempotent.
+# - This file should execute correctly with any system's /bin/sh
+# shell, not only with configure-time detected $CONFIG_SHELL.
# Be more Bourne compatible.
# (Snippet copied from configure's initialization in Autoconf 2.64)
@@ -75,7 +80,7 @@ FGREP='@FGREP@'
# this variable.
TEX='@TEX@'
-# Whether /bin/sh has working 'set -e' with exit trap.
+# Whether $SHELL has working 'set -e' with exit trap.
sh_errexit_works='@sh_errexit_works@'
# The amount we should wait after modifying files depends on the platform.
--
1.7.1
