Overhaul of tests/README.
* tests/README: General overhaul. Mention parallel test suite
execution, the test-suite.log file; document `Exit' vs. `exit',
`defs' vs. `defs-p', the naming of `parallel-tests' tests.
* README: Simplify, point to tests/README.
diff --git a/README b/README
index bb42a7f..0e23237 100644
--- a/README
+++ b/README
@@ -24,13 +24,7 @@ Automake has a test suite. Use
make check
to run it. Capture its output in case of failing tests. For more
-information, you can influence testing with the variables VERBOSE,
-MAKE, TESTS, keep_testdirs. So for example, to run tests/check.test
-verbosely using gmake and to inspect the test directory afterwards,
-enter the tests directory and use
-
- env VERBOSE=yes TESTS=check.test keep_testdirs=yes MAKE=gmake \
- gmake -e check
+information, please see the file tests/README.
Automake has a page on the web. See:
diff --git a/tests/README b/tests/README
index 4b63059..c4ff407 100644
--- a/tests/README
+++ b/tests/README
@@ -10,6 +10,9 @@ Running all tests
make check
+ You can use `-jN' for faster completion (it even helps on a
+ uniprocessor system, due to unavoidable sleep delays, as
+ noted below).
Interpretation
--------------
@@ -30,10 +33,13 @@ Getting details from failures
-----------------------------
Each test is a script. In a non-VPATH build you can run them
- directly, they will be verbose.
+ directly, they will be verbose. By default, verbose output of
+ a test foo.test is retained in the log file foo.log. A summary
+ log is created in the file test-suite.log.
- Otherwise, you can invoke make as follows, just replace the list
- of tests by those you want to check.
+ You can limit the set of files using the TESTS variable, and
+ enable detailed test output at the end of the test run with the
+ VERBOSE variable:
env VERBOSE=x TESTS='first.test second.test ...' make -e check
@@ -41,10 +47,10 @@ Getting details from failures
Reporting failures
------------------
- Send verbose output of failing tests to <bug-autom...@gnu.org>,
- along with the usual version numbers (which Automake, which
- Autoconf, which operating system, which make version, which shell,
- etc.)
+ Send verbose output, i.e., the contents of test-suite.log, of failing
+ tests to <bug-autom...@gnu.org>, along with the usual version numbers
+ (which Automake, which Autoconf, which operating system, which make
+ version, which shell, etc.)
@@ -68,7 +74,10 @@ Do
Use `required=...' for required tools.
- Include ./defs (see other tests).
+ Include ./defs (see other tests) for normal tests, ./defs-p for tests
+ that use the `parallel-tests' option. For the latter, use a name that
+ ends in `-p.test' and does not clash with any generated tests in the
+ suite.
Use `set -e' to catch failures you might not have thought of.
@@ -87,11 +96,16 @@ Do
Use `cat' or `grep' to display (part of) files that may be
interesting for debugging, so that when a user send a verbose
- output we don't have to ask him for more details.
+ output we don't have to ask him for more details. Display stderr
+ output on the stderr file descriptor. If some redirected command
+ is likely to fail, and `set -e' is in effect, display its output
+ even in the failure case, before exiting.
+
+ Use `Exit' rather than `exit' to abort a test.
It's more important to make sure that a feature works, than
make sure that Automake's output looks correct. It might look
- correct and still fails to work. In other words, prefer
+ correct and still fail to work. In other words, prefer
running `make' over grepping `Makefile.in' (or do both).
If you run $AUTOMAKE or $AUTOCONF several times in the same test
@@ -99,6 +113,7 @@ Do
rm -rf autom4te.cache
before the following runs. On fast machines the new `configure.in'
could otherwise have the same timestamp as the old `autom4te.cache'.
+ Alternatively, use `--force' for subsequent runs of the tools.
Use filenames with two consecutive spaces when testing that some
code preserves filenames with spaces. This will catch errors like
@@ -112,7 +127,7 @@ Do
Do not
------
- Do not test an Automake error with `$AUTOMAKE && exit 1', or in three
+ Do not test an Automake error with `$AUTOMAKE && Exit 1', or in three
years we'll discover that this test failed for some other bogus reason.
This happened many times. Better use something like
AUTOMAKE_fails
