Le 29 mars 09 à 00:04, Ralf Wildenhues a écrit :
Hi Akim,
Hi Ralf!
Agreed. While I won't change the TESTS=foo.test semantics, we can
publish that the user can use TEST_LOGS=foo.log to limit the tests
to be
run.
Never thought about that :) Thanks.
- LAZY_TEST_SUITE is not flexible enough
It is global, it should be per test-case.
Indeed. I like your idea, but I don't like the naming scheme too
much,
nor do I think we need to be backwards compatible with an unpublished
interface. I'm removing the LAZY_TEST_SUITE API and replacing it
with a
RECHECK_LOGS API (instead of your STRICT_TESTS).
Sorry if this causes trouble for you.
Not sure that there's an Eclipse component to rename Make macros, but
I can afford a query-replace :)
I often use
STRICT_TEST_LOGS = $(shell $(LIST_FAILED_TEST_LOGS))
which makes all failing test strict. In other words, successful
tests
are not rerun by "make check", but failing tests are. This is
because
our test suite sometimes hits the limit of the machine, and tests
timeout for no sound reason. So there is no reason to rerun
successful
tests, but failing test might pass if the machine has a lesser load.
This is what the 'recheck' target does, right?
Not exactly. make recheck runs only the tests that failed, while
here, in addition, the tests whose dependencies have changed will be
run again. Not to mention test that where not run at all.
One thing that one needs to look out for is, when overriding variables
in recursive `make' instances is that non-GNU make don't override by
default unless you use `make -e' and pass via the environment.
Requiring our users to do so if they want to override variables is ok
IMVHO, because then they are warned that their environment needs to be
clean. OTOH using -e internally is not ok at all, as the users may
not
be aware, and their environment could cause subtle breakage. So
consequently we can only transport overrides one recursion deep and
not
any further, unless each lower re-sets the variables on the make
command
line.
Wow, man, you *are* an expert...
I have not understood exactly what you are referring too, but why
don't we just pass TESTS=$(TESTS) explicitly? Or ask the users to add
his "precious macros" to AM_MAKEFLAGS?
This detail was BTW the reason that I have backed off of allowing
shell
globbing for TESTS or TEST_LOGS. It is just too complicated to get
right portably, causes other problems (e.g., I do not want to have a
rule that has both TESTS and TEST_LOGS expanded in one shell command:
it might overrun command line length limits), and can be emulated by
the
user on the command line or with a script (or, for GNU make, a simple
wrapper target). Thus this patch documents such an example.
Can't we have it when using GNU Make? Can't we have some sort of
#ifdef HAVE_GNU_MAKE sections in the output Makefile? I'm not
referring to AM_CONDITIONAL, but to some nice run-time trick.
+## Rerun all FAILed or XPASSed tests (as well as all whose logs are
out
+## of date or do not exist).
+recheck-TESTS:
+ @list='$(TEST_LOGS)'; \
+ logs=`for f in $$list; do \
+ if read line < $$f; then \
+ case $$line in FAIL*|XPASS*) echo $$f;; esac; \
+ else echo $$f; fi; \
+ done | tr '\012\015' ' '`; \
+ $(MAKE) $(AM_MAKEFLAGS) check-TESTS RECHECK_LOGS="$$logs"
Maybe on purpose, maybe not, you dropped recheck-html.
This is great!
