Simon Josefsson wrote:
> Most of my projects have multiple gnulib instances in
> them, which gnulib self-tests does not support.
Yes, this situation is not supported.
> I try to have gnulib tests enabled, but sometimes I disable them because
> having them enabled leads to problems that are too time-consuming to
> debug and fix. ...
>
> I've seen over the years a number of cases where old releases of my
> packages fail to 'make check' correctly only because of a gnulib
> self-test that was 1) simply buggy, or 2) the gnulib replacement code
> had bugs in them on some platform, or 3) the self-test tested a
> corner-case that newer platforms (for valid reasons) chose to behave
> differently for.
Failing tests can happen, yes. That's true for all test suites.
But it shouldn't be you as a package maintainer who debugs these tests.
(Only the tests for the modules that you contributed; i.e. wearing your
Gnulib contributor hat :-).
> I think there is room for improvements in this field
>From what you are telling, the first improvement should be to clarify
where to report failing tests. Done as follows:
2021-06-10 Bruno Haible <br...@clisp.org>
Clarify where to report test failures from Gnulib tests.
* gnulib-tool (func_emit_tests_Makefile_am): Emit a dependency of
'check-am' on 'check-notice' that prints a notice.
diff --git a/gnulib-tool b/gnulib-tool
index d41d58c..4f97e63 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -4275,6 +4275,14 @@ func_emit_tests_Makefile_am ()
echo " @echo '## You can ignore compiler warnings in this directory.
##'"
echo " @echo '## ----------------------------------------------------
##'"
echo
+ # Arrange to print a message before executing the tests in this directory.
+ echo "check-am: check-notice"
+ echo "check-notice:"
+ echo " @echo '## ----------------------------------------------------
##'"
+ echo " @echo '## ------------------- Gnulib tests -------------------
##'"
+ echo " @echo '## Please report test failures to <bug-gnulib@gnu.org>.
##'"
+ echo " @echo '## ----------------------------------------------------
##'"
+ echo
echo "# Clean up after Solaris cc."
echo "clean-local:"
echo " rm -rf SunWS_cache"
