tags 11897 + patch
severity 11897 minor
thanks
On 07/10/2012 01:44 PM, Max Horn wrote:
> Hi there,
>
Hi Max, thanks for the report.
> here is another test failure I am seeing on Mac OS X 10.7.4 with
> automake 1.12.2. Specifically, two tap test fail with errors.
> So when I execute
>
> make check TESTS="t/suffix8.tap t/suffix10.tap"
>
> then I get this (after removing some fluff):
>
>
> PASS: t/suffix8.tap 1 - libtoolize
> PASS: t/suffix8.tap 2 - aclocal
> PASS: t/suffix8.tap 3 - autoconf
> PASS: t/suffix8.tap 4 - automake
> PASS: t/suffix8.tap 5
> ERROR: t/suffix8.tap 5 - configure # OUT-OF-ORDER (expecting 6)
> ERROR: t/suffix8.tap 6 - make test0 # OUT-OF-ORDER (expecting 7)
> ERROR: t/suffix8.tap 7 - make test1 # OUT-OF-ORDER (expecting 8)
> ERROR: t/suffix8.tap 8 - make test2 # OUT-OF-ORDER (expecting 9)
> ERROR: t/suffix8.tap 9 - make all # OUT-OF-ORDER (expecting 10)
> ERROR: t/suffix8.tap 11 # UNPLANNED
> ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
> ERROR: t/suffix8.tap - too many tests run (expected 10, got 12)
> PASS: t/suffix10.tap 1 - libtoolize
> PASS: t/suffix10.tap 2 - aclocal
> PASS: t/suffix10.tap 3 - autoconf
> PASS: t/suffix10.tap 4 - automake
> PASS: t/suffix10.tap 5
> ERROR: t/suffix10.tap 5 - configure # OUT-OF-ORDER (expecting 6)
> ERROR: t/suffix10.tap 6 - make test # OUT-OF-ORDER (expecting 7)
> ERROR: t/suffix10.tap 7 - make all # UNPLANNED
> ERROR: t/suffix10.tap - too many tests run (expected 7, got 8)
>
>
> I think the problem is caused by the "configure" script spitting
> out these messages:
>
> checking for archiver @FILE support... rm: cannot remove `conftest.dSYM': Is
> a directory
> no
> checking for strip... strip
> checking for ranlib... ranlib
> checking command to parse /usr/bin/nm output from cc object... rm: cannot
> remove `conftest.dSYM': Is a directory
> ok
>
> Note the isolated "ok" on a line of its own; apparently this
> makes the test think that an unplanned test step occurred.
>
Your interpretation is absolutely correct. The attached patch should
take care of the issue. Could you give it a try?
> This in turn is caused by a "bug" in libtool -- or rather, there is
> a peculiar behavior of the Mac OS X compilers (namely to create
> conftest.dSYM directories under certain circumstances), and libtool
> hence needs to replace "rm -f conftest*" by "rm -rf conftest*". This
> was done some years ago, but apparently was forgotten in three cases.
>
> I already reported this to libtool:
> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11895>
>
> But since it directly affects the test suite of automake, I thought
> it best to report this here, too.
>
Well done. This is a spurious failure that shouldn't interfere with
our test results.
> Perhaps you would like to workaround it by making the test a bit
> more strict about how it detects a completed test step?
>
This is not possible; the '*.tap' tests in the Automake testsuite
use the TAP protocol, and that mandates which output lines must be
interpreted as declaration of test results. I think the workaround
implemented in the attached patch is the simplest and best fix.
Thanks,
Stefano
>From f2c18ffe38212751434b246d84c245e16c4dfd3d Mon Sep 17 00:00:00 2001
Message-Id: <f2c18ffe38212751434b246d84c245e16c4dfd3d.1341927056.git.stefano.lattar...@gmail.com>
From: Stefano Lattarini <stefano.lattar...@gmail.com>
Date: Tue, 10 Jul 2012 15:29:57 +0200
Subject: [PATCH] tests: avoid spurious TAP errors on Mac OS X 10.7
Fixes automake bug#1897. Reported by Max Horn.
* t/suffix8.tap: The libtool bug#11895 was causing the ./configure script
to output a stray "ok" string on a line of its own, confusing the TAP
driver into thinking this was an extra test result (which resulted in the
next, real test results being flagged as "OUT-OF-ORDER"). Fix this by
protecting configure output.
* t/suffix10.tap: Likewise.
* THANKS: Update.
Signed-off-by: Stefano Lattarini <stefano.lattar...@gmail.com>
---
THANKS | 1 +
t/suffix10.tap | 13 ++++++++++++-
t/suffix8.tap | 14 +++++++++++++-
3 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/THANKS b/THANKS
index 01b78c1..afdd33a 100644
--- a/THANKS
+++ b/THANKS
@@ -245,6 +245,7 @@ Matthew D. Langston langs...@slac.stanford.edu
Matthias Andree matthias.and...@gmx.de
Matthias Clasen cla...@mathematik.uni-freiburg.de
Matthias Klose d...@ubuntu.com
+Max Horn m...@quendi.de
Maxim Sinev g...@goods.ru
Maynard Johnson mayna...@us.ibm.com
Merijn de Jonge m.de.jo...@cwi.nl
diff --git a/t/suffix10.tap b/t/suffix10.tap
index a650924..7e19e04 100755
--- a/t/suffix10.tap
+++ b/t/suffix10.tap
@@ -57,11 +57,22 @@ cat > foo.x_ << 'END'
int foo (void) { return yyparse(); }
END
+# We must protect the TAP driver from the output of configure, since
+# that might output a stray "ok" on a line of its own (due to a
+# libtool bug on Mac OS X), thus causing a spurious test result to
+# be seen. See automake bug#11897.
+run_configure ()
+{
+ st=0; ./configure >output 2>&1 || st=1
+ sed 's/^/ /' output
+ test $st -eq 0
+}
+
command_ok_ "libtoolize" libtoolize --force
command_ok_ "aclocal" $ACLOCAL
command_ok_ "autoconf" $AUTOCONF
command_ok_ "automake" $AUTOMAKE --add-missing
-command_ok_ "configure" ./configure
+command_ok_ "configure" run_configure
command_ok_ "make test" $MAKE test
directive=''; make_can_chain_suffix_rules || directive=TODO
diff --git a/t/suffix8.tap b/t/suffix8.tap
index fdaf087..87d6530 100755
--- a/t/suffix8.tap
+++ b/t/suffix8.tap
@@ -75,11 +75,23 @@ END
echo 'int main (void) { return 0; }' > foo.x_
echo 'int bar (void) { return 0; }' > bar.x_
+
+# We must protect the TAP driver from the output of configure, since
+# that might output a stray "ok" on a line of its own (due to a
+# libtool bug on Mac OS X), thus causing a spurious test result to
+# be seen. See automake bug#11897.
+run_configure ()
+{
+ st=0; ./configure >output 2>&1 || st=1
+ sed 's/^/ /' output
+ test $st -eq 0
+}
+
command_ok_ "libtoolize" libtoolize
command_ok_ "aclocal" $ACLOCAL
command_ok_ "autoconf" $AUTOCONF
command_ok_ "automake" $AUTOMAKE -a
-command_ok_ "configure" ./configure
+command_ok_ "configure" run_configure
command_ok_ "make test0" env OBJEXT=foo $MAKE -e test0
command_ok_ "make test1" $MAKE test1
--
1.7.9.5
