bug#14528: remake-configure-dependencies test fails on fast systems without subsecond timestamps
Automake 1.13.2 failed t/remake-configure-dependencies.sh in two
consecutive attempts on Ubuntu autobuilders. On inspection I found that
our autobuilder network happened to pick the same backend machine each
time, which was a relatively old one still using ext3. (Our newer
builders use ext4.) On running this test in a loop-mounted ext3
filesystem on my laptop, I reproduced the failure first time:
Running from installcheck: no
Test Protocol: none
PATH =
/home/cjwatson/src/ubuntu/automake1.13/tmp/t/automake1.13-1.13.2/t/wrap:/home/cjwatson/src/ubuntu/automake1.13/tmp/t/automake1.13-1.13.2/t/ax:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ pwd
/home/cjwatson/src/ubuntu/automake1.13/tmp/t/automake1.13-1.13.2/t/remake-configure-dependencies.dir
+ cat
+ cat
+ cat
+ echo '#!/bin/sh'
+ echo 'printf %s 1.0'
+ chmod a+x print-version
+ aclocal-1.13 -Werror
+ autoconf
+ automake-1.13 --foreign -Werror -Wall
+ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
+ make test-1
test 1.0 = 1.0
+ echo '#!/bin/sh'
+ echo 'printf %s 2.1'
+ using_gmake
+ case $am__using_gmake in
+ make --version -v
+ grep GNU
GNU Make 3.81
+ am__using_gmake=yes
+ return 0
+ make test-2
cd . && /bin/bash
/home/cjwatson/src/ubuntu/automake1.13/tmp/t/automake1.13-1.13.2/t/remake-configure-dependencies.dir/missing
automake-1.13 --foreign
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash
/home/cjwatson/src/ubuntu/automake1.13/tmp/t/automake1.13-1.13.2/t/remake-configure-dependencies.dir/missing
autoconf
test 1.0 = 2.1
make: *** [test-2] Error 1
+ am_exit_trap 2
+ exit_status=2
+ set +e
+ cd /home/cjwatson/src/ubuntu/automake1.13/tmp/t/automake1.13-1.13.2
+ test none = tap
+ case $am_explicit_skips in
+ test 2 -eq 0
+ keep_testdirs=yes
+ am_keeping_testdirs
+ case $keep_testdirs in
+ return 0
+ set +x
remake-configure-dependencies: exit 2
I think perhaps a sprinkling of $sleep is called for in this test?
Thanks,
--
Colin Watson [cjwat...@ubuntu.com]
bug#14528: remake-configure-dependencies test fails on fast systems without subsecond timestamps
On Mon, Jun 03, 2013 at 11:29:08AM +0200, Stefano Lattarini wrote: > You are perfectly right. I thought the sleeps implicit in configure > invocations were enough to avoid this kind of issues, but after a more > careful consideration I realized I was mistaken (see commit message in > the patch below for an extended rationale). > > Can you confirm the patch works for you? Before this patch, I got 32 passes from 100 runs; after this patch, I get 100 passes from 100 runs. So it looks good to me. Thanks, -- Colin Watson [cjwat...@ubuntu.com]
Translated manual pages
I recently ported man-db to Automake (using version 1.10). The main difficulty I encountered was in handling translated manual pages. Automake only appears to support installing manual pages in $(mandir)/man%SECTION%. However, this is only appropriate for English manual pages. Pages in other languages should be installed in $(mandir)/$(LANGUAGE)/man%SECTION%. I also had the problem that my section 5 manual pages shouldn't go through program name transformation, because they describe configuration files, not programs. I think only sections 1, 6, and 8 should be transformed. I eventually worked around this using a stub Makefile.am for each language which looks like this (for English, LANGUAGE = .): LANGUAGE = de.UTF-8 man1_MANS = \ man1/apropos.1 \ man1/man.1 \ man1/manpath.1 \ man1/whatis.1 \ man1/zsoelim.1 man5_MANS = \ man5/manpath.5 man8_MANS = \ man8/accessdb.8 \ man8/catman.8 \ man8/mandb.8 include $(top_srcdir)/man/Rules.man ... and the following Rules.man, largely cribbed from mans.am and adjusted for my needs (the SED_ARGS bit existed beforehand and I could probably do this better some other way, maybe by making the generated pages be configure output files instead): ## Copyright (C) 1994, 1995 Graeme Wilford. ## Copyright (C) 2001, 2002, 2003, 2007 Colin Watson. ## Much of the following is based on automake/lib/am/mans.am, which is: ## Copyright (C) 1998, 2001, 2003, 2004, 2006 Free Software Foundation, Inc. ## ## This file is part of man-db. ## ## man-db is free software; you can redistribute it and/or modify it ## under the terms of the GNU General Public License as published by ## the Free Software Foundation; either version 2 of the License, or ## (at your option) any later version. ## ## man-db is distributed in the hope that it will be useful, but ## WITHOUT ANY WARRANTY; without even the implied warranty of ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ## GNU General Public License for more details. ## ## You should have received a copy of the GNU General Public License ## along with man-db; if not, write to the Free Software Foundation, ## Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA CLEANFILES = $(man1_MANS) $(man5_MANS) $(man8_MANS) SED_ARGS = \ -e 's,%pager%,$(pager),g' \ -e 's,%troff%,$(troff),g' \ -e 's,%date%,$(date),g' \ -e 's,%version%,$(VERSION),g' \ -e 's,%zsoelim%,$(TRANS_ZSOELIM),g' \ -e 's,%man%,$(TRANS_MAN),g' \ -e 's,%mandb%,$(TRANS_MANDB),g' \ -e 's,%manpath%,$(TRANS_MANPATH),g' \ -e 's,%catman%,$(TRANS_CATMAN),g' \ -e 's,%apropos%,$(TRANS_APROPOS),g' \ -e 's,%whatis%,$(TRANS_WHATIS),g' \ -e 's,%manconv%,$(TRANS_MANCONV),g' \ -e 's,%thzsoelim%,$(TRANS_ZSOELIM_UPPER),g' \ -e 's,%thman%,$(TRANS_MAN_UPPER),g' \ -e 's,%thmandb%,$(TRANS_MANDB_UPPER),g' \ -e 's,%thmanpath%,$(TRANS_MANPATH_UPPER),g' \ -e 's,%thcatman%,$(TRANS_CATMAN_UPPER),g' \ -e 's,%thapropos%,$(TRANS_APROPOS_UPPER),g' \ -e 's,%thwhatis%,$(TRANS_WHATIS_UPPER),g' \ -e 's,%thmanconv%,$(TRANS_MANCONV_UPPER),g' \ -e 's,%manpath_config_file%,$(config_file),g' .man1.1: $(MKDIR_P) man1 sed $(SED_ARGS) $< > $@ .man5.5: $(MKDIR_P) man5 sed $(SED_ARGS) $< > $@ .man8.8: $(MKDIR_P) man8 sed $(SED_ARGS) $< > $@ install-data-local: @set -e; \ for s in 1 5 8; do \ test -z "$(mandir)" || $(MKDIR_P) "$(DESTDIR)$(mandir)/$(LANGUAGE)/man$$s"; \ done; \ list='$(man1_MANS) $(man8_MANS)'; \ for i in $$list; do \ ext=`echo $$i | sed -e 's/^.*\\.//'`; \ inst=`echo $$i | sed -e 's/\\.[0-9a-z]*$$//'`; \ inst=`echo $$inst | sed -e 's/^.*\///'`; \ inst=`echo $$inst | sed '$(transform)'`.$$ext; \ echo " $(INSTALL_DATA) '$$i' '$(DESTDIR)$(mandir)/$(LANGUAGE)/$$inst'"; \ $(INSTALL_DATA) "$$i" "$(DESTDIR)$(mandir)/$(LANGUAGE)/$$inst"; \ done; \ list='$(man5_MANS)'; \ for i in $$list; do \ echo " $(INSTALL_DATA) '$$i' '$(DESTDIR)$(mandir)/$(LANGUAGE)/$$i'"; \ $(INSTALL_DATA) "$$i" "$(DESTDIR)$(mandir)/$(LANGUAGE)/$$i"; \ done uninstall-local: @list='$(man1_MANS) $(man8_MANS)'; \ for i in $$list; do \ ext=`echo $$i | sed -e 's/^.*\\.//'`;
aclocal.m4 not regenerated when Makefile.am changes
In a fresh unpacked copy of http://download.savannah.nongnu.org/releases/man-db/man-db-2.5.1.tar.gz: $ touch Makefile.am $ ./configure $ make configure.ac:7: version mismatch. This is Automake 1.10.1, configure.ac:7: but the definition used by this AM_INIT_AUTOMAKE configure.ac:7: comes from Automake 1.10. You should recreate configure.ac:7: aclocal.m4 with aclocal and run automake again. WARNING: `automake-1.10' is probably too old. You should only need it if you modified `Makefile.am', `acinclude.m4' or `configure.ac'. You might want to install the `Automake' and `Perl' packages. Grab them from any GNU archive site. Of course, I have upgraded automake on my system to 1.10.1 since producing that tarball. But why didn't the rebuild rules just run aclocal for me? This seems as simple as adding Makefile.am to the dependencies of $(ACLOCAL_M4). If nothing else, Makefile.am may contain ACLOCAL_AMFLAGS (and in fact does in this case), so IMO correct rebuild rules should re-run aclocal in case those flags changed; although in order to get that completely right it also ought to fish ACLOCAL_AMFLAGS out of Makefile.am à la autoreconf ... Thanks, -- Colin Watson [EMAIL PROTECTED]
'nonemtpy' typo in lib/am/tags.am
The attached patch fixes a typo I happened to notice while upgrading
man-db to Automake 1.10.1.
Thanks,
--
Colin Watson [EMAIL PROTECTED]
diff --git a/lib/am/tags.am b/lib/am/tags.am
index 5383e0d..d279b19 100644
--- a/lib/am/tags.am
+++ b/lib/am/tags.am
@@ -26,7 +26,7 @@ ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES)
## Handle VPATH correctly.
if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \
done | \
- $(AWK) '{ files[$$0] = 1; nonemtpy = 1; } \
+ $(AWK) '{ files[$$0] = 1; nonempty = 1; } \
END { if (nonempty) { for (i in files) print i; }; }'`; \
mkid -fID $$unique
Re: aclocal.m4 not regenerated when Makefile.am changes
On Sun, Feb 17, 2008 at 06:48:07PM +0100, Ralf Wildenhues wrote: > * Colin Watson wrote on Sun, Feb 17, 2008 at 04:23:32PM CET: > > In a fresh unpacked copy of > > http://download.savannah.nongnu.org/releases/man-db/man-db-2.5.1.tar.gz: > > > > $ touch Makefile.am > > $ ./configure > > $ make > > configure.ac:7: version mismatch. This is Automake 1.10.1, > > configure.ac:7: but the definition used by this AM_INIT_AUTOMAKE > > configure.ac:7: comes from Automake 1.10. You should recreate > > configure.ac:7: aclocal.m4 with aclocal and run automake again. > > WARNING: `automake-1.10' is probably too old. You should only need it if > >you modified `Makefile.am', `acinclude.m4' or `configure.ac'. > >You might want to install the `Automake' and `Perl' packages. > >Grab them from any GNU archive site. > > > > Of course, I have upgraded automake on my system to 1.10.1 since > > producing that tarball. But why didn't the rebuild rules just run > > aclocal for me? This seems as simple as adding Makefile.am to the > > dependencies of $(ACLOCAL_M4). > > But normally, aclocal.m4 does not need to be regenerated when > Makefile.am is updated. What you would really need in your case is a > dependency of aclocal.m4 on /usr/bin/aclocal-1.10. > > Since we don't have dependencies on installed packages, pragmatically > the best thing to do when you update system tools is to run autoreconf. Mm. I don't rely on the rebuild rules myself, of course; I have an autogen.sh script which calls autoreconf and a few other things. The reason I noticed this was in the context of ensuring that users get Makefile.in et al automatically upgraded if they need to change Makefile.am for some reason. I can arrange that they have at least some required version by way of a version number in AM_INIT_AUTOMAKE, but I can't arrange that they have exactly the same version I have. > > If nothing else, Makefile.am may contain ACLOCAL_AMFLAGS (and in fact > > does in this case), so IMO correct rebuild rules should re-run aclocal > > in case those flags changed; > > Good point. I don't know how to formulate this without making a blanket > dependency on Makefile.am, though; and that would, for the vast majority > of use cases, be much more work than is normally needed. Users would > complain about this. I see; fair point. I don't suppose it would be possible for automake, when it would normally emit the message quoted above, to run aclocal itself and then re-exec itself, perhaps with a guard against an infinite loop? > > although in order to get that completely > > right it also ought to fish ACLOCAL_AMFLAGS out of Makefile.am à la > > autoreconf ... > > This does happen though, when the aclocal rule is executes: it has > $(ACLOCAL) $(ACLOCAL_AMFLAGS) That's the previous value of ACLOCAL_AMFLAGS, though (the one in place when automake and configure were last run), rather than the current one in Makefile.am. Thanks, -- Colin Watson [EMAIL PROTECTED]
Re: Serial number formats
On Tue, Mar 17, 2009 at 01:14:33AM +0100, Bruno Haible wrote:
> Colin Watson wrote:
> > I would find it more elegant to install all the files separately and
> > have a defined ordering between them
>
> It may work for you. For the general developer, I think it opens too
> many pitfalls (missing -I options, confusion about which file is used,
> possibly empty directories after "make distrib", ...).
I take the point about 'make dist', at least.
How about the attached patch to autopoint which causes it to look
through all m4 directories in ACLOCAL_AMFLAGS and compare serial
numbers? That would avoid the problem of 'make dist' forgetting to ship
files even if you do use multiple m4 directories; the files will simply
not be copied into the package at all, which surely makes it easier to
understand what's going on.
> > Beyond the gettext compatibility issue, there are many files in gnulib
> > that have serial lines in a format not recognised by aclocal that have
> > nothing to do with gettext. Is there a good reason for these?
>
> The reason is transparency and ease of debugging: There are cases when
> 'aclocal' copies the contents of the .m4 file. When I have a configure file
> that does not work and look at the aclocal.m4 from which it was generated,
> I want to know from which .m4 file each snippet came from.
You can do this with aclocal's documented format too, since it permits
and ignores any trailing garbage after the serial number itself.
# serial 5 snprintf.m4
Thanks,
--
Colin Watson [cjwat...@debian.org]
Index: gettext-tools/misc/autopoint.in
===
RCS file: /sources/gettext/gettext/gettext-tools/misc/autopoint.in,v
retrieving revision 1.40
diff -p -u -r1.40 autopoint.in
--- gettext-tools/misc/autopoint.in 7 Dec 2008 16:20:56 - 1.40
+++ gettext-tools/misc/autopoint.in 17 Mar 2009 02:06:23 -
@@ -352,6 +352,7 @@ fi
# Check in which directory the *.m4 macros belong.
m4dir=m4
+all_m4dirs=
if test -f Makefile.am; then
# A package using automake.
# Extract the macro directory name from Makefile.am.
@@ -359,8 +360,11 @@ if test -f Makefile.am; then
m4dir_is_next=
for arg in $aclocal_amflags; do
if test -n "$m4dir_is_next"; then
- m4dir="$arg"
- break
+ if [ -z "$m4dir" ]; then
+m4dir="$arg"
+ fi
+ all_m4dirs="$all_m4dirs $arg"
+ m4dir_is_next=
else
if test "X$arg" = "X-I"; then
m4dir_is_next=yes
@@ -505,6 +509,30 @@ func_compare ()
fi
}
+# func_compare_all gettextfile
+# searches for all files in the package with the same name as gettextfile,
+# and runs func_compare on each. Returns 0 if the file from gettext can be
+# used, or 1 if it should be skipped.
+func_compare_all ()
+{
+ case "$1" in
+*.m4)
+ base=`echo "$1" | sed -e 's,.*/,,'`
+ for dir in $all_m4dirs; do
+if test -f "$dir/$base"; then
+ if ! func_compare "$dir/$base" "$1"; then
+return 1
+ fi
+fi
+ done
+ true
+ ;;
+*)
+ true
+ ;;
+ esac
+}
+
# If some files have been locally modified and we have not been requested
# to overwrite them, then bail out. This is better than leaving a source
# package around where half of the files are locally modified and half are
@@ -514,10 +542,10 @@ if test -z "$force"; then
func_tmpdir
mismatchfile="$tmp"/autopoint.diff
for file in `find "$work_dir/archive" -type f -print | sed -e "s,^$work_dir/archive/,," | LC_ALL=C sort`; do
-func_destfile "$file"
-if test -n "$destfile"; then
- if test -f "$destfile"; then
-if func_compare "$destfile" "$work_dir/archive/$file"; then
+if func_compare_all "$work_dir/archive/$file"; then
+ func_destfile "$file"
+ if test -n "$destfile"; then
+if test -f "$destfile"; then
if test -n "$sharedowner"; then
echo "autopoint: warning: File $destfile has been locally modified." 1>&2
else
@@ -581,11 +609,11 @@ func_backup ()
# Now copy the files.
for file in `find "$work_dir/archive" -type f -print | sed -e "s,^$work_dir/archive/,," | LC_ALL=C sort`; do
- func_destfile "$file"
mustcopy=
- if test -n "$destfile"; then
-if test -f "$destfile"; then
- if func_compare "$destfile" "$work_dir/archive/$file"; then
+ if func_compare_all "$work_dir/archive/$file"; then
+func_destfile "$file"
+if test -n "$destfile"; then
+ if test -f "$destfile"; then
if test -n "$force"; then
# Overwrite locally modified file.
mustcopy=yes
