All, I've noticed that 'make dist' tarballs can be different from 'make release' tarballs with gnulib, the difference is in aclocal.m4 and Makefile.in:
│ ├── libidn2-2.3.8/aclocal.m4 │ │ @@ -1198,33 +1198,14 @@ │ │ m4_include([unistring/m4/unictype_h.m4]) │ │ m4_include([unistring/m4/uninorm_h.m4]) │ │ m4_include([unistring/m4/wchar_h.m4]) │ │ -m4_include([m4/ax_ac_append_to_file.m4]) ... │ │ -m4_include([m4/progtest.m4]) │ │ m4_include([gl/m4/00gnulib.m4]) │ │ m4_include([gl/m4/__inline.m4]) │ │ m4_include([gl/m4/absolute-header.m4]) │ │ m4_include([gl/m4/alloca.m4]) │ │ m4_include([gl/m4/assert_h.m4]) │ │ m4_include([gl/m4/build-to-host.m4]) │ │ m4_include([gl/m4/c-bool.m4]) │ │ @@ -1294,7 +1275,26 @@ │ │ m4_include([gl/m4/valgrind-tests.m4]) │ │ m4_include([gl/m4/version-etc.m4]) │ │ m4_include([gl/m4/visibility.m4]) │ │ m4_include([gl/m4/warn-on-use.m4]) │ │ m4_include([gl/m4/warnings.m4]) │ │ m4_include([gl/m4/wint_t.m4]) │ │ m4_include([gl/m4/zzgnulib.m4]) │ │ +m4_include([m4/ax_ac_append_to_file.m4]) ... │ │ +m4_include([m4/progtest.m4]) │ ├── libidn2-2.3.8/doc/Makefile.in │ │ $(top_srcdir)/unistring/m4/unictype_h.m4 \ │ │ $(top_srcdir)/unistring/m4/uninorm_h.m4 \ │ │ $(top_srcdir)/unistring/m4/wchar_h.m4 \ │ │ - $(top_srcdir)/m4/ax_ac_append_to_file.m4 \ ... │ │ - $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \ │ │ $(top_srcdir)/gl/m4/00gnulib.m4 \ │ │ $(top_srcdir)/gl/m4/__inline.m4 \ │ │ $(top_srcdir)/gl/m4/absolute-header.m4 \ │ │ $(top_srcdir)/gl/m4/alloca.m4 $(top_srcdir)/gl/m4/assert_h.m4 \ │ │ $(top_srcdir)/gl/m4/build-to-host.m4 \ │ │ $(top_srcdir)/gl/m4/c-bool.m4 $(top_srcdir)/gl/m4/close.m4 \ │ │ $(top_srcdir)/gl/m4/codeset.m4 \ │ │ @@ -189,15 +176,29 @@ │ │ $(top_srcdir)/gl/m4/sys_types_h.m4 \ │ │ $(top_srcdir)/gl/m4/time_h.m4 $(top_srcdir)/gl/m4/unistd_h.m4 \ │ │ $(top_srcdir)/gl/m4/valgrind-tests.m4 \ │ │ $(top_srcdir)/gl/m4/version-etc.m4 \ │ │ $(top_srcdir)/gl/m4/visibility.m4 \ │ │ $(top_srcdir)/gl/m4/warn-on-use.m4 \ │ │ $(top_srcdir)/gl/m4/warnings.m4 $(top_srcdir)/gl/m4/wint_t.m4 \ │ │ - $(top_srcdir)/gl/m4/zzgnulib.m4 $(top_srcdir)/configure.ac │ │ + $(top_srcdir)/gl/m4/zzgnulib.m4 \ │ │ + $(top_srcdir)/m4/ax_ac_append_to_file.m4 \ ... │ │ + $(top_srcdir)/m4/po.m4 $(top_srcdir)/m4/progtest.m4 \ │ │ + $(top_srcdir)/configure.ac │ │ am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ │ │ $(ACLOCAL_M4) If the diff is unreadable, the summary is that it just moves around things. The ordering seems to be coming from the order that aclocal uses, which normally (with autoreconf) would use the configure.ac order, like this: AC_CONFIG_MACRO_DIRS([m4 gl/m4 unistring/m4]) However bootstrap-funclib.sh invokes it with an explicit -I like this: # Invoke autoreconf with --force --install to ensure upgrades of tools # such as ylwrap. AUTORECONFFLAGS="--verbose --install --force -I $m4_base $ACLOCAL_FLAGS" Here $m4_base is gnulib's M4 base, typically coming from bootstrap.conf. The problem happens because 'make release' invoke (via GNUmakefile '_version' target) autoreconf to re-generate files for the new version number. I believe this step is working fine, and uses the "right" m4 directory path as specified by configure.ac. It is bootstrap that is using a different m4 path ordering, which I suspect is "incorrect". These two different usages is causing the problem. Why is bootstrap-funclib.m4 adding that -I? To get reproducible tarballs, I could re-order the directories in AC_CONFIG_MACRO_DIRS so that gnulib's M4 directory comes first. Or to always run 'autoreconf' after running bootstrap. This leads to other issues or is inefficient. I prefer having the gnulib M4 directory further down the aclocal include search path, so that I can have a top-level m4/ with my own custom-maintained files and not have things cluttered with all the generated gnulib files. But there shouldn't be any requirement that the gnulib m4 directory is the first, should there? I don't recall seeing a requirement like that. This becomes complicated when you have multiple gnulib instances in the same package. Since the m4 search path affects tarball reproducibility, I think we need to be careful and always use the maintainer's preferred order. Gnulib already properly document that one has to add gnulib's m4 path to that order. (Another fix could be for aclocal to sort all *.m4 in a deterministic order before calling 'm4_include', but I suspect that may break things -- what if you have different versions of the same file in two different directories? You want the maintainer order of things, presumably.) What do you think about this change? - AUTORECONFFLAGS="--verbose --install --force -I $m4_base $ACLOCAL_FLAGS" + AUTORECONFFLAGS="--verbose --install --force $ACLOCAL_FLAGS" The worse that I can see happening is that someone who run ./bootstrap on a package before it is completely gnulib-ified will get some warning about missing M4 functions until they add the gnulib m4_base to aclocal -I and/or configure.ac AC_CONFIG_MACRO_DIRS. I have not tested this, but wanted to (attempt to) bring up a problem description first. If nobody objects to the idea of changing things here, I can try it more and see if it breaks anything else. /Simon
signature.asc
Description: PGP signature
