Today's changes:
2022-08-09 Bruno Haible <br...@clisp.org>
gnulib-tool.py: Finish implementing option --conditional-dependencies.
* gnulib-tool.py (main) Accept options --conditional-dependencies,
--no-conditional-dependencies.
* pygnulib/GLModuleSystem.py (GLModuleTable.addConditional): Use
str(module), not module, as key. Fix logic bug.
(GLModuleTable.getCondition): Simplify.
(GLModuleTable.transitive_closure): Show a warning when there are
duplicate dependencies. Fix logic bug.
(GLModuleTable.transitive_closure_separately): Simplify.
(GLModuleTable.add_dummy): Ignore tests modules. Cope with multiple
lib_SOURCES augmentation lines. Cope with comments at the end of a
lib_SOURCES augmentation line. Add the dummy module at the end of the
modules list.
* pygnulib/GLTestDir.py (GLTestDir.execute): Remove the code that forces
the dummy module to the end of the list.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippets): Add code to
terminate the shell functions. Add code for the dependencies from the
unconditional to the conditional modules. Don't emit AM_CONDITIONAL for
unconditional modules.
gnulib-tool.py: Don't do license replacements in the autoconf snippets.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Remove fileassistant
argument. Don't invoke the 'aux' transformer here. Don't produce Windows
CR-LFs on Windows.
(GLEmiter.autoconfSnippets): Remove fileassistant argument.
* pygnulib/GLImport.py (GLImport.gnulib_comp): Update all callers.
* pygnulib/GLTestDir.py (GLTestDir.execute): Likewise.
gnulib-tool.py: Fix some code generation details.
* pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): When removing a
lib_LDFLAGS line, remove also the newline. Fix regex that matches
lib_SOMETHING. Add a newline after each '## begin gnulib module' line.
Don't emit 'endif' lines without corresponding 'if'. When emitting a
'+=' augmentation, make sure it does not get emitted a second time.
Don't emit a blank line when there is no AM_CPPFLAGS augmentation.
Update after getLink() changed. In the value of DEFAULT_TEXT_DOMAIN,
backslash-escape the double-quotes. Don't produce Windows CR-LFs on
Windows. Simplify.
(GLEmiter.tests_Makefile_am): When removing a lib_LDFLAGS line, remove
also the newline. Fix regex that matches lib_SOMETHING. Don't remove a
blank line before EXTRA_DIST. Add a newline after each
'## begin gnulib module' line. Set uses_subdirs also when there is a .c
file in a subdir of tests/. When emitting a '+=' augmentation, make sure
it does not get emitted a second time. Don't produce Windows CR-LFs on
Windows. Simplify.
* pygnulib/GLImport.py (GLImport.execute): Update after getLink()
changed.
gnulib-tool.py: Fixes for conditional dependencies.
* pygnulib/GLModuleSystem.py (GLModule.shell_id_chars): New constant.
(GLModule.getShellFunc): Don't use md5 just because of an '_' character.
(GLModule.getShellVar): Likewise.
(GLModule.getConditionalName): Include a newline in the md5 input.
* pygnulib/constants.py (ALPHANUMERIC): Remove constant.
gnulib-tool.py: Refactor.
* pygnulib/GLModuleSystem.py (GLModule.getLicense): Separate the warning
logic from the result logic.
>From d0e094317bbf34413e458f9551543335b6eb8cef Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Mon, 8 Aug 2022 18:42:45 +0200
Subject: [PATCH 1/5] gnulib-tool.py: Refactor.
* pygnulib/GLModuleSystem.py (GLModule.getLicense): Separate the warning
logic from the result logic.
---
ChangeLog | 6 ++++++
pygnulib/GLModuleSystem.py | 21 +++++++++++----------
2 files changed, 17 insertions(+), 10 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 5dc57dc4f8..596f629316 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2022-08-09 Bruno Haible <br...@clisp.org>
+
+ gnulib-tool.py: Refactor.
+ * pygnulib/GLModuleSystem.py (GLModule.getLicense): Separate the warning
+ logic from the result logic.
+
2022-08-09 Paul Eggert <egg...@cs.ucla.edu>
largefile, year2038: simplify if Autoconf 2.72
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 19d13d213b..13d6a72231 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -651,23 +651,24 @@ class GLModule(object):
Get license and warn user if module lacks a license.'''
if 'license' not in self.cache:
- result = None
+ license = self.getLicense_Raw().strip()
+ # Warn if the License field is missing.
+ if not self.isTests():
+ if not license:
+ if self.config['errors']:
+ raise GLError(18, str(self))
+ else: # if not self.config['errors']
+ sys.stderr.write('gnulib-tool: warning: module %s lacks a License\n' % str(self))
if str(self) == 'parse-datetime':
# This module is under a weaker license only for the purpose of some
# users who hand-edit it and don't use gnulib-tool. For the regular
# gnulib users they are under a stricter license.
result = 'GPL'
else:
- license = self.getLicense_Raw().strip()
- if not self.isTests():
- if not license:
- if self.config['errors']:
- raise GLError(18, str(self))
- else: # if not self.config['errors']
- sys.stderr.write('gnulib-tool: warning: module %s lacks a license\n' % str(self))
- if not license:
- license = 'GPL'
result = license
+ # The default is GPL.
+ if not result:
+ result = 'GPL'
self.cache['license'] = result
return self.cache['license']
--
2.34.1
>From 501feeb5d595d8b4cea9a1a8e404277c27d2f5bf Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Mon, 8 Aug 2022 21:22:15 +0200
Subject: [PATCH 2/5] gnulib-tool.py: Fixes for conditional dependencies.
* pygnulib/GLModuleSystem.py (GLModule.shell_id_chars): New constant.
(GLModule.getShellFunc): Don't use md5 just because of an '_' character.
(GLModule.getShellVar): Likewise.
(GLModule.getConditionalName): Include a newline in the md5 input.
* pygnulib/constants.py (ALPHANUMERIC): Remove constant.
---
ChangeLog | 7 ++++
pygnulib/GLModuleSystem.py | 70 ++++++++++++++++++++------------------
pygnulib/constants.py | 3 --
3 files changed, 44 insertions(+), 36 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 596f629316..091af1f974 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
2022-08-09 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fixes for conditional dependencies.
+ * pygnulib/GLModuleSystem.py (GLModule.shell_id_chars): New constant.
+ (GLModule.getShellFunc): Don't use md5 just because of an '_' character.
+ (GLModule.getShellVar): Likewise.
+ (GLModule.getConditionalName): Include a newline in the md5 input.
+ * pygnulib/constants.py (ALPHANUMERIC): Remove constant.
+
gnulib-tool.py: Refactor.
* pygnulib/GLModuleSystem.py (GLModule.getLicense): Separate the warning
logic from the result logic.
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 13d6a72231..9bfd5bd266 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -183,6 +183,9 @@ class GLModule(object):
+ 'Makefile\\.am|Include|Link|License|Maintainer):$',
re.M)
+ # List of characters allowed in shell identifiers.
+ shell_id_chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_'
+
def __init__(self, config, path, patched=False):
'''GLModule.__init__(config, path[, patched]) -> GLModule
@@ -329,20 +332,19 @@ class GLModule(object):
Computes the shell function name that will contain the m4 macros for the
module.'''
- isalnum = True
macro_prefix = self.config['macro_prefix']
- for char in str(module):
- if char not in constants.ALPHANUMERIC:
- isalnum = False
+ valid_shell_id = True
+ for char in self.getName():
+ if char not in GLModule.shell_id_chars:
+ valid_shell_id = False
break
- if isalnum:
- module = str(self)
- else: # if not isalnum
- module = '%s\n' % str(self)
- if type(module) is str:
- module = module.encode(ENCS['default'])
- module = hashlib.md5(module).hexdigest()
- result = 'func_%s_gnulib_m4code_%s' % (macro_prefix, module)
+ identifier = None
+ if valid_shell_id:
+ identifier = self.getName()
+ else:
+ hash_input = '%s\n' % self.getName()
+ identifier = hashlib.md5(hash_input.encode(ENCS['default'])).hexdigest()
+ result = 'func_%s_gnulib_m4code_%s' % (macro_prefix, identifier)
return result
def getShellVar(self):
@@ -350,20 +352,19 @@ class GLModule(object):
Compute the shell variable name the will be set to true once the m4 macros
for the module have been executed.'''
- isalnum = True
macro_prefix = self.config['macro_prefix']
- for char in str(module):
- if char not in constants.ALPHANUMERIC:
- isalnum = False
+ valid_shell_id = True
+ for char in self.getName():
+ if char not in GLModule.shell_id_chars:
+ valid_shell_id = False
break
- if isalnum:
- module = str(self)
- else: # if not isalnum
- module = '%s\n' % str(self)
- if type(module) is str:
- module = module.encode(ENCS['default'])
- module = hashlib.md5(module).hexdigest()
- result = '%s_gnulib_enabled_%s' % (macro_prefix, module)
+ identifier = None
+ if valid_shell_id:
+ identifier = self.getName()
+ else:
+ hash_input = '%s\n' % self.getName()
+ identifier = hashlib.md5(hash_input.encode(ENCS['default'])).hexdigest()
+ result = '%s_gnulib_enabled_%s' % (macro_prefix, identifier)
return result
def getConditionalName(self):
@@ -372,15 +373,18 @@ class GLModule(object):
Return the automake conditional name.
GLConfig: macro_prefix.'''
macro_prefix = self.config['macro_prefix']
- nonascii = [ char
- for char in self.getName()
- if char not in 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_' ]
- if nonascii:
- name = self.getName().encode(ENCS['default'])
- name = hashlib.md5(name).hexdigest()
- conditional = '%s_GNULIB_ENABLED_%s' % (macro_prefix, name)
- else: # if not nonascii
- result = '%s_GNULIB_ENABLED_%s' % (macro_prefix, name)
+ valid_shell_id = True
+ for char in self.getName():
+ if char not in GLModule.shell_id_chars:
+ valid_shell_id = False
+ break
+ identifier = None
+ if valid_shell_id:
+ identifier = self.getName()
+ else:
+ hash_input = '%s\n' % self.getName()
+ identifier = hashlib.md5(hash_input.encode(ENCS['default'])).hexdigest()
+ result = '%s_GNULIB_ENABLED_%s' % (macro_prefix, identifier)
return result
def getDescription(self):
diff --git a/pygnulib/constants.py b/pygnulib/constants.py
index ba0ebc9942..ae27d8d41a 100644
--- a/pygnulib/constants.py
+++ b/pygnulib/constants.py
@@ -57,9 +57,6 @@ MODES = dict() # Modes
TESTS = dict() # Tests
NL = '''
''' # Newline character
-ALPHANUMERIC = 'abcdefghijklmnopqrstuvwxyz\
-ABCDEFGHIJKLMNOPQRSTUVWXYZ\
-0123456789' # Alphanumeric characters
# Set ENCS dictionary
if not hasattr(interpreter, '__file__'):
--
2.34.1
>From 98e7ca354a1108b0b99236e29bedbc6bfbd8593e Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Tue, 9 Aug 2022 16:11:09 +0200
Subject: [PATCH 3/5] gnulib-tool.py: Fix some code generation details.
* pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): When removing a
lib_LDFLAGS line, remove also the newline. Fix regex that matches
lib_SOMETHING. Add a newline after each '## begin gnulib module' line.
Don't emit 'endif' lines without corresponding 'if'. When emitting a
'+=' augmentation, make sure it does not get emitted a second time.
Don't emit a blank line when there is no AM_CPPFLAGS augmentation.
Update after getLink() changed. In the value of DEFAULT_TEXT_DOMAIN,
backslash-escape the double-quotes. Don't produce Windows CR-LFs on
Windows. Simplify.
(GLEmiter.tests_Makefile_am): When removing a lib_LDFLAGS line, remove
also the newline. Fix regex that matches lib_SOMETHING. Don't remove a
blank line before EXTRA_DIST. Add a newline after each
'## begin gnulib module' line. Set uses_subdirs also when there is a .c
file in a subdir of tests/. When emitting a '+=' augmentation, make sure
it does not get emitted a second time. Don't produce Windows CR-LFs on
Windows. Simplify.
* pygnulib/GLImport.py (GLImport.execute): Update after getLink()
changed.
---
ChangeLog | 20 +++++
pygnulib/GLEmiter.py | 203 +++++++++++++++++++++----------------------
pygnulib/GLImport.py | 16 ++--
3 files changed, 125 insertions(+), 114 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 091af1f974..50ef82d525 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2022-08-09 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix some code generation details.
+ * pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): When removing a
+ lib_LDFLAGS line, remove also the newline. Fix regex that matches
+ lib_SOMETHING. Add a newline after each '## begin gnulib module' line.
+ Don't emit 'endif' lines without corresponding 'if'. When emitting a
+ '+=' augmentation, make sure it does not get emitted a second time.
+ Don't emit a blank line when there is no AM_CPPFLAGS augmentation.
+ Update after getLink() changed. In the value of DEFAULT_TEXT_DOMAIN,
+ backslash-escape the double-quotes. Don't produce Windows CR-LFs on
+ Windows. Simplify.
+ (GLEmiter.tests_Makefile_am): When removing a lib_LDFLAGS line, remove
+ also the newline. Fix regex that matches lib_SOMETHING. Don't remove a
+ blank line before EXTRA_DIST. Add a newline after each
+ '## begin gnulib module' line. Set uses_subdirs also when there is a .c
+ file in a subdir of tests/. When emitting a '+=' augmentation, make sure
+ it does not get emitted a second time. Don't produce Windows CR-LFs on
+ Windows. Simplify.
+ * pygnulib/GLImport.py (GLImport.execute): Update after getLink()
+ changed.
+
gnulib-tool.py: Fixes for conditional dependencies.
* pygnulib/GLModuleSystem.py (GLModule.shell_id_chars): New constant.
(GLModule.getShellFunc): Don't use md5 just because of an '_' character.
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index a02b44bdd5..1a15fc4918 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -655,29 +655,34 @@ AC_DEFUN([%V1%_LIBSOURCES], [
if libtool:
libext = 'la'
perhapsLT = 'LT'
- LD_flags = False
+ eliminate_LDFLAGS = False
else: # if not libtool
libext = 'a'
perhapsLT = ''
- LD_flags = True
+ eliminate_LDFLAGS = True
if for_test:
# When creating a package for testing: Attempt to provoke failures,
# especially link errors, already during "make" rather than during
# "make check", because "make check" is not possible in a cross-compiling
# situation. Turn check_PROGRAMS into noinst_PROGRAMS.
- check_PROGRAMS = True
+ edit_check_PROGRAMS = True
else: # if not for_test
- check_PROGRAMS = False
+ edit_check_PROGRAMS = False
emit += "## DO NOT EDIT! GENERATED AUTOMATICALLY!\n"
emit += "## Process this file with automake to produce Makefile.in.\n"
emit += self.copyright_notice()
if actioncmd:
+ # The maximum line length (excluding the terminating newline) of
+ # any file that is to be preprocessed by config.status is 3070.
+ # config.status uses awk, and the HP-UX 11.00 awk fails if a line
+ # has length >= 3071; similarly, the IRIX 6.5 awk fails if a line
+ # has length >= 3072.
if len(actioncmd) <= 3000:
emit += "# Reproduce by: %s\n" % actioncmd
emit += '\n'
uses_subdirs = False
- # Modify allsnippets variable.
+ # Compute allsnippets variable.
allsnippets = ''
for module in modules:
if not module.isTests():
@@ -685,16 +690,16 @@ AC_DEFUN([%V1%_LIBSOURCES], [
amsnippet1 = module.getAutomakeSnippet_Conditional()
amsnippet1 = amsnippet1.replace('lib_LIBRARIES', 'lib%_LIBRARIES')
amsnippet1 = amsnippet1.replace('lib_LTLIBRARIES', 'lib%_LTLIBRARIES')
- if LD_flags:
- pattern = re.compile('lib_LDFLAGS[\t ]*\\+=(.*)$', re.M)
+ if eliminate_LDFLAGS:
+ pattern = re.compile('^(lib_LDFLAGS[\t ]*\\+=.*$\n)', re.M)
amsnippet1 = pattern.sub('', amsnippet1)
- pattern = re.compile('lib_([A-Z][A-Z](?:.*))', re.M)
+ pattern = re.compile('lib_([A-Z][A-Z]*)', re.M)
amsnippet1 = pattern.sub('%s_%s_\\1' % (libname, libext),
amsnippet1)
amsnippet1 = amsnippet1.replace('$(GNULIB_', '$(' + module_indicator_prefix + '_GNULIB_')
amsnippet1 = amsnippet1.replace('lib%_LIBRARIES', 'lib_LIBRARIES')
amsnippet1 = amsnippet1.replace('lib%_LTLIBRARIES', 'lib_LTLIBRARIES')
- if check_PROGRAMS:
+ if edit_check_PROGRAMS:
amsnippet1 = amsnippet1.replace('check_PROGRAMS', 'noinst_PROGRAMS')
amsnippet1 = amsnippet1.replace('${gl_include_guard_prefix}',
include_guard_prefix)
@@ -706,29 +711,33 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# Get unconditional snippet, edit it and save to amsnippet2.
amsnippet2 = module.getAutomakeSnippet_Unconditional()
- pattern = re.compile('lib_([A-Z][A-Z](?:.*))', re.M)
+ pattern = re.compile('lib_([A-Z][A-Z]*)', re.M)
amsnippet2 = pattern.sub('%s_%s_\\1' % (libname, libext),
amsnippet2)
amsnippet2 = amsnippet2.replace('$(GNULIB_',
'$(' + module_indicator_prefix + '_GNULIB_')
+ # Skip the contents if it's entirely empty.
if not (amsnippet1 + amsnippet2).isspace():
- allsnippets += '## begin gnulib module %s\n' % str(module)
+ allsnippets += '## begin gnulib module %s\n\n' % str(module)
if conddeps:
if moduletable.isConditional(module):
name = module.getConditionalName()
allsnippets += 'if %s\n' % name
allsnippets += amsnippet1
if conddeps:
- allsnippets += 'endif\n'
+ if moduletable.isConditional(module):
+ allsnippets += 'endif\n'
allsnippets += amsnippet2
allsnippets += '## end gnulib module %s\n\n' % str(module)
# Test whether there are some source files in subdirectories.
for file in module.getFiles():
- if (file.startswith('lib/') and file.endswith('.c')
+ if (file.startswith('lib/')
+ and file.endswith('.c')
and file.count('/') > 1):
uses_subdirs = True
break
+
if not makefile_name:
subdir_options = ''
# If there are source files in subdirectories, prevent collision of the
@@ -762,53 +771,50 @@ AC_DEFUN([%V1%_LIBSOURCES], [
emit += 'MAINTAINERCLEANFILES =\n'
# Execute edits that apply to the Makefile.am being generated.
- current_edit = int()
- makefile_am_edits = makefiletable.count()
- while current_edit != makefile_am_edits:
+ for current_edit in range(0, makefiletable.count()):
dictionary = makefiletable[current_edit]
if dictionary['var']:
- paths = list()
- paths += [joinpath(dictionary['dir'], 'Makefile.am')]
- paths += [os.path.normpath('./%s/Makefile.am' % dictionary['dir'])]
- paths = sorted(set(paths))
- if destfile in paths:
- emit += '%s += %s\n' % (dictionary['var'],
- dictionary['val'])
- current_edit += 1
+ if destfile == joinpath(dictionary['dir'], 'Makefile.am'):
+ emit += '%s += %s\n' % (dictionary['var'], dictionary['val'])
+ dictionary.pop('var')
# Define two parts of cppflags variable.
- emit += '\n'
cppflags_part1 = ''
- cppflags_part2 = ''
if witness_c_macro:
cppflags_part1 = ' -D%s=1' % witness_c_macro
+ cppflags_part2 = ''
if for_test:
cppflags_part2 = ' -DGNULIB_STRICT_CHECKING=1'
cppflags = '%s%s' % (cppflags_part1, cppflags_part2)
if not makefile_name:
+ emit += '\n'
emit += 'AM_CPPFLAGS =%s\n' % cppflags
emit += 'AM_CFLAGS =\n'
else: # if makefile_name
if cppflags:
+ emit += '\n'
emit += 'AM_CPPFLAGS +=%s\n' % cppflags
emit += '\n'
- # One of the snippets or the user's Makefile.am already specifies an
+ # Test if one of the snippets or the user's Makefile.am already specifies an
# installation location for the library. Don't confuse automake by saying
# it should not be installed.
# First test if allsnippets already specify an installation location.
- insnippets = False
- inmakefile = False
+ lib_gets_installed = False
regex = '^[a-zA-Z0-9_]*_%sLIBRARIES *\\+{0,1}= *%s\\.%s' % (perhapsLT, libname, libext)
pattern = re.compile(regex, re.M)
- insnippets = bool(pattern.findall(allsnippets))
- # Then test if $sourcebase/Makefile.am (if it exists) specifies it.
- path = joinpath(sourcebase, 'Makefile.am')
- if makefile_name and isfile(path):
- with codecs.open(path, 'rb', 'UTF-8') as file:
- data = file.read()
- inmakefile = bool(pattern.findall(data))
- if not any([insnippets, inmakefile]):
+ if pattern.findall(allsnippets):
+ lib_gets_installed = True
+ else:
+ # Then test if $sourcebase/Makefile.am (if it exists) specifies it.
+ if makefile_name:
+ path = joinpath(sourcebase, 'Makefile.am')
+ if isfile(path):
+ with codecs.open(path, 'rb', 'UTF-8') as file:
+ data = file.read()
+ if pattern.findall(data):
+ lib_gets_installed = True
+ if not lib_gets_installed:
# By default, the generated library should not be installed.
emit += 'noinst_%sLIBRARIES += %s.%s\n' % (perhapsLT, libname, libext)
@@ -824,22 +830,22 @@ AC_DEFUN([%V1%_LIBSOURCES], [
emit += '%s_%s_LDFLAGS += -no-undefined\n' % (libname, libext)
# Synthesize an ${libname}_${libext}_LDFLAGS augmentation by combining
# the link dependencies of all modules.
- links = [module.getLink()
- for module in modules if not module.isTests()]
- ulinks = list()
- for link in links:
- for lib in link:
- lib = constants.nlremove(lib)
- position = lib.find(' when linking with libtool')
- if position != -1:
- lib = lib[:position]
- ulinks += [lib]
- ulinks = sorted(set(ulinks))
- for link in ulinks:
- emit += '%s_%s_LDFLAGS += %s\n' % (libname, libext, link)
+ links = [ module.getLink()
+ for module in modules
+ if not module.isTests() ]
+ lines = [ line
+ for link in links
+ for line in link.split('\n')
+ if line != '' ]
+ pattern = re.compile(' when linking with libtool.*')
+ lines = [ pattern.sub('', line)
+ for line in lines ]
+ lines = sorted(set(lines))
+ for line in lines:
+ emit += '%s_%s_LDFLAGS += %s\n' % (libname, libext, line)
emit += '\n'
if pobase:
- emit += 'AM_CPPFLAGS += -DDEFAULT_TEXT_DOMAIN="%s-gnulib"\n' % podomain
+ emit += 'AM_CPPFLAGS += -DDEFAULT_TEXT_DOMAIN=\\"%s-gnulib\\"\n' % podomain
emit += '\n'
allsnippets = allsnippets.replace('$(top_srcdir)/build-aux/',
'$(top_srcdir)/%s/' % auxdir)
@@ -852,7 +858,6 @@ AC_DEFUN([%V1%_LIBSOURCES], [
emit += '\t fi; \\\n'
emit += '\tdone; \\\n'
emit += '\t:\n'
- emit = constants.nlconvert(emit)
result = tuple([emit, uses_subdirs])
return result
@@ -913,19 +918,19 @@ AC_DEFUN([%V1%_LIBSOURCES], [
if libtool:
libext = 'la'
perhapsLT = 'LT'
- LD_flags = False
+ eliminate_LDFLAGS = False
else: # if not libtool
libext = 'a'
perhapsLT = ''
- LD_flags = True
+ eliminate_LDFLAGS = True
if for_test:
# When creating a package for testing: Attempt to provoke failures,
# especially link errors, already during "make" rather than during
# "make check", because "make check" is not possible in a cross-compiling
# situation. Turn check_PROGRAMS into noinst_PROGRAMS.
- check_PROGRAMS = True
+ edit_check_PROGRAMS = True
else: # if not for_test
- check_PROGRAMS = False
+ edit_check_PROGRAMS = False
# Calculate testsbase_inverse
counter = int()
@@ -945,22 +950,22 @@ AC_DEFUN([%V1%_LIBSOURCES], [
longrun_snippets = ''
for module in modules:
if for_test and not single_configure:
- flag = module.isTests()
+ accept = module.isTests()
else: # if for_test and not single_configure
- flag = True
- if flag:
+ accept = True
+ if accept:
snippet = module.getAutomakeSnippet()
snippet = snippet.replace('lib_LIBRARIES', 'lib%_LIBRARIES')
snippet = snippet.replace('lib_LTLIBRARIES', 'lib%_LTLIBRARIES')
- if LD_flags:
- pattern = re.compile('lib_LDFLAGS[\t ]*\\+=(.*)$', re.M)
- snippet = pattern.sub('', snippet)
- pattern = re.compile('lib_([A-Z][A-Z](?:.*))', re.M)
+ if eliminate_LDFLAGS:
+ pattern = re.compile('^(lib_LDFLAGS[\t ]*\\+=.*$\n)', re.M)
+ amsnippet1 = pattern.sub('', snippet)
+ pattern = re.compile('lib_([A-Z][A-Z]*)', re.M)
snippet = pattern.sub('libtests_a_\\1', snippet)
snippet = snippet.replace('$(GNULIB_', '$(' + module_indicator_prefix + '_GNULIB_')
snippet = snippet.replace('lib%_LIBRARIES', 'lib_LIBRARIES')
snippet = snippet.replace('lib%_LTLIBRARIES', 'lib_LTLIBRARIES')
- if check_PROGRAMS:
+ if edit_check_PROGRAMS:
snippet = snippet.replace('check_PROGRAMS', 'noinst_PROGRAMS')
snippet = snippet.replace('${gl_include_guard_prefix}',
include_guard_prefix)
@@ -970,28 +975,20 @@ AC_DEFUN([%V1%_LIBSOURCES], [
snippet += 'libtests_a_DEPENDENCIES += @%sALLOCA@\n' % perhapsLT
# Skip the contents if it's entirely empty.
- if snippet.strip():
- # Check status of the module.
- statuses = module.getStatuses()
- islongrun = False
- for word in statuses:
- if word == 'longrunning-test':
- islongrun = True
- break
- if not islongrun:
- snippet = snippet.replace('\n\nEXTRA_DIST', '\nEXTRA_DIST')
- main_snippets += '## begin gnulib module %s\n' % str(module)
- main_snippets += snippet
- main_snippets += '## end gnulib module %s\n\n' % str(module)
- else: # if islongrunning
- snippet = snippet.replace('\n\nEXTRA_DIST', '\nEXTRA_DIST')
- longrun_snippets += '## begin gnulib module %s\n' % str(module)
+ if not snippet.isspace():
+ snippet = ('## begin gnulib module %s\n\n' % str(module)
+ + snippet
+ + '## end gnulib module %s\n\n' % str(module))
+ # Mention long-running tests at the end.
+ if 'longrunning-test' in module.getStatuses():
longrun_snippets += snippet
- longrun_snippets += '## end gnulib module %s\n' % str(module)
+ else:
+ main_snippets += snippet
# Test whether there are some source files in subdirectories.
for file in module.getFiles():
- if (file.startswith('lib/') and file.endswith('.c')
+ if ((file.startswith('lib/') or file.startswith('tests/'))
+ and file.endswith('.c')
and file.count('/') > 1):
uses_subdirs = True
break
@@ -1031,7 +1028,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# * https://debbugs.gnu.org/11030
# So we need this workaround.
pattern = re.compile('^pkgdata_DATA *\\+=', re.M)
- if bool(pattern.findall(main_snippets)) or bool(pattern.findall(longrun_snippets)):
+ if pattern.findall(main_snippets) or pattern.findall(longrun_snippets):
emit += 'pkgdata_DATA =\n'
emit += 'EXTRA_DIST =\n'
@@ -1044,20 +1041,12 @@ AC_DEFUN([%V1%_LIBSOURCES], [
emit += 'MAINTAINERCLEANFILES =\n'
# Execute edits that apply to the Makefile.am being generated.
- # Execute edits that apply to the Makefile.am being generated.
- current_edit = int()
- makefile_am_edits = makefiletable.count()
- while current_edit != makefile_am_edits:
+ for current_edit in range(0, makefiletable.count()):
dictionary = makefiletable[current_edit]
if dictionary['var']:
- paths = list()
- paths += [joinpath(dictionary['dir'], 'Makefile.am')]
- paths += [os.path.normpath('./%s/Makefile.am' % dictionary['dir'])]
- paths = sorted(set(paths))
- if destfile in paths:
- emit += '%s += %s\n' % (dictionary['var'],
- dictionary['val'])
- current_edit += 1
+ if destfile == joinpath(dictionary['dir'], 'Makefile.am'):
+ emit += '%s += %s\n' % (dictionary['var'], dictionary['val'])
+ dictionary.pop('var')
emit += '\nAM_CPPFLAGS = \\\n'
if for_test:
@@ -1071,6 +1060,8 @@ AC_DEFUN([%V1%_LIBSOURCES], [
emit += ' -I%s/%s -I$(srcdir)/%s/%s\n' % (testsbase_inverse, sourcebase, testsbase_inverse, sourcebase)
emit += '\n'
+ ldadd_before = ''
+ ldadd_after = ''
if libtests:
# All test programs need to be linked with libtests.a.
# It needs to be passed to the linker before ${libname}.${libext},
@@ -1081,11 +1072,13 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# voluntarily omitted).
# The LIBTESTS_LIBDEPS can be passed to the linker once or twice, it
# does not matter.
- emit += 'LDADD = libtests.a %s/%s/%s.%s libtests.a %s/%s/%s.%s libtests.a $(LIBTESTS_LIBDEPS)\n\n' \
- % (testsbase_inverse, sourcebase, libname, libext,
- testsbase_inverse, sourcebase, libname, libext)
- else:
- emit += 'LDADD = %s/%s/%s.%s\n\n' % (testsbase_inverse, sourcebase, libname, libext)
+ ldadd_before = ' libtests.a'
+ ldadd_after = ' libtests.a $(LIBTESTS_LIBDEPS)'
+ emit += 'LDADD =%s %s/%s/%s.%s libtests.a %s/%s/%s.%s%s\n\n' \
+ % (ldadd_before,
+ testsbase_inverse, sourcebase, libname, libext,
+ testsbase_inverse, sourcebase, libname, libext,
+ ldadd_after)
if libtests:
emit += 'libtests_a_SOURCES =\n'
# Here we use $(LIBOBJS), not @LIBOBJS@. The value is the same. However,
@@ -1099,11 +1092,10 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# EXEEXT is defined by AC_PROG_CC through autoconf.
# srcdir is defined by autoconf and automake.
emit += "TESTS_ENVIRONMENT += EXEEXT='@EXEEXT@' srcdir='$(srcdir)'\n\n"
- main_snippets = main_snippets.replace('$(top_srcdir)/build-aux/',
- '$(top_srcdir)/%s/' % auxdir)
- longrun_snippets = longrun_snippets.replace('$(top_srcdir)/build-aux/',
- '$(top_srcdir)/%s/' % auxdir)
- emit += main_snippets + longrun_snippets
+ all_snippets = main_snippets + longrun_snippets
+ all_snippets = all_snippets.replace('$(top_srcdir)/build-aux/',
+ '$(top_srcdir)/%s/' % auxdir)
+ emit += all_snippets
emit += '# Clean up after Solaris cc.\n'
emit += 'clean-local:\n'
emit += '\trm -rf SunWS_cache\n\n'
@@ -1114,6 +1106,5 @@ AC_DEFUN([%V1%_LIBSOURCES], [
emit += '\t fi; \\\n'
emit += '\tdone; \\\n'
emit += '\t:\n'
- emit = constants.nlconvert(emit)
result = tuple([emit, uses_subdirs])
return result
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 87d85c5181..0197e0671d 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -1398,18 +1398,18 @@ AC_DEFUN([%s_FILE_LIST], [\n''' % macro_prefix
# Get link directives.
links = [ module.getLink()
for module in self.moduletable['main'] ]
- ulinks = list()
- for link in links:
- for lib in link:
- ulinks += [lib]
- ulinks = sorted(set(ulinks))
- if ulinks:
+ lines = [ line
+ for link in links
+ for line in link.split('\n')
+ if line != '' ]
+ lines = sorted(set(lines))
+ if lines:
print('''
You may need to use the following Makefile variables when linking.
Use them in <program>_LDADD when linking a program, or
in <library>_a_LDFLAGS or <library>_la_LDFLAGS when linking a library.''')
- for link in ulinks:
- print(' %s' % link)
+ for line in lines:
+ print(' %s' % line)
# Print reminders.
print('')
--
2.34.1
>From bc3d94bb2df6fa0c766f6226814c2fc2a55ee049 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Tue, 9 Aug 2022 23:59:40 +0200
Subject: [PATCH 4/5] gnulib-tool.py: Don't do license replacements in the
autoconf snippets.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Remove fileassistant
argument. Don't invoke the 'aux' transformer here. Don't produce Windows
CR-LFs on Windows.
(GLEmiter.autoconfSnippets): Remove fileassistant argument.
* pygnulib/GLImport.py (GLImport.gnulib_comp): Update all callers.
* pygnulib/GLTestDir.py (GLTestDir.execute): Likewise.
---
ChangeLog | 8 +++++++
pygnulib/GLEmiter.py | 51 ++++++++++++-------------------------------
pygnulib/GLImport.py | 4 ++--
pygnulib/GLTestDir.py | 10 ++++-----
4 files changed, 29 insertions(+), 44 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 50ef82d525..642a7363e3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,13 @@
2022-08-09 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Don't do license replacements in the autoconf snippets.
+ * pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Remove fileassistant
+ argument. Don't invoke the 'aux' transformer here. Don't produce Windows
+ CR-LFs on Windows.
+ (GLEmiter.autoconfSnippets): Remove fileassistant argument.
+ * pygnulib/GLImport.py (GLImport.gnulib_comp): Update all callers.
+ * pygnulib/GLTestDir.py (GLTestDir.execute): Likewise.
+
gnulib-tool.py: Fix some code generation details.
* pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): When removing a
lib_LDFLAGS line, remove also the newline. Fix regex that matches
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index 1a15fc4918..9b54d5009e 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -95,18 +95,15 @@ class GLEmiter(object):
# Generated by gnulib-tool.\n"""
return constants.nlconvert(emit)
- def autoconfSnippet(self, module, fileassistant, toplevel,
+ def autoconfSnippet(self, module, toplevel,
disable_libtool, disable_gettext, replace_auxdir, indentation):
'''GLEmiter.autoconfSnippet(module, toplevel,
- disable_libtool, disable_gettext, replace_auxdir,
- indentation) -> str
+ disable_libtool, disable_gettext, replace_auxdir, indentation) -> str
Emit the autoconf snippet of a module.
GLConfig: include_guard_prefix.
module is a GLModule instance, which is processed.
- fileassistant is a GLFileAssistant instance, which is used to get temporary
- directories and sed transformer.
toplevel is a bool variable, False means a subordinate use of pygnulib.
disable_libtool is a bool variable; it tells whether to disable libtool
handling even if it has been specified through the GLConfig class.
@@ -115,13 +112,9 @@ class GLEmiter(object):
replace_auxdir is a bool variable; it tells whether to replace
'build-aux' directory in AC_CONFIG_FILES.
indentation is a string which contain spaces to prepend on each line.'''
- emit = ''
if type(module) is not GLModule:
raise TypeError('module must be a GLModule, not %s'
% type(module).__name__)
- if type(fileassistant) is not GLFileAssistant:
- raise TypeError('fileassistant must be a GLFileAssistant, not %s'
- % type(fileassistant).__name__)
if type(toplevel) is not bool:
raise TypeError('toplevel must be a bool, not %s'
% type(toplevel).__name__)
@@ -139,6 +132,7 @@ class GLEmiter(object):
auxdir = self.config['auxdir']
libtool = self.config['libtool']
include_guard_prefix = self.config['include_guard_prefix']
+ emit = ''
if str(module) in ['gnumakefile', 'maintainer-makefile']:
# These modules are meant to be used only in the top-level directory.
flag = toplevel
@@ -152,18 +146,8 @@ class GLEmiter(object):
for line in snippet.split('\n')
if line.strip() ]
snippet = '%s\n' % '\n'.join(lines)
- transformer = fileassistant.transformers.get('aux', '')
pattern = re.compile('^(.*)$', re.M)
snippet = pattern.sub('%s\\1' % indentation, snippet)
- if transformer:
- args = ['sed', '-e', transformer]
- path = fileassistant.tmpfilename('snippet')
- with codecs.open(path, 'wb', 'UTF-8') as file:
- file.write(snippet)
- stdin = codecs.open(path, 'rb', 'UTF-8')
- snippet = sp.check_output(args, stdin=stdin, shell=False)
- snippet = snippet.decode("UTF-8")
- os.remove(path)
if disable_libtool:
snippet = snippet.replace('$gl_cond_libtool', 'false')
snippet = snippet.replace('gl_libdeps', 'gltests_libdeps')
@@ -181,23 +165,21 @@ class GLEmiter(object):
emit += "LTALLOCA=`echo \"$ALLOCA\" | sed -e 's/\\.[^.]* /.lo /g;s/\\.[^.]*$/.lo/'`\n"
emit += 'changequote([, ])dnl\n'
emit += 'AC_SUBST([LTALLOCA])'
- if replace_auxdir:
- regex = 'AC_CONFIG_FILES\\(\\[(.*)\\:build-aux/(.*)\\]\\)'
- repl = 'AC_CONFIG_FILES([\\1:%s/\\2])' % auxdir
- pattern = re.compile(regex, re.M)
- emit = pattern.sub(repl, emit)
+ if replace_auxdir:
+ regex = 'AC_CONFIG_FILES\\(\\[(.*)\\:build-aux/(.*)\\]\\)'
+ repl = 'AC_CONFIG_FILES([\\1:%s/\\2])' % auxdir
+ pattern = re.compile(regex, re.M)
+ emit = pattern.sub(repl, emit)
lines = [ line
for line in emit.split('\n')
if line.strip() ]
emit = '%s\n' % '\n'.join(lines)
- emit = constants.nlconvert(emit)
return emit
- def autoconfSnippets(self, modules, moduletable, fileassistant,
+ def autoconfSnippets(self, modules, moduletable,
verifier, toplevel, disable_libtool, disable_gettext, replace_auxdir):
- '''GLEmiter.autoconfSnippets(modules, fileassistant,
- verifier, toplevel, disable_libtool, disable_gettext,
- replace_auxdir) -> str
+ '''GLEmiter.autoconfSnippets(modules,
+ verifier, toplevel, disable_libtool, disable_gettext, replace_auxdir) -> str
Collect and emit the autoconf snippets of a set of modules.
GLConfig: conddeps.
@@ -209,8 +191,6 @@ class GLEmiter(object):
be a GLModule instance.
moduletable is a GLModuleTable instance, which contains necessary
information about dependencies of the modules.
- fileassistant is a GLFileAssistant instance, which is used to get temporary
- directories and sed transformers.
verifier is an integer, which can be 0, 1 or 2.
if verifier == 0, then process every module;
if verifier == 1, then process only non-tests modules;
@@ -229,9 +209,6 @@ class GLEmiter(object):
if type(moduletable) is not GLModuleTable:
raise TypeError('moduletable must be a GLFileAssistant, not %s'
% type(moduletable).__name__)
- if type(fileassistant) is not GLFileAssistant:
- raise TypeError('fileassistant must be a GLFileAssistant, not %s'
- % type(fileassistant).__name__)
if type(verifier) is not int:
raise TypeError('verifier must be an int, not %s'
% type(verifier).__name__)
@@ -262,7 +239,7 @@ class GLEmiter(object):
elif verifier == 2:
solution = module.isTests()
if solution:
- emit += self.autoconfSnippet(module, fileassistant, toplevel,
+ emit += self.autoconfSnippet(module, toplevel,
disable_libtool, disable_gettext, replace_auxdir, ' ')
else: # if conddeps
# Emit the autoconf code for the unconditional modules.
@@ -275,7 +252,7 @@ class GLEmiter(object):
solution = module.isTests()
if solution:
if not moduletable.isConditional(module):
- emit += self.autoconfSnippet(module, fileassistant, toplevel,
+ emit += self.autoconfSnippet(module, toplevel,
disable_libtool, disable_gettext, replace_auxdir, ' ')
# Initialize the shell variables indicating that the modules are enabled.
for module in modules:
@@ -306,7 +283,7 @@ class GLEmiter(object):
emit += ' %s ()\n' % shellfunc
emit += ' {\n'
emit += ' if ! $%s; then\n' % shellvar
- emit += self.autoconfSnippet(module, fileassistant, toplevel,
+ emit += self.autoconfSnippet(module, toplevel,
disable_libtool, disable_gettext, replace_auxdir, ' ')
emit += ' %s=true\n' % shellvar
depmodules = module.getDependenciesWithoutConditions()
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 0197e0671d..114605dd8a 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -636,7 +636,7 @@ AC_DEFUN([%s_INIT],
emit += ' m4_pushdef([gl_MODULE_INDICATOR_CONDITION], [%s])\n' % witness_c_macro
# Emit main autoconf snippets.
emit += self.emiter.autoconfSnippets(moduletable['main'],
- moduletable, assistant, 0, True, False, True, replace_auxdir)
+ moduletable, 0, True, False, True, replace_auxdir)
if witness_c_macro:
emit += ' m4_popdef([gl_MODULE_INDICATOR_CONDITION])\n'
emit += ' # End of code from modules\n'
@@ -659,7 +659,7 @@ AC_DEFUN([%s_INIT],
emit += '[$gl_module_indicator_condition])\n'
# Emit tests autoconf snippets.
emit += self.emiter.autoconfSnippets(moduletable['tests'],
- moduletable, assistant, 0, True, True, True, replace_auxdir)
+ moduletable, 0, True, True, True, replace_auxdir)
emit += ' m4_popdef([gl_MODULE_INDICATOR_CONDITION])\n'
emit += self.emiter.initmacro_end('%stests' % macro_prefix)
# _LIBDEPS and _LTLIBDEPS variables are not needed if this library is
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index 746b815b49..3c4ad0b9dc 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -496,11 +496,11 @@ class GLTestDir(object):
# those of the tests.
emit += "gl_source_base='../%s'\n" % sourcebase
emit += self.emiter.autoconfSnippets(modules,
- moduletable, self.assistant, 1, False, False, False,
+ moduletable, 1, False, False, False,
replace_auxdir)
emit += "gl_source_base='.'"
emit += self.emiter.autoconfSnippets(modules,
- moduletable, self.assistant, 2, False, False, False,
+ moduletable, 2, False, False, False,
replace_auxdir)
emit += self.emiter.initmacro_end(macro_prefix)
# _LIBDEPS and _LTLIBDEPS variables are not needed if this library is
@@ -614,10 +614,10 @@ class GLTestDir(object):
emit += 'gl_source_base=\'%s\'\n' % sourcebase
if single_configure:
emit += self.emiter.autoconfSnippets(main_modules, moduletable,
- self.assistant, 0, False, False, False, replace_auxdir)
+ 0, False, False, False, replace_auxdir)
else: # if not single_configure
emit += self.emiter.autoconfSnippets(modules, moduletable,
- self.assistant, 1, False, False, False, replace_auxdir)
+ 1, False, False, False, replace_auxdir)
emit += self.emiter.initmacro_end(macro_prefix)
if single_configure:
emit += ' gltests_libdeps=\n'
@@ -631,7 +631,7 @@ class GLTestDir(object):
emit += ' m4_pushdef([gl_MODULE_INDICATOR_CONDITION], '
emit += '[$gl_module_indicator_condition])\n'
snippets = self.emiter.autoconfSnippets(tests_modules, moduletable,
- self.assistant, 1, True, False, False, replace_auxdir)
+ 1, True, False, False, replace_auxdir)
emit += snippets.strip()
emit += ' m4_popdef([gl_MODULE_INDICATOR_CONDITION])\n'
emit += self.emiter.initmacro_end('%stests' % macro_prefix)
--
2.34.1
>From 62fa8fc7b2c9db14d8c24d6ec5beedecb27b4802 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Wed, 10 Aug 2022 00:51:59 +0200
Subject: [PATCH 5/5] gnulib-tool.py: Finish implementing option
--conditional-dependencies.
* gnulib-tool.py (main) Accept options --conditional-dependencies,
--no-conditional-dependencies.
* pygnulib/GLModuleSystem.py (GLModuleTable.addConditional): Use
str(module), not module, as key. Fix logic bug.
(GLModuleTable.getCondition): Simplify.
(GLModuleTable.transitive_closure): Show a warning when there are
duplicate dependencies. Fix logic bug.
(GLModuleTable.transitive_closure_separately): Simplify.
(GLModuleTable.add_dummy): Ignore tests modules. Cope with multiple
lib_SOURCES augmentation lines. Cope with comments at the end of a
lib_SOURCES augmentation line. Add the dummy module at the end of the
modules list.
* pygnulib/GLTestDir.py (GLTestDir.execute): Remove the code that forces
the dummy module to the end of the list.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippets): Add code to
terminate the shell functions. Add code for the dependencies from the
unconditional to the conditional modules. Don't emit AM_CONDITIONAL for
unconditional modules.
---
ChangeLog | 20 ++++++++
gnulib-tool.py | 13 ++++-
gnulib-tool.py.TODO | 7 +--
pygnulib/GLEmiter.py | 44 +++++++++++++---
pygnulib/GLImport.py | 3 +-
pygnulib/GLModuleSystem.py | 100 +++++++++++++++++++++++++++----------
pygnulib/GLTestDir.py | 24 +--------
7 files changed, 148 insertions(+), 63 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 642a7363e3..0dfcb7cdd7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,25 @@
2022-08-09 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Finish implementing option --conditional-dependencies.
+ * gnulib-tool.py (main) Accept options --conditional-dependencies,
+ --no-conditional-dependencies.
+ * pygnulib/GLModuleSystem.py (GLModuleTable.addConditional): Use
+ str(module), not module, as key. Fix logic bug.
+ (GLModuleTable.getCondition): Simplify.
+ (GLModuleTable.transitive_closure): Show a warning when there are
+ duplicate dependencies. Fix logic bug.
+ (GLModuleTable.transitive_closure_separately): Simplify.
+ (GLModuleTable.add_dummy): Ignore tests modules. Cope with multiple
+ lib_SOURCES augmentation lines. Cope with comments at the end of a
+ lib_SOURCES augmentation line. Add the dummy module at the end of the
+ modules list.
+ * pygnulib/GLTestDir.py (GLTestDir.execute): Remove the code that forces
+ the dummy module to the end of the list.
+ * pygnulib/GLEmiter.py (GLEmiter.autoconfSnippets): Add code to
+ terminate the shell functions. Add code for the dependencies from the
+ unconditional to the conditional modules. Don't emit AM_CONDITIONAL for
+ unconditional modules.
+
gnulib-tool.py: Don't do license replacements in the autoconf snippets.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Remove fileassistant
argument. Don't invoke the 'aux' transformer here. Don't produce Windows
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 9988c84de9..71ef1c4f2c 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -298,6 +298,15 @@ def main():
default=None,
action='append',
nargs=1)
+ # conditional-dependencies
+ parser.add_argument('--conditional-dependencies',
+ dest='cond_dependencies',
+ default=None,
+ action="store_true")
+ parser.add_argument('--no-conditional-dependencies',
+ dest='cond_dependencies',
+ default=None,
+ action="store_false")
# libtool
parser.add_argument('--libtool',
dest='libtool',
@@ -659,6 +668,7 @@ def main():
lgpl = lgpl[-1]
if lgpl == None:
lgpl = True
+ cond_dependencies = cmdargs.cond_dependencies
libtool = cmdargs.libtool
makefile_name = cmdargs.makefile_name
if makefile_name != None:
@@ -682,7 +692,6 @@ def main():
lsymlink = cmdargs.lsymlink == True
single_configure = cmdargs.single_configure
docbase = None
- conddeps = None
# Create pygnulib configuration.
config = classes.GLConfig(
@@ -702,7 +711,7 @@ def main():
lgpl=lgpl,
makefile_name=makefile_name,
libtool=libtool,
- conddeps=conddeps,
+ conddeps=cond_dependencies,
macro_prefix=macro_prefix,
podomain=podomain,
witness_c_macro=witness_c_macro,
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index be6a2e2b10..da6df2ab90 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -15,15 +15,9 @@ The following commits to gnulib-tool have not yet been reflected in
--------------------------------------------------------------------------------
-Inline all 'sed' invocations.
-
---------------------------------------------------------------------------------
-
Implement the options:
--extract-recursive-dependencies
--extract-recursive-link-directive
- --conditional-dependencies
- --no-conditional-dependencies
--gnu-make
--tests-makefile-name
--automake-subdir
@@ -37,6 +31,7 @@ Remove exit() in GLImport.py.
Optimize:
- os.chdir around subprocess creation -> cwd=... argument instead.
+ - Inline all 'sed' invocations.
--------------------------------------------------------------------------------
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index 9b54d5009e..c0fd053077 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -202,7 +202,6 @@ class GLEmiter(object):
AM_GNU_GETTEXT invocations.
replace_auxdir is a bool variable; it tells whether to replace
'build-aux' directory in AC_CONFIG_FILES.'''
- emit = ''
for module in modules:
if type(module) is not GLModule:
raise TypeError('each module must be a GLModule instance')
@@ -229,6 +228,7 @@ class GLEmiter(object):
auxdir = self.config['auxdir']
conddeps = self.config['conddeps']
macro_prefix = self.config['macro_prefix']
+ emit = ''
if not conddeps:
# Ignore the conditions, and enable all modules unconditionally.
for module in modules:
@@ -290,7 +290,39 @@ class GLEmiter(object):
# Intersect dependencies with the modules list.
depmodules = [ dep
for dep in depmodules
- if dep in modules ]
+ if dep in modules ] # TODO should this be basemodules or modules?
+ for depmodule in depmodules:
+ if moduletable.isConditional(depmodule):
+ shellfunc = depmodule.getShellFunc()
+ condition = moduletable.getCondition(module, depmodule)
+ if condition != None:
+ emit += ' if %s; then\n' % condition
+ emit += ' %s\n' % shellfunc
+ emit += ' fi\n'
+ else: # if condition == None
+ emit += ' %s\n' % shellfunc
+ # if not moduletable.isConditional(depmodule)
+ else:
+ # The autoconf code for $dep has already been emitted above and
+ # therefore is already executed when this code is run.
+ pass
+ emit += ' fi\n'
+ emit += ' }\n'
+ # Emit the dependencies from the unconditional to the conditional modules.
+ for module in modules:
+ if verifier == 0:
+ solution = True
+ elif verifier == 1:
+ solution = module.isNonTests()
+ elif verifier == 2:
+ solution = module.isTests()
+ if solution:
+ if not moduletable.isConditional(module):
+ depmodules = module.getDependenciesWithoutConditions()
+ # Intersect dependencies with the modules list.
+ depmodules = [ dep
+ for dep in depmodules
+ if dep in modules ] # TODO should this be basemodules or modules?
for depmodule in depmodules:
if moduletable.isConditional(depmodule):
shellfunc = depmodule.getShellFunc()
@@ -316,14 +348,14 @@ class GLEmiter(object):
elif verifier == 2:
solution = module.isTests()
if solution:
- condname = module.getConditionalName()
- shellvar = module.getShellVar()
- emit += ' AM_CONDITIONAL([%s], [$%s])\n' % (condname, shellvar)
+ if moduletable.isConditional(module):
+ condname = module.getConditionalName()
+ shellvar = module.getShellVar()
+ emit += ' AM_CONDITIONAL([%s], [$%s])\n' % (condname, shellvar)
lines = [ line
for line in emit.split('\n')
if line.strip() ]
emit = '%s\n' % '\n'.join(lines)
- emit = constants.nlconvert(emit)
return emit
def preEarlyMacros(self, require, indentation, modules):
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 114605dd8a..5dd73fa8bd 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -831,9 +831,10 @@ AC_DEFUN([%s_FILE_LIST], [\n''' % macro_prefix
if libtests:
self.config.setLibtests(True)
- # Add dummy package if it is needed.
+ # Add the dummy module to the main module list if needed.
main_modules = self.moduletable.add_dummy(main_modules)
if libtests: # if we need to use libtests.a
+ # Add the dummy module to the tests-related module list if needed.
tests_modules = self.moduletable.add_dummy(tests_modules)
# Check license incompatibilities.
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 9bfd5bd266..f9b47a64a5 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -705,6 +705,18 @@ class GLModuleTable(object):
inc_all_indirect_tests = True if all kinds of problematic unit tests
among the unit tests of the dependencies
should be included
+
+ Methods for conditional dependencies:
+ - addUnconditional(B)
+ notes the presence of B as an unconditional module.
+ - addConditional(A, B. cond)
+ notes the presence of a conditional dependency from module A to module B,
+ subject to the condition that A is enabled and cond is true.
+ - isConditional(B)
+ tests whether module B is conditional.
+ - getCondition(A, B)
+ returns the condition when B should be enabled as a dependency of A,
+ once the m4 code for A has been executed.
'''
self.dependers = dict() # Dependencies
self.conditionals = dict() # Conditional modules
@@ -765,9 +777,11 @@ class GLModuleTable(object):
raise TypeError('condition must be a string or True, not %s'
% type(condition).__name__)
if not str(module) in self.unconditionals:
+ # No unconditional dependency to the given module is known at this point.
if str(module) not in self.dependers:
- self.dependers[module] = list()
- self.dependers[module] += [module]
+ self.dependers[str(module)] = list()
+ if str(parent) not in self.dependers[str(module)]:
+ self.dependers[str(module)].append(str(parent))
key = '%s---%s' % (str(parent), str(module))
self.conditionals[key] = condition
@@ -778,9 +792,9 @@ class GLModuleTable(object):
if type(module) is not GLModule:
raise TypeError('module must be a GLModule, not %s'
% type(module).__name__)
+ self.unconditionals[str(module)] = True
if str(module) in self.dependers:
self.dependers.pop(str(module))
- self.unconditionals[str(module)] = True
def isConditional(self, module):
'''GLModuleTable.isConditional(module) -> bool
@@ -804,9 +818,7 @@ class GLModuleTable(object):
raise TypeError('module must be a GLModule, not %s'
% type(module).__name__)
key = '%s---%s' % (str(parent), str(module))
- result = None
- if key in self.conditionals:
- result = self.conditionals[key]
+ result = self.conditionals.get(key, None)
return result
def transitive_closure(self, modules):
@@ -823,6 +835,11 @@ class GLModuleTable(object):
for module in modules:
if type(module) is not GLModule:
raise TypeError('each module must be a GLModule instance')
+ # In order to process every module only once (for speed), process an
+ # "input list" of modules, producing an "output list" of modules. During
+ # each round, more modules can be queued in the input list. Once a
+ # module on the input list has been processed, it is added to the
+ # "handled list", so we can avoid to process it again.
inc_all_tests = self.inc_all_direct_tests
handledmodules = list()
inmodules = modules
@@ -833,7 +850,7 @@ class GLModuleTable(object):
self.addUnconditional(module)
while inmodules:
inmodules_this_round = inmodules
- inmodules = list()
+ inmodules = list() # Accumulator, queue for next round
for module in inmodules_this_round:
if module not in self.avoids:
outmodules += [module]
@@ -842,6 +859,13 @@ class GLModuleTable(object):
module.getAutomakeSnippet_Conditional()
pattern = re.compile('^if', re.M)
if not pattern.findall(automake_snippet):
+ # A module whose Makefile.am snippet contains a
+ # reference to an automake conditional. If we were
+ # to use it conditionally, we would get an error
+ # configure: error: conditional "..." was never defined.
+ # because automake 1.11.1 does not handle nested
+ # conditionals correctly. As a workaround, make the
+ # module unconditional.
self.addUnconditional(module)
conditional = self.isConditional(module)
dependencies = module.getDependenciesWithConditions()
@@ -849,6 +873,16 @@ class GLModuleTable(object):
for pair in dependencies ]
conditions = [ pair[1]
for pair in dependencies ]
+ # Duplicate dependencies are harmless, but Jim wants a warning.
+ duplicate_depmodules = [ depmodule
+ for depmodule in set(depmodules)
+ if depmodules.count(depmodule) > 1 ]
+ if duplicate_depmodules:
+ duplicate_depmodule_names = [ str(depmodule)
+ for depmodule in duplicate_depmodules ]
+ message = ('gnulib-tool: warning: module %s has duplicated dependencies: %s\n'
+ % (module, duplicate_depmodule_names))
+ sys.stderr.write(message)
if self.config.checkInclTestCategory(TESTS['tests']):
testsname = module.getTestsName()
if self.modulesystem.exists(testsname):
@@ -856,6 +890,7 @@ class GLModuleTable(object):
depmodules += [testsmodule]
conditions += [None]
for depmodule in depmodules:
+ # Determine whether to include the dependency or tests module.
include = True
statuses = depmodule.getStatuses()
for word in statuses:
@@ -890,15 +925,15 @@ class GLModuleTable(object):
if self.config['conddeps']:
index = depmodules.index(depmodule)
condition = conditions[index]
+ if condition == True:
+ condition = None
if condition:
self.addConditional(module, depmodule, condition)
else: # if condition
if conditional:
self.addConditional(module, depmodule, True)
else: # if not conditional
- self.addUnconditional(module)
- listing = list() # Create empty list
- inmodules = sorted(set(inmodules))
+ self.addUnconditional(depmodule)
handledmodules = sorted(set(handledmodules + inmodules_this_round))
# Remove handledmodules from inmodules.
inmodules = [module
@@ -934,17 +969,24 @@ class GLModuleTable(object):
for module in finalmodules:
if type(module) is not GLModule:
raise TypeError('each module must be a GLModule instance')
+ # Determine main module list.
saved_inctests = self.config.checkInclTestCategory(TESTS['tests'])
self.config.disableInclTestCategory(TESTS['tests'])
main_modules = self.transitive_closure(basemodules)
self.config.setInclTestCategory(TESTS['tests'], saved_inctests)
+ # Determine tests-related module list.
tests_modules = \
[ m
for m in finalmodules
- if m not in main_modules ] \
- + [ m
- for m in main_modules
- if m.getApplicability() != 'main' ]
+ if not (m in main_modules and m.getApplicability() == 'main') ]
+ # Note: Since main_modules is (hopefully) a subset of finalmodules, this
+ # ought to be the same as
+ # [ m
+ # for m in finalmodules
+ # if m not in main_modules ] \
+ # + [ m
+ # for m in main_modules
+ # if m.getApplicability() != 'main' ]
tests_modules = sorted(set(tests_modules))
result = tuple([main_modules, tests_modules])
return result
@@ -957,24 +999,30 @@ class GLModuleTable(object):
GLConfig: auxdir, ac_version.'''
auxdir = self.config['auxdir']
ac_version = self.config['ac_version']
- have_lib_sources = False
for module in modules:
if type(module) is not GLModule:
raise TypeError('each module must be a GLModule instance')
- snippet = module.getAutomakeSnippet()
- snippet = constants.remove_backslash_newline(snippet)
- pattern = re.compile('^lib_SOURCES[\t ]*\\+=[\t ]*(.*)$', re.M)
- files = pattern.findall(snippet)
- if files: # if source files were found
- files = files[-1].split(' ')
- for file in files:
- if not file.endswith('.h'):
- have_lib_sources = True
- break
+ # Determine whether any module provides a lib_SOURCES augmentation.
+ have_lib_sources = False
+ for module in modules:
+ if not module.isTests():
+ snippet = module.getAutomakeSnippet()
+ # Extract the value of "lib_SOURCES += ...".
+ snippet = constants.remove_backslash_newline(snippet)
+ pattern = re.compile('^lib_SOURCES[\t ]*\\+=([^#]*).*$', re.M)
+ for matching_rhs in pattern.findall(snippet):
+ files = matching_rhs.split(' ')
+ for file in files:
+ # Ignore .h files since they are not compiled.
+ if not file.endswith('.h'):
+ have_lib_sources = True
+ break
+ # Add the dummy module, to make sure the library will be non-empty.
if not have_lib_sources:
dummy = self.modulesystem.find('dummy')
if dummy not in self.avoids:
- modules = sorted(set(modules + [dummy]))
+ if dummy not in modules:
+ modules = sorted(set(modules)) + [dummy]
return list(modules)
def filelist(self, modules):
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index 3c4ad0b9dc..bf00099283 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -265,33 +265,13 @@ class GLTestDir(object):
self.config.setLibtests(True)
if single_configure:
- # Add dummy package if it is needed.
+ # Add the dummy module to the main module list if needed.
main_modules = moduletable.add_dummy(main_modules)
- if 'dummy' in [ str(module)
- for module in main_modules ]:
- main_modules = [ m
- for m in main_modules
- if str(m) != 'dummy' ]
- dummy = self.modulesystem.find('dummy')
- main_modules = sorted(set(main_modules)) + [dummy]
if libtests: # if we need to use libtests.a
+ # Add the dummy module to the tests-related module list if needed.
tests_modules = moduletable.add_dummy(tests_modules)
- if 'dummy' in [ str(module)
- for module in tests_modules ]:
- tests_modules = [ m
- for m in tests_modules
- if str(m) != 'dummy' ]
- dummy = self.modulesystem.find('dummy')
- tests_modules = sorted(set(tests_modules)) + [dummy]
else: # if not single_configure
modules = moduletable.add_dummy(modules)
- if 'dummy' in [ str(module)
- for module in modules ]:
- modules = [ m
- for m in modules
- if str(m) != 'dummy' ]
- dummy = self.modulesystem.find('dummy')
- modules = sorted(set(modules)) + [dummy]
# Show banner notice of every module.
if single_configure:
--
2.34.1
