Today's changes:
2022-08-07 Bruno Haible <br...@clisp.org>
gnulib-tool.py: Finish implementing option --extract-test-module.
* gnulib-tool.py (main): Accept option --extract-tests-module.
gnulib-tool.py: Fix handling of nonexistent module names in --extract-*.
* gnulib-tool.py (main): To test whether a module exists, just call
GLModuleSystem.find and test its return value.
gnulib-tool.py: Fix --extract-dependencies result.
* pygnulib/GLModuleSystem.py (GLModule.getDependencies): Return a
snippet, not a list. Implement dependency of ${module}-tests on
${module}.
(GLModule.getDependenciesWithoutConditions,
GLModule.getDependenciesWithConditions): New methods.
(GLModuleTable.transitive_closure): Call getDependenciesWithConditions.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippets): Call
getDependenciesWithoutConditions.
* gnulib-tool.py (main) [--extract-dependencies]: Update.
gnulib-tool.py: Rename a method.
* pygnulib/GLModuleSystem.py (GLModule.getAutoconfEarlySnippet): Renamed
from GLModule.getAutoconfSnippet_Early.
* pygnulib/GLImport.py: Update.
* pygnulib/GLTestDir.py: Likewise.
gnulib-tool.py: Fix section extraction from module descriptions.
The code with self.content.split(section)[-1]
was broken because it recognizes an indented section label.
Similar code with ('\n' + self.content).split('\n' + section)[-1]
would still be broken because it recognizes an indented section label
in the first line of the file.
The code with section_label_regex
was broken because sometimes it returns the second-to-last section with
the given label, not the last one.
Also, whitespace after the colon was not ignored.
* pygnulib/GLModuleSystem.py (GLModule.__init__): Dissect the module
description's contents immediately, once only, in a reliable way.
(GLModule.getDescription, GLModule.getComment): Simplify.
(GLModule.getStatus): Simplify. Return a string.
(GLModule.getStatuses): New function. Return a list.
(GLModule.getNotice, GLModule.getApplicability, GLModule.getFiles,
GLModule.getDependencies, GLModules.getAutoconfSnippet_Early,
GLModules.getAutoconfSnippet, GLModule.getAutomakeSnippet_Conditional,
GLModule.getInclude, GLModule.getLink, GLModule.getLicense_Raw):
Simplify.
(GLModule.getLicense): Remove whitespace after calling getLicense_Raw.
(GLModule.getMaintainer): Simplify.
(GLModuleTable.transitive_closure): Call getStatuses() instead of
getStatus().
* pygnulib/GLEmiter.py: Likewise.
* gnulib-tool.py (main): For --extract-description, --extract-comment,
--extract-status, --extract-notice, --extract-autoconf-snippet,
--extract-automake-snippet, --extract-include-directive,
--extract-link-directive, --extract-maintainer, don't add an extra
newline after the snippet.
gnulib-tool.py: Improve field naming.
* pygnulib/GLModuleSystem.py (GLModule): Rename field 'module' to
'path'. Fix a typo in a TypeError message.
gnulib-tool.py: Simplify.
* pygnulib/GLModuleSystem.py (GLModule): Convert Windows newlines right
after reading the module description, not in every accessor.
gnulib-tool.py: Reduce code duplication.
* pygnulib/GLModuleSystem.py (GLModule): Declare two regexes are class
variables.
gnulib-tool.py: Implement option --find.
* pygnulib/GLModuleSystem.py (GLModuleSystem.file_is_module): New
method.
(GLModuleSystem.list): Filter the listing in memory; don't use a 'sed'
subprocess.
* gnulib-tool.py (main): Handle mode 'find'.
gnulib-tool.py: Fix some regexes.
* pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): Use an equivalent
regex as gnulib-tool.
* pygnulib/GLTestDir.py (GLTestDir.execute): Likewise.
gnulib-tool.py: Fix some regex uses.
* pygnulib/GLModuleSystem.py (GLModuleTable.transitive_closure): Match
the regex against all lines of the snippet, not only the first line.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Likewise.
gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/constants.py: Don't use the "minimal matching" *? construct
when it makes no difference (because we're matching a single line only
and flag re.M is not specified).
* pygnulib/GLModuleSystem.py: Likewise.
gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/GLModuleSystem.py: Don't use flag re.S on regular expressions
that are meant to match a single line only, and remove the use of the
"minimal matching" *? construct whose only purpose was to neutralize the
re.S flag.
* pygnulib/GLEmiter.py: Likewise.
* pygnulib/GLImport.py: Likewise.
* pygnulib/GLTestDir.py: Likewise.
gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/GLEmiter.py: Don't use flag re.S on regular expressions on
regular expressions with no '.'.
* pygnulib/GLImport.py: Likewise.
2022-08-06 Bruno Haible <br...@clisp.org>
gnulib-tool.py: Fix output in --dry-run mode.
* pygnulib/GLImport.py (GLImport._update_ignorelist_): In dry-run mode,
say "Update", not "Updating".
gnulib-tool.py: Finish implementing options --vc-files, --no-vc-files.
* gnulib-tool.py (main): Accept options --vc-files, --no-vc-files.
* pygnulib/GLImport.py (GLImport.__init__): Correct parsing of
gl_VC_FILES directive.
(GLImport.gnulib_cache): Don't treat the value False like None.
(GLImport.execute): Skip the .gitignore file manipulations if vc_files
is False.
gnulib-tool.py: Finish implementing option --witness-c-macro.
* gnulib-tool.py (main): Accept option --witness-c-macro.
* pygnulib/GLConfig.py (GLConfig.__init__): Remove wrong type check of
witness_c_macro argument.
>From 860df12df83dd27ae2b83e0ed6b485c4ffbb640e Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sat, 6 Aug 2022 21:54:04 +0200
Subject: [PATCH 01/19] gnulib-tool.py: Finish implementing option
--witness-c-macro.
* gnulib-tool.py (main): Accept option --witness-c-macro.
* pygnulib/GLConfig.py (GLConfig.__init__): Remove wrong type check of
witness_c_macro argument.
---
ChangeLog | 7 +++++++
gnulib-tool.py | 12 ++++++++++--
gnulib-tool.py.TODO | 1 -
pygnulib/GLConfig.py | 9 +--------
4 files changed, 18 insertions(+), 11 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index dc968b8982..0043103914 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2022-08-06 Bruno Haible <br...@clisp.org>
+
+ gnulib-tool.py: Finish implementing option --witness-c-macro.
+ * gnulib-tool.py (main): Accept option --witness-c-macro.
+ * pygnulib/GLConfig.py (GLConfig.__init__): Remove wrong type check of
+ witness_c_macro argument.
+
2022-08-05 Bruno Haible <br...@clisp.org>
gnulib-tool.py: Don't initialize local variables too early.
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 78bb326a69..0bf72bf3a0 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -358,6 +358,11 @@ def main():
dest='podomain',
default=None,
nargs=1)
+ # witness-c-macro
+ parser.add_argument('--witness-c-macro',
+ dest='witness_c_macro',
+ default=None,
+ nargs=1)
# single-configure
parser.add_argument('--single-configure',
dest='single_configure',
@@ -547,7 +552,8 @@ def main():
or cmdargs.excl_unportable_tests != None
or cmdargs.avoids != None or cmdargs.lgpl != None
or cmdargs.makefile_name != None
- or cmdargs.macro_prefix != None or cmdargs.podomain != None))):
+ or cmdargs.macro_prefix != None or cmdargs.podomain != None
+ or cmdargs.witness_c_macro != None))):
message = '%s: *** ' % constants.APP['name']
message += 'invalid options for --%s mode\n' % mode
message += 'Try \'gnulib-tool --help\' for more information.\n'
@@ -640,6 +646,9 @@ def main():
podomain = cmdargs.podomain
if podomain != None:
podomain = podomain[0]
+ witness_c_macro = cmdargs.witness_c_macro
+ if witness_c_macro != None:
+ witness_c_macro = witness_c_macro[0]
avoids = cmdargs.avoids
if avoids != None:
avoids = [ module
@@ -650,7 +659,6 @@ def main():
single_configure = cmdargs.single_configure
docbase = None
conddeps = None
- witness_c_macro = None
vc_files = None
# Create pygnulib configuration.
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index b59e89cc59..c37fcbeebf 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -29,7 +29,6 @@ Implement the options:
--gnu-make
--tests-makefile-name
--automake-subdir
- --witness-c-macro
--vc-files
--no-vc-files
-h | --hardlink
diff --git a/pygnulib/GLConfig.py b/pygnulib/GLConfig.py
index e571dff2b3..5f30a80797 100644
--- a/pygnulib/GLConfig.py
+++ b/pygnulib/GLConfig.py
@@ -153,14 +153,7 @@ class GLConfig(object):
# witness_c_macro
self.resetWitnessCMacro()
if witness_c_macro != None:
- if type(witness_c_macro) is bool:
- if not witness_c_macro:
- self.setWitnessCMacro()
- else: # if witness_c_macro
- self.resetWitnessCMacro()
- else: # if type(witness_c_macro) is not bool
- raise TypeError('witness_c_macro must be a bool, not %s'
- % type(witness_c_macro).__name__)
+ self.setWitnessCMacro(witness_c_macro)
# vc_files
self.resetVCFiles()
if vc_files != None:
--
2.34.1
>From 9812027a9afa87b12eb3b7e3c28600e6eff7ca98 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sat, 6 Aug 2022 23:01:36 +0200
Subject: [PATCH 02/19] gnulib-tool.py: Finish implementing options --vc-files,
--no-vc-files.
* gnulib-tool.py (main): Accept options --vc-files, --no-vc-files.
* pygnulib/GLImport.py (GLImport.__init__): Correct parsing of
gl_VC_FILES directive.
(GLImport.gnulib_cache): Don't treat the value False like None.
(GLImport.execute): Skip the .gitignore file manipulations if vc_files
is False.
---
ChangeLog | 8 ++++++
gnulib-tool.py | 13 ++++++++--
gnulib-tool.py.TODO | 4 +--
pygnulib/GLConfig.py | 2 +-
pygnulib/GLImport.py | 62 ++++++++++++++++++++++----------------------
5 files changed, 53 insertions(+), 36 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 0043103914..29da7a3a6e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,13 @@
2022-08-06 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Finish implementing options --vc-files, --no-vc-files.
+ * gnulib-tool.py (main): Accept options --vc-files, --no-vc-files.
+ * pygnulib/GLImport.py (GLImport.__init__): Correct parsing of
+ gl_VC_FILES directive.
+ (GLImport.gnulib_cache): Don't treat the value False like None.
+ (GLImport.execute): Skip the .gitignore file manipulations if vc_files
+ is False.
+
gnulib-tool.py: Finish implementing option --witness-c-macro.
* gnulib-tool.py (main): Accept option --witness-c-macro.
* pygnulib/GLConfig.py (GLConfig.__init__): Remove wrong type check of
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 0bf72bf3a0..bb763a2245 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -363,6 +363,15 @@ def main():
dest='witness_c_macro',
default=None,
nargs=1)
+ # vc-files
+ parser.add_argument('--vc-files',
+ dest='vc_files',
+ default=None,
+ action='store_true')
+ parser.add_argument('--no-vc-files',
+ dest='vc_files',
+ default=None,
+ action='store_false')
# single-configure
parser.add_argument('--single-configure',
dest='single_configure',
@@ -553,7 +562,7 @@ def main():
or cmdargs.avoids != None or cmdargs.lgpl != None
or cmdargs.makefile_name != None
or cmdargs.macro_prefix != None or cmdargs.podomain != None
- or cmdargs.witness_c_macro != None))):
+ or cmdargs.witness_c_macro != None or cmdargs.vc_files != None))):
message = '%s: *** ' % constants.APP['name']
message += 'invalid options for --%s mode\n' % mode
message += 'Try \'gnulib-tool --help\' for more information.\n'
@@ -649,6 +658,7 @@ def main():
witness_c_macro = cmdargs.witness_c_macro
if witness_c_macro != None:
witness_c_macro = witness_c_macro[0]
+ vc_files = cmdargs.vc_files
avoids = cmdargs.avoids
if avoids != None:
avoids = [ module
@@ -659,7 +669,6 @@ def main():
single_configure = cmdargs.single_configure
docbase = None
conddeps = None
- vc_files = None
# Create pygnulib configuration.
config = classes.GLConfig(
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index c37fcbeebf..53915662b8 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -29,14 +29,14 @@ Implement the options:
--gnu-make
--tests-makefile-name
--automake-subdir
- --vc-files
- --no-vc-files
-h | --hardlink
--local-hardlink
-S | --more-symlinks
-H | --more-hardlinks
--help (same output)
+Remove exit() in GLImport.py.
+
--------------------------------------------------------------------------------
commit 76c7703cb2e9e0e803d1296618d8ab9e86e13d6c
diff --git a/pygnulib/GLConfig.py b/pygnulib/GLConfig.py
index 5f30a80797..216f2e05fa 100644
--- a/pygnulib/GLConfig.py
+++ b/pygnulib/GLConfig.py
@@ -276,7 +276,7 @@ class GLConfig(object):
elif key in ['libtool', 'lgpl', 'conddeps', 'symbolic', 'lsymbolic',
'libtests', 'dryrun']:
return False
- if key == 'vc_files':
+ elif key == 'vc_files':
return None
elif key == 'errors':
return True
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 9de4b3b93c..b06fc5b603 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -141,9 +141,6 @@ class GLImport(object):
if 'gl_CONDITIONAL_DEPENDENCIES' in data:
self.cache.setCondDeps(True)
data = data.replace('gl_CONDITIONAL_DEPENDENCIES', '')
- if 'gl_VC_FILES' in data:
- self.cache.setVCFiles(True)
- data = data.replace('gl_VC_FILES', '')
if 'gl_WITH_TESTS' in data:
self.cache.enableInclTestCategory(TESTS['tests'])
data = data.replace('gl_WITH_TESTS', '')
@@ -202,6 +199,8 @@ class GLImport(object):
self.cache.setPoDomain(cleaner(tempdict['gl_PO_DOMAIN']))
if tempdict['gl_WITNESS_C_MACRO']:
self.cache.setWitnessCMacro(cleaner(tempdict['gl_WITNESS_C_MACRO']))
+ if tempdict['gl_VC_FILES']:
+ self.cache.setVCFiles(cleaner(tempdict['gl_VC_FILES']))
# Get cached filelist from gnulib-comp.m4.
destdir, m4base = self.config.getDestDir(), self.config.getM4Base()
@@ -542,7 +541,7 @@ class GLImport(object):
emit += 'gl_MACRO_PREFIX([%s])\n' % macro_prefix
emit += 'gl_PO_DOMAIN([%s])\n' % podomain
emit += 'gl_WITNESS_C_MACRO([%s])\n' % witness_c_macro
- if vc_files:
+ if vc_files != None:
emit += 'gl_VC_FILES([%s])\n' % vc_files
return constants.nlconvert(emit)
@@ -1334,33 +1333,34 @@ AC_DEFUN([%s_FILE_LIST], [\n''' % macro_prefix
if isfile(tmpfile):
os.remove(tmpfile)
- # Update the .cvsignore and .gitignore files.
- ignorelist = list()
- filetable['added'] = sorted(set(filetable['added']))
- filetable['removed'] = sorted(set(filetable['added']))
- for file in filetable['added']:
- directory, basename = os.path.split(file)
- ignorelist += [tuple([directory, '|A|', basename])]
- for file in filetable['removed']:
- directory, basename = os.path.split(file)
- ignorelist += [tuple([directory, '|R|', basename])]
- last_dir = ''
- last_dirs_added = list()
- last_dirs_removed = list()
- for row in ignorelist:
- next_dir = row[0]
- operand = row[1]
- filename = row[2]
- if next_dir != last_dir:
- self._done_dir_(last_dir, last_dirs_added, last_dirs_removed)
- last_dir = next_dir
- last_dirs_added = list()
- last_dirs_removed = list()
- if operand == '|A|':
- last_dirs_added += [filename]
- elif operand == '|R|':
- last_dirs_removed += [filename]
- self._done_dir_(last_dir, last_dirs_added, last_dirs_removed)
+ if vc_files != False:
+ # Update the .cvsignore and .gitignore files.
+ ignorelist = list()
+ filetable['added'] = sorted(set(filetable['added']))
+ filetable['removed'] = sorted(set(filetable['added']))
+ for file in filetable['added']:
+ directory, basename = os.path.split(file)
+ ignorelist += [tuple([directory, '|A|', basename])]
+ for file in filetable['removed']:
+ directory, basename = os.path.split(file)
+ ignorelist += [tuple([directory, '|R|', basename])]
+ last_dir = ''
+ last_dirs_added = list()
+ last_dirs_removed = list()
+ for row in ignorelist:
+ next_dir = row[0]
+ operand = row[1]
+ filename = row[2]
+ if next_dir != last_dir:
+ self._done_dir_(last_dir, last_dirs_added, last_dirs_removed)
+ last_dir = next_dir
+ last_dirs_added = list()
+ last_dirs_removed = list()
+ if operand == '|A|':
+ last_dirs_added += [filename]
+ elif operand == '|R|':
+ last_dirs_removed += [filename]
+ self._done_dir_(last_dir, last_dirs_added, last_dirs_removed)
exit()
# Finish the work.
--
2.34.1
>From 07d41642fbad4a36a7ece9ee33db6a4d641cb585 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sat, 6 Aug 2022 23:10:14 +0200
Subject: [PATCH 03/19] gnulib-tool.py: Fix output in --dry-run mode.
* pygnulib/GLImport.py (GLImport._update_ignorelist_): In dry-run mode,
say "Update", not "Updating".
---
ChangeLog | 4 ++++
pygnulib/GLImport.py | 2 +-
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index 29da7a3a6e..e5af4e34c0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2022-08-06 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix output in --dry-run mode.
+ * pygnulib/GLImport.py (GLImport._update_ignorelist_): In dry-run mode,
+ say "Update", not "Updating".
+
gnulib-tool.py: Finish implementing options --vc-files, --no-vc-files.
* gnulib-tool.py (main): Accept options --vc-files, --no-vc-files.
* pygnulib/GLImport.py (GLImport.__init__): Correct parsing of
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index b06fc5b603..13418400af 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -739,7 +739,7 @@ AC_DEFUN([%s_FILE_LIST], [\n''' % macro_prefix
with codecs.open(srcpath, 'ab', 'UTF-8') as file:
file.write(destdata)
else: # if self.config['dryrun']
- print('Updating %s (backup in %s)' % (srcpath, backupname))
+ print('Update %s (backup in %s)' % (srcpath, backupname))
else: # if not isfile(srcpath)
if dirs_added:
if not self.config['dryrun']:
--
2.34.1
>From a68f103cc3863310e1c1b4dd9e05b7924381d9a1 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 13:11:07 +0200
Subject: [PATCH 05/19] gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/GLEmiter.py: Don't use flag re.S on regular expressions on
regular expressions with no '.'.
* pygnulib/GLImport.py: Likewise.
---
ChangeLog | 7 +++++++
pygnulib/GLEmiter.py | 4 ++--
pygnulib/GLImport.py | 6 +++---
3 files changed, 12 insertions(+), 5 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 5396f935a3..bcaedf98e4 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2022-08-07 Bruno Haible <br...@clisp.org>
+
+ gnulib-tool.py: Make regex uses more straightforward.
+ * pygnulib/GLEmiter.py: Don't use flag re.S on regular expressions on
+ regular expressions with no '.'.
+ * pygnulib/GLImport.py: Likewise.
+
2022-08-06 Bruno Haible <br...@clisp.org>
gnulib-tool: In the VC files messages, omit the destination directory.
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index b6c31fa603..f85ef30d52 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -750,7 +750,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# * https://debbugs.gnu.org/10997
# * https://debbugs.gnu.org/11030
# So we need this workaround.
- pattern = re.compile('^pkgdata_DATA *\\+=', re.S | re.M)
+ pattern = re.compile('^pkgdata_DATA *\\+=', re.M)
if pattern.findall(allsnippets):
emit += 'pkgdata_DATA =\n'
emit += 'EXTRA_DIST =\n'
@@ -1032,7 +1032,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# * https://debbugs.gnu.org/10997
# * https://debbugs.gnu.org/11030
# So we need this workaround.
- pattern = re.compile('^pkgdata_DATA *\\+=', re.S | re.M)
+ pattern = re.compile('^pkgdata_DATA *\\+=', re.M)
if bool(pattern.findall(main_snippets)) or bool(pattern.findall(longrun_snippets)):
emit += 'pkgdata_DATA =\n'
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 935a662593..7a6312ec39 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -100,7 +100,7 @@ class GLImport(object):
if match:
result = cleaner(match)[0]
self.cache.setAuxDir(joinpath(self.config['destdir'], result))
- pattern = re.compile(r'A[CM]_PROG_LIBTOOL', re.S | re.M)
+ pattern = re.compile(r'A[CM]_PROG_LIBTOOL', re.M)
guessed_libtool = bool(pattern.findall(data))
if self.config['auxdir'] == None:
self.config.setAuxDir(self.cache['auxdir'])
@@ -1439,9 +1439,9 @@ in <library>_a_LDFLAGS or <library>_la_LDFLAGS when linking a library.''')
with codecs.open(configure_ac, 'rb', 'UTF-8') as file:
data = file.read()
match_result1 = \
- bool(re.compile('^ *AC_PROG_CC_STDC', re.S | re.M).findall(data))
+ bool(re.compile('^ *AC_PROG_CC_STDC', re.M).findall(data))
match_result2 = \
- bool(re.compile('^ *AC_PROG_CC_C99', re.S | re.M).findall(data))
+ bool(re.compile('^ *AC_PROG_CC_C99', re.M).findall(data))
if match_result1:
position_early_after = 'AC_PROG_CC_STDC'
elif match_result2:
--
2.34.1
>From 54bf87fe01941049356177a1902bdf1a3eb49f7f Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 13:29:59 +0200
Subject: [PATCH 06/19] gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/GLModuleSystem.py: Don't use flag re.S on regular expressions
that are meant to match a single line only, and remove the use of the
"minimal matching" *? construct whose only purpose was to neutralize the
re.S flag.
* pygnulib/GLEmiter.py: Likewise.
* pygnulib/GLImport.py: Likewise.
* pygnulib/GLTestDir.py: Likewise.
---
ChangeLog | 9 +++++++++
pygnulib/GLEmiter.py | 18 +++++++++---------
pygnulib/GLImport.py | 4 ++--
pygnulib/GLModuleSystem.py | 6 +++---
pygnulib/GLTestDir.py | 20 ++++++++++----------
5 files changed, 33 insertions(+), 24 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index bcaedf98e4..0e162e8d0a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,14 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Make regex uses more straightforward.
+ * pygnulib/GLModuleSystem.py: Don't use flag re.S on regular expressions
+ that are meant to match a single line only, and remove the use of the
+ "minimal matching" *? construct whose only purpose was to neutralize the
+ re.S flag.
+ * pygnulib/GLEmiter.py: Likewise.
+ * pygnulib/GLImport.py: Likewise.
+ * pygnulib/GLTestDir.py: Likewise.
+
gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/GLEmiter.py: Don't use flag re.S on regular expressions on
regular expressions with no '.'.
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index f85ef30d52..02fa6654e7 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -153,7 +153,7 @@ class GLEmiter(object):
if line.strip() ]
snippet = '%s\n' % '\n'.join(lines)
transformer = fileassistant.transformers.get('aux', '')
- pattern = re.compile('(^.*?$)', re.S | re.M)
+ pattern = re.compile('^(.*)$', re.M)
snippet = pattern.sub('%s\\1' % indentation, snippet)
if transformer:
args = ['sed', '-e', transformer]
@@ -182,9 +182,9 @@ class GLEmiter(object):
emit += 'changequote([, ])dnl\n'
emit += 'AC_SUBST([LTALLOCA])'
if replace_auxdir:
- regex = 'AC_CONFIG_FILES\\(\\[(.*?)\\:build-aux/(.*?)\\]\\)'
+ regex = 'AC_CONFIG_FILES\\(\\[(.*)\\:build-aux/(.*)\\]\\)'
repl = 'AC_CONFIG_FILES([\\1:%s/\\2])' % auxdir
- pattern = re.compile(regex, re.S | re.M)
+ pattern = re.compile(regex, re.M)
emit = pattern.sub(repl, emit)
lines = [ line
for line in emit.split('\n')
@@ -688,9 +688,9 @@ AC_DEFUN([%V1%_LIBSOURCES], [
amsnippet1 = amsnippet1.replace('lib_LIBRARIES', 'lib%_LIBRARIES')
amsnippet1 = amsnippet1.replace('lib_LTLIBRARIES', 'lib%_LTLIBRARIES')
if LD_flags:
- pattern = re.compile('lib_LDFLAGS[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('lib_LDFLAGS[\t ]*\\+=(.*)$', re.M)
amsnippet1 = pattern.sub('', amsnippet1)
- pattern = re.compile('lib_([A-Z][A-Z](?:.*?))', re.S | re.M)
+ pattern = re.compile('lib_([A-Z][A-Z](?:.*))', re.M)
amsnippet1 = pattern.sub('%s_%s_\\1' % (libname, libext),
amsnippet1)
amsnippet1 = amsnippet1.replace('$(GNULIB_', '$(' + module_indicator_prefix + '_GNULIB_')
@@ -708,7 +708,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# Get unconditional snippet, edit it and save to amsnippet2.
amsnippet2 = module.getAutomakeSnippet_Unconditional()
- pattern = re.compile('lib_([A-Z][A-Z](?:.*?))', re.S | re.M)
+ pattern = re.compile('lib_([A-Z][A-Z](?:.*))', re.M)
amsnippet2 = pattern.sub('%s_%s_\\1' % (libname, libext),
amsnippet2)
amsnippet2 = amsnippet2.replace('$(GNULIB_',
@@ -802,7 +802,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
insnippets = False
inmakefile = False
regex = '^[a-zA-Z0-9_]*_%sLIBRARIES *\\+{0,1}= *%s.%s' % (perhapsLT, libname, libext)
- pattern = re.compile(regex, re.S | re.M)
+ pattern = re.compile(regex, re.M)
insnippets = bool(pattern.findall(allsnippets))
# Then test if $sourcebase/Makefile.am (if it exists) specifies it.
path = joinpath(sourcebase, 'Makefile.am')
@@ -955,9 +955,9 @@ AC_DEFUN([%V1%_LIBSOURCES], [
snippet = snippet.replace('lib_LIBRARIES', 'lib%_LIBRARIES')
snippet = snippet.replace('lib_LTLIBRARIES', 'lib%_LTLIBRARIES')
if LD_flags:
- pattern = re.compile('lib_LDFLAGS[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('lib_LDFLAGS[\t ]*\\+=(.*)$', re.M)
snippet = pattern.sub('', snippet)
- pattern = re.compile('lib_([A-Z][A-Z](?:.*?))', re.S | re.M)
+ pattern = re.compile('lib_([A-Z][A-Z](?:.*))', re.M)
snippet = pattern.sub('libtests_a_\\1', snippet)
snippet = snippet.replace('$(GNULIB_', '$(' + module_indicator_prefix + '_GNULIB_')
snippet = snippet.replace('lib%_LIBRARIES', 'lib_LIBRARIES')
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 7a6312ec39..818f3d57c0 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -95,7 +95,7 @@ class GLImport(object):
self.config.setAutoconfFile(path)
with codecs.open(path, 'rb', 'UTF-8') as file:
data = file.read()
- pattern = re.compile(r'^AC_CONFIG_AUX_DIR\((.*?)\)$', re.S | re.M)
+ pattern = re.compile(r'^AC_CONFIG_AUX_DIR\((.*)\)$', re.M)
match = pattern.findall(data)
if match:
result = cleaner(match)[0]
@@ -106,7 +106,7 @@ class GLImport(object):
self.config.setAuxDir(self.cache['auxdir'])
# Guess autoconf version.
- pattern = re.compile(r'.*AC_PREREQ\((.*?)\)', re.S | re.M)
+ pattern = re.compile(r'.*AC_PREREQ\((.*)\)', re.M)
versions = cleaner(pattern.findall(data))
if versions:
version = sorted(set([ float(version)
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 6b8f20e54f..afc7a472cf 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -705,7 +705,7 @@ Include:|Link:|License:|Maintainer:)'
# TODO: unconditional automake snippet for nontests modules
snippet = self.getAutomakeSnippet_Conditional()
snippet = constants.combine_lines(snippet)
- pattern = re.compile('^lib_SOURCES[\t ]*\\+=[\t ]*(.*?)$', re.S | re.M)
+ pattern = re.compile('^lib_SOURCES[\t ]*\\+=[\t ]*(.*)$', re.M)
mentioned_files = pattern.findall(snippet)
if mentioned_files != list():
mentioned_files = mentioned_files[-1].split(' ')
@@ -780,7 +780,7 @@ Include:|Link:|License:|Maintainer:)'
parts += [line]
result = ''.join(parts)
result = result.strip()
- pattern = re.compile('^(["<].*?[>"])', re.S | re.M)
+ pattern = re.compile('^(["<].*[>"])', re.M)
result = pattern.sub('#include \\1', result)
self.cache['include'] = result
return self.cache['include']
@@ -1164,7 +1164,7 @@ class GLModuleTable(object):
raise TypeError('each module must be a GLModule instance')
snippet = module.getAutomakeSnippet()
snippet = constants.remove_backslash_newline(snippet)
- pattern = re.compile('^lib_SOURCES[\t ]*\\+=[\t ]*(.*?)$', re.S | re.M)
+ pattern = re.compile('^lib_SOURCES[\t ]*\\+=[\t ]*(.*)$', re.M)
files = pattern.findall(snippet)
if files: # if source files were found
files = files[-1].split(' ')
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index 757c494011..f8d9910ab7 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -299,7 +299,7 @@ class GLTestDir(object):
notice = module.getNotice()
if notice:
print('Notice from module %s:' % str(module))
- pattern = re.compile('^(.*?)$', re.S | re.M)
+ pattern = re.compile('^(.*)$', re.M)
notice = pattern.sub(' \\1', notice)
print(notice)
else: # if not single_configure
@@ -307,7 +307,7 @@ class GLTestDir(object):
notice = module.getNotice()
if notice:
print('Notice from module %s:' % str(module))
- pattern = re.compile('^(.*?)$', re.S | re.M)
+ pattern = re.compile('^(.*)$', re.M)
notice = pattern.sub(' \\1', notice)
print(notice)
@@ -461,7 +461,7 @@ class GLTestDir(object):
for line in snippet.split('\n')
if line.strip() ]
snippet = '\n'.join(lines)
- pattern = re.compile('AC_REQUIRE\\(\\[([^()].*?)\\]\\)', re.S | re.M)
+ pattern = re.compile('AC_REQUIRE\\(\\[([^()].*)\\]\\)', re.M)
snippet = pattern.sub('\\1', snippet)
snippet = snippet.strip()
snippets += [snippet]
@@ -578,7 +578,7 @@ class GLTestDir(object):
for line in snippet.split('\n')
if line.strip() ]
snippet = '\n'.join(lines)
- pattern = re.compile('AC_REQUIRE\\(\\[([^()].*?)\\]\\)', re.S | re.M)
+ pattern = re.compile('AC_REQUIRE\\(\\[([^()].*)\\]\\)', re.M)
snippet = pattern.sub('\\1', snippet)
snippet = snippet.strip()
snippets += [snippet]
@@ -749,9 +749,9 @@ class GLTestDir(object):
# Extract the value of "CLEANFILES += ..." and "MOSTLYCLEANFILES += ...".
regex_find = list()
- pattern = re.compile('^CLEANFILES[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('^CLEANFILES[\t ]*\\+=(.*)$', re.M)
regex_find += pattern.findall(snippet)
- pattern = re.compile('^MOSTLYCLEANFILES[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('^MOSTLYCLEANFILES[\t ]*\\+=(.*)$', re.M)
regex_find += pattern.findall(snippet)
regex_find = [ line.strip()
for line in regex_find
@@ -765,7 +765,7 @@ class GLTestDir(object):
# Extract the value of "BUILT_SOURCES += ...". Remove variable references
# such $(FOO_H) because they don't refer to distributed files.
regex_find = list()
- pattern = re.compile('^BUILT_SOURCES[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('^BUILT_SOURCES[\t ]*\\+=(.*)$', re.M)
regex_find += pattern.findall(snippet)
regex_find = [ line.strip()
for line in regex_find
@@ -791,9 +791,9 @@ class GLTestDir(object):
# Extract the value of "CLEANFILES += ..." and "MOSTLYCLEANFILES += ...".
regex_find = list()
- pattern = re.compile('^CLEANFILES[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('^CLEANFILES[\t ]*\\+=(.*)$', re.M)
regex_find += pattern.findall(snippet)
- pattern = re.compile('^MOSTLYCLEANFILES[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('^MOSTLYCLEANFILES[\t ]*\\+=(.*)$', re.M)
regex_find += pattern.findall(snippet)
regex_find = [ line.strip()
for line in regex_find
@@ -808,7 +808,7 @@ class GLTestDir(object):
# such $(FOO_H) because they don't refer to distributed files.
regex_find = list()
tests_built_sources = list()
- pattern = re.compile('^BUILT_SOURCES[\t ]*\\+=(.*?)$', re.S | re.M)
+ pattern = re.compile('^BUILT_SOURCES[\t ]*\\+=(.*)$', re.M)
regex_find += pattern.findall(snippet)
regex_find = [ line.strip()
for line in regex_find
--
2.34.1
>From 9d722ddced2e46a06255be8eb24a54015f611af7 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 13:40:07 +0200
Subject: [PATCH 07/19] gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/constants.py: Don't use the "minimal matching" *? construct
when it makes no difference (because we're matching a single line only
and flag re.M is not specified).
* pygnulib/GLModuleSystem.py: Likewise.
---
ChangeLog | 6 ++++++
pygnulib/GLModuleSystem.py | 2 +-
pygnulib/constants.py | 2 +-
3 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 0e162e8d0a..f50e756d60 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Make regex uses more straightforward.
+ * pygnulib/constants.py: Don't use the "minimal matching" *? construct
+ when it makes no difference (because we're matching a single line only
+ and flag re.M is not specified).
+ * pygnulib/GLModuleSystem.py: Likewise.
+
gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/GLModuleSystem.py: Don't use flag re.S on regular expressions
that are meant to match a single line only, and remove the use of the
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index afc7a472cf..e1b62ec56c 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -280,7 +280,7 @@ Include:|Link:|License:|Maintainer:)'
'''GLModule.getName() -> str
Return the name of the module.'''
- pattern = re.compile(joinpath('modules', '(.*?)$'))
+ pattern = re.compile(joinpath('modules', '(.*)$'))
result = pattern.findall(self.module)[0]
return result
diff --git a/pygnulib/constants.py b/pygnulib/constants.py
index 46c0cbc152..ba0ebc9942 100644
--- a/pygnulib/constants.py
+++ b/pygnulib/constants.py
@@ -412,7 +412,7 @@ def filter_filelist(separator, filelist,
listing = list()
for filename in filelist:
if filename.startswith(prefix) and filename.endswith(suffix):
- pattern = re.compile('^%s(.*?)%s$'
+ pattern = re.compile('^%s(.*)%s$'
% (removed_prefix, removed_suffix))
result = pattern.sub('%s\\1%s'
% (added_prefix, added_suffix), filename)
--
2.34.1
>From 5c087011efa2817a7e857f69a1cdd22c21a5053d Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 13:46:15 +0200
Subject: [PATCH 08/19] gnulib-tool.py: Fix some regex uses.
* pygnulib/GLModuleSystem.py (GLModuleTable.transitive_closure): Match
the regex against all lines of the snippet, not only the first line.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Likewise.
---
ChangeLog | 5 +++++
pygnulib/GLEmiter.py | 2 +-
pygnulib/GLModuleSystem.py | 2 +-
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index f50e756d60..a44504ae20 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,10 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix some regex uses.
+ * pygnulib/GLModuleSystem.py (GLModuleTable.transitive_closure): Match
+ the regex against all lines of the snippet, not only the first line.
+ * pygnulib/GLEmiter.py (GLEmiter.autoconfSnippet): Likewise.
+
gnulib-tool.py: Make regex uses more straightforward.
* pygnulib/constants.py: Don't use the "minimal matching" *? construct
when it makes no difference (because we're matching a single line only
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index 02fa6654e7..9aa638d693 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -174,7 +174,7 @@ class GLEmiter(object):
else:
# Don't indent AM_GNU_GETTEXT_VERSION line, as that confuses
# autopoint through at least GNU gettext version 0.18.2.
- snippet = re.compile('^ *AM_GNU_GETTEXT_VERSION').sub('AM_GNU_GETTEXT_VERSION', snippet)
+ snippet = re.compile('^ *AM_GNU_GETTEXT_VERSION', re.M).sub('AM_GNU_GETTEXT_VERSION', snippet)
emit += snippet
if str(module) == 'alloca' and libtool and not disable_libtool:
emit += 'changequote(,)dnl\n'
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index e1b62ec56c..75316515b8 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -1041,7 +1041,7 @@ class GLModuleTable(object):
if self.config['conddeps']:
automake_snippet = \
module.getAutomakeSnippet_Conditional()
- pattern = re.compile('^if')
+ pattern = re.compile('^if', re.M)
if not pattern.findall(automake_snippet):
self.addUnconditional(module)
conditional = self.isConditional(module)
--
2.34.1
>From 6a795cb6b5c9f2b42058f444c61291f751c356e2 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 13:49:49 +0200
Subject: [PATCH 09/19] gnulib-tool.py: Fix some regexes.
* pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): Use an equivalent
regex as gnulib-tool.
* pygnulib/GLTestDir.py (GLTestDir.execute): Likewise.
---
ChangeLog | 5 +++++
pygnulib/GLEmiter.py | 2 +-
pygnulib/GLTestDir.py | 4 ++--
3 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index a44504ae20..b052583c01 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,10 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix some regexes.
+ * pygnulib/GLEmiter.py (GLEmiter.lib_Makefile_am): Use an equivalent
+ regex as gnulib-tool.
+ * pygnulib/GLTestDir.py (GLTestDir.execute): Likewise.
+
gnulib-tool.py: Fix some regex uses.
* pygnulib/GLModuleSystem.py (GLModuleTable.transitive_closure): Match
the regex against all lines of the snippet, not only the first line.
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index 9aa638d693..e0164ed7cc 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -801,7 +801,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# First test if allsnippets already specify an installation location.
insnippets = False
inmakefile = False
- regex = '^[a-zA-Z0-9_]*_%sLIBRARIES *\\+{0,1}= *%s.%s' % (perhapsLT, libname, libext)
+ regex = '^[a-zA-Z0-9_]*_%sLIBRARIES *\\+{0,1}= *%s\\.%s' % (perhapsLT, libname, libext)
pattern = re.compile(regex, re.M)
insnippets = bool(pattern.findall(allsnippets))
# Then test if $sourcebase/Makefile.am (if it exists) specifies it.
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index f8d9910ab7..9a3fde66ea 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -461,7 +461,7 @@ class GLTestDir(object):
for line in snippet.split('\n')
if line.strip() ]
snippet = '\n'.join(lines)
- pattern = re.compile('AC_REQUIRE\\(\\[([^()].*)\\]\\)', re.M)
+ pattern = re.compile('AC_REQUIRE\\(\\[([^()]*)\\]\\)', re.M)
snippet = pattern.sub('\\1', snippet)
snippet = snippet.strip()
snippets += [snippet]
@@ -578,7 +578,7 @@ class GLTestDir(object):
for line in snippet.split('\n')
if line.strip() ]
snippet = '\n'.join(lines)
- pattern = re.compile('AC_REQUIRE\\(\\[([^()].*)\\]\\)', re.M)
+ pattern = re.compile('AC_REQUIRE\\(\\[([^()]*)\\]\\)', re.M)
snippet = pattern.sub('\\1', snippet)
snippet = snippet.strip()
snippets += [snippet]
--
2.34.1
>From 72ac023526e84e7847cc67788be9afca2f3204fb Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 17:02:21 +0200
Subject: [PATCH 11/19] gnulib-tool.py: Implement option --find.
* pygnulib/GLModuleSystem.py (GLModuleSystem.file_is_module): New
method.
(GLModuleSystem.list): Filter the listing in memory; don't use a 'sed'
subprocess.
* gnulib-tool.py (main): Handle mode 'find'.
---
ChangeLog | 7 +++++
gnulib-tool.py | 60 +++++++++++++++++++++++++++++++++++++-
gnulib-tool.py.TODO | 5 +++-
pygnulib/GLModuleSystem.py | 59 +++++++++++++++++--------------------
4 files changed, 97 insertions(+), 34 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index e259a09333..acb82e903c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,12 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Implement option --find.
+ * pygnulib/GLModuleSystem.py (GLModuleSystem.file_is_module): New
+ method.
+ (GLModuleSystem.list): Filter the listing in memory; don't use a 'sed'
+ subprocess.
+ * gnulib-tool.py (main): Handle mode 'find'.
+
gnulib-tool: Fix option --find in combination with option --local-dir.
* gnulib-tool (func_prefixed_modules_in_dir): New function.
(find): Use it, and filter the directory names away after the 'grep'
diff --git a/gnulib-tool.py b/gnulib-tool.py
index bb763a2245..032deee6e0 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -43,15 +43,18 @@ import codecs
import random
import argparse
import subprocess as sp
+import shlex
from tempfile import mktemp
from pygnulib import constants
from pygnulib import classes
+from pygnulib import GLError
#===============================================================================
# Define global constants
#===============================================================================
APP = constants.APP
+DIRS = constants.DIRS
ENCS = constants.ENCS
UTILS = constants.UTILS
MODES = constants.MODES
@@ -422,6 +425,7 @@ def main():
# Determine when user tries to combine modes.
args = [
cmdargs.mode_list,
+ cmdargs.mode_find,
cmdargs.mode_import,
cmdargs.mode_add_import,
cmdargs.mode_remove_import,
@@ -460,6 +464,9 @@ def main():
files = None
if cmdargs.mode_list != None:
mode = 'list'
+ if cmdargs.mode_find != None:
+ mode = 'find'
+ files = list(cmdargs.non_option_arguments)
if cmdargs.mode_import != None:
mode = 'import'
modules = list(cmdargs.non_option_arguments)
@@ -701,13 +708,64 @@ def main():
)
# Work in the given mode.
- if mode in ['list']:
+ if mode == 'list':
modulesystem = classes.GLModuleSystem(config)
listing = modulesystem.list()
result = '\n'.join(listing)
os.rmdir(config['tempdir'])
print(result)
+ elif mode == 'find':
+ # Prepare GLModuleSystem.find to throw an exception.
+ config.setErrors(True)
+ modulesystem = classes.GLModuleSystem(config)
+ for filename in files:
+ if (isfile(joinpath(DIRS['root'], filename))
+ or (localpath != None
+ and any([ isfile(joinpath(localdir, filename))
+ for localdir in localpath ]))):
+ # Convert the file name to a POSIX basic regex.
+ # Needs to handle . [ \ * ^ $.
+ filename_regex = filename.replace('\\', '\\\\').replace('[', '\\[').replace('^', '\\^')
+ filename_regex = re.compile('([.*$])').sub('[\\1]', filename_regex)
+ filename_line_regex = '^' + filename_regex + '$'
+ # Read module candidates from gnulib root directory.
+ command = "find modules -type f -print | xargs -n 100 grep -l %s /dev/null | sed -e 's,^modules/,,'" % shlex.quote(filename_line_regex)
+ os.chdir(constants.DIRS['root'])
+ with sp.Popen(command, shell=True, stdout=sp.PIPE) as proc:
+ result = proc.stdout.read().decode("UTF-8")
+ os.chdir(DIRS['cwd'])
+ # Read module candidates from local directories.
+ if localpath != None and len(localpath) > 0:
+ command = "find modules -type f -print | xargs -n 100 grep -l %s /dev/null | sed -e 's,^modules/,,' -e 's,\\.diff$,,'" % shlex.quote(filename_line_regex)
+ for localdir in localpath:
+ os.chdir(localdir)
+ with sp.Popen(command, shell=True, stdout=sp.PIPE) as proc:
+ result += proc.stdout.read().decode("UTF-8")
+ os.chdir(DIRS['cwd'])
+ listing = [ line
+ for line in result.split('\n')
+ if line.strip() ]
+ # Remove modules/ prefix from each file name.
+ pattern = re.compile('^modules/')
+ listing = [ pattern.sub('', line)
+ for line in listing ]
+ # Filter out undesired file names.
+ listing = [ line
+ for line in listing
+ if modulesystem.file_is_module(line) ]
+ module_candidates = sorted(set(listing))
+ for module in module_candidates:
+ try:
+ if filename in modulesystem.find(module).getFiles():
+ print(module)
+ except GLError:
+ # Ignore module candidates that don't actually exist.
+ pass
+ else:
+ message = '%s: warning: file %s does not exist\n' % (constants.APP['name'], filename)
+ sys.stderr.write(message)
+
elif mode in ['import', 'add-import', 'remove-import', 'update']:
mode = MODES[mode]
if not destdir:
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index 53915662b8..a46da5e2ad 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -20,7 +20,6 @@ Inline all 'sed' invocations.
--------------------------------------------------------------------------------
Implement the options:
- --find
--extract-recursive-dependencies
--extract-recursive-link-directive
--extract-tests-module
@@ -37,6 +36,10 @@ Implement the options:
Remove exit() in GLImport.py.
+Optimize:
+ - GLModuleSystem: Parse each module description only once.
+ - os.chdir around subprocess creation -> cwd=... argument instead.
+
--------------------------------------------------------------------------------
commit 76c7703cb2e9e0e803d1296618d8ab9e86e13d6c
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 75316515b8..3383b22b28 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -43,6 +43,7 @@ DIRS = constants.DIRS
ENCS = constants.ENCS
TESTS = constants.TESTS
joinpath = constants.joinpath
+subend = constants.subend
isdir = os.path.isdir
isfile = os.path.isfile
filter_filelist = constants.filter_filelist
@@ -113,6 +114,18 @@ class GLModuleSystem(object):
sys.stderr.write('gnulib-tool: warning: ')
sys.stderr.write('file %s does not exist\n' % str(module))
+ def file_is_module(self, filename):
+ '''Given the name of a file in the modules/ directory, return true
+ if should be viewed as a module description file.'''
+ return not (filename == 'ChangeLog' or filename.endswith('/ChangeLog')
+ or filename == 'COPYING' or filename.endswith('/COPYING')
+ or filename == 'README' or filename.endswith('/README')
+ or filename == 'TEMPLATE'
+ or filename == 'TEMPLATE-EXTENDED'
+ or filename == 'TEMPLATE-TESTS'
+ or filename.startswith('.')
+ or filename.endswith('~'))
+
def list(self):
'''GLModuleSystem.list() -> list
@@ -123,23 +136,6 @@ class GLModuleSystem(object):
listing = list()
localpath = self.config['localpath']
find_args = ['find', 'modules', '-type', 'f', '-print']
- sed_args = \
- [
- 'sed',
- '-e', r's,^modules/,,',
- '-e', r'/^ChangeLog$/d',
- '-e', r'/\/ChangeLog$/d',
- '-e', r'/^COPYING$/d',
- '-e', r'/\/COPYING$/d',
- '-e', r'/^README$/d',
- '-e', r'/\/README$/d',
- '-e', r'/^TEMPLATE$/d',
- '-e', r'/^TEMPLATE-EXTENDED$/d',
- '-e', r'/^TEMPLATE-TESTS$/d',
- '-e', r'/^\..*/d',
- '-e', r'/~$/d',
- '-e', r'/-tests$/d',
- ]
# Read modules from gnulib root directory.
os.chdir(constants.DIRS['root'])
@@ -154,24 +150,23 @@ class GLModuleSystem(object):
find = sp.Popen(find_args, stdout=sp.PIPE)
result += find.stdout.read().decode("UTF-8")
os.chdir(DIRS['cwd'])
- sed_args += ['-e', r's,\.diff$,,']
-
- # Save the list of the modules to file.
- path = joinpath(self.config['tempdir'], 'list')
- with codecs.open(path, 'wb', 'UTF-8') as file:
- file.write(result)
-
- # Filter the list of the modules.
- stdin = codecs.open(path, 'rb', 'UTF-8')
- sed = sp.Popen(sed_args, stdin=stdin, stdout=sp.PIPE)
- result = sed.stdout.read().decode("UTF-8")
- stdin.close()
- os.remove(path)
+
listing = [ line
for line in result.split('\n')
if line.strip() ]
- listing = sorted(set(listing))
- return listing
+ if len(localpath) > 0:
+ listing = [ subend('.diff', '', line)
+ for line in listing ]
+ # Remove modules/ prefix from each file name.
+ pattern = re.compile('^modules/')
+ listing = [ pattern.sub('', line)
+ for line in listing ]
+ # Filter out undesired file names.
+ listing = [ line
+ for line in listing
+ if self.file_is_module(line) and not line.endswith('-tests') ]
+ modules = sorted(set(listing))
+ return modules
#===============================================================================
--
2.34.1
>From 8341b4a206b8f1f6ec14966292afb5134de2a4c2 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 17:29:16 +0200
Subject: [PATCH 12/19] gnulib-tool.py: Reduce code duplication.
* pygnulib/GLModuleSystem.py (GLModule): Declare two regexes are class
variables.
---
ChangeLog | 4 ++
pygnulib/GLModuleSystem.py | 84 +++++++++++---------------------------
2 files changed, 27 insertions(+), 61 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index acb82e903c..a8d020edab 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Reduce code duplication.
+ * pygnulib/GLModuleSystem.py (GLModule): Declare two regexes are class
+ variables.
+
gnulib-tool.py: Implement option --find.
* pygnulib/GLModuleSystem.py (GLModuleSystem.file_is_module): New
method.
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 3383b22b28..33495f9c07 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -177,6 +177,15 @@ class GLModule(object):
path. GLModule can get all information about module, get its dependencies,
files, etc.'''
+ section_label_regex = '(?:Description:|Comment:|Status:|Notice:|Applicability:|\
+Files:|Depends-on:|configure\\.ac-early:|configure\\.ac:|Makefile\\.am:|\
+Include:|Link:|License:|Maintainer:)'
+
+ section_label_pattern = \
+ re.compile('^(Description|Comment|Status|Notice|Applicability|'
+ + 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
+ + 'Makefile\\.am|Include|Link|License|Maintainer):$')
+
def __init__(self, config, module, patched=False):
'''GLModule.__init__(config, module[, patched]) -> GLModule
@@ -202,9 +211,6 @@ class GLModule(object):
self.modulesystem = GLModuleSystem(self.config)
with codecs.open(module, 'rb', 'UTF-8') as file:
self.content = file.read()
- self.regex = '(?:Description:|Comment:|Status:|Notice:|Applicability:|\
-Files:|Depends-on:|configure\\.ac-early:|configure\\.ac:|Makefile\\.am:|\
-Include:|Link:|License:|Maintainer:)'
def __eq__(self, module):
'''x.__eq__(y) <==> x==y'''
@@ -379,7 +385,7 @@ Include:|Link:|License:|Maintainer:)'
if section not in self.content:
result = ''
else: # if section in self.content
- pattern = '^%s[\t ]*(.*?)%s' % (section, self.regex)
+ pattern = '^%s[\t ]*(.*?)%s' % (section, GLModule.section_label_regex)
pattern = re.compile(pattern, re.S | re.M)
result = pattern.findall(self.content)
if type(result) is list:
@@ -400,7 +406,7 @@ Include:|Link:|License:|Maintainer:)'
if section not in self.content:
result = ''
else: # if section in self.content
- pattern = '^%s[\t ]*(.*?)%s' % (section, self.regex)
+ pattern = '^%s[\t ]*(.*?)%s' % (section, GLModule.section_label_regex)
pattern = re.compile(pattern, re.S | re.M)
result = pattern.findall(self.content)
if type(result) is list:
@@ -427,11 +433,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -456,11 +458,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -483,11 +481,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -521,11 +515,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -555,11 +545,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -599,11 +585,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -626,11 +608,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -667,11 +645,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -765,11 +739,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -793,11 +763,7 @@ Include:|Link:|License:|Maintainer:)'
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
@@ -838,7 +804,7 @@ Include:|Link:|License:|Maintainer:)'
if section not in self.content:
result = ''
else: # if section in self.content
- pattern = '^%s[\t ]*(.*?)%s' % (section, self.regex)
+ pattern = '^%s[\t ]*(.*?)%s' % (section, GLModule.section_label_regex)
pattern = re.compile(pattern, re.S | re.M)
result = pattern.findall(self.content)
if type(result) is list:
@@ -865,11 +831,7 @@ Include:|Link:|License:|Maintainer:)'
for line in snippet.split('\n') ]
parts = list()
for line in lines:
- regex = '^(Description|Comment|Status|Notice|Applicability|'
- regex += 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- regex += 'Makefile\\.am|Include|Link|License|Maintainer):$'
- pattern = re.compile(regex)
- findflag = pattern.findall(line)
+ findflag = GLModule.section_label_pattern.findall(line)
if findflag:
break
parts += [line]
--
2.34.1
>From 9b2a7c800a82de247a92d43c10cc926fdcc11031 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 19:26:51 +0200
Subject: [PATCH 13/19] gnulib-tool.py: Simplify.
* pygnulib/GLModuleSystem.py (GLModule): Convert Windows newlines right
after reading the module description, not in every accessor.
---
ChangeLog | 4 ++++
pygnulib/GLModuleSystem.py | 13 +------------
2 files changed, 5 insertions(+), 12 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index a8d020edab..fceb1538a8 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Simplify.
+ * pygnulib/GLModuleSystem.py (GLModule): Convert Windows newlines right
+ after reading the module description, not in every accessor.
+
gnulib-tool.py: Reduce code duplication.
* pygnulib/GLModuleSystem.py (GLModule): Declare two regexes are class
variables.
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 33495f9c07..b736bf2ed1 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -210,7 +210,7 @@ Include:|Link:|License:|Maintainer:)'
self.filesystem = GLFileSystem(self.config)
self.modulesystem = GLModuleSystem(self.config)
with codecs.open(module, 'rb', 'UTF-8') as file:
- self.content = file.read()
+ self.content = file.read().replace('\r\n', '\n')
def __eq__(self, module):
'''x.__eq__(y) <==> x==y'''
@@ -428,7 +428,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -453,7 +452,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -476,7 +474,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -510,7 +507,6 @@ Include:|Link:|License:|Maintainer:)'
result = list()
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -540,7 +536,6 @@ Include:|Link:|License:|Maintainer:)'
depmodules = list()
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -580,7 +575,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -603,7 +597,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -640,7 +633,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -734,7 +726,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
@@ -759,7 +750,6 @@ Include:|Link:|License:|Maintainer:)'
parts = list()
if section in self.content:
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
for line in lines:
@@ -826,7 +816,6 @@ Include:|Link:|License:|Maintainer:)'
result = ''
else: # if section in self.content
snippet = self.content.split(section)[-1]
- snippet = snippet.replace('\r\n', '\n')
lines = [ '%s\n' % line
for line in snippet.split('\n') ]
parts = list()
--
2.34.1
>From 586000e597d4ef7cba8de869a15ad5c922a2010c Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 20:04:56 +0200
Subject: [PATCH 14/19] gnulib-tool.py: Improve field naming.
* pygnulib/GLModuleSystem.py (GLModule): Rename field 'module' to
'path'. Fix a typo in a TypeError message.
---
ChangeLog | 4 ++++
pygnulib/GLModuleSystem.py | 38 ++++++++++++++++++--------------------
2 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index fceb1538a8..36962c2dd2 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Improve field naming.
+ * pygnulib/GLModuleSystem.py (GLModule): Rename field 'module' to
+ 'path'. Fix a typo in a TypeError message.
+
gnulib-tool.py: Simplify.
* pygnulib/GLModuleSystem.py (GLModule): Convert Windows newlines right
after reading the module description, not in every accessor.
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index b736bf2ed1..ec2ff0c35c 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -186,11 +186,11 @@ Include:|Link:|License:|Maintainer:)'
+ 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
+ 'Makefile\\.am|Include|Link|License|Maintainer):$')
- def __init__(self, config, module, patched=False):
- '''GLModule.__init__(config, module[, patched]) -> GLModule
+ def __init__(self, config, path, patched=False):
+ '''GLModule.__init__(config, path[, patched]) -> GLModule
- Create new GLModule instance. Arguments are module and patched, where
- module is a string representing the path to the module and patched is a
+ Create new GLModule instance. Arguments are path and patched, where
+ path is a string representing the path to the module and patched is a
bool indicating that module was created after applying patch.'''
self.args = dict()
self.cache = dict()
@@ -198,25 +198,25 @@ Include:|Link:|License:|Maintainer:)'
if type(config) is not GLConfig:
raise TypeError('config must be a GLConfig, not %s'
% type(config).__name__)
- if type(module) is not str:
- raise TypeError('module must be a string, not %s'
- % type(module).__name__)
+ if type(path) is not str:
+ raise TypeError('path must be a string, not %s'
+ % type(path).__name__)
if type(patched) is not bool:
raise TypeError('patched must be a bool, not %s'
- % type(module).__name__)
- self.module = module
+ % type(patched).__name__)
+ self.path = path
self.patched = patched
self.config = config
self.filesystem = GLFileSystem(self.config)
self.modulesystem = GLModuleSystem(self.config)
- with codecs.open(module, 'rb', 'UTF-8') as file:
+ with codecs.open(path, 'rb', 'UTF-8') as file:
self.content = file.read().replace('\r\n', '\n')
def __eq__(self, module):
'''x.__eq__(y) <==> x==y'''
result = bool()
if type(module) is GLModule:
- if self.module == module.module:
+ if self.path == module.path:
result = True
return result
@@ -224,7 +224,7 @@ Include:|Link:|License:|Maintainer:)'
'''x.__ne__(y) <==> x!=y'''
result = bool()
if type(module) is GLModule:
- if self.module != module.module:
+ if self.path != module.path:
result = True
return result
@@ -232,7 +232,7 @@ Include:|Link:|License:|Maintainer:)'
'''x.__ge__(y) <==> x>=y'''
result = bool()
if type(module) is GLModule:
- if self.module >= module.module:
+ if self.path >= module.path:
result = True
return result
@@ -240,22 +240,20 @@ Include:|Link:|License:|Maintainer:)'
'''x.__gt__(y) <==> x>y'''
result = bool()
if type(module) is GLModule:
- if self.module > module.module:
+ if self.path > module.path:
result = True
return result
def __hash__(self):
'''x.__hash__() <==> hash(x)'''
- module = hash(self.module)
- patched = hash(self.patched)
- result = module ^ patched
+ result = hash(self.path) ^ hash(self.patched)
return result
def __le__(self, module):
'''x.__le__(y) <==> x<=y'''
result = bool()
if type(module) is GLModule:
- if self.module <= module.module:
+ if self.path <= module.path:
result = True
return result
@@ -263,7 +261,7 @@ Include:|Link:|License:|Maintainer:)'
'''x.__lt__(y) <==> x<y'''
result = bool()
if type(module) is GLModule:
- if self.module < module.module:
+ if self.path < module.path:
result = True
return result
@@ -282,7 +280,7 @@ Include:|Link:|License:|Maintainer:)'
Return the name of the module.'''
pattern = re.compile(joinpath('modules', '(.*)$'))
- result = pattern.findall(self.module)[0]
+ result = pattern.findall(self.path)[0]
return result
def isPatched(self):
--
2.34.1
>From 81b8c4d5565dbbea10eb3561063d2e8da52148d7 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 22:53:32 +0200
Subject: [PATCH 15/19] gnulib-tool.py: Fix section extraction from module
descriptions.
The code with self.content.split(section)[-1]
was broken because it recognizes an indented section label.
Similar code with ('\n' + self.content).split('\n' + section)[-1]
would still be broken because it recognizes an indented section label
in the first line of the file.
The code with section_label_regex
was broken because sometimes it returns the second-to-last section with
the given label, not the last one.
Also, whitespace after the colon was not ignored.
* pygnulib/GLModuleSystem.py (GLModule.__init__): Dissect the module
description's contents immediately, once only, in a reliable way.
(GLModule.getDescription, GLModule.getComment): Simplify.
(GLModule.getStatus): Simplify. Return a string.
(GLModule.getStatuses): New function. Return a list.
(GLModule.getNotice, GLModule.getApplicability, GLModule.getFiles,
GLModule.getDependencies, GLModules.getAutoconfSnippet_Early,
GLModules.getAutoconfSnippet, GLModule.getAutomakeSnippet_Conditional,
GLModule.getInclude, GLModule.getLink, GLModule.getLicense_Raw):
Simplify.
(GLModule.getLicense): Remove whitespace after calling getLicense_Raw.
(GLModule.getMaintainer): Simplify.
(GLModuleTable.transitive_closure): Call getStatuses() instead of
getStatus().
* pygnulib/GLEmiter.py: Likewise.
* gnulib-tool.py (main): For --extract-description, --extract-comment,
--extract-status, --extract-notice, --extract-autoconf-snippet,
--extract-automake-snippet, --extract-include-directive,
--extract-link-directive, --extract-maintainer, don't add an extra
newline after the snippet.
---
ChangeLog | 31 +++
gnulib-tool.py | 19 +-
gnulib-tool.py.TODO | 1 -
pygnulib/GLEmiter.py | 4 +-
pygnulib/GLModuleSystem.py | 387 ++++++++++---------------------------
5 files changed, 140 insertions(+), 302 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 36962c2dd2..305d02be51 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,36 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix section extraction from module descriptions.
+ The code with self.content.split(section)[-1]
+ was broken because it recognizes an indented section label.
+ Similar code with ('\n' + self.content).split('\n' + section)[-1]
+ would still be broken because it recognizes an indented section label
+ in the first line of the file.
+ The code with section_label_regex
+ was broken because sometimes it returns the second-to-last section with
+ the given label, not the last one.
+ Also, whitespace after the colon was not ignored.
+ * pygnulib/GLModuleSystem.py (GLModule.__init__): Dissect the module
+ description's contents immediately, once only, in a reliable way.
+ (GLModule.getDescription, GLModule.getComment): Simplify.
+ (GLModule.getStatus): Simplify. Return a string.
+ (GLModule.getStatuses): New function. Return a list.
+ (GLModule.getNotice, GLModule.getApplicability, GLModule.getFiles,
+ GLModule.getDependencies, GLModules.getAutoconfSnippet_Early,
+ GLModules.getAutoconfSnippet, GLModule.getAutomakeSnippet_Conditional,
+ GLModule.getInclude, GLModule.getLink, GLModule.getLicense_Raw):
+ Simplify.
+ (GLModule.getLicense): Remove whitespace after calling getLicense_Raw.
+ (GLModule.getMaintainer): Simplify.
+ (GLModuleTable.transitive_closure): Call getStatuses() instead of
+ getStatus().
+ * pygnulib/GLEmiter.py: Likewise.
+ * gnulib-tool.py (main): For --extract-description, --extract-comment,
+ --extract-status, --extract-notice, --extract-autoconf-snippet,
+ --extract-automake-snippet, --extract-include-directive,
+ --extract-link-directive, --extract-maintainer, don't add an extra
+ newline after the snippet.
+
gnulib-tool.py: Improve field naming.
* pygnulib/GLModuleSystem.py (GLModule): Rename field 'module' to
'path'. Fix a typo in a TypeError message.
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 032deee6e0..0e888e6fd8 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -974,29 +974,28 @@ def main():
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getDescription())
+ sys.stdout.write(module.getDescription())
elif mode == 'extract-comment':
modulesystem = classes.GLModuleSystem(config)
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getComment())
+ sys.stdout.write(module.getComment())
elif mode == 'extract-status':
modulesystem = classes.GLModuleSystem(config)
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- status = module.getStatus()
- print('\n'.join(status))
+ sys.stdout.write(module.getStatus())
elif mode == 'extract-notice':
modulesystem = classes.GLModuleSystem(config)
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getNotice())
+ sys.stdout.write(module.getNotice())
elif mode == 'extract-applicability':
modulesystem = classes.GLModuleSystem(config)
@@ -1039,28 +1038,28 @@ def main():
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getAutoconfSnippet())
+ sys.stdout.write(module.getAutoconfSnippet())
elif mode == 'extract-automake-snippet':
modulesystem = classes.GLModuleSystem(config)
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getAutomakeSnippet())
+ sys.stdout.write(module.getAutomakeSnippet())
elif mode == 'extract-include-directive':
modulesystem = classes.GLModuleSystem(config)
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getInclude())
+ sys.stdout.write(module.getInclude())
elif mode == 'extract-link-directive':
modulesystem = classes.GLModuleSystem(config)
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getLink())
+ sys.stdout.write(module.getLink())
elif mode == 'extract-license':
modulesystem = classes.GLModuleSystem(config)
@@ -1074,7 +1073,7 @@ def main():
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- print(module.getMaintainer())
+ sys.stdout.write(module.getMaintainer())
elif mode == 'extract-tests-module':
modulesystem = classes.GLModuleSystem(config)
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index a46da5e2ad..9efcda510d 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -37,7 +37,6 @@ Implement the options:
Remove exit() in GLImport.py.
Optimize:
- - GLModuleSystem: Parse each module description only once.
- os.chdir around subprocess creation -> cwd=... argument instead.
--------------------------------------------------------------------------------
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index e0164ed7cc..61cabc92fa 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -974,9 +974,9 @@ AC_DEFUN([%V1%_LIBSOURCES], [
# Skip the contents if it's entirely empty.
if snippet.strip():
# Check status of the module.
- status = module.getStatus()
+ statuses = module.getStatuses()
islongrun = False
- for word in status:
+ for word in statuses:
if word == 'longrunning-test':
islongrun = True
break
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index ec2ff0c35c..32a133009a 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -177,14 +177,11 @@ class GLModule(object):
path. GLModule can get all information about module, get its dependencies,
files, etc.'''
- section_label_regex = '(?:Description:|Comment:|Status:|Notice:|Applicability:|\
-Files:|Depends-on:|configure\\.ac-early:|configure\\.ac:|Makefile\\.am:|\
-Include:|Link:|License:|Maintainer:)'
-
section_label_pattern = \
re.compile('^(Description|Comment|Status|Notice|Applicability|'
+ 'Files|Depends-on|configure\\.ac-early|configure\\.ac|'
- + 'Makefile\\.am|Include|Link|License|Maintainer):$')
+ + 'Makefile\\.am|Include|Link|License|Maintainer):$',
+ re.M)
def __init__(self, config, path, patched=False):
'''GLModule.__init__(config, path[, patched]) -> GLModule
@@ -209,8 +206,20 @@ Include:|Link:|License:|Maintainer:)'
self.config = config
self.filesystem = GLFileSystem(self.config)
self.modulesystem = GLModuleSystem(self.config)
+ # Read the module description file into memory.
with codecs.open(path, 'rb', 'UTF-8') as file:
self.content = file.read().replace('\r\n', '\n')
+ # Dissect it into sections.
+ self.sections = dict()
+ last_section_label = None
+ last_section_start = 0
+ for match in GLModule.section_label_pattern.finditer(self.content):
+ if last_section_label != None:
+ self.sections[last_section_label] = self.content[last_section_start : match.start()]
+ last_section_label = match.group(1)
+ last_section_start = match.end() + 1
+ if last_section_label != None:
+ self.sections[last_section_label] = self.content[last_section_start:]
def __eq__(self, module):
'''x.__eq__(y) <==> x==y'''
@@ -378,117 +387,51 @@ Include:|Link:|License:|Maintainer:)'
'''GLModule.getDescription() -> str
Return description of the module.'''
- section = 'Description:'
- if 'description' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- pattern = '^%s[\t ]*(.*?)%s' % (section, GLModule.section_label_regex)
- pattern = re.compile(pattern, re.S | re.M)
- result = pattern.findall(self.content)
- if type(result) is list:
- if not result:
- result = ''
- else: # if result
- result = result[-1]
- result = result.strip()
- self.cache['description'] = result
- return self.cache['description']
+ return self.sections.get('Description', '')
def getComment(self):
'''GLModule.getComment() -> str
Return comment to module.'''
- section = 'Comment:'
- if 'comment' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- pattern = '^%s[\t ]*(.*?)%s' % (section, GLModule.section_label_regex)
- pattern = re.compile(pattern, re.S | re.M)
- result = pattern.findall(self.content)
- if type(result) is list:
- if not result:
- result = ''
- else: # if result
- result = result[-1]
- result = result.strip()
- self.cache['comment'] = result
- return self.cache['comment']
+ return self.sections.get('Comment', '')
def getStatus(self):
'''GLModule.getStatus() -> str
Return module status.'''
- section = 'Status:'
- if 'status' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = [ part.strip()
- for part in parts
- if part.strip() ]
- self.cache['status'] = list(result)
- return list(self.cache['status'])
+ return self.sections.get('Status', '')
+
+ def getStatuses(self):
+ '''GLModule.getStatuses() -> list
+
+ Return module status.'''
+ if 'statuses' not in self.cache:
+ snippet = self.getStatus()
+ result = [ line.strip()
+ for line in snippet.split('\n')
+ if line.strip() ]
+ self.cache['statuses'] = result
+ return self.cache['statuses']
def getNotice(self):
'''GLModule.getNotice() -> str
Return notice to module.'''
- section = 'Notice:'
- if 'notice' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = ''.join(parts)
- self.cache['notice'] = result
- return self.cache['notice']
+ return self.sections.get('Notice', '')
def getApplicability(self):
'''GLModule.getApplicability() -> str
Return applicability of module.'''
- section = 'Applicability:'
if 'applicability' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- parts = [ part.strip()
- for part in parts ]
- result = ''.join(parts)
- if not result.strip():
- if self.getName().endswith('-tests'):
+ result = self.sections.get('Applicability', '')
+ result = result.strip()
+ if not result:
+ # The default is 'main' or 'tests', depending on the module's name.
+ if self.isTests():
result = 'tests'
- else: # if not self.getName().endswith('-tests')
+ else:
result = 'main'
- result = result.strip()
self.cache['applicability'] = result
return self.cache['applicability']
@@ -497,115 +440,56 @@ Include:|Link:|License:|Maintainer:)'
Return list of files.
GLConfig: ac_version.'''
- ac_version = self.config['ac_version']
- section = 'Files:'
- result = list()
if 'files' not in self.cache:
- if section not in self.content:
- result = list()
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = [ part.strip()
- for part in parts
- if part.strip() ]
- result += [joinpath('m4', '00gnulib.m4')]
- result += [joinpath('m4', 'zzgnulib.m4')]
- result += [joinpath('m4', 'gnulib-common.m4')]
- self.cache['files'] = list(result)
- return list(self.cache['files'])
+ snippet = self.sections.get('Files', '')
+ result = [ line.strip()
+ for line in snippet.split('\n')
+ if line.strip() ]
+ result.append(joinpath('m4', '00gnulib.m4'))
+ result.append(joinpath('m4', 'zzgnulib.m4'))
+ result.append(joinpath('m4', 'gnulib-common.m4'))
+ self.cache['files'] = result
+ return self.cache['files']
def getDependencies(self):
'''GLModule.getDependencies() -> list
Return list of dependencies.
GLConfig: localpath.'''
- result = list()
- section = 'Depends-on:'
if 'dependencies' not in self.cache:
- if section not in self.content:
- depmodules = list()
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- modules = ''.join(parts)
- modules = [ line
- for line in modules.split('\n')
- if line.strip() ]
- modules = [ module
- for module in modules
- if not module.startswith('#') ]
- for line in modules:
- split = [ part
- for part in line.split(' ')
- if part.strip() ]
- if len(split) == 1:
- module = line.strip()
- condition = None
- else: # if len(split) != 1
- module = split[0]
- condition = split[1]
- result += [tuple([self.modulesystem.find(module), condition])]
+ snippet = self.sections.get('Depends-on', '')
+ modules = [ line.strip()
+ for line in snippet.split('\n')
+ if line.strip() ]
+ modules = [ module
+ for module in modules
+ if not module.startswith('#') ]
+ result = list()
+ for line in modules:
+ split = [ part
+ for part in line.split(' ')
+ if part.strip() ]
+ if len(split) == 1:
+ module = line.strip()
+ condition = None
+ else: # if len(split) != 1
+ module = split[0]
+ condition = split[1]
+ result += [tuple([self.modulesystem.find(module), condition])]
self.cache['dependencies'] = result
- return list(self.cache['dependencies'])
+ return self.cache['dependencies']
def getAutoconfSnippet_Early(self):
'''GLModule.getAutoconfSnippet_Early() -> str
Return autoconf-early snippet.'''
- section = 'configure.ac-early:'
- if 'autoconf-early' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = ''.join(parts)
- self.cache['autoconf-early'] = result
- return self.cache['autoconf-early']
+ return self.sections.get('configure.ac-early', '')
def getAutoconfSnippet(self):
'''GLModule.getAutoconfSnippet() -> str
Return autoconf snippet.'''
- section = 'configure.ac:'
- if 'autoconf' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = ''.join(parts)
- self.cache['autoconf'] = result
- return self.cache['autoconf']
+ return self.sections.get('configure.ac', '')
def getAutomakeSnippet(self):
'''getAutomakeSnippet() -> str
@@ -625,23 +509,7 @@ Include:|Link:|License:|Maintainer:)'
'''GLModule.getAutomakeSnippet_Conditional() -> str
Return conditional automake snippet.'''
- section = 'Makefile.am:'
- if 'makefile-conditional' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = ''.join(parts)
- self.cache['makefile-conditional'] = result
- return self.cache['makefile-conditional']
+ return self.sections.get('Makefile.am', '')
def getAutomakeSnippet_Unconditional(self):
'''GLModule.getAutomakeSnippet_Unconditional() -> str
@@ -718,24 +586,10 @@ Include:|Link:|License:|Maintainer:)'
'''GLModule.getInclude() -> str
Return include directive.'''
- section = 'Include:'
if 'include' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = ''.join(parts)
- result = result.strip()
- pattern = re.compile('^(["<].*[>"])', re.M)
- result = pattern.sub('#include \\1', result)
+ snippet = self.sections.get('Include', '')
+ pattern = re.compile('^(["<])', re.M)
+ result = pattern.sub('#include \\1', snippet)
self.cache['include'] = result
return self.cache['include']
@@ -743,64 +597,36 @@ Include:|Link:|License:|Maintainer:)'
'''GLModule.getLink() -> str
Return link directive.'''
- section = 'Link:'
- if 'link' not in self.cache:
- parts = list()
- if section in self.content:
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- parts = [ part.strip()
- for part in parts
- if part.strip() ]
- # result = ' '.join(parts)
- self.cache['link'] = parts
- return self.cache['link']
-
- def getLicense(self):
- '''GLModule.getLicense(self) -> str
-
- Get license and warn user if module lacks a license.'''
- if str(self) == 'parse-datetime':
- # This module is under a weaker license only for the purpose of some
- # users who hand-edit it and don't use gnulib-tool. For the regular
- # gnulib users they are under a stricter license.
- return 'GPL'
- else:
- license = self.getLicense_Raw()
- if not self.isTests():
- if not license:
- if self.config['errors']:
- raise GLError(18, str(self))
- else: # if not self.config['errors']
- sys.stderr.write('gnulib-tool: warning: module %s lacks a license\n' % str(self))
- if not license:
- license = 'GPL'
- return license
+ return self.sections.get('Link', '')
def getLicense_Raw(self):
'''GLModule.getLicense_Raw() -> str
Return module license.'''
- section = 'License:'
+ return self.sections.get('License', '')
+
+ def getLicense(self):
+ '''GLModule.getLicense(self) -> str
+
+ Get license and warn user if module lacks a license.'''
if 'license' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- pattern = '^%s[\t ]*(.*?)%s' % (section, GLModule.section_label_regex)
- pattern = re.compile(pattern, re.S | re.M)
- result = pattern.findall(self.content)
- if type(result) is list:
- if not result:
- result = ''
- else: # if result
- result = result[-1]
- result = result.strip()
+ result = None
+ if str(self) == 'parse-datetime':
+ # This module is under a weaker license only for the purpose of some
+ # users who hand-edit it and don't use gnulib-tool. For the regular
+ # gnulib users they are under a stricter license.
+ result = 'GPL'
+ else:
+ license = self.getLicense_Raw().strip()
+ if not self.isTests():
+ if not license:
+ if self.config['errors']:
+ raise GLError(18, str(self))
+ else: # if not self.config['errors']
+ sys.stderr.write('gnulib-tool: warning: module %s lacks a license\n' % str(self))
+ if not license:
+ license = 'GPL'
+ result = license
self.cache['license'] = result
return self.cache['license']
@@ -808,24 +634,7 @@ Include:|Link:|License:|Maintainer:)'
'''GLModule.getMaintainer() -> str
Return maintainer directive.'''
- section = 'Maintainer:'
- if 'maintainer' not in self.cache:
- if section not in self.content:
- result = ''
- else: # if section in self.content
- snippet = self.content.split(section)[-1]
- lines = [ '%s\n' % line
- for line in snippet.split('\n') ]
- parts = list()
- for line in lines:
- findflag = GLModule.section_label_pattern.findall(line)
- if findflag:
- break
- parts += [line]
- result = ''.join(parts)
- result = result.strip()
- self.cache['maintainer'] = result
- return self.cache['maintainer']
+ return self.sections.get('Maintainer', '')
#===============================================================================
@@ -1002,8 +811,8 @@ class GLModuleTable(object):
conditions += [None]
for depmodule in depmodules:
include = True
- status = depmodule.getStatus()
- for word in status:
+ statuses = depmodule.getStatuses()
+ for word in statuses:
if word == 'obsolete':
if not self.config.checkInclTestCategory(TESTS['obsolete']):
include = False
--
2.34.1
>From d528738ac3449ce1c4897882e6032eb3a3e929e2 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Sun, 7 Aug 2022 22:59:08 +0200
Subject: [PATCH 16/19] gnulib-tool.py: Rename a method.
* pygnulib/GLModuleSystem.py (GLModule.getAutoconfEarlySnippet): Renamed
from GLModule.getAutoconfSnippet_Early.
* pygnulib/GLImport.py: Update.
* pygnulib/GLTestDir.py: Likewise.
---
ChangeLog | 6 ++++++
pygnulib/GLImport.py | 2 +-
pygnulib/GLModuleSystem.py | 4 ++--
pygnulib/GLTestDir.py | 4 ++--
4 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 305d02be51..1278d8e4d9 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,11 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Rename a method.
+ * pygnulib/GLModuleSystem.py (GLModule.getAutoconfEarlySnippet): Renamed
+ from GLModule.getAutoconfSnippet_Early.
+ * pygnulib/GLImport.py: Update.
+ * pygnulib/GLTestDir.py: Likewise.
+
gnulib-tool.py: Fix section extraction from module descriptions.
The code with self.content.split(section)[-1]
was broken because it recognizes an indented section label.
diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index 818f3d57c0..87d85c5181 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -606,7 +606,7 @@ AC_DEFUN([%s_EARLY],
emit += ' AC_REQUIRE([AM_PROG_CC_C_O])\n'
for module in moduletable['final']:
emit += ' # Code from module %s:\n' % str(module)
- snippet = module.getAutoconfSnippet_Early()
+ snippet = module.getAutoconfEarlySnippet()
lines = [ line
for line in snippet.split(constants.NL)
if line != '' ]
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 32a133009a..14bc089b27 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -479,8 +479,8 @@ class GLModule(object):
self.cache['dependencies'] = result
return self.cache['dependencies']
- def getAutoconfSnippet_Early(self):
- '''GLModule.getAutoconfSnippet_Early() -> str
+ def getAutoconfEarlySnippet(self):
+ '''GLModule.getAutoconfEarlySnippet() -> str
Return autoconf-early snippet.'''
return self.sections.get('configure.ac-early', '')
diff --git a/pygnulib/GLTestDir.py b/pygnulib/GLTestDir.py
index 9a3fde66ea..746b815b49 100644
--- a/pygnulib/GLTestDir.py
+++ b/pygnulib/GLTestDir.py
@@ -456,7 +456,7 @@ class GLTestDir(object):
pass
# if str(module) not in ['gnumakefile', 'maintainer-makefile']
else:
- snippet = module.getAutoconfSnippet_Early()
+ snippet = module.getAutoconfEarlySnippet()
lines = [ line
for line in snippet.split('\n')
if line.strip() ]
@@ -573,7 +573,7 @@ class GLTestDir(object):
else: # if not single_configure
solution = module.isNonTests()
if solution:
- snippet = module.getAutoconfSnippet_Early()
+ snippet = module.getAutoconfEarlySnippet()
lines = [ line
for line in snippet.split('\n')
if line.strip() ]
--
2.34.1
>From 276c20ee288873b519be96ffe9b3a98bfe3bb8f1 Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Mon, 8 Aug 2022 00:02:59 +0200
Subject: [PATCH 17/19] gnulib-tool.py: Fix --extract-dependencies result.
* pygnulib/GLModuleSystem.py (GLModule.getDependencies): Return a
snippet, not a list. Implement dependency of ${module}-tests on
${module}.
(GLModule.getDependenciesWithoutConditions,
GLModule.getDependenciesWithConditions): New methods.
(GLModuleTable.transitive_closure): Call getDependenciesWithConditions.
* pygnulib/GLEmiter.py (GLEmiter.autoconfSnippets): Call
getDependenciesWithoutConditions.
* gnulib-tool.py (main) [--extract-dependencies]: Update.
---
ChangeLog | 11 +++++
gnulib-tool.py | 10 +----
pygnulib/GLEmiter.py | 4 +-
pygnulib/GLModuleSystem.py | 83 ++++++++++++++++++++++++++++----------
4 files changed, 75 insertions(+), 33 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 1278d8e4d9..beb60578d0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,16 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix --extract-dependencies result.
+ * pygnulib/GLModuleSystem.py (GLModule.getDependencies): Return a
+ snippet, not a list. Implement dependency of ${module}-tests on
+ ${module}.
+ (GLModule.getDependenciesWithoutConditions,
+ GLModule.getDependenciesWithConditions): New methods.
+ (GLModuleTable.transitive_closure): Call getDependenciesWithConditions.
+ * pygnulib/GLEmiter.py (GLEmiter.autoconfSnippets): Call
+ getDependenciesWithoutConditions.
+ * gnulib-tool.py (main) [--extract-dependencies]: Update.
+
gnulib-tool.py: Rename a method.
* pygnulib/GLModuleSystem.py (GLModule.getAutoconfEarlySnippet): Renamed
from GLModule.getAutoconfSnippet_Early.
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 0e888e6fd8..0cd5accd21 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -1013,7 +1013,6 @@ def main():
print('\n'.join(files))
elif mode == 'extract-dependencies':
- result = ''
if avoids:
message = '%s: *** ' % constants.APP['name']
message += 'cannot combine --avoid and --extract-dependencies\n'
@@ -1024,14 +1023,7 @@ def main():
modules = [ modulesystem.find(module)
for module in modules ]
for module in modules:
- dependencies = module.getDependencies()
- if dependencies:
- for depmodule, condition in dependencies:
- if condition == None:
- result += '%s\n' % str(depmodule)
- else: # if condition != None
- result += '%s\t%s' % (str(depmodule), condition)
- print(result)
+ sys.stdout.write(module.getDependencies())
elif mode == 'extract-autoconf-snippet':
modulesystem = classes.GLModuleSystem(config)
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index 61cabc92fa..a02b44bdd5 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -309,9 +309,7 @@ class GLEmiter(object):
emit += self.autoconfSnippet(module, fileassistant, toplevel,
disable_libtool, disable_gettext, replace_auxdir, ' ')
emit += ' %s=true\n' % shellvar
- dependencies = module.getDependencies()
- depmodules = [ pair[0]
- for pair in dependencies ]
+ depmodules = module.getDependenciesWithoutConditions()
# Intersect dependencies with the modules list.
depmodules = [ dep
for dep in depmodules
diff --git a/pygnulib/GLModuleSystem.py b/pygnulib/GLModuleSystem.py
index 14bc089b27..19d13d213b 100644
--- a/pygnulib/GLModuleSystem.py
+++ b/pygnulib/GLModuleSystem.py
@@ -452,33 +452,74 @@ class GLModule(object):
return self.cache['files']
def getDependencies(self):
- '''GLModule.getDependencies() -> list
+ '''GLModule.getDependencies() -> str
- Return list of dependencies.
+ Return list of dependencies, as a snippet.
GLConfig: localpath.'''
if 'dependencies' not in self.cache:
+ result = ''
+ # ${module}-tests implicitly depends on ${module}, if that module exists.
+ if self.isTests():
+ main_module = subend('-tests', '', self.getName())
+ if self.modulesystem.exists(main_module):
+ result += '%s\n' % main_module
+ # Then the explicit dependencies listed in the module description.
snippet = self.sections.get('Depends-on', '')
- modules = [ line.strip()
- for line in snippet.split('\n')
- if line.strip() ]
- modules = [ module
- for module in modules
- if not module.startswith('#') ]
- result = list()
- for line in modules:
- split = [ part
- for part in line.split(' ')
- if part.strip() ]
- if len(split) == 1:
- module = line.strip()
- condition = None
- else: # if len(split) != 1
- module = split[0]
- condition = split[1]
- result += [tuple([self.modulesystem.find(module), condition])]
+ # Remove comment lines.
+ snippet = re.compile('^#.*$[\n]', re.M).sub('', snippet)
+ result += snippet
self.cache['dependencies'] = result
return self.cache['dependencies']
+ def getDependenciesWithoutConditions(self):
+ '''GLModule.getDependenciesWithoutConditions() -> list
+
+ Return list of dependencies, as a list of GLModule objects.
+ GLConfig: localpath.'''
+ if 'dependenciesWithoutCond' not in self.cache:
+ snippet = self.getDependencies()
+ lines = [ line.strip()
+ for line in snippet.split('\n')
+ if line.strip() ]
+ pattern = re.compile(' *\\[.*$')
+ lines = [ pattern.sub('', line)
+ for line in lines ]
+ result = [ self.modulesystem.find(module)
+ for module in lines
+ if module != '' ]
+ self.cache['dependenciesWithoutCond'] = result
+ return self.cache['dependenciesWithoutCond']
+
+ def getDependenciesWithConditions(self):
+ '''GLModule.getDependenciesWithConditions() -> list
+
+ Return list of dependencies, as a list of pairs (GLModule object, condition).
+ The "true" condition is denoted by None.
+ GLConfig: localpath.'''
+
+ if 'dependenciesWithCond' not in self.cache:
+ snippet = self.getDependencies()
+ lines = [ line.strip()
+ for line in snippet.split('\n')
+ if line.strip() ]
+ pattern = re.compile(' *\\[')
+ result = []
+ for line in lines:
+ match = pattern.search(line)
+ if match:
+ module = line[0 : match.start()]
+ condition = line[match.end() :]
+ condition = subend(']', '', condition)
+ else:
+ module = line
+ condition = None
+ if module != '':
+ if condition == 'true':
+ condition = None
+ result.append(tuple([self.modulesystem.find(module), condition]))
+ self.cache['dependenciesWithCond'] = result
+ return self.cache['dependenciesWithCond']
+
def getAutoconfEarlySnippet(self):
'''GLModule.getAutoconfEarlySnippet() -> str
@@ -798,7 +839,7 @@ class GLModuleTable(object):
if not pattern.findall(automake_snippet):
self.addUnconditional(module)
conditional = self.isConditional(module)
- dependencies = module.getDependencies()
+ dependencies = module.getDependenciesWithConditions()
depmodules = [ pair[0]
for pair in dependencies ]
conditions = [ pair[1]
--
2.34.1
>From 6d29a15180d7ba0360d8fb56793a0766926b0dee Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Mon, 8 Aug 2022 00:43:25 +0200
Subject: [PATCH 18/19] gnulib-tool.py: Fix handling of nonexistent module
names in --extract-*.
* gnulib-tool.py (main): To test whether a module exists, just call
GLModuleSystem.find and test its return value.
---
ChangeLog | 4 ++
gnulib-tool.py | 133 ++++++++++++++++++++++++-------------------------
2 files changed, 68 insertions(+), 69 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index beb60578d0..f3a37cea1c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,9 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Fix handling of nonexistent module names in --extract-*.
+ * gnulib-tool.py (main): To test whether a module exists, just call
+ GLModuleSystem.find and test its return value.
+
gnulib-tool.py: Fix --extract-dependencies result.
* pygnulib/GLModuleSystem.py (GLModule.getDependencies): Return a
snippet, not a list. Implement dependency of ${module}-tests on
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 0cd5accd21..0abe8c5ca7 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -47,7 +47,6 @@ import shlex
from tempfile import mktemp
from pygnulib import constants
from pygnulib import classes
-from pygnulib import GLError
#===============================================================================
@@ -716,8 +715,6 @@ def main():
print(result)
elif mode == 'find':
- # Prepare GLModuleSystem.find to throw an exception.
- config.setErrors(True)
modulesystem = classes.GLModuleSystem(config)
for filename in files:
if (isfile(joinpath(DIRS['root'], filename))
@@ -754,14 +751,12 @@ def main():
listing = [ line
for line in listing
if modulesystem.file_is_module(line) ]
- module_candidates = sorted(set(listing))
- for module in module_candidates:
- try:
- if filename in modulesystem.find(module).getFiles():
- print(module)
- except GLError:
- # Ignore module candidates that don't actually exist.
- pass
+ candidates = sorted(set(listing))
+ for name in candidates:
+ module = modulesystem.find(name)
+ if module: # Ignore module candidates that don't actually exist.
+ if module.getFiles():
+ print(name)
else:
message = '%s: warning: file %s does not exist\n' % (constants.APP['name'], filename)
sys.stderr.write(message)
@@ -971,46 +966,46 @@ def main():
elif mode == 'extract-description':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getDescription())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getDescription())
elif mode == 'extract-comment':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getComment())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getComment())
elif mode == 'extract-status':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getStatus())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getStatus())
elif mode == 'extract-notice':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getNotice())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getNotice())
elif mode == 'extract-applicability':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- print(module.getApplicability())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ print(module.getApplicability())
elif mode == 'extract-filelist':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- files = module.getFiles()
- print('\n'.join(files))
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ files = module.getFiles()
+ print('\n'.join(files))
elif mode == 'extract-dependencies':
if avoids:
@@ -1020,60 +1015,60 @@ def main():
sys.stderr.write(message)
sys.exit(1)
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getDependencies())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getDependencies())
elif mode == 'extract-autoconf-snippet':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getAutoconfSnippet())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getAutoconfSnippet())
elif mode == 'extract-automake-snippet':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getAutomakeSnippet())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getAutomakeSnippet())
elif mode == 'extract-include-directive':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getInclude())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getInclude())
elif mode == 'extract-link-directive':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getLink())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getLink())
elif mode == 'extract-license':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- print(module.getLicense())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ print(module.getLicense())
elif mode == 'extract-maintainer':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- sys.stdout.write(module.getMaintainer())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ sys.stdout.write(module.getMaintainer())
elif mode == 'extract-tests-module':
modulesystem = classes.GLModuleSystem(config)
- modules = [ modulesystem.find(module)
- for module in modules ]
- for module in modules:
- if module.getTestsModule():
- print(module.getTestsName())
+ for name in modules:
+ module = modulesystem.find(name)
+ if module:
+ if module.getTestsModule():
+ print(module.getTestsName())
elif mode == 'copy-file':
srcpath = files[0]
--
2.34.1
>From a7bcb91088e7fa6a533528aec076fc536789b96f Mon Sep 17 00:00:00 2001
From: Bruno Haible <br...@clisp.org>
Date: Mon, 8 Aug 2022 00:46:13 +0200
Subject: [PATCH 19/19] gnulib-tool.py: Finish implementing option
--extract-test-module.
* gnulib-tool.py (main): Accept option --extract-tests-module.
---
ChangeLog | 3 +++
gnulib-tool.py | 8 ++++++++
gnulib-tool.py.TODO | 1 -
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index f3a37cea1c..63691ebfb6 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,8 @@
2022-08-07 Bruno Haible <br...@clisp.org>
+ gnulib-tool.py: Finish implementing option --extract-test-module.
+ * gnulib-tool.py (main): Accept option --extract-tests-module.
+
gnulib-tool.py: Fix handling of nonexistent module names in --extract-*.
* gnulib-tool.py (main): To test whether a module exists, just call
GLModuleSystem.find and test its return value.
diff --git a/gnulib-tool.py b/gnulib-tool.py
index 0abe8c5ca7..9988c84de9 100755
--- a/gnulib-tool.py
+++ b/gnulib-tool.py
@@ -180,6 +180,10 @@ def main():
dest='mode_xmaintainer',
default=None,
action='store_true')
+ parser.add_argument('--extract-tests-module',
+ dest='mode_xtests',
+ default=None,
+ action='store_true')
# copy-file
parser.add_argument('--copy-file',
dest='mode_copy_file',
@@ -446,6 +450,7 @@ def main():
cmdargs.mode_xlink,
cmdargs.mode_xlicense,
cmdargs.mode_xmaintainer,
+ cmdargs.mode_xtests,
cmdargs.mode_copy_file,
]
overflow = [ arg
@@ -535,6 +540,9 @@ def main():
if cmdargs.mode_xmaintainer != None:
mode = 'extract-maintainer'
modules = list(cmdargs.non_option_arguments)
+ if cmdargs.mode_xtests != None:
+ mode = 'extract-tests-module'
+ modules = list(cmdargs.non_option_arguments)
if cmdargs.mode_copy_file != None:
mode = 'copy-file'
if len(cmdargs.non_option_arguments) < 1 or len(cmdargs.non_option_arguments) > 2:
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index 9efcda510d..be6a2e2b10 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -22,7 +22,6 @@ Inline all 'sed' invocations.
Implement the options:
--extract-recursive-dependencies
--extract-recursive-link-directive
- --extract-tests-module
--conditional-dependencies
--no-conditional-dependencies
--gnu-make
--
2.34.1
