Hi Bruno,
I have attached two updated patches for 0003 and 0004.
On 3/10/24 6:18 AM, Bruno Haible wrote:
> In the function _eliminate_NMD, the list accumulation expressions are a bit
> complex.
Sorry, fixed. I am a big fan of Python's list comprehensions [1]. :)
I will concede that they can get confusing though. I usually get
confused by nested lists '[num for elem in vec for num in elem]'.
> - A function _eliminate_NMD_from_line that takes a single line as argument
> and return a line or None.
> - A function _eliminate_NMD that invokes _eliminate_NMD_from_line on each
> line of the snippet, and combines the results?
I tried to implement it as you described it here. Feel free to let me
know if I missed anything.
> Also, please don't drop comments that are present in the original code, such
> as:
>
> # Replace NMD, so as to remove redundant "$(MKDIR_P) '.'" invocations.
> # The logic is similar to how we define gl_source_base_prefix.
These comments were above the assignments to 'sed_eliminate_NMD' in
the shell script. I've decided to place them in the body of
GLEmiter.lib_Makefile_am and GLEmiter.tests_Makefile_am above calls to
_eliminate_NMD. This would mean that grep'ing for the comment would
take you to a similar place in the code. I also added separate
docstrings to each new function.
I tested these patches by creating a testdir with the shell and python
script and then importing a few modules and tests, then diffing them.
The output seems correct (as far as these patches go) to me with
--automake-subdir enabled and disabled.
I have also been adding type hints to new functions. Many docstrings
have similar information like so:
def autoconfSnippets(self, modules, referenceable_modules, moduletable,
verifier, toplevel, disable_libtool, disable_gettext,
replace_auxdir):
'''GLEmiter.autoconfSnippets(modules,
verifier, toplevel, disable_libtool, disable_gettext, replace_auxdir)
-> str
Because _eliminate_NMD_from_line can return a string or None, I feel
it would be misleading to *not* use the Union type hint like so:
def _eliminate_NMD_from_line(line: str, automake_subdir: bool) -> str |
None:
Since we have discussed the type hints before and decided we can
assume Python version >= 3.7, I felt it was okay to add with this
patch.
Python version 3.7 requires 'from __future__ import annotations'
before this syntax is used [2]. I have confirmed this by running
./bootstrap && ./configure && make all && make check in Coreutils with
both Python version 3.7 and 3.12.
[1] https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions
[2] https://docs.python.org/3/library/__future__.html
CollinFrom 49596fc1f103f5bdff2529c3506c6eb783f4de4d Mon Sep 17 00:00:00 2001
From: Collin Funk <[email protected]>
Date: Mon, 11 Mar 2024 00:06:18 -0700
Subject: [PATCH 3/4] gnulib-tool.py: Follow gnulib-tool changes, part 53.
Follow gnulib-tool change
2021-12-21 Bruno Haible <[email protected]>
Optimize redundant 'mkdir -p .' invocations.
* pygnulib/GLEmiter.py (_eliminate_NMD_from_line): New private function
to remove @NMD@ occurrences from the line of an Automake snippet.
(_eliminate_NMD): New private function which uses
_eliminate_NMD_from_line on each line of the Automake snippet.
(GLEmiter.lib_Makefile_am, GLEmiter.tests_Makefile_am): Use
_eliminate_NMD on the Automake snippet.
---
ChangeLog | 13 +++++++
gnulib-tool.py.TODO | 81 --------------------------------------------
pygnulib/GLEmiter.py | 29 ++++++++++++++++
3 files changed, 42 insertions(+), 81 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 34323c66e8..864a036916 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2024-03-11 Collin Funk <[email protected]>
+
+ gnulib-tool.py: Follow gnulib-tool changes, part 53.
+ Follow gnulib-tool change
+ 2021-12-21 Bruno Haible <[email protected]>
+ Optimize redundant 'mkdir -p .' invocations.
+ * pygnulib/GLEmiter.py (_eliminate_NMD_from_line): New private function
+ to remove @NMD@ occurrences from the line of an Automake snippet.
+ (_eliminate_NMD): New private function which uses
+ _eliminate_NMD_from_line on each line of the Automake snippet.
+ (GLEmiter.lib_Makefile_am, GLEmiter.tests_Makefile_am): Use
+ _eliminate_NMD on the Automake snippet.
+
2024-03-10 Collin Funk <[email protected]>
gnulib-tool.py: Follow gnulib-tool changes, part 52.
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index e6c94c8bb8..c3d5d65f5c 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -207,87 +207,6 @@ Date: Fri Dec 24 17:22:00 2021 -0800
--------------------------------------------------------------------------------
-commit 83a0d5891db50c00e283e70df694b6003b1f485d
-Author: Bruno Haible <[email protected]>
-Date: Tue Dec 21 09:59:17 2021 +0100
-
- Optimize redundant 'mkdir -p .' invocations.
-
- Reported by Paul Eggert.
-
- * gnulib-tool (func_emit_lib_Makefile_am, func_emit_tests_Makefile_am):
- Eliminate @NMD@ tokens.
- * modules/alloca-opt (Makefile.am): Prefix the "$(MKDIR_P) '%reldir%'"
- line with @NMD@.
- * modules/argz (Makefile.am): Likewise.
- * modules/assert-h (Makefile.am): Likewise.
- * modules/byteswap (Makefile.am): Likewise.
- * modules/configmake (Makefile.am): Likewise.
- * modules/ctype (Makefile.am): Likewise.
- * modules/dirent (Makefile.am): Likewise.
- * modules/errno (Makefile.am): Likewise.
- * modules/execinfo (Makefile.am): Likewise.
- * modules/fcntl-h (Makefile.am): Likewise.
- * modules/float (Makefile.am): Likewise.
- * modules/fnmatch-h (Makefile.am): Likewise.
- * modules/getopt-posix (Makefile.am): Likewise.
- * modules/glob-h (Makefile.am): Likewise.
- * modules/iconv-h (Makefile.am): Likewise.
- * modules/ieee754-h (Makefile.am): Likewise.
- * modules/inttypes-incomplete (Makefile.am): Likewise.
- * modules/langinfo (Makefile.am): Likewise.
- * modules/libtextstyle-optional (Makefile.am): Likewise.
- * modules/limits-h (Makefile.am): Likewise.
- * modules/locale (Makefile.am): Likewise.
- * modules/malloc-h (Makefile.am): Likewise.
- * modules/math (Makefile.am): Likewise.
- * modules/monetary (Makefile.am): Likewise.
- * modules/netdb (Makefile.am): Likewise.
- * modules/openmp-init (Makefile.am): Likewise.
- * modules/poll-h (Makefile.am): Likewise.
- * modules/posix-shell (Makefile.am): Likewise.
- * modules/pthread-h (Makefile.am): Likewise.
- * modules/pty (Makefile.am): Likewise.
- * modules/sched (Makefile.am): Likewise.
- * modules/search (Makefile.am): Likewise.
- * modules/signal-h (Makefile.am): Likewise.
- * modules/sigsegv (Makefile.am): Likewise.
- * modules/snippet/link-warning (Makefile.am): Likewise.
- * modules/spawn (Makefile.am): Likewise.
- * modules/stdalign (Makefile.am): Likewise.
- * modules/stdarg (Makefile.am): Likewise.
- * modules/stdbool (Makefile.am): Likewise.
- * modules/stddef (Makefile.am): Likewise.
- * modules/stdint (Makefile.am): Likewise.
- * modules/stdio (Makefile.am): Likewise.
- * modules/stdlib (Makefile.am): Likewise.
- * modules/stdnoreturn (Makefile.am): Likewise.
- * modules/string (Makefile.am): Likewise.
- * modules/strings (Makefile.am): Likewise.
- * modules/sysexits (Makefile.am): Likewise.
- * modules/termios (Makefile.am): Likewise.
- * modules/threads-h (Makefile.am): Likewise.
- * modules/time (Makefile.am): Likewise.
- * modules/uchar (Makefile.am): Likewise.
- * modules/unicase/base (Makefile.am): Likewise.
- * modules/uniconv/base (Makefile.am): Likewise.
- * modules/unictype/base (Makefile.am): Likewise.
- * modules/unigbrk/base (Makefile.am): Likewise.
- * modules/unilbrk/base (Makefile.am): Likewise.
- * modules/uniname/base (Makefile.am): Likewise.
- * modules/uninorm/base (Makefile.am): Likewise.
- * modules/unistd (Makefile.am): Likewise.
- * modules/unistdio/base (Makefile.am): Likewise.
- * modules/unistr/base (Makefile.am): Likewise.
- * modules/unitypes (Makefile.am): Likewise.
- * modules/uniwbrk/base (Makefile.am): Likewise.
- * modules/uniwidth/base (Makefile.am): Likewise.
- * modules/utime-h (Makefile.am): Likewise.
- * modules/wchar (Makefile.am): Likewise.
- * modules/wctype-h (Makefile.am): Likewise.
-
---------------------------------------------------------------------------------
-
commit 4bdc327dbda59dcdbfa0f983a4f35c4a4ec3578c
Author: Bruno Haible <[email protected]>
Date: Sun Dec 19 12:49:16 2021 +0100
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index a904aee8ad..f0ae4627e6 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -13,6 +13,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
+from __future__ import annotations
#===============================================================================
# Define global imports
#===============================================================================
@@ -65,6 +66,27 @@ def _convert_to_gnu_make(snippet: str) -> str:
return snippet
+def _eliminate_NMD_from_line(line: str, automake_subdir: bool) -> str | None:
+ '''Eliminate occurrences of @NMD@ from the given line. The modified
+ line is returned or None if the line should be removed.'''
+ if line.startswith('@NMD@'):
+ if automake_subdir:
+ line = line.replace('@NMD@', '')
+ else:
+ line = None
+ return line
+
+
+def _eliminate_NMD(snippet: str, automake_subdir: bool) -> str:
+ '''Return the Automake snippet with occurrences of @NMD@ removed.'''
+ result = []
+ for line in snippet.splitlines():
+ line = _eliminate_NMD_from_line(line, automake_subdir)
+ if line != None:
+ result.append(line)
+ return '\n'.join(result) + '\n'
+
+
#===============================================================================
# Define GLEmiter class
#===============================================================================
@@ -694,6 +716,7 @@ AC_DEFUN([%V1%_LIBSOURCES], [
auxdir = self.config['auxdir']
gnu_make = self.config['gnu_make']
makefile_name = self.config['makefile_name']
+ automake_subdir = self.config['automake_subdir']
libtool = self.config['libtool']
macro_prefix = self.config['macro_prefix']
podomain = self.config['podomain']
@@ -747,6 +770,9 @@ AC_DEFUN([%V1%_LIBSOURCES], [
if eliminate_LDFLAGS:
pattern = re.compile('^(lib_LDFLAGS[\t ]*\\+=.*$\n)', re.M)
amsnippet1 = pattern.sub('', amsnippet1)
+ # Replace NMD, so as to remove redundant "$(MKDIR_P) '.'" invocations.
+ # The logic is similar to how we define gl_source_base_prefix.
+ amsnippet1 = _eliminate_NMD(amsnippet1, automake_subdir)
pattern = re.compile('lib_([A-Z][A-Z]*)', re.M)
amsnippet1 = pattern.sub('%s_%s_\\1' % (libname, libext),
amsnippet1)
@@ -1047,6 +1073,9 @@ AC_DEFUN([%V1%_LIBSOURCES], [
if eliminate_LDFLAGS:
pattern = re.compile('^(lib_LDFLAGS[\t ]*\\+=.*$\n)', re.M)
amsnippet1 = pattern.sub('', amsnippet1)
+ # Replace NMD, so as to remove redundant "$(MKDIR_P) '.'" invocations.
+ # The logic is similar to how we define gl_source_base_prefix.
+ amsnippet1 = _eliminate_NMD(amsnippet1, False)
pattern = re.compile('lib_([A-Z][A-Z]*)', re.M)
amsnippet1 = pattern.sub('libtests_a_\\1', amsnippet1)
amsnippet1 = amsnippet1.replace('$(GNULIB_', '$(' + module_indicator_prefix + '_GNULIB_')
--
2.44.0
From 8389eae1678bd870c3199718d8d24202539402cc Mon Sep 17 00:00:00 2001
From: Collin Funk <[email protected]>
Date: Mon, 11 Mar 2024 00:42:57 -0700
Subject: [PATCH 4/4] gnulib-tool.py: Follow gnulib-tool changes, part 54.
Follow gnulib-tool change
2021-12-24 Paul Eggert <[email protected]>
maint: avoid empty lines in recipes
* pygnulib/GLEmiter.py (_eliminate_NMD_from_line): Eliminate occurrences
of @!NMD@ too. Document parameters in docstring.
(_eliminate_NMD): Update docstring to reflect changes. Document
parameters.
---
ChangeLog | 11 +++++++++++
gnulib-tool.py.TODO | 38 --------------------------------------
pygnulib/GLEmiter.py | 32 ++++++++++++++++++++++++--------
3 files changed, 35 insertions(+), 46 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 864a036916..b335768a16 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2024-03-11 Collin Funk <[email protected]>
+
+ gnulib-tool.py: Follow gnulib-tool changes, part 54.
+ Follow gnulib-tool change
+ 2021-12-24 Paul Eggert <[email protected]>
+ maint: avoid empty lines in recipes
+ * pygnulib/GLEmiter.py (_eliminate_NMD_from_line): Eliminate occurrences
+ of @!NMD@ too. Document parameters in docstring.
+ (_eliminate_NMD): Update docstring to reflect changes. Document
+ parameters.
+
2024-03-11 Collin Funk <[email protected]>
gnulib-tool.py: Follow gnulib-tool changes, part 53.
diff --git a/gnulib-tool.py.TODO b/gnulib-tool.py.TODO
index c3d5d65f5c..69d3cf7d72 100644
--- a/gnulib-tool.py.TODO
+++ b/gnulib-tool.py.TODO
@@ -169,44 +169,6 @@ Date: Sat Dec 25 12:19:13 2021 +0100
--------------------------------------------------------------------------------
-commit b4c57b4240992832fa89a02dc620c4fa5ac36973
-Author: Paul Eggert <[email protected]>
-Date: Fri Dec 24 17:22:00 2021 -0800
-
- maint: avoid empty lines in recipes
-
- AIX 7.2 ‘make’ complains about recipe lines that are empty after
- macro expansion, and I suppose there’s a good chance some
- non-POSIX ‘make’ would prohibit them. Rework macros so
- that we can avoid them.
- * gnulib-tool (func_emit_lib_Makefile_am)
- (func_emit_lib_Makefile_am): Support @!NMD@ too.
- * modules/gen-header (@gl_V_at): New macro.
- * modules/alloca-opt, modules/argz, modules/assert-h:
- * modules/byteswap, modules/configmake, modules/ctype:
- * modules/dirent, modules/errno, modules/execinfo, modules/fcntl-h:
- * modules/float, modules/fnmatch-h, modules/getopt-posix:
- * modules/glob-h, modules/iconv-h, modules/ieee754-h:
- * modules/inttypes-incomplete, modules/langinfo:
- * modules/libtextstyle-optional, modules/limits-h, modules/locale:
- * modules/malloc-h, modules/math, modules/monetary, modules/netdb:
- * modules/openmp-init, modules/poll-h, modules/posix-shell:
- * modules/pthread-h, modules/pty, modules/sched, modules/search:
- * modules/signal-h, modules/sigsegv, modules/snippet/link-warning:
- * modules/spawn, modules/stdalign, modules/stdarg, modules/stdbool:
- * modules/stddef, modules/stdint, modules/stdio, modules/stdlib:
- * modules/stdnoreturn, modules/string, modules/strings:
- * modules/sysexits, modules/termios, modules/threads-h:
- * modules/time, modules/uchar, modules/unicase/base:
- * modules/uniconv/base, modules/unictype/base, modules/unigbrk/base:
- * modules/unilbrk/base, modules/uniname/base, modules/uninorm/base:
- * modules/unistd, modules/unistdio/base, modules/unistr/base:
- * modules/unitypes, modules/uniwbrk/base, modules/uniwidth/base:
- * modules/utime-h, modules/wchar, modules/wctype-h:
- Use it.
-
---------------------------------------------------------------------------------
-
commit 4bdc327dbda59dcdbfa0f983a4f35c4a4ec3578c
Author: Bruno Haible <[email protected]>
Date: Sun Dec 19 12:49:16 2021 +0100
diff --git a/pygnulib/GLEmiter.py b/pygnulib/GLEmiter.py
index f0ae4627e6..fd6451cad9 100644
--- a/pygnulib/GLEmiter.py
+++ b/pygnulib/GLEmiter.py
@@ -67,18 +67,34 @@ def _convert_to_gnu_make(snippet: str) -> str:
def _eliminate_NMD_from_line(line: str, automake_subdir: bool) -> str | None:
- '''Eliminate occurrences of @NMD@ from the given line. The modified
- line is returned or None if the line should be removed.'''
- if line.startswith('@NMD@'):
- if automake_subdir:
- line = line.replace('@NMD@', '')
- else:
- line = None
+ '''Eliminate occurrences of @NMD@ and @!NMD@ from the given line. The
+ modified line is returned or None if the line should be removed.
+
+ line is the current line in the snippet being operated on.
+ automake_subdir is a bool that is True if --automake-subdir is in use,
+ else False.'''
+ if automake_subdir:
+ clean = '@NMD@'
+ eliminate = '@!NMD@'
+ else:
+ clean = '@!NMD@'
+ eliminate = '@NMD@'
+ # Check if we should eliminate the line from the output.
+ if line.startswith(eliminate):
+ return None
+ # Check if we should clean the mark but keep the line.
+ if line.startswith(clean):
+ return line.replace(clean, '')
return line
def _eliminate_NMD(snippet: str, automake_subdir: bool) -> str:
- '''Return the Automake snippet with occurrences of @NMD@ removed.'''
+ '''Return the Automake snippet with occurrences of @NMD@ and @!NMD@
+ removed.
+
+ snippet is the Automake snippet being operated on.
+ automake_subdir is a bool that is True if --automake-subdir is in use,
+ else False.'''
result = []
for line in snippet.splitlines():
line = _eliminate_NMD_from_line(line, automake_subdir)
--
2.44.0
