Re: Parallelization of shell scripts for 'configure' etc.
On Tue, Jun 14, 2022 at 11:15 PM Ángel wrote: > > On 2022-06-13 at 18:32 -0700, Paul Eggert wrote: > > Yes, all that could be done in theory, but it'd take a lot of > > hacking and it's been decades and it hasn't happened. > > > > I'd rather have shell scripts "just work" in parallel with a minimum > > of fuss. > > If this hasn't happened, that might be because nobody found it > important enough to dive into such task, relatively to the effort > required (or estimated). > > Everyone wishes their configure scripts to finish quickly but, first > and foremost, the result must be right. Something which is easier with > serial script than with a parallel implementation. > > And in the case of configure, they are a mix of boilerplate code, m4 > and shell scripting, so not many people would be, in addition to > motivated by that task, proficient enough in all of those. Furthermore, > configure typically target really classic dialects, which makes even > more difficult to use modern features which could speed up the script. > > Optimizing that at bash would likely be even more complex, since it > could only automatically optimize the cases that would never have side > effects. > > I think that most gainings should probably come from autoconf blocks > with optimizations. Still, it might be possible through some loadable > builtin to improve the configure times. > > Do you have any handy example of configure that takes too long to run? Try Emacs. Jeff
Auto-update program cache feature
Hi Everyone, I noticed a fair number of new Linux users have trouble with stale program caches. Users install a package from a package manager or sources and then are confused when the new package is not used. They do not realize they need to run 'bash -r'; and most don't know where to begin searching. I think a useful feature for Bash would be to automatically update the program cache after an install. Posix does not standardize an install command (cf, http://pubs.opengroup.org/onlinepubs/009696699/utilities/contents.html), so monitoring of common commands seems like a sensible way to implement the feature. A single terminal can monitor for a regex that looks for 'make install' and perhaps other common installation commands. Multiple terminals seems like a trickier case, and could use a scheme where the source terminal broadcasts an 'update cache' message to other open terminals. Some folks will argue that a person who installs a new program or updates an existing program does not want to use the newly installed program. They will argue it to the point it displaces the common case (with the common case being a user wants to actually use the newly installed program). For users who do not want to use the newly installed program a means to disable the behavior should be provided. Jeff
Re: Auto-update program cache feature
On Wed, Oct 3, 2018 at 9:33 PM Eduardo A. Bustamante López wrote: > > On Wed, Oct 03, 2018 at 04:45:44PM -0400, Jeffrey Walton wrote: > > Hi Everyone, > > > > I noticed a fair number of new Linux users have trouble with stale > > program caches. Users install a package from a package manager or > > sources and then are confused when the new package is not used. They > > do not realize they need to run 'bash -r'; and most don't know where > > to begin searching. > > You mean `hash -r' ? > > > (...) > > so monitoring of common commands seems like a sensible way to > > implement the feature. > > > A single terminal can monitor for a regex that looks for 'make > > install' and perhaps other common installation commands. Multiple > > terminals seems like a trickier case, and could use a scheme where the > > source terminal broadcasts an 'update cache' message to other open > > terminals. > > A shell is not a terminal. A terminal is a hardware (or emulated) device that > provides input/output capabilities. Nowadays most terminals are emulated > (gnome-terminal, xterm, urxvt, ...), but there are still physical terminals in > use. It seems quite complicated to have all terminal emulators and physical > terminals introduce this functionality. > > And even if you manage to do that... how would that work? Terminals are only > aware of the input typed by the user, and the output provided by the programs > that run attached to that terminal device... that excludes a bunch of cases > like... How Bash achieves it is an implementation detail left to the experts. I made a few suggestions that don't seem to fit well. That's OK because Bash internals is not my area of expertise. Architecturally each instance of Bash is an object could receive a "clear cache" message. How it is achieved does not matter to me. What is important is, the right thing is done to relieve users from the extra work for the common case. > If you don't care about performance, why don't you just run `hash -r' through > PROMPT_COMMAND? That seems simple enough? This was answered in the problem statement: new users don't know what they have to do, and don't know where to go looking for the answer when things don't work as expected. It would be helpful if programs like Autotools and CMake finished with a prompt like, "Run 'hash -r' to tell users their next step but they don't. (It also would not help the multiple sessions case, but it would go a long way in helping users with the problem). Jeff
How to tell Bash multiple values are success?
Hi Everyone, I have a systemd service that automatically applies updates. Looking at the service history, the service reports failure when it installs updates due to this: if dnf -y update &>/dev/null then echo "Upgraded system" else echo "Failed to upgrade system" exit 1 fi The problems seems to be 0, 100 and 200 are success. When updates are installed either 100 or 200 is returned. Confer, https://dnf.readthedocs.io/en/latest/command_ref.html . I looked through Bash Variables, but I did not see a way to signal the information to Bash. Confer, https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html . How do I tell Bash 0, 100 and 200 are success? (Or, how to tell Bash 1 is the only failure?) Thanks in advance
Create an alias on the fly?
Hi Everyone, I'm testing some software from Master. My testing machines sometimes lack the distro tools like makeinfo. It results in things like this: ./bootstrap: 255: makeinfo: not found ./bootstrap: Error: 'makeinfo' not found I tried the MAKEINFO=true tricks but they did not work. How can I create an alias on the fly using code like this in Bash? if ! ./bootstrap; then echo "Failed to bootstrap Wget2" exit 1 fi Thanks in advance.
Re: Create an alias on the fly?
On Mon, Mar 30, 2020 at 4:38 AM Andreas Kusalananda Kähäri wrote: > > On Mon, Mar 30, 2020 at 04:24:07AM -0400, Jeffrey Walton wrote: > > Hi Everyone, > > > > I'm testing some software from Master. My testing machines sometimes > > lack the distro tools like makeinfo. It results in things like this: > > > > ./bootstrap: 255: makeinfo: not found > > ./bootstrap: Error: 'makeinfo' not found > > > > I tried the MAKEINFO=true tricks but they did not work. > > > > How can I create an alias on the fly using code like this in Bash? > > > > if ! ./bootstrap; > > then > > echo "Failed to bootstrap Wget2" > > exit 1 > > fi > > > > Thanks in advance. > > This does not seem to be a bug in the bash shell. Maybe you meant > to post this the help-bash list? My bad. I did not realize there was a separate user list. > Also, why not just install the > prerequisites for the software you're building? Some of the systems I test on are too old and don't have working repos to install things from. Others are resource constrained like ARM dev boards. The dev boards don't have the storage so I would prefer to skip the docs. Jeff
Segfault in Bash
Hi Everyone, I'm working on a script to find all shared objects in a directory. A filename should match the RE '*.so$'. I thought I would pipe it to grep: $ ./audit-libs.sh /home/jwalton/tmp/ok2delete/lib ./audit-libs.sh: line 17: 22929 Segmentation fault (core dumped) $(echo "$file" | grep -E "*.so$") ./audit-libs.sh: line 17: 22934 Segmentation fault (core dumped) $(echo "$file" | grep -E "*.so$") ./audit-libs.sh: line 17: 22939 Segmentation fault (core dumped) $(echo "$file" | grep -E "*.so$") ... My code is broken at the moment. I know I am the cause of Bash's crash. But I feel like Bash should not segfault. IFS="" find "$dir" -name '*.so' -print | while read -r file do if ! $(echo "$file" | grep -E "*.so$"); then continue; fi echo "library: $file" done Are you guys interested in the segfault?
Use of `mktemp' is dangerous, better use `mkstemp'...
Hi Everyone, I'm building Bash 5.1 from sources. This may be of interest: /usr/bin/ld: ./lib/sh/libsh.a(tmpfile.o): in function `sh_mktmpname': /home/jwalton/Build-Scripts/bash-5.1/lib/sh/tmpfile.c:160: warning: the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp' Jeff
Bash 5.1 and Ncurses
Hi Everyone, I configured Bash 5.1 with prefix=/usr/local, --with-curses and --enable-multibyte. The link failed with undefined references, like tputs. I have the wide char Ncurses built and installed in /usr/local. I found I needed to add -ltinfow to LIBS and LDLIBS for the build to succeed. I'm pretty sure we are supposed to use the wide character version of Ncurses nowadays. I believe the wide character version supplies both multibyte and single byte functions, so it should just work with all software nowadays. It seems like configure should be able to find the Ncurses library. Maybe Ncurses needs one of those --with-libncurses-prefix. Jeff
[no subject]
Hi Everyone, I'm testing on Ubuntu 18.05 x86_64 fully patched. It looks like the man2html recipe is not using LDFLAGS: /usr/bin/cc -DHAVE_CONFIG_H -DSHELL -I/home/jwalton/Build-Scripts/bash-5.1 -I.. -I/home/jwalton/ok2delete-asan/include -DNDEBUG -DTEST_ASAN=1 -g2 -O2 -fsanitize=address -fno-omit-frame-pointer -march=native -fPIC -pthread man2html.o -o man2html -ldl -ltinfow -ldl -lpthread /usr/bin/ld: cannot find -ltinfow collect2: error: ld returned 1 exit status Makefile:80: recipe for target 'man2html' failed make[1]: *** [man2html] Error 1 make[1]: Leaving directory '/home/jwalton/Build-Scripts/bash-5.1/support' Makefile:753: recipe for target 'support/man2html' failed make: *** [support/man2html] Error 1 make: *** Waiting for unfinished jobs LDFLAGS has the path to -ltinfow. Jeff
Re:
On Fri, Jan 15, 2021 at 4:33 PM Jeffrey Walton wrote:
>
> Hi Everyone,
>
> I'm testing on Ubuntu 18.05 x86_64 fully patched. It looks like the
> man2html recipe is not using LDFLAGS:
>
> /usr/bin/cc -DHAVE_CONFIG_H -DSHELL
> -I/home/jwalton/Build-Scripts/bash-5.1 -I..
> -I/home/jwalton/ok2delete-asan/include -DNDEBUG -DTEST_ASAN=1 -g2 -O2
> -fsanitize=address -fno-omit-frame-pointer -march=native -fPIC
> -pthread man2html.o -o man2html -ldl -ltinfow -ldl -lpthread
> /usr/bin/ld: cannot find -ltinfow
> collect2: error: ld returned 1 exit status
> Makefile:80: recipe for target 'man2html' failed
> make[1]: *** [man2html] Error 1
> make[1]: Leaving directory '/home/jwalton/Build-Scripts/bash-5.1/support'
> Makefile:753: recipe for target 'support/man2html' failed
> make: *** [support/man2html] Error 1
> make: *** Waiting for unfinished jobs
>
> LDFLAGS has the path to -ltinfow.
This patch cleared the issue.
# Written and placed in public domain by Jeffrey Walton.
--- support/Makefile.in
+++ support/Makefile.in
@@ -48,6 +48,8 @@
CFLAGS_FOR_BUILD = @CFLAGS_FOR_BUILD@
CPPFLAGS = @CPPFLAGS@
CPPFLAGS_FOR_BUILD = @CPPFLAGS_FOR_BUILD@
+LDFLAGS = @LDFLAGS@
+LDFLAGS_FOR_BUILD = ${LDFLAGS}
LOCAL_CFLAGS = @LOCAL_CFLAGS@
DEFS = @DEFS@
LOCAL_DEFS = @LOCAL_DEFS@
@@ -77,7 +79,7 @@
all: man2html$(EXEEXT)
man2html$(EXEEXT): $(OBJ1)
-$(CC_FOR_BUILD) $(CCFLAGS_FOR_BUILD) $(OBJ1) -o $@ ${LIBS_FOR_BUILD}
+$(CC_FOR_BUILD) $(CCFLAGS_FOR_BUILD) $(OBJ1) -o $@
${LDFLAGS_FOR_BUILD} ${LIBS_FOR_BUILD}
clean:
$(RM) man2html$(EXEEXT) $(OBJ1)
Jeff
Crash in malloc.c : internal_malloc under Asan
Hi Everyone, I'm trying to use an Asan-instrumented version of Bash for my shell. Bash is dying in startup code. The source file is malloc.c (missing from Bash sources at https://git.savannah.gnu.org/cgit/bash.git/tree/), and the function is internal_malloc. The Asan folks have a suspicion of malloc.c. Also see "Crash in application's startup code around Asan red zone", https://groups.google.com/g/address-sanitizer/c/oB8opTbjkuw. Where is malloc.c coming from? Where can I find it? Jeff
