Bash 5.1 fails parallel builds from source
I occasionally encounter the error: gcc -DPROGRAM='"bash"' -DCONF_HOSTTYPE='"x86_64"' -DCONF_OSTYPE='"linux-musl"' -DCONF_MACHTYPE='"x86_64-pc-linux-musl"' -DCONF_VENDOR='"pc"' -DLOCALEDIR='"/usr/local/share/locale"' -DPACKAGE='"bash"' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib-g -O2 -Wno-parentheses -Wno-format-security -c list.c bashline.c:65:10: fatal error: builtins/builtext.h: No such file or directory 65 | #include "builtins/builtext.h" /* for read_builtin */ | ^ compilation terminated. That's because bashline.o does not have a dependency on builtins/builtext.h in Makefile, unlike sig,o, subst.o etc -- Fazal Majid www.majid.info
Re: Bash 5.1 fails parallel builds from source
On 12/7/20 7:31 AM, Fazal Majid wrote: I occasionally encounter the error: gcc -DPROGRAM='"bash"' -DCONF_HOSTTYPE='"x86_64"' -DCONF_OSTYPE='"linux-musl"' -DCONF_MACHTYPE='"x86_64-pc-linux-musl"' -DCONF_VENDOR='"pc"' -DLOCALEDIR='"/usr/local/share/locale"' -DPACKAGE='"bash"' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib-g -O2 -Wno-parentheses -Wno-format-security -c list.c bashline.c:65:10: fatal error: builtins/builtext.h: No such file or directory 65 | #include "builtins/builtext.h" /* for read_builtin */ | ^ compilation terminated. That's because bashline.o does not have a dependency on builtins/builtext.h in Makefile, unlike sig,o, subst.o etc Thanks for the report. I've never actually encountered this error. Just lucky, I guess. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Bash-5.1 release available
Introduction The first public release of bash-5.1 is now available with the URLs ftp://ftp.cwru.edu/pub/bash/bash-5.1.tar.gz ftp://ftp.gnu.org/pub/gnu/bash/bash-5.1.tar.gz and from the master branch of the bash git repository (http://git.savannah.gnu.org/cgit/bash.git/log/) and the usual GNU mirror sites. Bash is the GNU Project's Bourne Again SHell, a complete implementation of the POSIX shell spec, but also with interactive command line editing, job control on architectures that support it, csh-like features such as history substitution and brace expansion, and a slew of other features. For more information on the features of Bash that are new to this type of shell, see the file `doc/bashref.texi'. There is also a large Unix-style man page. The man page is the definitive description of the shell's features. This tar file includes the formatted documentation (pdf, postscript, dvi, info, and html, plus nroffed versions of the manual pages). Please use `bashbug' to report bugs with this version. It is built and installed at the same time as bash. Installation Please read the README file first. Installation instructions are provided in the INSTALL file. New Features This is the fifth major release of bash. Read the file NEWS in the bash-5.1 distribution for a complete description of the new features. A copy of the relevant portions is included below. This release fixes several outstanding bugs in bash-5.0 and introduces several new features. The most significant change is a return to the bash-4.4 behavior of not performing pathname expansion on a word that contains backslashes but does not contain any unquoted globbing special characters. This comes after a long POSIX discussion that resulted in a change to the standard. There are several changes regarding trap handling while reading from the terminal (e.g, for `read' and `select'.) There are a number of bug fixes, including several bugs that caused the shell to crash. Details are appended. The most notable new features are in the random number engine. There is a new variable, SRANDOM, which gets its random data from the system's entropy engine and so is not linear and cannot be reseeded to get an identical random sequence. The PROMPT_COMMANDS array variable can be used to run multiple commands before printing the primary prompt. Associative arrays may be assigned using a series of key-value pairs within a compound assignment. `wait' has a new `-p' option which stores PID information about the process reaped by `wait -n'. Process substitution is now available in posix mode. There are new parameter transformation operators. There is no new `compat50' option; use the BASH_COMPAT variable to select the compatibility level. All the new features are described below. The most visible new feature is in Readline: the addition of `faces', which highlights the text between the point and mark (the region, so this is also called the 'active region'). This was added to show visibly the text inserted by bracketed paste, and also marks the text found by incremental and non-incremental history searches. Faces are currently tied to bracketed paste and are enabled and disabled along with bracketed paste mode. Bracketed paste mode is enabled by default. There are a few incompatible changes between bash-5.0 and bash-5.1. The change to pathname expansion means that words containing backslashes, but no special globbing characters, will not undergo pathname expansion. While the bash-5.0 behavior was POSIX-conformant, the change was not well-received. Changes to the random number engines mean that seeding RANDOM will produce a different numeric sequence. Set the compatibility level to 50 to revert to the bash-5.0 behavior. Bash can be linked against an already-installed Readline library rather than the private version in lib/readline if desired. Only readline-8.1 and later versions are able to provide all of the symbols that bash-5.1 requires; earlier versions of the Readline library will not work correctly. A complete list of changes between bash-5.0 and bash-5.1 is available in the file CHANGES; the complete list is too large to include in this message. Readline Also available is a new release of the standalone Readline library, version 8.1, with its own configuration scripts and Makefiles. It can be retrieved with the URLs ftp://ftp.cwru.edu/pub/bash/readline-8.1.tar.gz ftp://ftp.gnu.org/pub/gnu/readline/readline-8.1.tar.gz and from the master branch of the GNU readline git repository (http://git.savannah.gnu.org/cgit/readline.git/log/) and the usual GNU mirror sites. The formatted Readline documentation is included in the readline distribution tar file. A separate announcement listing the changes in Readline is being distributed. As always, thanks for your help. Chet +== NEWS ==+ This is a terse description of the new features added to bash-5.1 since the release of bas
Readline-8.1 release available
The first public release of the GNU Readline library, version 8.1, is now available for FTP with the URLs ftp://ftp.cwru.edu/pub/bash/readline-8.1.tar.gz ftp://ftp.gnu.org/pub/gnu/readline/readline-8.1.tar.gz and from the master branch in the readline git repository (http://git.savannah.gnu.org/cgit/readline.git/log) and the usual GNU mirrors. This distribution is essentially a standalone version of the readline library that appears in bash-5.1 together with an `autoconf' framework. The documentation has been updated and is current. Postscript, DVI, and Info versions of the Readline and History manuals are included. A list of changes in this release is appended to this announcement. This release accompanies the simultaneous release of bash-5.1. There are more improvements in the programming interface and new user-visible variables and bindable commands. There are a several new public API functions, but there should be no incompatible changes to existing APIs. The most visible new feature is the addition of `faces', which highlights the text between the point and mark (the region, so this is also called `active region'). This was added to show visibly the text inserted by bracketed paste, and also marks the text found by incremental and non-incremental history searches. Several additional commands set the mark to take advantage of this feature. Bracketed paste mode is supported in more places (e.g., in vi `overstrike' mode). Faces are currently tied to bracketed paste and are enabled and disabled along with bracketed paste mode. Bracketed paste mode is enabled by default; there is a configure-time option to make the default setting disabled. There are a few bug fixes in the redisplay code when the prompt string contains invisible multibyte characters, and several in vi mode. Full details are below. GNU Readline is a library which provides programs with an input facility including command-line editing and history. Editing commands similar to both emacs and vi are included. The GNU History library, which provides facilities for managing a list of previously-typed command lines and an interactive command line recall facility similar to that provided by csh, is also present. The history library is built as part of the readline as well as separately. Please send readline bug reports to bug-readl...@gnu.org. As always, thanks for your help. Chet +== CHANGES ==+ This document details the changes between this version, readline-8.1, and the previous version, readline-8.0. 1. Changes to Readline a. There are a number of fixes that were found as the result of fuzzing with random input. b. Changed the revert-all-at-newline behavior to make sure to start at the end of the history list when doing it, instead of the line where the user hit return. c. When parsing `set' commands from the inputrc file or an application, readline now allows trailing whitespace. d. Fixed a bug that left a file descriptor open to the history file if the file size was 0. e. Fixed a problem with binding key sequences containing meta characters. f. Fixed a bug that caused the wrong line to be displayed if the user tried to move back beyond the beginning of the history list, or forward past the end of the history list. g. If readline catches SIGTSTP, it now sets a hook that allows the calling application to handle it if it desires. h. Fixed a redisplay problem with a prompt string containing embedded newlines. i. Fixed a problem with completing filenames containing invalid multibyte sequences when case-insensitive comparisons are enabled. j. Fixed a redisplay problem with prompt strings containing invisible multibyte characters. k. Fixed a problem with multibyte characters mapped to editing commands that modify the search string in incremental search. l. Fixed a bug with maintaining the key sequence while resolving a bound command in the presence of ambiguous sequences (sequences with a common prefix), in most cases while attempting to unbind it. m. Fixed several buffer overflows found as the result of fuzzing. n. Reworked backslash handling when translating key sequences for key binding to be more uniform and consistent, which introduces a slight backwards incompatibility. o. Fixed a bug with saving the history that resulted in errors not being propagated to the calling application when the history file is not writable. p. Readline only calls chown(2) on a newly-written history file if it really needs to, instead of having it be a no-op. q. Readline now behaves better when operate-and-get-next is used when the history list is `full': when there are already $HISTSIZE entries. r. Fixed a bug that could cause vi redo (`.') of a replace command not to work correctly in the C or POSIX locale. s. Fixed a bug with vi-mode digit arguments that caused the last command to be set incorrectly. This prevents yank-last-arg from working as intended
Re: Bash 5.1 fails parallel builds from source
> On Dec 7, 2020, at 15:37, Chet Ramey wrote: > > On 12/7/20 7:31 AM, Fazal Majid wrote: >> I occasionally encounter the error: >> gcc -DPROGRAM='"bash"' -DCONF_HOSTTYPE='"x86_64"' >> -DCONF_OSTYPE='"linux-musl"' -DCONF_MACHTYPE='"x86_64-pc-linux-musl"' >> -DCONF_VENDOR='"pc"' -DLOCALEDIR='"/usr/local/share/locale"' >> -DPACKAGE='"bash"' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib >> -g -O2 -Wno-parentheses -Wno-format-security -c list.c >> bashline.c:65:10: fatal error: builtins/builtext.h: No such file or directory >>65 | #include "builtins/builtext.h" /* for read_builtin */ >> | ^ >> compilation terminated. >> That's because bashline.o does not have a dependency on builtins/builtext.h >> in Makefile, unlike sig,o, subst.o etc > > Thanks for the report. I've never actually encountered this error. Just > lucky, I guess. It’s a race condition. The machine I run it on has 6 cores and HT, so I run it with a `make -j 12`, and even then, it’s not consistently reproducible. -- Fazal Majidwww.majid.info
Re: Bash 5.1 fails parallel builds from source
On 12/7/20 1:44 PM, Fazal Majid wrote: On Dec 7, 2020, at 15:37, Chet Ramey wrote: Thanks for the report. I've never actually encountered this error. Just lucky, I guess. It’s a race condition. The machine I run it on has 6 cores and HT, so I run it with a `make -j 12`, and even then, it’s not consistently reproducible. Sure. I run make -j 4 at a minimum, make -j 8 sometimes, and I've never seen it. Not very reproducible. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1 release available
On 12/7/20 1:47 PM, Benno Schulenberg wrote: Hello Chet, The first public release of bash-5.1 is now available with the URLs ftp://ftp.cwru.edu/pub/bash/bash-5.1.tar.gz ftp://ftp.gnu.org/pub/gnu/bash/bash-5.1.tar.gz It would be nice if the Translation Project would get a CC for the rc1 or rc2 release of bash, not for the final release. I copied coordina...@translationproject.org for bash-5.1-alpha 17 June 2020 and assumed that would start the update process. The translatable strings change very little, if at all, after an alpha release. I can add you for release candidates as well if you'd like. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1 release available
Hello Chet, > The first public release of bash-5.1 is now available with the URLs > > ftp://ftp.cwru.edu/pub/bash/bash-5.1.tar.gz > ftp://ftp.gnu.org/pub/gnu/bash/bash-5.1.tar.gz It would be nice if the Translation Project would get a CC for the rc1 or rc2 release of bash, not for the final release. Now the translators had no chance to update their translations before the final release. :| Benno
bash doesn't display user-typed characters; can interfere with COPY/PASTE
If I type in ( + are keypresses) if [[ '' == $'\t' ]]; then echo ok; else echo notok; fi bash displays: if [[ ' ' == $'\t' ]]; then echo ok; else echo notok; fi ok if I now copy the 'if' line and paste it if [[ ' ' == $'\t' ]]; then echo ok; else echo notok; fi notok if I take the same line from an editor like gvim, it works. If the test line is in a file, and I use 'cat file' and copy/past the resulting line, it works. It is only when bash is displaying the line that it doesn't work. The problem is that bash isn't displaying a 'tab' character where one was typed. With many (most?) terminal windows these days, especially Unicode-enabled ones, the terminal has to read what is on the screen to be able to read the binary code of whatever is displayed on the screen, Otherwise, it wouldn't be able to read typed unicode. Can this be fixed -- maybe with an option in 'shopt', for those who might be using a non-expanding terminal, but anyone using an xterm/linux compatible terminal should get the expansions from their terminal. Where this can be even more annoying is if your terminal's response to a tab is different than that used on old-hardware terminals. Thanks, -l
Bash 5.1: rl_readline_version = 0x801 (shouldn't it be 0x0801 ?)
Configuration Information [Automatically generated, do not change]: Machine: aarch64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security uname output: Linux raspberrypi 5.4.72-v8+ #1356 SMP PREEMPT Thu Oct 22 13:58:52 BST 2020 aarch64 G$ Machine Type: aarch64-unknown-linux-gnu Bash Version: 5.1 Patch Level: 0 Release Status: release Description: I just built Bash 5.1 upon its official release today — December 7, 2020. To confirm the version of readline that's bundled with Bash 5.1, I ran "gdb bash" and entered "print /x (int) rl_readline_version". I get "0x801" as the output. If I do the same thing with Bash 5.0, I get "0x800". I believe these values represent readline 8.1 and 8.0, respectively. However, readline's online documentation at https://tiswww.case.edu/php/chet/readline/readline.html#SEC25 — and the "rltech.texi" file in the source code — both indicate that the version code format should be slightly different: -- readline.h defines a C preprocessor variable that should be treated as an integer, RL_READLINE_VERSION, which may be used to conditionally compile application code depending on the installed Readline version. The value is a hexadecimal encoding of the major and minor version numbers of the library, of the form 0xMMmm. MM is the two-digit major version number; mm is the two-digit minor version number. For Readline 4.2, for example, the value of RL_READLINE_VERSION would be 0x0402. -- If the version number for readline follows the "0xMMmm" format, shouldn't the value of "rl_readline_version" be "0x0801" for Bash 5.1 and "0x0800" for Bash 5.0? In other words, the "leading 0" after the "x" appears to be missing. I'm not sure if this is a bug — or if it's an artifact of using gdb to extract readline's version information. If it is an artifact, is there a better way to determine the bundled version of readline that comes with Bash? Thanks, Chet, for your outstanding work!
Re: Bash 5.1: rl_readline_version = 0x801 (shouldn't it be 0x0801 ?)
On Tue, Dec 8, 2020 at 10:04 AM Testing Purposes < raspberry.teststr...@gmail.com> wrote: > -- > readline.h defines a C preprocessor variable that should be treated as an > integer, As it says it's an integer. > RL_READLINE_VERSION, which may be used to conditionally compile > application code depending on the installed Readline version. The value is > a hexadecimal encoding of the major and minor version numbers of the > library, of the form 0xMMmm. MM is the two-digit major version number; mm > is the two-digit minor version number. For Readline 4.2, for example, the > value of RL_READLINE_VERSION would be 0x0402. > -- > > If the version number for readline follows the "0xMMmm" format, shouldn't > the value of "rl_readline_version" be "0x0801" for Bash 5.1 and "0x0800" > for Bash 5.0? In other words, the "leading 0" after the "x" appears to be > missing. The integer 0x801 is the same as 0x0801. # echo $(( 0x801 )) 2049 # echo $(( 0x0801 )) 2049 # echo $(( 0x0801 )) 2049 #
Re: Bash 5.1: rl_readline_version = 0x801 (shouldn't it be 0x0801 ?)
On Mon, Dec 7, 2020 at 9:30 PM Eastern Time, Clark Wang wrote: The integer 0x801 is the same as 0x0801. > > # echo $(( 0x801 )) > 2049 > # echo $(( 0x0801 )) > 2049 > # echo $(( 0x0801 )) > 2049 > # > >From an integer standpoint, I know that 08 (with one leading zero) is the same as 8. But Readline's official documentation specifically chose an example with a leading zero — 0x0402. It says that Readline 4.2 should have a version value of 0x0402, not 0x402. Therefore, based on the documentation, there should still be a way to extract the full, 0xMMmm-formatted value for rl_readline_version. The Readline documentation explicitly states that the value should consist of a "*two-digit* major version number" (MM) plus a "*two-digit* minor version number" (mm). In other words, the output format should always be 0xMMmm. All contemporary versions of Readline consist of a single-digit integer for the major version number. That won't change until we get to version 10 of Readline. That means, for now, all major version numbers of Readline, when formatted as a two-digit number, would always contain a leading zero — so there's nothing unusual about this scenario. So there should be a way to extract the full and proper 0xMMmm format, as the example in the documentation indicates. At a minimum, this is important to avoid confusion when people are simply trying to determine what version of Readline came bundled with their build of Bash. If anyone knows how to do this, please let the mailing list know.
Re: Bash 5.1: rl_readline_version = 0x801 (shouldn't it be 0x0801 ?)
On Tue, Dec 8, 2020 at 12:36 PM Testing Purposes < raspberry.teststr...@gmail.com> wrote: > But Readline's official documentation specifically chose an example with a > leading zero — 0x0402. It says that Readline 4.2 should have a version > value of 0x0402, not 0x402. > Could you point me to where it says 0x402 is wrong? > > Therefore, based on the documentation, there should still be a way to > extract the full, 0xMMmm-formatted value for rl_readline_version. > > The Readline documentation explicitly states that the value should consist > of a "*two-digit* major version number" (MM) plus a "*two-digit* minor > version number" (mm). In other words, the output format should always > be 0xMMmm. > I'd interpret the two-digit as "at most two digits". For 0x402, MM is "04" and mm is "02". > > All contemporary versions of Readline consist of a single-digit integer > for the major version number. That won't change until we get to version 10 > of Readline. > It sounds to me that the "two digits" are in HEX format (0xMMmm) so when it reaches version 10 it'll be 0x0a00. > That means, for now, all major version numbers of Readline, when formatted > as a two-digit number, would always contain a leading zero — so there's > nothing unusual about this scenario. > > So there should be a way to extract the full and proper 0xMMmm format, as > the example in the documentation indicates. > You can $ printf '0x%04x\n' 0x402 0x0402 $ printf '0x%04x\n' 0x801 0x0801 $ >
Re: Bash 5.1: rl_readline_version = 0x801 (shouldn't it be 0x0801 ?)
2020年12月8日(火) 10:04 Testing Purposes : > Description: > > I just built Bash 5.1 [...], I ran "gdb bash" and entered > "print /x (int) rl_readline_version". I get "0x801" as the output. > If I do the same thing with Bash 5.0, I get "0x800". > > However, readline's online documentation at > https://tiswww.case.edu/php/chet/readline/readline.html#SEC25 - and > the "rltech.texi" file in the source code - both indicate that the > version code format should be slightly different: 1) First of all, the value of `rl_readline_version' is embedded in `bash' in the binary format, so it is meaningless to discuss whether that binary data is `0xMMmm' or `0xMmm'. 2) Next, the manual talks about the literal contained in the macro `RL_READLINE_VERSION' (defined in `lib/readline/readline.h'), which is nothing to do with the binary value of `rl_readline_version'. In fact, the macro value of `RL_READLINE_VERSION' in `readline.h' is correctly in the format of 0xMMmm as described in the manual. $ grep RL_READLINE_VERSION lib/readline/readline.h #define RL_READLINE_VERSION 0x0801 /* Readline 8.0 */ Because the information on the literal format used in the source code is lost in the executable `bash', you need to explicitly specify the format `0xMMmm' (i.e. %04x) to gdb like this: (gdb) printf "%04x\n", (int) rl_readline_version 0x0801 -- Koichi
Re: Bash 5.1: rl_readline_version = 0x801 (shouldn't it be 0x0801 ?)
On Tue, Dec 8, 2020 at 1:35 AM Eastern Time, Koichi Murase wrote: > Because the information on the literal format used in the source code > is lost in the executable `bash', you need to explicitly specify the > format `0xMMmm' (i.e. %04x) to gdb like this: > > (gdb) printf "%04x\n", (int) rl_readline_version > 0x0801 > > -- > Koichi > I like your answer, Koichi. You explained the full, "hidden" story behind the output — and now it all makes sense. For those who would like a single one-liner without using gdb in interactive mode, you can simply run this command line: gdb bash -batch -ex 'printf "%04x\n", (int) rl_readline_version' In my case, since I'm using Bash 5.1, it generates this output: 0801 Which means that my brand-new build of Bash 5.1 is correctly using Readline 8.1
