Re: simple prob?
On Tue, 29 Jun 2021 16:35:28 -0400
Greg Wooledge wrote:
> On Tue, Jun 29, 2021 at 01:21:44PM -0700, L A Walsh wrote:
> > I hope a basic question isn't too offtopic.
>
> More of a help-bash question, in my opinion, but close enough.
>
> > Say I have some number of jobs running:
> >
> > > jobs|wc -l
> > 3
>
> OK.
>
> > Would like to pass a varname to njobs to store the answer in, like:
> > > njobs n
> > echo "$n"
> > 3
>
> "How do I pass a value back from a function to its caller, you know,
> like a function in any other programming language can do" is one of the
> holy grails of shell programming. There is no sane answer. You appear
> to be going down the "pass a variable name by reference" path, so:
>
> unicorn:~$ jobs | wc -l
> 3
> unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
> unicorn:~$ njobs walsh
> unicorn:~$ echo "$walsh"
> 3
>
> Now you just need to add sanity-checking on the argument of njobs, to
> avoid whatever code injection the malicious caller wants to perform.
I can't fathom the switch to eval there. Why not printf -v "$1" %s "$_n", for
example? It even rejects invalid identifiers.
--
Kerin Millar
Re: simple prob?
On Tue, 29 Jun 2021 17:02:16 -0400
Greg Wooledge wrote:
> On Tue, Jun 29, 2021 at 09:47:30PM +0100, Kerin Millar wrote:
> > On Tue, 29 Jun 2021 16:35:28 -0400
> > Greg Wooledge wrote:
> >
> > > unicorn:~$ njobs() { local _n=$(jobs | wc -l); eval "$1=\$_n"; }
> > > unicorn:~$ njobs walsh
> > > unicorn:~$ echo "$walsh"
> > > 3
> > >
> > > Now you just need to add sanity-checking on the argument of njobs, to
> > > avoid whatever code injection the malicious caller wants to perform.
> >
> > I can't fathom the switch to eval there. Why not printf -v "$1" %s "$_n",
> > for example? It even rejects invalid identifiers.
>
> declare, printf -v, local -n, eval -- they're mostly equivalent. Some
> of them may prevent *some* possible code injections, but none of them
> prevent *all* possible code injections.
>
> unicorn:~$ njobs2() { printf -v "$1" %s 42; }
> unicorn:~$ njobs2 'x[0$(date >&2)]'
> Tue Jun 29 17:00:29 EDT 2021
>
> No matter which one of these you choose, you still have to sanity-check
> the input. Or else declare that you do not care if the user shoots their
> own foot off (which is a valid stance as long as your code is never used
> in a context where the user can elevate their privileges/capabilites).
>
I see. Thanks.
--
Kerin Millar
Re: parameter expansion with `:` does not work
On Thu, 8 Jul 2021 02:54:06 +0200 (CEST)
lisa-as...@perso.be wrote:
> As I was in it, have also changed
>
>
>
> fdir=${fdir:-$PWD}
This makes sense.
>
>
>
> to
>
>
>
> fdir=${fdir:=$PWD}
Here, you are using a form of parameter expansion that intrinsically performs
variable assignment, only to redundantly assign the result of the expansion to
the same variable once again. In a roundabout fashion, it ends up proving
Greg's point. I'd suggest temporarily disavowing yourself of the
${parameter:=word} form, for the time being.
--
Kerin Millar
Re: Bug concatenating member of large array with string
On Tue, 10 Aug 2021 14:26:56 +0100 Joe Pater wrote: > The attached bash script (named 'test'), when run on my laptop, > produces the following output: > > xyzerg > > Instead of the expected: > > Aabergxyz Try piping your script to "od -An -t x1 -c" and also taking a look at https://mywiki.wooledge.org/BashFAQ/052. -- Kerin Millar
Re: An alias named `done` breaks for loops
On Sat, 14 Aug 2021 15:59:38 -0700
George Nachman wrote:
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: darwin20.5.0
> Compiler: gcc
> Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
> uname output: Darwin Georges-Mac-Pro.local 20.5.0 Darwin Kernel Version
> 20.5.0: Sat May 8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64
> x86_64
> Machine Type: x86_64-apple-darwin20.5.0
>
> Bash Version: 5.1
> Patch Level: 8
> Release Status: release
>
> Description:
> Defining an alias named `done` breaks parsing a for loop that does not have
> an `in word` clause.
>
>
> Repeat-By:
>
> Run the following script. It fails with this error:
>
> myscript.bash: line 7: syntax error near unexpected token `done'
> myscript.bash: line 7: ` done'
>
>
> #!/bin/bash
>
> alias done=""
>
> f() {
> for var; do
> done
> }
This does not constitute a valid test case for two reasons. Firstly, aliases
have no effect in scripts unless the expand_aliases shell option is set.
Secondly, the syntax is invalid on account of not having specified a list where
the for keyword expects to find one. In fact, this is the cause of the
"unexpected token" error that you encountered, rather than the ineffective
alias. Below is a revision of your test case that demonstrates the point that
you were trying to make.
#!/bin/bash
shopt -s expand_aliases
alias done=""
f() {
for var; do
:
done
}
Running this script results in the following error.
line 8: syntax error near unexpected token `}'
line 8: `}'
Not defining the alias does, indeed, allow for the revised script to execute
successfully.
--
Kerin Millar
Re: Exclamation mark when using character classes
On Fri, 20 Aug 2021 19:28:25 +
hancooper via Bug reports for the GNU Bourne Again SHell
wrote:
> I am using EPOCHREALTIME and then computing the corresponding human readable
> form, that can handle
> changes in locale
>
> now=$EPOCHREALTIME
> printf -v second '%(%S)T.%s' "${now%[^[:digit:]]*}" "${now#*[^[:digit:]]}"
> printf -v minute '%(%M)T' "${now%[^[:digit:]]*}"
> printf -v hour '%(%H)T' "${now%[^[:digit:]]*}"
>
> Incidentally, [![:digit:]] does not work there, you need to use the
> POSIX-specified caret (^) instead of an
> exclamation mark when using character classes. I'm not sure if this is
> intentional or a bug in bash; man
> page doesn't seem to mention it.
"If an open bracket introduces a bracket expression as in XBD RE Bracket
Expression, [...] the character ( '!' ) shall replace the
character ( '^' ) in its role in a non-matching list in the
regular expression notation."
So says POSIX on the matter of pattern matching notation. In other words, only
the exclamation-mark is POSIX-specified, although bash happens to tolerate the
use of a circumflex, in which case it should behave in the exact same way. Are
you able to show a concrete example of one behaving differently from the other?
--
Kerin Millar
Re: Exclamation mark when using character classes
On Fri, 20 Aug 2021 23:33:38 +
hancooper wrote:
> ‐‐‐ Original Message ‐‐‐
> On Friday, August 20, 2021 8:00 PM, Kerin Millar wrote:
>
> > On Fri, 20 Aug 2021 19:28:25 +
> > hancooper via Bug reports for the GNU Bourne Again SHell bug-bash@gnu.org
> > wrote:
> >
> > > I am using EPOCHREALTIME and then computing the corresponding human
> > > readable form, that can handle
> > > changes in locale
> > > now=$EPOCHREALTIME
> > > printf -v second '%(%S)T.%s' "${now%[^[:digit:]]}" "${now#[^[:digit:]]}"
> > > printf -v minute '%(%M)T' "${now%[^[:digit:]]}"
> > > printf -v hour '%(%H)T' "${now%[^[:digit:]]}"Incidentally, [![:digit:]]
> > > does not work there, you need to use the POSIX-specified caret (^)
> > > instead of an
> > > exclamation mark when using character classes. I'm not sure if this is
> > > intentional or a bug in bash; man
> > > page doesn't seem to mention it.
> >
> > "If an open bracket introduces a bracket expression as in XBD RE Bracket
> > Expression, [...] the character ( '!' ) shall replace
> > the character ( '^' ) in its role in a non-matching list in
> > the regular expression notation."
> >
> > So says POSIX on the matter of pattern matching notation. In other words,
> > only the exclamation-mark is POSIX-specified, although bash happens to
> > tolerate the use of a circumflex, in which case it should behave in the
> > exact same way. Are you able to show a concrete example of one behaving
> > differently from the other?
>
>
> I could do with some help separating the integer and the fractional part from
> a floating point number,
> particularly for the situation where the number starts with a period.
>
> t="13.357713"
>
> # integer part of time interval
> ts="${t%%[![:digit:]]+([:digit:])}" # remove longest match from back
> # 13.877597, remove .877597
>
> # fractional part of time interval
> fr="${t##*([:digit:])[![:digit:]]}" # remove longest match from front
> # 13.877597, remove 13.
"[:digit:]" is missing the additional pair of square brackets that would have
it be treated as a character class name. Assuming that the extglob option has
been enabled, try this.
ts=${t%%[![:digit:]]+([[:digit:]])}
fr=${t##*([[:digit:]])[![:digit:]]}
If it's only the value of EPOCHREALTIME that you are concerned with, your
approach could perhaps be simplified on the basis that it has a well-defined
format of %u.%06u, where "." may also be some other radix character.
t=$EPOCHREALTIME
ts=${t:0:-7}
fr=${t:(-6)}
--
Kerin Millar
Re: EPOCHREALTIME does not behave correctly before 1970
On Sun, 22 Aug 2021 16:13:28 -0400 Chet Ramey wrote: > On 8/21/21 1:28 AM, Emanuele Torre wrote: > > > Bash Version: 5.1 > > Patch Level: 8 > > Release Status: release > > > > > get_epochrealtime() casts tv.tv_sec (a time_t a.k.a. int) to unsigned > > int causing EPOCHREALTIME to not behave correctly before the Unix Epoch. > > The definition is seconds since the Unix epoch. It's not surprising that it > doesn't pay attention to dates before that. The problem with this statement is that EPOCHREALTIME is permitted to get it wrong, yet EPOCHSECONDS is permitted to get it right. The discrepancy certainly came as a surprise to me. $ TZ=UTC faketime '1970-01-01' bash -c 'declare -p EPOCHSECONDS' declare -- EPOCHSECONDS="0" $ TZ=UTC faketime '1969-12-31' bash -c 'declare -p EPOCHSECONDS' declare -- EPOCHSECONDS="-86400" -- Kerin Millar
Re: unexpected exit code for integer operation
On Wed, 27 Oct 2021 21:09:29 +0200 Toralf Förster wrote: > Hi, > > expected: > $ i=0; ((i = i + 1)); echo $? > 0 > > expected: > $ i=0; ((++i)); echo $? > 0 This makes use of a pre-increment operator. The evaluated number is 1. > > unexpected: > $ i=0; ((i++)); echo $? > 1 This makes use of a post-increment operator. The evaluated number is 0. > > i is always set to 0, the result is always non-zero, but the return code > differs. This is to be expected. It works the same way as in other languages, such as C. You should use the operator that reflects your intent. -- Kerin Millar
Re: Arbitrary command execution from test on a quoted string
On Thu, 28 Oct 2021 20:33:22 + elettrino via Bug reports for the GNU Bourne Again SHell wrote: > The following shows an example of bash testing a quoted string and as a > result executing a command embedded in the string. > > Here I used the command "id" to stand as an example of a command. The output > of id on this machine was as follows: > > user@machine:~$ id > uid=1519(user) gid=1519(user) groups=1519(user),100(users) > user@machine:~$ > > So to demonstrate: > > user@machine:~$ USER_INPUT='x[$(id>&2)]' > user@machine:~$ test -v "$USER_INPUT" > uid=1519(user) gid=1519(user) groups=1519(user),100(users) > user@machine:~$ > > This means that if variable USER_INPUT was indeed input from a user, the user > could execute an arbitrary command. > > Also unexpected (and presumably related), bash is expanding the construct > inside single quote marks: > > user@machine:~$ test -v 'x[$(id>&2)]' > uid=1519(user) gid=1519(user) groups=1519(user),100(users) > user@machine:~$ > > user@machine:~$ echo $BASH_VERSION > 4.4.20(1)-release > user@machine:~$ > > I don't know whether this happens with anything other than the -v option with > test; I have not seen it happen under any other circumstance. > > Sent with [ProtonMail](https://protonmail.com) Secure Email. It is a regrettable consequence of the arithmetical context. See https://mywiki.wooledge.org/BashProgramming/05#Arithmetic_Expansion for some other examples. As things stand, your only option is to validate or sanitise abitrarily specified array indices before proceeding to use them in such a context. -- Kerin Millar
Re: I've found a vulnerability in bash
On 19/11/2021 10:53, Marshall Whittaker wrote: You could argue that bash should parse filenames globbed from * that start with - and exclude them specifically, so I'll have to respectfully disagree. One could, but it would not make for a compelling argument. Define GLOBIGNORE, if you insist. Also, it is not the programs doing the parsing of *, that is a function of bash. Try typing * in just your terminal/command line and see what happens Yes. However, the presented 'exploit' hinges upon the behaviour of a selected external program. Luckily for you, any that uses getopt(3) will support -- as a means of concluding option recognition, rm(1) included. In the case that you are using a program where option arguments cannot be reliably separated from non-option arguments, specifying the glob as ./* will commonly suffice. A short whitepaper on it has been made public at: https://oxagast.org/posts/bash-wildcard-expansion-arbitrary-command-line-arguments-0day/ complete with a mini Po It's perplexing that your post relies upon the use of -- to get the point across, without acknowledging its import. At any rate, this does not constitute a vulnerability on the part of bash, much less a zero-day. -- Kerin Millar
Re: I've found a vulnerability in bash
(Copying the list back in ...) On Fri, 19 Nov 2021 07:19:29 -0500 Marshall Whittaker wrote: > Though I do disagree with you, this is the only message in this thread that > even makes sense. Firstly, rm * is a valid - albeit unsafe - simple command, and one that is easily rectified. Secondly, the manner in which * expands is in accordance with the documented behaviour. Thirdly, the manner in which simple commands are processed is in accordance with the documented behaviour. In the event that you can falsify the second and/or third of these assertions, then - and only then - will you have discovered a bug in bash. As far as I can tell, your contention is that the default mode of pathname expansion should be changed in order to paper over the first point. Very well. Let's consider what would happen if, say, GLOBIGNORE=".*:-*" were to be a default. * bash would violate POSIX [1] * bash would be incompatible with other shells on this point * existing scripts would change their behaviour and/or break (note: file names beginning with a dash are perfectly legal) * the behavioural change would not conclusively address the issue What do I mean by not addressing the issue? Fundamentally, the issue is one of passing unsanitised input to an program, with the input taking the form of an argument vector. Programs are free to act upon their arguments in whatever way they see fit. While it may be common for argument beginning with a dash to denote an option, this is by no means a cast-iron rule. If the words produced by the expansion of a glob be arbitrary in nature, it is your responsibility to understand how to convey them safely to a given program. In many cases, assuming that responsibility turns out not to be overly hard. The point is that it is beyond the purview of the shell. [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 -- Kerin Millar
Re: I've found a vulnerability in bash
On Fri, 19 Nov 2021 10:05:39 -0500
Marshall Whittaker wrote:
> Fair. I'm not saying anyone has to change it, but I will call out what I
> think is a design flaw. But this is going to turn into some philosophical
> discussion as to whether it should have been done this way from the start.
> That I don't know, and hold no responsibility for, as I'm not a bash dev,
> I'm an exploit dev. Maybe an asshole too.
You appear to be missing the implication; it has nothing in particular to do
with bash. Consider the following Perl program. At no point is a shell involved.
@args = glob('*');
system('rm', '-f', @args); # bad
So, would it be Perl's fault that the programmer failed to demarcate the
non-option arguments to rm(1)? Did I just publish a zero-day? No. Absolutely
not.
system('rm', '-f', '--', @args); # better
Granted, one would normally use the native unlink function but that's besides
the point.
--
Kerin Millar
Re: I've found a vulnerability in bash
(Copying the list back in ...)
On Fri, 19 Nov 2021 17:37:54 +0100
Andreas Kusalananda Kähäri wrote:
> On Fri, Nov 19, 2021 at 03:56:21PM +0000, Kerin Millar wrote:
> > On Fri, 19 Nov 2021 10:05:39 -0500
> > Marshall Whittaker wrote:
> >
> > > Fair. I'm not saying anyone has to change it, but I will call out what I
> > > think is a design flaw. But this is going to turn into some philosophical
> > > discussion as to whether it should have been done this way from the start.
> > > That I don't know, and hold no responsibility for, as I'm not a bash dev,
> > > I'm an exploit dev. Maybe an asshole too.
> >
> > You appear to be missing the implication; it has nothing in particular to
> > do with bash. Consider the following Perl program. At no point is a shell
> > involved.
>
> I believe system() in Perl may well invoke sh -c depending on the
> arguments given. See "perldoc -f system".
Yes, but there would need to be "one scalar argument".
> > @args = glob('*');
> > system('rm', '-f', @args); # bad
At least two arguments are given there. Granted, the win32 port is an outlier
but the sample clearly isn't intended for it.
--
Kerin Millar
Re: I've found a vulnerability in bash
On Fri, 19 Nov 2021 13:40:16 -0600
Eric Blake wrote:
> On Fri, Nov 19, 2021 at 03:56:21PM +0000, Kerin Millar wrote:
> > On Fri, 19 Nov 2021 10:05:39 -0500
> > Marshall Whittaker wrote:
> >
> > > Fair. I'm not saying anyone has to change it, but I will call out what I
> > > think is a design flaw. But this is going to turn into some philosophical
> > > discussion as to whether it should have been done this way from the start.
> > > That I don't know, and hold no responsibility for, as I'm not a bash dev,
> > > I'm an exploit dev. Maybe an asshole too.
> >
> > You appear to be missing the implication; it has nothing in particular to
> > do with bash. Consider the following Perl program. At no point is a shell
> > involved.
> >
> > @args = glob('*');
> > system('rm', '-f', @args); # bad
>
> I had to double-check you via 'strace -f -e execve ...', but you are
> right, for this particular example. But according to 'perldoc -f
> system', there ARE instances where perl's system() involves a shell:
>
>Note that argument processing varies depending on the
> number of arguments. If there is more than one argument in LIST,
> or if LIST is an array with more than one value, starts the
> program given by the first element of the list with arguments
> given by the rest of the list. If there is only one scalar
> argument, the argument is checked for shell metacharacters, and
> if there are any, the entire argument is passed to the system's
> command shell for parsing (this is "/bin/sh -c" on Unix
>
> although /bin/sh is not always bash. But that brings up a bug in perl(1):
>
> $ strace -f -e execve perl -e 'system("echo \$HOME")'
> execve("/usr/bin/perl", ["perl", "-e", "system(\"echo \\$HOME\")"],
> 0x7ffc3e642e58 /* 72 vars */) = 0
> strace: Process 1248831 attached
> [pid 1248831] execve("/bin/sh", ["sh", "-c", "echo $HOME"], 0x55d3099d69d0 /*
> 72 vars */) = 0
> /home/eblake
> [pid 1248831] +++ exited with 0 +++
> --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1248831,
> si_uid=14986, si_status=0, si_utime=0, si_stime=0} ---
> +++ exited with 0 +++
>
> According to POSIX, perl should REALLY be passing a "--" argument
> between "-c" and the scalar string given by the user; see
> https://www.austingroupbugs.net/view.php?id=1440
While perl's system is not an implementation of system(3) per se, I concur.
Good find.
$ perl -E 'system("-h &")'
sh: 0: Illegal option -h
That does nobody any good. Indeed, it serves to remind me that even shebangs
can be unsafe.
$ mkdir -- -h
$ printf '%s\n' '#!/bin/sh' > -h/script
$ chmod +x -- -h/script
$ -h/script
sh: 0: Illegal option -h
That's with dash as /bin/sh. The only defense would be to write the shebang as
"#!/bin/sh --".
--
Kerin Millar
Re: Greek messages output to iOS ssh clients
On Wed, 24 Nov 2021 11:55:50 -0500 (EST) jer...@jrandall.duckdns.org wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: darwin17.7.0 > Compiler: /usr/bin/clang > Compilation CFLAGS: -pipe -Os -DSSH_SOURCE_BASHRC -arch x86_64 > uname output: Darwin imac11.local 17.7.0 Darwin Kernel Version 17.7.0: Fri > Oct 30 13:34:27 PDT 2020; root:xnu-4570.71.82.8~1/RELEASE_X86_64 x86_64 > Machine Type: x86_64-apple-darwin17.7.0 > > Bash Version: 5.1 > Patch Level: 8 > Release Status: release > > Description: > Upon ssh to bash 5.1.8 from a mobile device running iOS 15.1, error and > status messages are echoed by bash to the screen in the Greek alphabet and > language. > > Repeat-By: > From an iPad or iPhone running iOS 15, use any ssh client app to > connect to and obtain a login shell on a Mac running OS 13.6. Enter, e.g., > the following sequence: > /bin/sleep 300 & > jobs > In the output the sleep command will appear in English, but the "running" > status will be given in Greek. > Then attempt to disconnect with the mistyped command > exir > The message explaining that the command is unknown will likewise be echoed in > Greek. > > Fix: > [Description of how to fix the problem. If you don't know a > fix for the problem, don't include this section.] > I'd look at the source code that tells bash how to look up and find, on an > SSH_CLIENT running iOS, the locale key needed to select output messages for > echoing to the screen. There's nothing here that implies a bug in bash. Try running "locale" in your remote shell. The meaning of the variables printed by that command is defined by https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html. Should the effective value of LC_MESSAGES not be to your liking, you are free to change it, be it in your operating system, in your login shell environment or in your scripts. Incidentally, by default, the sshd_config(5) shipped by macOS defines "AcceptEnv LANG LC_*", meaning that it honours all of the environment variables pertaining to the locale, as conveyed by the ssh(1) client. -- Kerin Millar
Re: bash manual on interactive shell
On Sun, 12 Dec 2021 23:51:22 -0800 Mallika wrote: > Hi! > > Thanks for the quick response! And for clearing up what 'option arguments' > means. > > I'm not sure I understood the last bit, though > >Something like "with only option arguments" could easily be misunderstood > as "with one or more option arguments". > > Are you saying that the 'only' could be easily ignored? i.e. "with one or > more option arguments" is incorrect because it fails to specify that *no* > non-option arguments may be used? > Oh! You're saying the language used includes the case of no arguments at > all, whereas the language I'm proposing may be interpreted as requiring > option-arguments (which is what I had strugglingly understood it to mean) > in that case, may I simply suggest: > "started without any arguments, or with option arguments only" Just to muddy the waters further, the POSIX Utility Syntax Guidlines [1] employs a different terminology. What the bash manual refers to as a "non-option argument" is, instead, an "operand". On the other hand, an "option-argument" (note the hypen) would be defined as the argument that follows an option that expects one. For instance, in the case of `sh -c date`, the string, "date", would be an option-argument! In the absence of a formal definition, the import of any term constitutes a form of tribal knowledge. While the GNU glibc manual speaks of "non-option arguments" in its Program Argument Syntax Conventions [2], it does, unfortunately, lack the formality and rigour of the aformentioned POSIX document. Still, one can infer the meaning by also considering its first bullet point, not that I would expect anyone reading the bash manual to necessarily think of looking there. [1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html [2] https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html > > It is certainly more words, but unless you already have a solid > understanding of non-option arguments (commands?) - in which case you're > only having to perform one negation in your head - I think this > construction makes it significantly easier to follow along. > > Ultimately, of course, it's up to you. I've made as much of a case for > re-wording as I could :) You mentioned being confused by how the and's and or's combine. The wording of the bash man page seems marginally less ambiguous in that regard, partly owing to its sparse use of the comma. "An interactive shell is one started without non-option arguments (unless -s is specified) and without the -c option whose standard input and error are both connected to terminals (as determined by isatty(3)), or one started with the -i option." Although, I would say that the absence of a comma before whose is a grammatical error. In any case, I wanted to suggest the use of the word, either. For example:- "An interactive shell is either one started without non-option arguments (unless -s is specified) and without the -c option whose standard input and error are both connected to terminals (as determined by isatty(3)), or one started with the -i option." I think that the addition of this word might increase the probability of the sentence being (correctly) interpreted in the fashion of "an interactive shell is either one that fulfils ( criteria A ) or ( criteria B )", keeping in mind that the -i option always renders bash interactive. -- Kerin Millar
Re: in-line calls to functions cause "exit" in the function to act like "return"
On Sat, 18 Dec 2021 13:07:03 +
yesxorno via Bug reports for the GNU Bourne Again SHell
wrote:
> Description:
>
> When 'exit' is executed in a function called to "in-line" its output to
> stdout, the 'exit' acts like return, leaving the function scope, but not
> terminating the bash process.
It does, in fact, exit the subshell in question.
>
> Repeat-By:
>
> Sample script:
>
> #!/bin/bash
>
> set -x
>
> function bar()
> {
> exit 1
> }
>
> foo="$(bar)"
Here, the use of a command substitution causes a subshell to be created. The
command, exit 1, is not run by the shell that the kernel loaded to interpret
your script.
--
Kerin Millar
Re: bash completion mangles file names with newlines
On Wed, 23 Feb 2022 11:34:52 -0500 "Ian! D. Allen" wrote: > # Now, load the completion scripts and watch it break: > bash-5.0$ source /usr/share/bash-completion/bash_completion Issues regarding completion scripts should be reported to their respective authors, such as https://github.com/scop/bash-completion. As such, you will need to discern the offending script so as to establish its provenance. -- Kerin Millar
Re: Unfortunate error message for invalid executable
On Thu, 26 May 2022 11:27:32 -0700 AA via Bug reports for the GNU Bourne Again SHell wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -g -O2 > -fdebug-prefix-map=/build/bash-Smvct5/bash-5.0=. > -fstack-protector-strong -Wformat -Werror=format-security -Wall > -Wno-parentheses -Wno-format-security > uname output: Linux ip-172-31-20-239 5.13.0-1021-aws #23~20.04.2-Ubuntu > SMP Thu Mar 31 11:36:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-pc-linux-gnu > > Bash Version: 5.0 > Patch Level: 17 > Release Status: release > > Description: > When a user attempts to execute an executable that is not > recognized as an executable by the system, the generated error is "No > such file or directory" > > Repeat-By: > Obtain an i386 executable instead of an amd64 executable and > attempt to run it via ./name_of_executable > > Fix: > If the file exists but cannot be executed, issue an error > message that is not so misleading. The file does exist, so "No such file > or directory" seems plain wrong. While potentially confusing, it isn't wrong. Your kernel has CONFIG_IA32 compiled in so, in fact, it is considered as a valid executable. If that were not the case, it would have raised ENOEXEC (Exec format error) instead. Instead, the execve() syscall raises ENOENT (No such file or directory) because the 32-bit instance of ld-linux.so is missing on your system. You may run "man 8 ld-linux.so" to learn more about it. The reason that this file is missing in your OS is because it's not presently set up to be multilib-capable. Still, all of this is outside of the purview of the shell. What bash is doing in response to the failed syscall is to report the exact error that it raised. It doesn't seem reasonable to expect for it to do anything else. -- Kerin Millar
Re: Heredoc leading tab/space behaviour
Hi Gerold, On Fri, 24 Jun 2022 13:53:07 +0200 (CEST) "Ing. Gerold Broser" via Bug reports for the GNU Bourne Again SHell wrote: > Hello, > > Configuration Information 1: > > Machine: x86_64 > OS: linux-gnu > Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.3 > -L/home/abuild/rpmbuild/BUILD/bash-4.3/../readline-6.3 > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' > -DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL > -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -fmessage-length=0 > -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 > -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g > -D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g -Wuninitialized -Wextra > -Wno-unprototyped-calls -Wno-switch-enum -Wno-unused-variable > -Wno-unused-parameter -Wno-parentheses -ftree-loop-linear -pipe -DBNC382214=0 > -DIMPORT_FUNCTIONS_DEF=0 -fprofile-use > uname output: Linux <...host name...> 4.12.14-122.116-default #1 SMP Tue Apr > 5 13:16:39 UTC 2022 (b42b08b) x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-suse-linux-gnu > > Bash Version: 4.3 > Patch Level: 48 > Release Status: release > > Configuration Information 2 (Win 10 Enterprise, Git Bash v2.34.1): > > Machine: x86_64 > OS: msys > Compiler: gcc > Compilation CFLAGS: -DPROGRAM='bash.exe' -DCONF_HOSTTYPE='x86_64' > -DCONF_OSTYPE='msys' -DCONF_MACHTYPE='x86_64-pc-msys' -DCONF_VENDOR='pc' > -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H > -DRECYCLES_PIDS -I. -I. -I./include -I./lib -DWORDEXP_OPTION > -Wno-discarded-qualifiers -march=x86-64 -mtune=generic -O2 -pipe > -Wno-parentheses -Wno-format-security -D_STATIC_BUILD -g > uname output: MINGW64_NT-10.0-19042 <...host name...> 3.1.7-340.x86_64 > 2021-10-12 16:29 UTC x86_64 Msys > Machine Type: x86_64-pc-msys > > Bash Version: 4.4 > Patch Level: 23 > Release Status: release > > > Description: > > The Bash Reference Manual says about Heredoc's redirection operator '<<-': > > "If the redirection operator is ‘<<-’, then all leading tab characters are > stripped from input lines and the line containing delimiter." > – https://www.gnu.org/software/bash/manual/bash.html#Here-Documents > > This is only half true since apparently also leading spaces are stripped from > input lines (while they are not from the delimiter line, and lead to an error > there). They're not, though. $ printf '%s\n' 'cat <<-EOF' $'\x20\x20foo' $'\t\tbar' EOF | bash | od -a 000 sp sp f o o nl b a r nl 012 Note that the two space characters preceding foo are preserved, whereas the two leading tab characters preceding bar are not. > > > Repeat-By: > > Note > Unicode characters: > - blank (U+2423) for space (U+20) > and > - tab (U+2B73) + 7 spaces for Tab (U+9) > used for clarity in the following. > -- > > heredoc-error.sh: > > #!/bin/bash > sed -f - heredoc-error.sh <<-SCRIPT > ␣␣s/something/else/ > ⭲ s/more/else/ > ␣␣SCRIPT > > $ . heredoc.sh > bash: warning: here-document at line 2 delimited by end-of-file (wanted > `SCRIPT') > sed: file - line 3: unknown command: `S' > > > $ . heredoc-ok.sh > #!/bin/bash > sed -f - heredoc-ok.sh <<-SCRIPT > ␣␣s/else/else/ > ⭲ s/else/else/ > ⭲ SCRIPT > > > Fix: > > I think it'd be convenient if Heredoc supports indentation with spaces at the > delimiter line, too. One issue with this proposal is that it would violate the rules of the Shell Command Language, per https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04. -- Kerin Millar
Warning raised if trying to set LC_ALL to a locale bearing a character type other than UTF-8 and C
Hello,
I was experimenting with locale variables in bash and noticed something that
appears odd. This is with bash 5.1.16 and glibc 2.35.
$ locale -a | grep -E '^en_(US|GB)'
en_GB.utf8
en_US
en_US.iso88591
en_US.utf8
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
$ ( LC_ALL=en_US.ISO-8859-1 )
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.ISO-8859-1)
$ ( LC_ALL=en_GB.UTF-8 )
$ : setting LC_ALL to any of the UTF-8 locales does not raise a warning
Taking Perl as a counter-example, there is no problem issuing the equivalent
setlocale call.
$ perl -MPOSIX=locale_h -E 'if (setlocale(LC_ALL, "en_US.ISO-8859-1")) { say
"Successfully changed LC_ALL to ", setlocale(LC_ALL) }'
Successfully changed LC_ALL to en_US.ISO-8859-1
Is there a particular reason that bash appears to reject the attempt to assign
"en_US.ISO-8859-1" to LC_CTYPE?
--
Kerin Millar
Re: Revisiting Error handling (errexit)
On Fri, 8 Jul 2022 13:22:46 +0300
Oğuz wrote:
> 8 Temmuz 2022 Cuma tarihinde Yair Lenga yazdı:
> >
> > Practical Example - real life. A job has to copy 3 critical data files. It
> > then sends notification via email (non-critical).
> >
> > #! /bin/bash
> > set -o errfail
> > function copy-files {
> > # ALL files are critical, script will not cont
> > cp /new/file1 /production/path/
> > cp /new/file2 /production/path/
> > # Use the files for something - e.g., count the lines.
> > important-job /production/path/file1 /production/path/file2
> >
> > ls -l /production/path | mail -s "all-good" not...@company.com ||
> > true# Not critical
> > }
> >
> > if copy-files ; then
> > more-critical-jobs
> > echo "ALL GOOD"
> > else
> > mail -s "PROBLEM" nor...@company.com < /dev/null
> > fi
> >
> > What is the difference ? consider the case where /new/file1 does not
> > exists, which is critical error.
> > * Without errfail, an error message will be sent to script stderr, but the
> > script will continue to copy the 2nd file, and to perform the
> > important-job, even though the data is not ready.
>
>
> How is this any better than doing `cp ... && cp ... && important-job ...'?
Just to elaborate on the premise of the question, rather than address the
question itself, the sample could have been written as:
#!/bin/bash
copy_files() {
cp /new/file1 /production/path/ &&
cp /new/file2 /production/path/ &&
important-job /production/path/file1 /production/path/file2
}
if copy_files; then
ls -l /production/path | mail -s "all-good" not...@company.com
echo "ALL GOOD"
else
mail -s "PROBLEM" not...@company.com < /dev/null
false # not in the original but one probably wanted either this or exit
1
fi
Alternatively, if it is considered necessary to issue ls from within the
function:
copy_files() {
cp /new/file1 /production/path/ &&
cp /new/file2 /production/path/ &&
important-job /production/path/file1 /production/path/file2 || return
ls -l /production/path | mail -s "all-good" not...@company.com
true
}
Ultimately, which approach is easier to reason with? Consider it as an open
question.
--
Kerin Millar
Re: functions can fully unset local vars in other scopes
Hi Emanuele,
On Fri, 29 Jul 2022 03:59:07 +0200
Emanuele Torre wrote:
> Description:
> `bash' does not let `unset' fully undeclare local variables. (so that
> they can be used later as `local' variables without needing to
> redeclare them I assume.)
>
> bash-5.1$ x=abc; declare -p x; unset -v x; declare -p x
> declare -- x="abc"
> bash: declare: x: not found
>
> bash-5.1$ a () { local x=abc; local -p x; unset -v x; local -p x ;}
> bash-5.1$ a
> declare -- x="abc"
> declare -- x
>
> However, other functions are allowed to delete those variables:
>
> bash-5.1$ a () { local x=abc; b; local -p x ;}
> bash-5.1$ b () { unset -v x ;}
> bash-5.1$ a
> bash: local: x: not found
>
> This enables defininng a "really_unset" function like so:
>
> really_unset () { unset "$@" ;}
>
> Which may be useful I guess.
>
> But I think allowing functions to unset local variables from other
> functions defeats the whole purpose of having that `unset'
> behaviour. This enables `local' variable to unexpectedly become global
> after a function is called.
Though it only goes halfway towards addressing your concerns, 5.0 added a
localvar_unset option that impedes the behaviour of 'popping' from the
outermost scope.
$ bash -c 'x=foo; f1() { local x=bar; f2; x=baz; }; f2() { unset x; declare -p
x; }; f1; declare -p x'
declare -- x="foo"
declare -- x="baz"
$ bash -O localvar_unset -c 'x=foo; f1() { local x=bar; f2; x=baz; }; f2() {
unset x; declare -p x; }; f1; declare -p x'
declare -- x
declare -- x="foo"
>
> Fix:
> I think calling `unset -v x' (where `x' is a local variable not in the
> current scope) should behave as if it was called in the scope of `x',
> so `x' should remain declared in that scope with no attributes and no
> value.
>
> It may be nice to also add a "force" option for `unset' that makes it
> actually unset the variable if it is `local'. Since this could be
> useful in some cases and it won't be possible after the behaviour is
> changed.
>
--
Kerin Millar
Re: functions can fully unset local vars in other scopes
Hi Martin,
On Sat, 30 Jul 2022 11:47:01 +1000
Martin D Kealey wrote:
> On Fri, 29 Jul 2022 at 18:51, Kerin Millar wrote:
>
> > 5.0 added a localvar_unset option that impedes the behaviour of 'popping'
> > from the outermost scope.
I would have preferred for the sentence to have been quoted in full.
> >
> > $ bash -c 'x=foo; f1() { local x=bar; f2; x=baz; }; f2() { unset x;
> > declare -p x; }; f1; declare -p x'
> > declare -- x="foo"
> > declare -- x="baz"
> >
> > $ bash -O localvar_unset -c 'x=foo; f1() { local x=bar; f2; x=baz; }; f2()
> > { unset x; declare -p x; }; f1; declare -p x'
> > declare -- x
> > declare -- x="foo"
> >
>
> Whilst I have serious misgivings about the old behaviour, I find this
> approach *deeply* unsettling, introducing yet more action at a distance:
> modifying the behaviour of *existing* code by changing a global option that
> didn't even exist when the code was written. Whilst it's possible to
> localize the effect of shopt so it doesn't leak out, that's an extra step,
> making it likely to be overlooked.
>
> I'll grant that this new shopt is significantly better than simply changing
> the default behaviour and then requiring compat44 to get the old behaviour
> back (*1), and that version control is "hard", but adding yet another
> global setting seems to be going in entirely the wrong direction.
I'm inclined to agree.
>
> I'm aware that there are historical choices that later turned out to be
> suboptimal, but it seems like the better way to fix them is not to
> unwittingly change the behaviour of existing code, but rather to introduce
> new commands or options to get the new behaviours (*2).
>
> What consideration was given to creating “local --unset *VAR*” or “unset
> --local *VAR*” or some other equivalent that doesn't overload any existing
> command form?
Obviously, only Chet can meaningfully respond to this. I mentioned the
existence of the option because it appeared likely that Emanuele was unaware of
it, not to attempt to obviate any of his suggestions, nor endorse the status
quo.
--
Kerin Millar
Re: Bash-5.2-rc3 available
Hi Chet, On Fri, 26 Aug 2022 14:28:00 -0400 Chet Ramey wrote: > 1. Changes to Bash > > a. Added a compatibility mode feature that causes the parser to parse command >substitutions as if extglob were enabled. If it is enabled before > execution, >parse at execution will succeed. If not, the subsequent execution parse > will >fail. I harbour some concerns as to both the utility and robustness of this. $ declare -p BASH_VERSION declare -- BASH_VERSION="5.2.0(1)-rc3" $ shopt extglob extglob off $ [[ '' = $(shopt extglob >&2) ]] extglob on $ shopt extglob extglob on $ grep -o extglob <<<"$BASHOPTS" | wc -c 0 For extglob to be arbitrarily enabled in either context is unexpected (and undesirable). For BASHOPTS not to be in agreement with the output of shopt is also unexpected. I presume that this is as a consequence of the above-mentioned change. Paradoxically, this breaks one of the QA tests implemented by portage. I say paradoxically because I'm aware that Gentoo requested a concession pertaining to the relationship between the extglob option and the newly recursive parser. To put it in perspective, it transpired that only one ebuild (package) in its 20620-strong repo was affected and that none of the portions of portage that are written in bash were affected (until now). My concern is that these ostensible concessions to backward-compatibility will end up causing more problems than will be solved. -- Kerin Millar
Re: Bash-5.2-rc3 available
On Mon, 29 Aug 2022 09:53:39 -0400
Chet Ramey wrote:
> On 8/26/22 9:29 PM, Kerin Millar wrote:
> > Hi Chet,
> >
> > On Fri, 26 Aug 2022 14:28:00 -0400
> > Chet Ramey wrote:
> >
> >> 1. Changes to Bash
> >>
> >> a. Added a compatibility mode feature that causes the parser to parse
> >> command
> >> substitutions as if extglob were enabled. If it is enabled before
> >> execution,
> >> parse at execution will succeed. If not, the subsequent execution
> >> parse will
> >> fail.
> >
> > I harbour some concerns as to both the utility and robustness of this.
>
> Thanks for the report. This was an issue with both constructs using the
> same mechanism for parser error recovery and colliding, an easy fix.
Good! That's certainly a relief.
>
>
> > For extglob to be arbitrarily enabled in either context is unexpected (and
> > undesirable).
>
> The conditional command behavior is compatible with ksh93, and has been
> this way for nearly twenty years. It's documented to behave that way. One
> would think people are used to it by now.
I am aware of how the == and != operators work, and that the pattern on the
right hand side is always treated as an extglob in the shell performing the
test [*]. I don't see what bearing that has on my report or its wording. Might
I enquire as to what it is that I am supposed to be used to on account of the
passage of those twenty years? Certainly not for a command substitution on the
right hand-side of the == operator to implicitly perform the equivalent of
shopt -s extglob within the implied subshell without so much as a by-your-leave
on the part of the programmer, much less doing so in the shell that specified
the command substitution. To make that observation was neither contentious, nor
indicative of a need to study the manual or the CHANGES file more astutely.
For the benefit of anyone else reading, the following amounts to an error in
bash 5.1.16, as it should.
$ shopt extglob
extglob off
$ [[ '' == $( : !() ) ]]
bash: command substitution: line 3: syntax error near unexpected token `('
bash: command substitution: line 3: ` : !() )'
>
> The parser change to enable this for backwards compatibility is controlled
> by the compatibility mode, and requires an explicit action to enable. It's
> a choice to enable compatibility mode, and choices have consequences;
> compatibility mode is intended to provide an opportunity to update scripts,
> not provide perfect emulation of a previous version.
I understand that you are striving to do right by a majority of your users at
all times. The reported issue was a consequence of a regression in a late-stage
release candidate introduced by a new backward compatibility feature that I had
not elected to enable. I found this regression somewhat startling. I hope that
you can understand where my concern stems from, at least.
>
> I understand from Sam James that Gentoo globally forces the compatibility
> mode to 5.0. That, too, is a choice.
Yes, that's right - subject to the "EAPI" level defined by a given ebuild. I
would aver that all of this amounted to a storm in a teacup. In only one single
instance was a pathname expansion found that was incorrectly attempting to
enable the extglob option after parsing has occurred, as was learned in the
course of Sam and I proceeding to work together on this issue. I must emphasise
that it's not that I don't think that backward compatibility is important. Far
from it. Yet, I can't shake the feeling I had from the outset that, in this
particular instance, the compromise may simply not be worth it. I considered
the behaviour of the prior release candidates to be a step in the right
direction. In any case, I have already expressed as much so I'll leave it at
that and shall look forward to the next release candidate, should there be one.
>
> > Paradoxically, this breaks one of the QA tests implemented by portage.
>
> It's certainly ironic.
In case you were wondering, it's a test that entails stashing the output of
shopt then later comparing, so as to determine whether an ebuild/eclass changed
any options without restoring their prior values. Coercing extglob on results
in a constant stream of false positives.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
[*] Notwithstanding the change introduced by 4.1, whereby it was no longer
necessary to either enable extglob first or store the pattern in a variable
that was subsequently expanded on the right hand side of the operator.
--
Kerin Millar
Re: Bash-5.2-rc3 available
On Wed, 31 Aug 2022 10:55:58 -0400
Chet Ramey wrote:
> On 8/29/22 12:28 PM, Kerin Millar wrote:
>
> >>> For extglob to be arbitrarily enabled in either context is unexpected
> >>> (and undesirable).
> >>
> >> The conditional command behavior is compatible with ksh93, and has been
> >> this way for nearly twenty years. It's documented to behave that way. One
> >> would think people are used to it by now.
> >
> > I am aware of how the == and != operators work, and that the pattern on the
> > right hand side is always treated as an extglob in the shell performing the
> > test [*]. I don't see what bearing that has on my report or its wording.
>
> OK. You wrote "either context" above. There are two contexts in which
> extglob is enabled internally: conditional commands and (now) command
> substitutions. The former has been the case for a long time; the latter
> is new.
>
> How am I supposed to interpret "either context?" A reasonable person could
> assume that you're objecting to established behavior.
I disagree. The sample clearly demonstrated extglob unexpectedly being turned
on in two contexts, that of the initial shell and that of the subshell.
Nevertheless, I shall attempt to be more specific next time.
>
> Might I enquire as to what it is that I am supposed to be used to on
> account of the passage of those twenty years? Certainly not for a command
> substitution on the right hand-side of the == operator to implicitly
> perform the equivalent of shopt -s extglob within the implied subshell
> without so much as a by-your-leave on the part of the programmer, much less
> doing so in the shell that specified the command substitution.
>
>
> Wow, that was quite a sentence. In any event, you should report it, as you
> did.
>
> >
> > For the benefit of anyone else reading, the following amounts to an error
> > in bash 5.1.16, as it should.
> >
> > $ shopt extglob
> > extglob off
> > $ [[ '' == $( : !() ) ]]
> > bash: command substitution: line 3: syntax error near unexpected token `('
> > bash: command substitution: line 3: ` : !() )'
>
> It will produce exactly the same error in bash-5.2. On the other hand,
Very good. In the meantime, I tried -rc2 and saw that it behaves as expected
there.
> there are valid constructs that bash-5.1 did not handle, and no amount of
> coercion could make it do so. I included one in a previous message thread.
Interesting. I will look for this thread.
>
> The ad-hoc comsub parser in previous versions didn't understand extglob
> patterns at all, period, whether or not extglob was enabled. That resulted
> in it accepting bad code, invalid constructs that later resulted in errors,
> and valid patterns that were later accepted when the command was executed,
> as in a shell function definition. The current compatibility mode behavior
> is a concession to that previous reality. The problem is that neither you
> nor I know how many instances exist.
Fair enough.
>
> In this case, the backwards compatibility setting doesn't matter, of course --
> parsing the word on the rhs of the ==/!= operator forces extglob on, and
> the command substitution parsing inherits that. That is not new behavior.
Indeed it is not. I find "forces extglob on" to be a loaded phrase, however.
$ shopt extglob
extglob off
$ [[ a == @(a|b) ]] # the rhs is treated as if extglob were enabled
$ shopt extglob
extglob off # I wouldn't refer to this outcome as extglob being forced
on; it's still off
In my estimation, it makes more sense to say that the rhs of ==/!== is taken to
be an extglob, irrespective of the state of the extglob shopt. Indeed, the
manual states that it behaves "[...] as if the extglob shell option were
enabled", which makes perfect sense. If it were written as "[...] forcing the
extglob option on" instead, it would be misleading, wouldn't it? On the other
hand, to say that a comsub as the rhs of ==/!= forces extglob on in 5.2-rc3
rather captures the essence of my report.
>
>
> > I understand that you are striving to do right by a majority of your users
> > at all times. The reported issue was a consequence of a regression in a
> > late-stage release candidate introduced by a new backward compatibility
> > feature that I had not elected to enable.
>
> Yes, it's good that you reported it. There will probably be another release
> candidate as a result.
>
> >>
> >> I understand from Sam James that Gentoo globally forces the compatibility
> >> mode to 5.0. That, too, is a choice.
> >
extglob can be erroneously enabled in bash-5.2 through comsub nesting
Hi, I am writing to report that the 5.2 release does not correctly address the regressions introduced by 5.2-rc3 (see https://lists.gnu.org/archive/html/bug-bash/2022-08/msg00104.html for the context). Given a sufficient degree of command substitution nesting, it remains possible for the extglob shell option to be switched on, without having instructed bash to do so. $ declare -p BASH_VERSION declare -- BASH_VERSION="5.2.0(1)-release" $ BASH_COMPAT=50; shopt extglob; : $(: $(: $(:))); shopt extglob extglob off extglob on When the bug occurs, all levels are affected, as is demonstrated by the attached script. See, also, the original bug report at https://bugs.gentoo.org/873931. -- Kerin Millar gentoo-bug-873931-reproducer-r1.bash Description: Binary data
Re: extglob can be erroneously enabled in bash-5.2 through comsub nesting
On Thu, 6 Oct 2022 15:49:26 -0400 Chet Ramey wrote: > On 10/2/22 4:51 AM, Kerin Millar wrote: > > > $ declare -p BASH_VERSION > > declare -- BASH_VERSION="5.2.0(1)-release" > > $ BASH_COMPAT=50; shopt extglob; : $(: $(: $(:))); shopt extglob > > extglob off > > extglob on > > Thanks for the report. I've attached the patch I applied to fix this. Thanks for the patch. It is probably sufficient for the downstream bug report to be closed. Unfortunately, it remains the case that the >=5.2-rc3 parser is buggy. Consider the following, as conducted using 5.2.2 with said patch applied. $ declare -p BASH_VERSION declare -- BASH_VERSION="5.2.2(1)-release" $ BASH_COMPAT=50 $ [[ foo = $(: $(shopt extglob >&2)) ]] extglob off $ shopt extglob extglob off $ [[ foo = $(: $(shopt extglob >&2) ]] > ^C $ shopt extglob extglob on Note that, within the second test, the comsub parentheses are deliberately imbalanced, causing bash to display the PS2 prompt and wait for further input. I then interrupt bash with ^C and check on the status of the extglob option, only to find that - yet again - it has been unexpectedly enabled. This is perfectly reproducible, provided that those exact steps are carried out as shown. In particular, one cannot skip the first (syntactically correct) test, which must nest at least one comsub within another. -- Kerin Millar
Re: Reportting a BUG in Heredoc
On Tue, 25 Oct 2022 19:32:33 + Abla OUHAGA wrote: > From: abouhaga > To: bug-bash@gnu.org > Subject: Reporting a BUG in Heredoc. > > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: darwin18.7.0 > Compiler: clang > Compilation CFLAGS: -DSSH_SOURCE_BASHRC > uname output: Darwin e2r9p9.1337.ma 18.7.0 Darwin Kernel Version 18.7.0: Tue > Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64 > Machine Type: x86_64-apple-darwin18.7.0 > > Bash Version: 5.2 > Patch Level: 2 > Release Status: release > > Description: > While working to make my own shell that reproduces the behavior of > bash for some cases, I tested my work and compared it to the output of Bash. > For Heredoc I gave it different delimiters as test cases, then I encountered > a Bug for the case of ( << "$USER ) as input. > > Repeat-By: > For the described case above, the heredoc never gets out with any > given delimiter. I do understand that I should close the double quote first > in order to get in the heredoc maybe, But I find it ambiguous. I spent hours > trying so many delimiters but it never gets out of it unless I enter CTRL+D > (EOF). I also upgraded my BASH to check if it's the same behavior in the > latest version, and I encounter the same BUG. based on that I did not know > how to implement this test case in my own shell. Finally, that is my reason > for reporting it. > Thank you. > > Fix: > I guess that the correct behavior is to give a prompt to close the > double quote, then proceed to close the heredoc with an unquoted delimiter. > > PS: It is the case for a single quote too. Below is a test case that concretely demonstrates the issue. The warning implies that the delimiter is correctly taken to be $'EOF\n', yet bash does not recognise the delimiter at the point that it occurs. $ printf '%s\n' 'cat <<"EOF' '"' line EOF '' | bash bash: line 5: warning: here-document at line 2 delimited by end-of-file (wanted `EOF ') line EOF $ For comparative purposes, I tried dash-0.5.11.5, ksh-2020 and busybox-1.34.1 (ash), all of which appear to get it right. $ printf '%s\n' 'cat <<"EOF' '"' line EOF '' | dash line $ $ printf '%s\n' 'cat <<"EOF' '"' line EOF '' | ksh line $ $ printf '%s\n' 'cat <<"EOF' '"' line EOF '' | busybox sh line $ -- Kerin Millar
Re: Reportting a BUG in Heredoc
On Wed, 26 Oct 2022 11:03:57 +0700 Robert Elz wrote: > There is no bash bug here, though it could perhaps be a little > kinder to users who make this simple mistake. > > The rules for heredocs say that the end delimiter is a line. So they do. Somehow, I missed that upon my reading. > A line cannot contain an embedded newline - then it would be > two lines. > > The here doc end delimiter given after the << redirect operator > is just a shell word. Those can contain embedded newlines, either > by having an unbalanced number of unquoted single or double quote > characters, or by using $'\n' to create one. > > If the user gives such a heredoc delim after the << they have put > themselves in a situation where it is impossible to ever end > the here doc. That is until the input stream ends (EOF). > > bash could issue a warning when this occurs, rather than just > the one new versions issue when EOF is reached without the end > delimiter being located (which can happen for other reasons as well > of course). It has chosen not to. Yes, it could. While not unwelcome, the present warning came across as confusing because it expressed the would-be delimiter verbatim, newline and all. It purports to "want" something that cannnot susbequently be detected. > > Shells are allowed to extend the standard - bash has many of those > extensions. Other shells do various things when encountering a > here doc delimiter with an embedded \n - one is to look for a multi > line delimiter to end the here doc, the shell could also simply > truncate the expected end delim at the first embedded newline. > Or treat this as a syntax error. Or almost anything else. In this > case bash has chosen not to extend the standard at all - if the user > gives an end delim it is impossible to find, bash just looks for it > anyway. > > If the intent is to duplicate bash exactly (which I would not > recommend, some of what bash does is historical, and has little > tporecommend it) then this would be something that would have to > be copied. Doing so is pointless however, no real scripts, as > distinct from test cases, are going to be using a here doc > delim with embedded newlines, so this would be a case to do better. Indeed. Thanks for the clarification. -- Kerin Millar
Re: Bash-5.2 official patch 9
On Tue, 8 Nov 2022 09:50:51 -0500 Chet Ramey wrote: >BASH PATCH REPORT >= > > Bash-Release: 5.2 > Patch-ID: bash52-009 Are there any plans to backport the "fixes for extended glob in compat mode" in the near future? -- Kerin Millar
Re: Regression in pattern substitution with compat42
On Fri, 10 Feb 2023 11:32:54 +1300 (NZDT) Martin D Kealey wrote: > On 8 Feb 2023 at 18:50Z Tom Briden wrote: > > Bash Version: 5.2 > > Patch Level: 15 > > Release Status: release > > > > As of version 5.2-beta, replacing a single backslash with a double > > backslash is no longer possible when using BASH_COMPAT=4.2. > > On 9 Feb 2023 at 15:57Z Chet Ramey wrote: > > shopt -u patsub_replacement > > Surely that should be automatic with compat51 and earlier, since it's a > breaking change to previous behaviour? Somewhat related is https://bugs.gentoo.org/881383. -- Kerin Millar
Re: Regression in pattern substitution with compat42
On Wed, 15 Feb 2023 12:48:10 +1000 Martin D Kealey wrote: > > There are still Linux versions shipping with bash-4.2, which was released > > in 2010. > > > > Is anyone surprised by this, when new versions cannot be trusted? It is, perhaps, telling that two of the most notable rolling Linux distributions continue to hold off from issuing 5.2 in their stable channels. That includes Arch, the very same Arch that routinely performs same-day releases from upstream without any semblance of QA - including for glibc - and generally doesn't bother to backport any critical fixes between upstream releases. > So if I make "Bash6" as a fork of Bash 5.1, with the following guarantees, > would anyone be interested in adopting it? > It seems like a lot to take on but count me in as being interested. -- Kerin Millar
On the behaviour of the checkwinsize shopt from within a subshell
I am writing to report that the documented behaviour of the checkwinsize shopt
can be impeded by the use of the subshell.
Firstly, some preliminaries.
$ declare -p BASH_VERSION
declare -- BASH_VERSION="5.2.15(1)-release"
$ shopt checkwinsize
checkwinsizeon
$ sz() { /bin/true; declare -p COLUMNS LINES; }
In both of the following two cases, I manually resized the terminal window
during the time that bash was waiting for sleep to exit.
$ sz; sleep 3; sz
declare -- COLUMNS="86"
declare -- LINES="29"
declare -- COLUMNS="95"
declare -- LINES="34"
In this case, the values of the variables were updated as expected.
$ ( sz; sleep 3; sz )
declare -- COLUMNS="95"
declare -- LINES="34"
declare -- COLUMNS="95"
declare -- LINES="34"
$ declare -p COLUMNS LINES
declare -- COLUMNS="85"
declare -- LINES="31"
In this case, the values of the variables were not updated from within the
subshell, only the initial shell. I consider this to be surprising because the
manual does not mention any such limitation. Consequently, the utility of the
checkwinsize feature suffers in terms of being a potential alternative to the
invocation of stty(1) with the "size" operand (an operand which will eventually
be standard, but isn't yet).
--
Kerin Millar
Re: On the behaviour of the checkwinsize shopt from within a subshell
On Tue, 21 Feb 2023 15:44:34 -0500 Chet Ramey wrote: > On 2/21/23 4:18 AM, Kerin Millar wrote: > > I am writing to report that the documented behaviour of the checkwinsize > > shopt can be impeded by the use of the subshell. > > It hasn't worked that way since at least bash-3.2, when I quit looking, so > at least 16+ years. I'll take it as a new feature request and take a look. Thanks. -- Kerin Millar
Re: Behaviour of an empty conditional expression is undefined?
On Thu, 23 Feb 2023 16:26:43 +0800 (CST) "Qingsheng Huang" wrote: > Here shows the behavious of empty conditional expressions: > > > infinite:~# [[ ]] > -bash: syntax error near unexpected token `]]' > infinite:~# [[ ]] > infinite:~# [[ "$A" ]] > -bash: syntax error near unexpected token `"$A"' > infinite:~# [[ "$A" ]] > infinite:~# [[ "$A" ]] > infinite:~# [[ ]] > infinite:~# [[ "$A" ]] > -bash: syntax error near unexpected token `"$A"' > infinite:~# echo $? > 2 > infinite:~# [[ "$A" ]] > infinite:~# echo $? > 1 > infinite:~# > infinite:~# bash --version > bash --version > GNU bash, version 5.2.15(1)-release (x86_64-pc-linux-gnu) > > > The variable $A has never been defined. > Double brackets surrounding 1) a few spaces or 2) an empty variable even > quoted behave illogically. I've also try different lengths of spaces, and > also an empty variable without quotes, the same illogical. > > I am curious if I am the first guy encounter such a situation. Well observed. I can confirm this strange behaviour. I just want to add that 5.1.16 behaves sensibly, with [[ ]] always raising an error in an interactive shell. -- Kerin Millar
Re: Behaviour of an empty conditional expression is undefined?
On Thu, 23 Feb 2023 11:07:52 +0100 alex xmb ratchev wrote: > i [[ daily , never had such.. This observation isn't helpful unless you account for which version of bash you are using. In any case, the report is accurate and it describes a post-5.1 regression. -- Kerin Millar
Re: IFS field splitting doesn't conform with POSIX
On Thu, 30 Mar 2023 07:51:59 -0600
Felipe Contreras wrote:
> On Thu, Mar 30, 2023 at 5:23 AM Greg Wooledge wrote:
> >
> > On Thu, Mar 30, 2023 at 05:12:46AM -0600, Felipe Contreras wrote:
> > > IFS=,
> > > str='foo,bar,,roo,'
> > > printf '"%s"\n' $str
> > >
> > > There is a discrepancy between how this is interpreted between bash
> > > and zsh: in bash the last comma doesn't generate a field and is
> > > ignored,
> >
> > ... which is correct according to POSIX (but not sensible).
> >
> > > in zsh a last empty field is generated. Initially I was going
> > > to report the bug in zsh, until I read what the POSIX specification
> > > says about field splitting [1].
> >
> > You seem to have misinterpreted whatever you read.
> >
> > https://mywiki.wooledge.org/BashPitfalls#pf47
> >
> > Unbelievable as it may seem, POSIX requires the treatment of IFS as
> > a field terminator, rather than a field separator. What this means
> > in our example is that if there's an empty field at the end of the
> > input line, it will be discarded:
> >
> > $ IFS=, read -ra fields <<< "a,b,"
> > $ declare -p fields
> > declare -a fields='([0]="a" [1]="b")'
> >
> > Where did the empty field go? It was eaten for historical reasons
> > ("because it's always been that way"). This behavior is not unique
> > to bash; all conformant shells do it.
>
> If you think in terms of terminators instead of separators, then the
> above code makes sense because if you add ',' at the end of each field
> (terminate it), you get the original string:
>
> printf '%s,' ${fields[@]}
>
> But you can't replicate 'a,b' that way, because b does not have a
> terminator. Obviously we'll want 'b' as a field, therefore one has to
> assume either 1) the end of the string is considered an implicit
> terminator, or 2) the terminator in the last field is optional.
> Neither of these two things is specified in POSIX.
>
> If we consider 1) the end of the string is considered an implicit
> terminator, then 'a' contains a valid field, but then 'a,' contains
> *two* fields. Making these terminators indistinguishable from
> separators.
>
> We can go for 2) of course, but this is not specified anywhere in
> POSIX, that's just common practice.
You may find these interesting; the second link in particular.
- https://lists.gnu.org/archive/html/bug-bash/2006-12/msg00033.html
- https://lists.gnu.org/archive/html/bug-bash/2006-12/msg00035.html
- http://std.dkuug.dk/JTC1/SC22/WG15/docs/rr/9945-2/9945-2-98.html
Though I was aware of these behaviours, I do find the POSIX wording to be
unclear as concerns the observations made by the second link, to say the least.
I would add that it is possible to have it both ways, so to speak, though the
means of going about it are no less confusing than the topic at large.
$ IFS=,
$ str="a,b"
$ arr=($str""); declare -p arr
declare -a arr=([0]="a" [1]="b")
$ str="a,b,"
$ arr=($str""); declare -p arr # duly coercing an empty field that some may
expect or wish for
declare -a arr=([0]="a" [1]="b" [2]="")
--
Kerin Millar
Re: IFS field splitting doesn't conform with POSIX
On Thu, 30 Mar 2023 11:52:06 -0600 Felipe Contreras wrote: > Chet wrote: > > Alternately, you can think of the NUL at the end of the string as an > > additional field terminator, > > Except if you do that, then 'a,' has two fields since the end of the > string is an additional field terminator, as I explained. > > > but one that follows the adjacency rules and doesn't create any empty > > fields. > > So it's a *very special* field terminator that is mentioned nowhere in > the POSIX specification. I can only suggest issuing a formal request for clarification. Clearly, there exists a prevailing consenus across implementations (bash included). For the matter not to be broached by the specification - at least, not by my reading - seems irregular. -- Kerin Millar
Re: IFS field splitting doesn't conform with POSIX
On Sat, 1 Apr 2023 19:44:10 -0400 Saint Michael wrote: > There is an additional problem with IFS and the command read > > Suppose I have variable $line with a string "a,b,c,d" > IFS=',' read -r x1 <<< $line > Bash will assign the whole line to x1 > echo $x1 > line="a,b,c,d";IFS=',' read -r x1 <<< $line;echo $x1; > a,b,c,d > but if I use two variables > line="a,b,c,d";IFS=',' read -r x1 x2 <<< $line;echo "$x1 ---> $x2"; > a ---> b,c,d > this is incorrect. If IFS=",", then a read -r statement must assign the No it isn't. > first value to the single variable, and disregard the rest. No it musn't. Read https://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html and pay particular attention to the definition of what must happen where there are fewer vars (names) than fields encountered. -- Kerin Millar
Re: IFS field splitting doesn't conform with POSIX
On Sun, 02 Apr 2023 09:32:20 +0700 Robert Elz wrote: > Date:Sat, 1 Apr 2023 19:44:10 -0400 > From:Saint Michael > Message-ID: > > > | The compelling reason is: I may not know how many values are stored in the > | comma-separated list. > > Others have told you you're wrong, but this is not any kind of compelling > reason - you simply give one more variable name than you expected to need > (than you would have used otherwise) and then all the extra fields that > you wanted the shell to ignore will be assigned to it - which you are free > to ignore if you like, or you can test to see if anything is there, and > issue an error message (or something) if more fields were given than you > were expecting. Much better behaviour than the shell simply ignoring > data (silently). I would add to this that bash affords one the luxury of using read -a before proceeding to check the size of the resulting array. -- Kerin Millar
5.2 fails to treat a missing conditional expression as an error of syntax
As below. $ bash -c 'declare -p BASH_VERSION' declare -- BASH_VERSION="5.2.15(1)-release" $ bash -c '[[ ]]; echo fin'; echo $? 0 $ bash -c 'declare -p BASH_VERSION' declare -- BASH_VERSION="5.1.16(1)-release" $ bash -c '[[ ]]; echo fin'; echo $? bash: -c: line 1: syntax error near `;' bash: -c: line 1: `[[ ]]; echo fin' 2 The development branch behaves as 5.1 does. Being uncertain as to whether this is a result of the issue having been identified prior, I determined that it was worth reporting. -- Kerin Millar
Re: 5.2 fails to treat a missing conditional expression as an error of syntax
On Fri, 16 Jun 2023 14:25:54 -0400 Chet Ramey wrote: > On 6/15/23 8:57 PM, Kerin Millar wrote: > > As below. > > > > $ bash -c 'declare -p BASH_VERSION' > > declare -- BASH_VERSION="5.2.15(1)-release" > > $ bash -c '[[ ]]; echo fin'; echo $? > > 0 > > https://lists.gnu.org/archive/html/bug-bash/2022-10/msg00103.html > describes the fix.1 > > We had this same discussion back in February: > > https://lists.gnu.org/archive/html/bug-bash/2023-02/msg00156.html Thanks. > > where I answered in more detail: > > https://lists.gnu.org/archive/html/bug-bash/2023-02/msg00163.html > > You participated. Good heavens. This entirely slipped my mind. -- Kerin Millar
Re: Alias in command substitution
On Wed, 21 Jun 2023 13:40:56 +0530 LitHack wrote: > Normally declaring a alias inside the command substitution would not > reflect in main shell like > lithack@aura:~$ `alias l=ls` > lithack@aura:~$ l > l: command not found > But using this command . > lithack@aura:~$ `alias l=ls;alias` > lithack@aura:~$ l > api Desktop Downloads Music __py__ v v.zip > Arjun dir idafree82_linux.run Pictures snap vi > BurpSuiteCommunity Documents 'IDA Freeware 8.2.desktop' Public > Templates Videos > It runs in main shell. Is this a bug? No. This is what happens. 1) an alias is declared in the subshell incurred by the `command substitution` 2) the alias builtin is executed, which prints out a declaration of the just-declared alias 3) the command substitution causes that which was just printed to be evaluated in the original shell 4) an alias is thus declared in the original shell As Lawrence advised in a different thread, such questions would be more appropriately posed at the help-bash list. -- Kerin Millar
Re: Alias in command substitution
On Wed, 21 Jun 2023 14:11:28 +0530 LitHack wrote: > But why this not worked in zsh. I don't use zsh. A fairly plausible explanation would be that its alias builtin ignores the request to declare an alias, if issued in a subshell. -- Kerin Millar
Re: Alias in command substitution
On Wed, 21 Jun 2023 09:01:20 -0400 Greg Wooledge wrote: > On Wed, Jun 21, 2023 at 09:54:30AM +0100, Kerin Millar wrote: > > On Wed, 21 Jun 2023 14:11:28 +0530 > > LitHack wrote: > > > > > But why this not worked in zsh. > > > > I don't use zsh. A fairly plausible explanation would be that its alias > > builtin ignores the request to declare an alias, if issued in a subshell. > > I don't normally use it either, so let's try it and see: > > unicorn:~$ zsh > greg@unicorn ~ % alias > which-command=whence > greg@unicorn ~ % (alias) > which-command=whence > greg@unicorn ~ % (alias x=date; alias) > which-command=whence > x=date > > The alias definition occurs in the subshell, but the output of the "alias" > command differs from bash's. Running zsh's "alias" output as a command > would only give you a variable assignment, not an alias assignment. Cf. > bash: > > unicorn:~$ alias > alias cls='tput clear; tput cup 99 0' > [...] > > Bash's output includes the "alias" command name before the x=y bits. > Zsh's outpt does not. Ah. I had to look it up to be certain but it transpires that this is the POSIXLY correct thing to do, so to speak. Bash does the same with -o posix. -- Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Mon, 26 Jun 2023 17:09:47 +1000
Martin D Kealey wrote:
> Hi Eli
>
> How about using the shell itself to parse the output of "typeset" (an alias
> for "declare"), but redefining "declare" to do something different. This is
> a bit verbose but it works cleanly:
>
> ```
> (
> function declare {
> while [[ $1 = -* ]] ; do shift ; done
> printf %s\\n "${@%%=*}"
> }
> eval "$( typeset -p )"
> )
> ```
Unfortunately, this is defective.
$ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval "declare -a
BASH_ARGC=()"'; echo $?
1
In fact, bash cannot successfully execute the output of declare -p in full.
$ declare -p | grep BASH_ARGC
declare -a BASH_ARGC=([0]="0")
$ declare -a BASH_ARGC=([0]="0"); echo $? # echo is never reached
While it is understandable that an attempt to assign to certain shell variables
would be treated as an error, the combination of not printing a diganostic
message and inducing a non-interactive shell to exit is rather confusing.
Further, declare is granted special treatment, even after having been defined
as a function (which might be a bug).
$ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval "declare -a
BASH_ARGC=()"'; echo $?
1
$ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval "declare -a
BASH_ARG=()"'; echo $?
BASH_ARG
0
$ bash -c 'f() { shift; printf %s\\n "${1%%=*}"; }; eval "f -a BASH_ARGC=()"';
echo $?
bash: eval: line 1: syntax error near unexpected token `('
bash: eval: line 1: `f -a BASH_ARGC=()'
2
$ bash -c 'f() { shift; printf %s\\n "${1%%=*}"; }; eval "f -a BASH_ARG=()"';
echo $?
bash: eval: line 1: syntax error near unexpected token `('
bash: eval: line 1: `f -a BASH_ARG=()'
2
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Mon, 26 Jun 2023 11:47:21 +0200
alex xmb ratchev wrote:
> On Mon, Jun 26, 2023, 00:25 Eli Schwartz wrote:
>
> > compgen is a useful builtin for inspecting information about the shell
> > context e.g. in scripts -- a good example of this is compgen -A function
> > or compgen -A variable.
> >
> > But it's not always available depending on how bash is built, which
> > results in people lacking confidence that it can / should be used in
> > scripts. See e.g. https://bugs.gentoo.org/909148
>
>
> i read kerin s try to parse declare -p
>
> for me it works ..
>
> bash-5.2$ { FOO=$'\nBAR BAZ QUUX='; declare -p; } |
> while IFS=' =' read -r _ _ name _; do printf '%s\n' "$name"; done |
> grep QUUX
> bash-5.2$ echo $?
> 1
> bash-5.2$
Yes. That's because you're using 5.2, which improves the quoting strategy for
declare/typeset. Unfortunately, it's not something that I can rely upon in
production (not least, because I can't yet assume that everyone is using bash
>=5.2).
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Mon, 26 Jun 2023 11:51:58 +0200
alex xmb ratchev wrote:
> On Mon, Jun 26, 2023, 11:33 Kerin Millar wrote:
>
> > On Mon, 26 Jun 2023 17:09:47 +1000
> > Martin D Kealey wrote:
> >
> > > Hi Eli
> > >
> > > How about using the shell itself to parse the output of "typeset" (an
> > alias
> > > for "declare"), but redefining "declare" to do something different. This
> > is
> > > a bit verbose but it works cleanly:
> > >
> > > ```
> > > (
> > > function declare {
> > > while [[ $1 = -* ]] ; do shift ; done
> > > printf %s\\n "${@%%=*}"
> > > }
> > > eval "$( typeset -p )"
> > > )
> > > ```
> >
> > Unfortunately, this is defective.
> >
> > $ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval "declare -a
> > BASH_ARGC=()"'; echo $?
> > 1
> >
> > In fact, bash cannot successfully execute the output of declare -p in full.
> >
> > $ declare -p | grep BASH_ARGC
> > declare -a BASH_ARGC=([0]="0")
> > $ declare -a BASH_ARGC=([0]="0"); echo $? # echo is never reached
> >
> > While it is understandable that an attempt to assign to certain shell
> > variables would be treated as an error, the combination of not printing a
> > diganostic message and inducing a non-interactive shell to exit is rather
> > confusing. Further, declare is granted special treatment, even after having
> > been defined as a function (which might be a bug).
> >
> > $ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval "declare -a
> > BASH_ARGC=()"'; echo $?
> > 1
> >
> > $ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval "declare -a
> > BASH_ARG=()"'; echo $?
> > BASH_ARG
> > 0
> >
> > $ bash -c 'f() { shift; printf %s\\n "${1%%=*}"; }; eval "f -a
> > BASH_ARGC=()"'; echo $?
> > bash: eval: line 1: syntax error near unexpected token `('
> > bash: eval: line 1: `f -a BASH_ARGC=()'
> > 2
> >
> > $ bash -c 'f() { shift; printf %s\\n "${1%%=*}"; }; eval "f -a
> > BASH_ARG=()"'; echo $?
> > bash: eval: line 1: syntax error near unexpected token `('
> > bash: eval: line 1: `f -a BASH_ARG=()'
> > 2
> >
>
> you forgot
> see u cmd foo bar=()
> u still need as always escape ( and )
I didn't forget anything. Martin's proposal was intended to work by evaluating
the unmodified output of typeset -p. That ( and ) normally need to be escaped
simply demonstrates further that it is untenable as a solution.
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Mon, 26 Jun 2023 11:55:03 +0200 alex xmb ratchev wrote: > by the way , what does this nonsense code supposed to do > separate between vars for future list / deletement ? Print all declared variable names reliably (which it doesn't). It's not as easy as you might think, by the way. -- Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Mon, 26 Jun 2023 12:03:47 +0200
alex xmb ratchev wrote:
> On Mon, Jun 26, 2023, 12:01 Kerin Millar wrote:
>
> > On Mon, 26 Jun 2023 11:51:58 +0200
> > alex xmb ratchev wrote:
> >
> > > On Mon, Jun 26, 2023, 11:33 Kerin Millar wrote:
> > >
> > > > On Mon, 26 Jun 2023 17:09:47 +1000
> > > > Martin D Kealey wrote:
> > > >
> > > > > Hi Eli
> > > > >
> > > > > How about using the shell itself to parse the output of "typeset" (an
> > > > alias
> > > > > for "declare"), but redefining "declare" to do something different.
> > This
> > > > is
> > > > > a bit verbose but it works cleanly:
> > > > >
> > > > > ```
> > > > > (
> > > > > function declare {
> > > > > while [[ $1 = -* ]] ; do shift ; done
> > > > > printf %s\\n "${@%%=*}"
> > > > > }
> > > > > eval "$( typeset -p )"
> > > > > )
> > > > > ```
> > > >
> > > > Unfortunately, this is defective.
> > > >
> > > > $ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval
> > "declare -a
> > > > BASH_ARGC=()"'; echo $?
> > > > 1
> > > >
> > > > In fact, bash cannot successfully execute the output of declare -p in
> > full.
> > > >
> > > > $ declare -p | grep BASH_ARGC
> > > > declare -a BASH_ARGC=([0]="0")
> > > > $ declare -a BASH_ARGC=([0]="0"); echo $? # echo is never reached
> > > >
> > > > While it is understandable that an attempt to assign to certain shell
> > > > variables would be treated as an error, the combination of not
> > printing a
> > > > diganostic message and inducing a non-interactive shell to exit is
> > rather
> > > > confusing. Further, declare is granted special treatment, even after
> > having
> > > > been defined as a function (which might be a bug).
> > > >
> > > > $ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval
> > "declare -a
> > > > BASH_ARGC=()"'; echo $?
> > > > 1
> > > >
> > > > $ bash -c 'declare() { shift; printf %s\\n "${1%%=*}"; }; eval
> > "declare -a
> > > > BASH_ARG=()"'; echo $?
> > > > BASH_ARG
> > > > 0
> > > >
> > > > $ bash -c 'f() { shift; printf %s\\n "${1%%=*}"; }; eval "f -a
> > > > BASH_ARGC=()"'; echo $?
> > > > bash: eval: line 1: syntax error near unexpected token `('
> > > > bash: eval: line 1: `f -a BASH_ARGC=()'
> > > > 2
> > > >
> > > > $ bash -c 'f() { shift; printf %s\\n "${1%%=*}"; }; eval "f -a
> > > > BASH_ARG=()"'; echo $?
> > > > bash: eval: line 1: syntax error near unexpected token `('
> > > > bash: eval: line 1: `f -a BASH_ARG=()'
> > > > 2
> > > >
> > >
> > > you forgot
> > > see u cmd foo bar=()
> > > u still need as always escape ( and )
> >
> > I didn't forget anything. Martin's proposal was intended to work by
> > evaluating the unmodified output of typeset -p. That ( and ) normally need
> > to be escaped simply demonstrates further that it is untenable as a
> > solution.
>
>
> 1. making bash eat func foo=() never worked without escaping ( ) .. same
> err msg
> 2. cab u paste me the specific msg / url to that declare -p .. im parser
> pro ..
With all due respect, you are taking this thread way out into the weeds. The
given solution doesn't work, and may have inadvertently revealed a bug in bash.
I have absolutely nothing more to add that hasn't already been stated in my
reply to Martin. Everything you need to know is in that post.
If you are interested in a solution that is clumsy - but that actually works -
see https://bugs.gentoo.org/909148#c10.
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Tue, 27 Jun 2023 02:23:23 +0700
Robert Elz wrote:
> Date:Mon, 26 Jun 2023 10:32:19 +0100
> From: Kerin Millar
> Message-ID: <20230626103219.0f74c089c616248cee6ab...@plushkava.net>
>
>
> | Further, declare is granted special treatment, even after having been
> | defined as a function (which might be a bug).
>
> That will be because of the way that "declaration" utilities (which get
> special syntax rules for parsing their args) are detected. POSIX allows
> (and I think most shells which implement that absurd idea, for this there
> is very little choice) those to be detected merely by examining the
> characters that make up the command word, well before any actual
> interpretation of them is attempted.
>
> In bash, "declare" (the built-in) is a a declaration utility, so its
> args always get special treatment by the parser, regardless of whether
> or not the declare that will be run is the declaration utility, or
> something else (eg: a function).
>
> In POSIX, this is not a problem, as the declaration utilities are all
> special built-in commands, and those are recognised before functions
> (ie: it is not meaningfully possible to define a function to replace
> a special built-in). Bash, at least in non-posix mode, doesn't have
> that restriction.
>
> Avoiding the declaration utility nonsense arg parsing semantics if
> desired is easy, all that's needed is something like
>
> E=
> $E declare ...
>
> and then since "$E" is the command word position, and is not a
> declaration utility (regardless of what E is set to, an alternative
> that does the same thing is:
>
> D=declare
> $D ...
>
> as "$D" is not a declaration utility either - before it is expanded).
>
> Both of them run "declare" with the args processed in the same way
> that would be used for any other utility (like echo, printf, or awk).
>
> Unfortunately, it can be (fractionally) simpler to write code where
> the magic declaration utility parsing rules are used, though as best
> I can tell it is never the only way - to avoid it, one simply needs
> to separate the declaration from the assignment, putting one before
> the other (which order depends upon the declaration utility, for export
> it doesn't matter, either order works, for readonly the assignment
> must appear before the readonly, for declare the declaration probably
> usually needs to precede the assignment). Simple variable assignments
> (as distinct from things which appear to be variable assignments but
> which are actually just args to a utility, regardless of what that
> utility does with them) are always processed with the special arg
> processing syntax rules.
>
> Of course, none of this is relevant to finding a solution to the
> original problem - but is to deciding whether or not the way that
> bash gives special treatment to "declare" is a bug or not. My
> impression is that it is not, and that if an application (ie: you)
> decide to define a function with the same name as a declaration
> utility, you need to understand that it is still going to get
> declaration utility arg parsing applied to it, rather than what
> happens to everything else.
>
> Of course, Chet might decide to change that - shells aren't
> required to only use the chars that make up the command word
> position (before anything is expanded) in order to determine
> the expansion rules - another way (much more complex to code,
> and probably slower to execute) would be the expand the words
> in a simple command, one by one, finding the (expanded) word
> which will actually be the command that is run, and then
> parsing and expanding (in POSIX it is only expanding that
> matters - the parsing rules do not alter, as posix has no arrays
> and hence '(' is always an operator, never just a part of a
> variable value) the remainder of the command line according to
> the rules appropriate for the command that will actually be run.
> Of course, if that were done, the trivial mechanisms to avoid
> declaration utility semantics being applied given above ($E or $D)
> no longer work. But separating the declaration from the
> assignment should always be possible - if it isn't, something is
> going badly wrong (it might sometimes mean 3 commands need to be
> executed instead of one - one declaration before the assignment,
> and another after, but that's OK for more rational and understandable
> syntax).
I thank you for your insight. I am by no means certain that it should be
considered a bug, though I did find it surprising (this behaviour does not
appear to be documented). What was more surprising was the eventual realisation
that bash silently exits upon any attempt to assign to any of a particular
subset of its variables. That certainly complicated the matter of debugging
Martin's code.
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Tue, 27 Jun 2023 18:37:37 +0200
Fabien Orjollet wrote:
> I'm far from having the skills of the people here. However, I found the
> problem interesting. I think I've come up with a reasonable solution
> (you tell me). Although it's not as short as Kerin Millar's, I think it
> offers some improvements. I hope there are no particular weaknesses.
> If it's of any use to anyone.
>
>
> declare-P() {
> local curVar
> declare -a curVars
>
> readarray -t curVars <<<"$1"
> curVars=( "${curVars[@]%%=*}" )
> curVars=( "${curVars[@]##* }" )
>
> for curVar in "${curVars[@]}"; do
>### unfortunately, we cannot use [[ -v "$curVar" ]]
>[[ "${curVar//[a-zA-Z0-9_]}" || \
> "${curVar:0:1}" == [0-9] || \
> ! "$curVar" =~ $2 ]] || printf '%s\n' "$curVar"
> done
> }
>
> declare-P "$(declare -p)"
> echo "##"
> declare-P "$(declare -p)" "TERM"
> echo "##"
> declare-P "$(declare -p)" "^BASH|^SHELL"
>
It doesn't work at all for >=5.2. The reason for this is interesting and I may
make a separate post about it.
Prior to 5.2, it can easily be tricked into printing names that do not exist.
$ VAR=$'\nNONEXISTENT=' ./declare-P | grep ^NONEXISTENT
NONEXISTENT
All of this lends further credence to Eli's post. Parsing declare -F is a minor
nuisance, whereas parsing declare -p is broken by design. While the format of
declare -p improved for 5.2, there is no guarantee of forward-compatibility.
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Tue, 27 Jun 2023 21:52:53 +0200
of1 wrote:
> On 27/06/2023 21:05, Kerin Millar wrote:
> > It doesn't work at all for >=5.2. The reason for this is interesting and I
> > may make a separate post about it.
> >
> > Prior to 5.2, it can easily be tricked into printing names that do not
> > exist.
> >
> > $ VAR=$'\nNONEXISTENT=' ./declare-P | grep ^NONEXISTENT
> > NONEXISTENT
> >
> Thank you. I was just reading the discussion in Gentoo forum and
> realizing that I've been to quickly: it doesn't pass the
> FOO=$'\nBAR BAZ QUUX=' test. But what about this? (I also quickly
> tested with Bash 5.2.15).
>
>
> FOO=$'\nBAR BAZ QUUX='
> VAR=$'\nNONEXISTENT='
>
> declare-P() {
> local curVar
> declare -a curVars
>
> readarray -t curVars <<<"$1"
> curVars=( "${curVars[@]%%=*}" )
> curVars=( "${curVars[@]##* }" )
> for curVar in "${curVars[@]}"; do
>### we can use [[ -v "$curVar" ]] at some point!
>[[ "${curVar//[a-zA-Z0-9_]}" || \
> "${curVar:0:1}" == [0-9] || \
> ! -v "$curVar" || \
> ! "$curVar" =~ $2 ]] || printf '%s\n' "$curVar"
> done
> }
>
> declare-P "$(declare -p)"
> echo "##"
> declare-P "$(declare -p)" "QU|^NON|VAR"r
>
This use of test -v is probably sufficient to address that particular issue. It
is just as well that the test occurs after determining that the string is a
valid identifier because test -v has the power to facilitate arbitrary code
execution.
My assertion that your code is broken in 5.2 was incorrect. Rather, the problem
seems to be that regcomp(3) behaviour differs across platforms, where given an
empty expression to compile. To address that, you could determine whether a
second positional parameter was given and is non-empty before proceeding to use
it.
Curiously, repeated invocation may lead to obvious issues of resource
consumption.
# This is excruciatingly expensive
time for i in {1..100}; do
declare-P "$(declare -p)"
done
# This is not so expensive (!)
time for i in {1..100}; do
declare-P "$(declare -p)"
:
done
At present, I do not have a concrete explanation as to why this is, though it
may well have something to do with the value of the _ parameter. Suffice to say
that I'll be sticking to multiple ${!prefix*} expansions until such time as
bash offers a better way of going about it that doesn't also require readline.
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Wed, 28 Jun 2023 12:42:16 +0200
Fabien Orjollet wrote:
> On 28/06/2023 00:40, Kerin Millar wrote:
> > On Tue, 27 Jun 2023 21:52:53 +0200
> > of1 wrote:
> >
> >> On 27/06/2023 21:05, Kerin Millar wrote:
> >>> It doesn't work at all for >=5.2. The reason for this is interesting and
> >>> I may make a separate post about it.
> >>>
> >>> Prior to 5.2, it can easily be tricked into printing names that do not
> >>> exist.
> >>>
> >>> $ VAR=$'\nNONEXISTENT=' ./declare-P | grep ^NONEXISTENT
> >>> NONEXISTENT
> >>>
> >> Thank you. I was just reading the discussion in Gentoo forum and
> >> realizing that I've been to quickly: it doesn't pass the
> >> FOO=$'\nBAR BAZ QUUX=' test. But what about this? (I also quickly
> >> tested with Bash 5.2.15).
> >>
> >>
> >> FOO=$'\nBAR BAZ QUUX='
> >> VAR=$'\nNONEXISTENT='
> >>
> >> declare-P() {
> >> local curVar
> >> declare -a curVars
> >>
> >> readarray -t curVars <<<"$1"
> >> curVars=( "${curVars[@]%%=*}" )
> >> curVars=( "${curVars[@]##* }" )
> >> for curVar in "${curVars[@]}"; do
> >> ### we can use [[ -v "$curVar" ]] at some point!
> >> [[ "${curVar//[a-zA-Z0-9_]}" || \
> >>"${curVar:0:1}" == [0-9] || \
> >>! -v "$curVar" || \
> >>! "$curVar" =~ $2 ]] || printf '%s\n' "$curVar"
> >> done
> >> }
> >>
> >> declare-P "$(declare -p)"
> >> echo "##"
> >> declare-P "$(declare -p)" "QU|^NON|VAR"r
> >>
> >
> > This use of test -v is probably sufficient to address that particular
> > issue. It is just as well that the test occurs after determining that the
> > string is a valid identifier because test -v has the power to facilitate
> > arbitrary code execution.
> >
> > My assertion that your code is broken in 5.2 was incorrect. Rather, the
> > problem seems to be that regcomp(3) behaviour differs across platforms,
> > where given an empty expression to compile. To address that, you could
> > determine whether a second positional parameter was given and is non-empty
> > before proceeding to use it.
>
> Thank you, good to know. I think that ${2:-.} should be correct.
>
> >
> > Curiously, repeated invocation may lead to obvious issues of resource
> > consumption.
> >
> > # This is excruciatingly expensive
> > time for i in {1..100}; do
> > declare-P "$(declare -p)"
> > done
> >
> > # This is not so expensive (!)
> > time for i in {1..100}; do
> > declare-P "$(declare -p)"
> > :
> > done
>
> I made some testing by adding
> echo -n "$(( ++COUNT ))>$(wc -c <<<"$1") "
> echo "$1" >"/tmp/declare-P$COUNT"
> to the function:
>
> 1>4651 2>9785 3>15283 4>21504 5>29173 6>39739 7>56095 8>84036
> 9>135145 10>232592 11>422710 12>798174 13>1544326 14>3031854
> 15>6002134 16>11937918 ^C
>
> ls -rtnh /tmp/declare-P*
> -rw-r--r-- 1 1000 1000 4,6K 28 juin 11:09 /tmp/declare-P1
> -rw-r--r-- 1 1000 1000 9,6K 28 juin 11:09 /tmp/declare-P2
> -rw-r--r-- 1 1000 1000 15K 28 juin 11:09 /tmp/declare-P3
> -rw-r--r-- 1 1000 1000 21K 28 juin 11:09 /tmp/declare-P4
> -rw-r--r-- 1 1000 1000 29K 28 juin 11:09 /tmp/declare-P5
> -rw-r--r-- 1 1000 1000 39K 28 juin 11:09 /tmp/declare-P6
> -rw-r--r-- 1 1000 1000 55K 28 juin 11:09 /tmp/declare-P7
> -rw-r--r-- 1 1000 1000 83K 28 juin 11:09 /tmp/declare-P8
> -rw-r--r-- 1 1000 1000 132K 28 juin 11:09 /tmp/declare-P9
> -rw-r--r-- 1 1000 1000 228K 28 juin 11:09 /tmp/declare-P10
> -rw-r--r-- 1 1000 1000 413K 28 juin 11:09 /tmp/declare-P11
> -rw-r--r-- 1 1000 1000 780K 28 juin 11:09 /tmp/declare-P12
> -rw-r--r-- 1 1000 1000 1,5M 28 juin 11:09 /tmp/declare-P13
> -rw-r--r-- 1 1000 1000 2,9M 28 juin 11:09 /tmp/declare-P14
> -rw-r--r-- 1 1000 1000 5,8M 28 juin 11:09 /tmp/declare-P15
> -rw-r--r-- 1 1000 1000 12M 28 juin 11:09 /tmp/declare-P16
>
> But by adding set -p to the subshell:
Your best option would probably be to run declare -p within the function while
avoiding the use of command substitution altogether, so as not to 'pollute' the
_ variable.
--
Kerin Millar
Bash silently exits where attempting to assign an array to certain built-in variables using declare
This report is based on an observation made within the depths of this thread: https://lists.gnu.org/archive/html/bug-bash/2023-06/msg00094.html. Attempting to assign an array to any of the following variables with the declare builtin causes bash to immediately exit with no diagnostic message being issued. BASH_ARGC BASH_ARGV BASH_LINENO BASH_SOURCE GROUPS Here is an example. $ bash -c 'declare -p BASH_VERSION; declare BASH_ARGC=(); echo FIN'; echo $? declare -- BASH_VERSION="5.2.15(1)-release" It does not happen if trying to assign a string. $ bash -c 'declare BASH_ARGC=1; echo FIN' FIN There are various other variables bearing the readonly attribute for which this also happens. In the following case, bash does, at least, complain that the variable is readonly. $ bash -c 'declare BASHOPTS=(); echo FIN' bash: line 1: BASHOPTS: readonly variable However: $ bash -c 'declare BASHOPTS=1; echo FIN' bash: line 1: BASHOPTS: readonly variable FIN This seems rather inconsistent. Also, it is confusing for bash to quit without indicating why it did so. -- Kerin Millar
Re: Bash silently exits where attempting to assign an array to certain built-in variables using declare
On Thu, 29 Jun 2023 08:51:58 -0400 Chet Ramey wrote: > On 6/28/23 1:14 PM, Kerin Millar wrote: > > This report is based on an observation made within the depths of this > > thread: https://lists.gnu.org/archive/html/bug-bash/2023-06/msg00094.html. > > > > Attempting to assign an array to any of the following variables with the > > declare builtin causes bash to immediately exit with no diagnostic message > > being issued. > > > >BASH_ARGC > >BASH_ARGV > >BASH_LINENO > >BASH_SOURCE > >GROUPS > > These are all `noassign' variables; assignments to them are ignored. > The bash debugger variables cannot be unset either. Other noassign > variables can be unset; that's why they're not readonly. > > (Before you ask, noassign variables have been in bash since 1996.) Thanks for this information. While I had no expectation of them being assignable, some of this might have a valid place in the manual. > > In this case, assignment to the noassign variable is being treated like an > assignment error, which aborts the current command (a compound command in > your examples) but does not exit the shell. If you were to separate the > commands with a newline, you'd see the difference. Ah. I thought that I had tried it at some point. Evidently, my manner of testing was faulty. > > This came up in 2021 in the GROUPS case: > > https://lists.gnu.org/archive/html/bug-bash/2021-08/msg00013.html > > The other variables in that list inherited their noassign property from > changes to support the bash debugger. > > > It does not happen if trying to assign a string. > > It should be consistent, at least. I think the string assignment behavior > is the most reasonable: assignments return 1 but there's no assignment > error. I'll look at how compound assignments are different. Thanks. I would just add that 'noassign' variables are not consistently indicated in the manual. For example, it is said for GROUPS that "Assignments to GROUPS have no effect" but there is no such wording for the others that I mentioned. That might be worth addressing, even if only by inserting similarly worded sentences where appropriate. > > > > >$ bash -c 'declare BASH_ARGC=1; echo FIN' > >FIN > > > > There are various other variables bearing the readonly attribute for which > > this also happens. In the following case, bash does, at least, complain > > that the variable is readonly. > > > >$ bash -c 'declare BASHOPTS=(); echo FIN' > >bash: line 1: BASHOPTS: readonly variable > > Attempted assignment to readonly variables is an assignment error. > > > This seems rather inconsistent. Also, it is confusing for bash to quit > > without indicating why it did so. > > Technically, it exited because it hit EOF after aborting the compound > command. I see. -- Kerin Millar
Re: maybe a bug in bash?
On Thu, 29 Jun 2023 11:55:12 +0200
Sebastian Luhnburg wrote:
> #!/usr/bin/env bash
>
> initial_password="\$abc&xyz"
> echo "initial password: " $initial_password
> printf -v password '%q' $initial_password
> echo "initial password with escaped characters: " $password
> bash << EOF
> echo "password in here document: " ${password@Q}
> /bin/bash -c "echo 'password in subshell in here document: ' ${password@Q}"
> EOF
While Dominique has already responded adequately, I have several things to add.
One is that you should quote the expansion of initial_password in your printf
statement to impede word splitting and pathname expansion. Another is that you
should not requote the string with %q then proceed to do so a second time by
using the ${param@Q} form of expansion. Instead, use one or the other.
Traversing multiple quoting layers is hard and I would suggest simply not doing
it, if it can be helped. That being said, you could get the results that you
expect by conveying password as a discrete argument to bash (which is not a
subshell, by the way). Below is an example.
#!/bin/bash
initial_password='$abc&xyz'
printf -v password %q "$initial_password"
bash <
Re: Bash silently exits where attempting to assign an array to certain built-in variables using declare
On Thu, 29 Jun 2023 12:10:37 -0400 Chet Ramey wrote: > On 6/29/23 8:51 AM, Chet Ramey wrote: > > > It should be consistent, at least. I think the string assignment behavior > > is the most reasonable: assignments return 1 but there's no assignment > > error. I'll look at how compound assignments are different. > > I have reconsidered this. Making assignments that are supposed to be > ignored set $? to 1 means that a shell with errexit enabled will exit. > I don't think that's desirable. I don't think attempted assignments to > noassign variables should change $?. That seems reasonable to me. It would also make it less likely that the evaluation of the prior output of declare -p - be in whole or in part - affects the value of $?, which seems like a win. -- Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Thu, 29 Jun 2023 16:39:52 -0400
Chet Ramey wrote:
> On 6/25/23 2:38 PM, Eli Schwartz wrote:
> > compgen is a useful builtin for inspecting information about the shell
> > context e.g. in scripts -- a good example of this is compgen -A function
> > or compgen -A variable.
> >
> > But it's not always available depending on how bash is built, which
> > results in people lacking confidence that it can / should be used in
> > scripts. See e.g. https://bugs.gentoo.org/909148
>
> It's dependent on programmable completion and readline, which are features
> that are enabled by default. Who builds a version of bash with those turned
> off? What's the rationale for doing that?
This is discussed in the referenced bug.
To begin with, readline is disabled during gentoo's bootstrapping stage. I'm
not one of its release engineers, but I presume that there is a cromulent
reason for doing so. Sam might be able to chime in on this. Meanwhile, portage
tries to run "compgen -A function" when the time comes to save the "ebuild
environment", which is always stashed in the "VDB" for an installed package
(under /var/db/pkg). Naturally, these two things do not go well together. As
such, the bug was filed with the intent of eliminating this (sole) use of
compgen.
Now for my part in this. Without going into the details of portage's internals,
some of the code concerning the processing of the ebuild environment is ...
well, let me just put it politely by saying that it is far from ideal. I have
been working on cleaning much of this up. Consequently, I have come to rely
upon compgen quite heavily, duly compelling me to pose the same question: what
is the reason for disabling readline support in the first place? In fact, not
only is it unavailable during bootstrapping but users do also have the option
to disable it. Admittedly, they never do in practice; it's an option that is
made deliberately difficult to take advantage of. In any case, comments #7 and
#8 were the catalyst for bringing the matter up here on the list, if you're
interested.
As it happens, I have submitted a patch that replaces the use of "compgen -A
function" with a "declare -F" parser, which is all well and good. The thing is
that portage also has a legitimate stake in needing to enumerate all variable
names. To that end, I had been using "compgen -A variable" in my fork. The
implication of the bug is that I must refrain from using it, at which point the
prospect of parsing "declare -p" is a tremendously unappealing one, to say the
least.
Ultimately, depending on whether you entertain any of this or not, the outcome
will be that I rely upon my eval-using hack - which I had thought of by comment
#10 - in all perpetuity or look forward to a cleaner way of going about it that
doesn't necessarily depend upon compgen being available. One possibility would
be to extend declare in such a manner that it is able to print only variable
names, just as declare -F prints only function names. Another might be to
extend the "${!prefix*}" and "${!prefix@}" syntax in such a way that it is
possible to match any name. To be clear, I am not as concerned with the matter
as I was at the time that the bug was initially filed but would still welcome
any potential improvement, if it is at all feasible.
--
Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Sat, 01 Jul 2023 00:19:41 +0700 Robert Elz wrote: > Date:Thu, 29 Jun 2023 23:05:38 +0100 > From: Kerin Millar > Message-ID: <20230629230538.cbef14a75694143ccf034...@plushkava.net> > > | The thing is that portage also has a legitimate stake in needing > | to enumerate all variable names. > > Why isn't "set" good enough for that? > > (set -o posix; set) | sed 's/=.*$//' | whatever_processes_them This can be trivially foiled. -- Kerin Millar
Re: Enable compgen even when programmable completions are not available?
On Sat, 01 Jul 2023 02:25:33 +0700 Robert Elz wrote: > Date:Fri, 30 Jun 2023 18:35:34 +0100 > From: Kerin Millar > Message-ID: <20230630183534.85da7986a24855126bfea...@plushkava.net> > > | This can be trivially foiled. > > You mean it doesn't give you all the variable names? Why not? > Does bash have a bug in this area that I am unaware of? > > Or do you mean that it will sometimes (due to newlines in the values) > be able to be persuaded to give you more than just the var names? > > If the latter, why do you care? The processing can check for each variable Well, I don't in particular. However, it is also the case that: - every single solution posted in this thread has been broken in its initial form, save for that which I had devised even before Eli posted - I have to yet to see one working solution that provides any tangible advantage over mine - I am not soliciting solutions in any case; they have been nothing but a distraction - I am tired of being told things that I already know and am frankly now tired of this entire thread; nothing is going to come of it -- Kerin Millar
Re: suggestion: shell option for echo to not interpret any argument as an option
On Wed, 26 Jul 2023, at 1:42 PM, Zachary Santer wrote:
> bash's echo command is broken - YouTube
> <https://www.youtube.com/watch?v=lq98MM2ogBk>
>
> To restate what's in the video, you can't safely use echo to print the
> contents of a variable that could be arbitrary, because the variable could
> consist entirely of '-n', '-e', or '-E', and '--' is not interpreted as the
> end of options, but rather, something to print.
>
> I recognized this and replaced all of my calls to echo with printf some
> time ago.
>
> If POSIX mandates that '--' not be taken as the end of options, then the
> safe thing would be to simply not have echo take any options. Obviously,
> that would break backwards compatibility, so you'd want this to be optional
> behavior that the shell programmer can enable if desired.
echo() { local IFS=' '; printf '%s\n' "$*"; }
--
Kerin Millar
Re: String replacement drops leading '-e' if replacing char is a space
On Mon, 14 Aug 2023 02:11:27 +
pphick via Bug reports for the GNU Bourne Again SHell wrote:
> If a string starts with '-e' the replacement operators ${x//,/ } and ${x/, /}
> drop the '-e'.
> The behaviour seems to be very specific: the string must start with '-e' and
> the replacing character has to be a space.
>
> Repeat-By:
>
> x='-e,b,c'
> echo ${x//,/ }
> b c
> echo ${x/,/ }
> b,c
This is to be expected. Given that you haven't quoted the expansion, word
splitting occurs, after which echo is run with three arguments. The first of
these arguments is -e, which is treated as an option.
--
Kerin Millar
Re: test -v difference between bash 5.1 and 5.2
Hi,
On Tue, 29 Aug 2023 16:32:36 +0200
Christian Schneider wrote:
> Hi all,
>
> not sure if this intended or not, but in bash 5.2-p15 one of our scripts
> is broken. it is related to test -v, that checks, if a variable is set
> together with arrays.
>
> I condensed it to following example:
> #!/bin/bash
>
> declare -A foo
> foo=(["a"]="b" ["c"]="d")
> declare -a bar
> bar=("a" "b" "c")
> declare -a baz
> baz=("foo" "bar")
> for i in "${baz[@]}" ; do
> echo $i
> if [ ! -v "$i"[@] ] ; then
> echo "$i not set"
> fi
> done
>
>
> with bash 5.2-p15 the output of this script is
> foo
> foo not set
> bar
It pertains to the following change.
j. Associative array assignment and certain instances of referencing (e.g.,
`test -v' now allow `@' and `*' to be used as keys.
For now, you have the option of setting the compatibility level to 51.
Incidentally, your code is defective to begin with. That is, it doesn't
actually prove that an array variable is set, even with 5.1.
$ declare -p BASH_VERSION
declare -- BASH_VERSION="5.1.16(1)-release
$ declare -A map; [[ -v 'map[@]' ]]; echo $?
1
Frankly, the only interface that I would trust for this is declare -p, which is
a wasteful one; there is no way to instruct the declare builtin to refrain from
writing out the elements of an array.
--
Kerin Millar
Re: test -v difference between bash 5.1 and 5.2
On Tue, 29 Aug 2023 11:24:43 -0400
Chet Ramey wrote:
> If you want to check whether an array variable is set, you can check
> whether it has any set elements:
>
> (( ${#assoc[@]} > 0 ))
This doesn't check whether an "array variable is set".
Not only that, but the test will be true in the case that assoc has been
defined as a variable that is not an array.
$ unset -v assoc; assoc=; (( ${#assoc[@]} > 0 )); echo $?
0
--
Kerin Millar
Re: test -v difference between bash 5.1 and 5.2
On Tue, 29 Aug 2023 11:34:21 -0400
Chet Ramey wrote:
> On 8/29/23 11:30 AM, Kerin Millar wrote:
> > Hi,
> >
> > On Tue, 29 Aug 2023 16:32:36 +0200
> > Christian Schneider wrote:
> >
> >> Hi all,
> >>
> >> not sure if this intended or not, but in bash 5.2-p15 one of our scripts
> >> is broken. it is related to test -v, that checks, if a variable is set
> >> together with arrays.
> >>
> >> I condensed it to following example:
> >> #!/bin/bash
> >>
> >> declare -A foo
> >> foo=(["a"]="b" ["c"]="d")
> >> declare -a bar
> >> bar=("a" "b" "c")
> >> declare -a baz
> >> baz=("foo" "bar")
> >> for i in "${baz[@]}" ; do
> >> echo $i
> >> if [ ! -v "$i"[@] ] ; then
> >> echo "$i not set"
> >> fi
> >> done
> >>
> >>
> >> with bash 5.2-p15 the output of this script is
> >> foo
> >> foo not set
> >> bar
> >
> > It pertains to the following change.
> >
> > j. Associative array assignment and certain instances of referencing (e.g.,
> > `test -v' now allow `@' and `*' to be used as keys.
> >
> > For now, you have the option of setting the compatibility level to 51.
> >
> > Incidentally, your code is defective to begin with. That is, it doesn't
> > actually prove that an array variable is set, even with 5.1.
> >
> > $ declare -p BASH_VERSION
> > declare -- BASH_VERSION="5.1.16(1)-release
> > $ declare -A map; [[ -v 'map[@]' ]]; echo $?
> > 1
>
> That isn't set; you have to assign a value to set a variable. For instance,
Fair enough.
> `export foo' does not result in `foo' being set. What he's really
> interested in is whether the array has any set elements.
Yes, hopefully.
--
Kerin Millar
Re: test -v difference between bash 5.1 and 5.2
On Tue, 29 Aug 2023 11:44:13 -0400
Chet Ramey wrote:
> On 8/29/23 11:38 AM, Kerin Millar wrote:
> > On Tue, 29 Aug 2023 11:24:43 -0400
> > Chet Ramey wrote:
> >
> >> If you want to check whether an array variable is set, you can check
> >> whether it has any set elements:
> >>
> >> (( ${#assoc[@]} > 0 ))
> >
> > This doesn't check whether an "array variable is set".
>
> It checks whether there are any set elements. You have to assign a value
> to set a variable.
I conflated the property of being set with that of being declared. Sorry about
that. So, what I really meant to say was that the existing test does not prove
that it's declared. I initially thought that Christian might be concerned with
that distinction, but do hope it's not the case.
>
> > Not only that, but the test will be true in the case that assoc has been
> > defined as a variable that is not an array.
>
> One hopes that the shell programmer knows what variable types he's
> using, and uses the appropriate constructs.
Some elect to source shell code masquerading as configuration data (or are
using programs that elect to do so). Otherwise, yes, definitely.
--
Kerin Millar
Re: Some incorrect behaviour for BASH arrays
On Thu, 31 Aug 2023 19:39:09 +0700
Victor Pasko wrote:
> Thanks for prompt response. But see my comments below.
>
> Well, instead of the following line
>
> RESULT=''
>
> done
>
> declare -p RESULT
>
> without initialization, because otherwise I got the following error "-bash:
> declare: RESULT=: not found".
declare -p RESULT='' translates as declare -p RESULT=, which instructs bash to
attempt to dump the contents of a variable named RESULT=. There is no variable
named RESULT=, so it prints an error to that effect. Put another way, you're
not supposed to be using declare -p as a substitute for assignment syntax.
While declare can also be used to set variables, that's not what the -p option
is for.
>
> And after deleting/commenting the following fragment in badrev
>
> #if [ $i -ge $QSTRING ] ; then
> #echo "XXX-break i=$i result=$RESULT size=${#RESULT}"
> #break
> #fi
>
> and correcting end echo-line to
>
> echo " result=${RESULT[0]} size=${#RESULT[0]}"
${RESULT[0]} will expand as the first element of the array. ${#RESULT[0]} will
expand as the length of the first element of the array (not the length of the
array itself).
>
> i can see strange output for badrev:
> declare -a RESULT=([0]="987654321" [1]="B" [2]="1" [3]="2" [4]="3" [5]="\\"
> [6]="-" [7]="4" [8]="5" [9]="6" [10]="7" [11]="\\" [12]="-" [13]="8")
> result=9 size=1
> Still incorrect output and size:( How to get correct output? ${RESULT[0]}
> ???
What would be correct to you?
>
> result=${RESULT[0]} size=${#RESULT[0]} ### does not work :(
What would it be doing for you to be satisfied that it works?
Your script contains no explanation as to what it is intended to do. Comments
to the effect of "not working" do not help to discern your intent.
>
> BTW, in the presented link to the manual with *6.7 Arrays*
> there is no explanation for *declare -p *:(
It's a builtin command, so it's listed within the SHELL BUILTIN COMMANDS
section. Also, you may use the help builtin to display some (less detailed)
documentation.
$ help declare | grep -- -p
-pdisplay the attributes and value of each NAME
--
Kerin Millar
Re: Some incorrect behaviour for BASH arrays
On Thu, 31 Aug 2023 01:12:48 +0700
Victor Pasko wrote:
> Hi,
>
> On my Win-7 64-bit using Cygwin
>
>
> *% bash --version*GNU bash, version 5.2.15(3)-release (x86_64-pc-cygwin)
> Copyright (C) 2022 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
> >
>
> This is free software; you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
>
> *% echo LANG=$LANG; echo LC_ALL=$LC_ALL*LANG=
> LC_ALL=
>
> Attached please find bug.bash to reproduce incorrect BASH behaviour for
> BASH arrays
> and look at ways to run it
This script is a mess because its functions begin by defining RESULT as an
ordinary, non-array variable (if not yet declared).
$ RESULT='string'
$ declare -p RESULT # not yet an array variable
declare -- RESULT='string'
It then proceeds to operate on the variable in such a way that it will be
transformed to an array.
$ RESULT[1]='second element'
$ declare -p RESULT
declare -a RESULT=([0]="string" [1]="second element")
Now, would RESULT='' empty this array? No, it would not.
$ RESULT='' # no different from RESULT[0]=''
$ declare -p RESULT
declare -a RESULT=([0]="" [1]="second element")
A correct way to initialise an empty array variable is RESULT=().
$ RESULT=()
$ declare -p RESULT # now an empty array
declare -a RESULT=()
You might also consider using the "local" builtin to declare the variable with
a function-local scope.
$ f() { local RESULT=(); RESULT+=("append me"); declare -p RESULT; }; unset -v
RESULT; f; declare -p RESULT
declare -a RESULT=([0]="append me")
bash: declare: RESULT: not found
Note that the second declare command correctly raises an error because RESULT
is not visible in its scope. Used judiciously, local can help to avoid the
writing of needless bugs.
--
Kerin Millar
Re: Inner Command Lists fail in Bash 5.2.15
On Thu, 31 Aug 2023 12:05:21 -0400 (EDT)
dkro...@gmail.com wrote:
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: darwin20.6.0
> Compiler: clang
> Compilation CFLAGS: -DSSH_SOURCE_BASHRC
> uname output: Darwin San-Francisco-iMac.local 20.6.0 Darwin Kernel Version
> 20.6.0: Thu Jul 6 22:12:47 PDT 2023;
> root:xnu-7195.141.49.702.12~1/RELEASE_X86_64 x86_64
> Machine Type: x86_64-apple-darwin20.6.0
>
> Bash Version: 5.2
> Patch Level: 15
> Release Status: release
>
> Description:
> I've run into a regression in one of my scripts after upgrading Bash
> from 5.1.16 to 5.2.15 via HomeBrew on a Mac.
> I was able to narrow it down to the Bash upgrade itself. The problem seems to
> be a pretty serious one that can affect a lot of scripts.
>
> Repeat-By:
>
> # Bash 5.1.16: success
> # Bash 5.2.15: syntax error near unexpected token `}'
> var2=$(time { echo foo; echo bar; })
This looks like a casualty of the changes made for 5.2 that have it validate
syntax recursively. The error message suggests that "time" is not considered to
be a keyword there, even though it is. In any case, here is a temporary
workaround for this regression.
$ declare -- BASH_VERSION="5.2.15(1)-release"
$ var2=$(:; time { echo foo; echo bar; })
real0m0.000s
user0m0.000s
sys 0m0.000s
--
Kerin Millar
Re: Fwd: Some incorrect behaviour for BASH arrays
On Fri, 1 Sep 2023 14:44:49 +0700
Victor Pasko wrote:
> Just forward my response to all who was involved in discussion of my request
>
> -- Forwarded message -
> From: Victor Pasko
> Date: Fri, Sep 1, 2023 at 2:23 PM
> Subject: Re: Some incorrect behaviour for BASH arrays
> To: Kerin Millar
>
>
> Thanks for the detailed explanations of *declare *.
>
> As to the idea behind of my request:
> 1) I need local variable RESULT as empty string (not array)
> 2) this RESULT should collect symbols taken from other strings using
> ${STRING:i:1} for one symbol or ${STRING:i:k} for k-symbols
> So, the main question is: how to save such sub-strings in RESULT at needed
> j-place?
> With another words - I need RESULT as C-char-array to use it something like
> this
>
> RESULT[j]="${STRING:i:1}"
You would have to reassemble RESULT from the appropriate substrings.
$ RESULT=cat; STRING=moo; i=1 j=1;
RESULT=${RESULT:0:j}${STRING:i:1}${RESULT:j+1}; declare -p RESULT
declare -- RESULT="cot"
Alternatively, use an array then finish by joining its elements, using the [*]
subscript.
f() {
local RESULT=(c a t)
local STRING=moo
local IFS=
local i=1 j=1
RESULT[j]=${STRING:i:1}
printf '%s\n' "${RESULT[*]}" # joins elements by first char of IFS (empty
string)
}
--
Kerin Millar
Re: Inner Command Groups fail in Bash 5.2
On Fri, 1 Sep 2023 09:52:17 -0400 Chet Ramey wrote: > On 8/31/23 1:21 PM, Dima Korobskiy wrote: > > > Bash Version: 5.2 > > Patch Level: 15 > > Release Status: release > > > > Description: > > One of my Bash scripts started to fail all of a sudden and I was > > able to narrow down this problem to Bash upgrade itself (via Homebrew on a > > Mac): from 5.1.16 to 5.2.15. > > It looks to me pretty serious: a lot of scripts might be potentially > > affected. > > Thanks for the report. It's an easy fix: > > > *** ../bash-20230818/parse.y Wed Aug 23 09:56:19 2023 > --- parse.y Fri Sep 1 09:29:27 2023 > *** > *** 3224,3227 > --- 3224,3229 >case TIMEOPT: /* time -p time pipeline */ >case TIMEIGN: /* time -p -- ... */ > + case DOLPAREN: > + case DOLBRACE: > return 1; >default: Would you mind supplying a diff for 5.2.15? For that version, I get: ./parse.y: In function ‘time_command_acceptable’: ./parse.y:3139:14: error: ‘DOLBRACE’ undeclared (first use in this function); did you mean ‘Q_DOLBRACE’ -- Kerin Millar
Re: Inner Command Groups fail in Bash 5.2
On Fri, 1 Sep 2023 10:29:29 -0400 Chet Ramey wrote: > On 9/1/23 10:27 AM, Kerin Millar wrote: > > > Would you mind supplying a diff for 5.2.15? For that version, I get: > > > > ./parse.y: In function ‘time_command_acceptable’: > > ./parse.y:3139:14: error: ‘DOLBRACE’ undeclared (first use in this > > function); did you mean ‘Q_DOLBRACE’ > > Remove `DOLBRACE'. It's for nofork comsubs. Thanks. -- Kerin Millar
Re: Inner Command Lists fail in Bash 5.2.15
On Fri, 1 Sep 2023 12:52:14 -0400 Dima Korobskiy wrote: > Kerin, > > thanks for the workaround. > > Just to clarify, is the issue specific to the `time` command? I think that you would have to get an answer from Chet because yacc/bison grammar is a little over my head. To that end, I have taken the liberty of copying the bug-bash list back into this discussion. That being said, it appears so. After reading your report, I tried to break the parser in other, similar, ways but was not able to do so. I suppose that the time keyword is special in so far as it expects to be given a pipeline. -- Kerin Millar
Re: Warn upon "declare -ax"
On Mon, 4 Sep 2023 14:46:08 +0200 Léa Gris wrote: > Le 04/09/2023 à 14:18, Dan Jacobson écrivait : > > Shouldn't "declare -ax" print a warning that it is useless? > > There don's seem to be any warning system in Bash or other shells. As > long as it is not a fatal error condition and errexit is not set, > execution continue. > > There are static analysis tools like Shellcheck which might be expanded > to ware of such incompatible flags but that's it. Curiously, ARRAY_EXPORT can be defined in config-top.h. It's probably safe to say that nobody uses it (nor should anybody wish to upon realising how it works). -- Kerin Millar
Re: Warn upon "declare -ax"
On Tue, 5 Sep 2023 16:04:50 +0200
alex xmb ratchev wrote:
> On Mon, Sep 4, 2023, 15:19 Kerin Millar wrote:
>
> > On Mon, 4 Sep 2023 14:46:08 +0200
> > Léa Gris wrote:
> >
> > > Le 04/09/2023 à 14:18, Dan Jacobson écrivait :
> > > > Shouldn't "declare -ax" print a warning that it is useless?
> > >
> > > There don's seem to be any warning system in Bash or other shells. As
> > > long as it is not a fatal error condition and errexit is not set,
> > > execution continue.
> > >
> > > There are static analysis tools like Shellcheck which might be expanded
> > > to ware of such incompatible flags but that's it.
> >
> > Curiously, ARRAY_EXPORT can be defined in config-top.h. It's probably safe
> > to say that nobody uses it (nor should anybody wish to upon realising how
> > it works).
> >
>
> does it make too big copies or wha ..
My pet name for it is arrayshock.
$ arr=(foo bar baz)
$ export arr
$ env | grep ^BASH_ARRAY_
BASH_ARRAY_arr%%=([0]="foo" [1]="bar" [2]="baz")
$ ./bash -c 'declare -p arr'
declare -ax arr=([0]="foo" [1]="bar" [2]="baz")
It's not particularly reliable. The following is to be expected because the
prospective environment ends up being too large.
$ arr=({1..10}); /bin/true
bash: /bin/true: Argument list too long
However, emptying the array does not remedy the situation (unsetting does).
$ arr=(); /bin/true
bash: /bin/true: Argument list too long
--
Kerin Millar
Re: Prompt messed up if PS1 contains ANSI escape sequences
On Thu, 7 Sep 2023 15:53:03 +0200
alex xmb ratchev wrote:
> On Thu, Sep 7, 2023, 15:46 Gioele Barabucci wrote:
>
> > On 07/09/23 15:00, alex xmb ratchev wrote:
> > > u have to \[ esc-seq \]
> > > eg inside \[ and \]
> > >
> > > PS1=$'\u\[\e[1m\]\h\[\e[0m- '
> > >
> > > should display hostname bold
> >
> > Thanks for the suggestion, but adding \] does not really fix the
> > problem, it just masks it in many cases (better than nothing).
> >
> > Try:
> >
> > $ long_name="$(printf 'abcdef0123456789/%.0s' {0..20})"
> > $ mkdir -p /tmp/$long_name
> > $ cd /tmp/$long_name
> > $ PS1='\n\[\e[1m\]\w\[\e[m\] \$ '
> >
>
> foo=$' .. '
> not
> foo=' .. '
$'' quoting is not required for that particular definition of PS1.
--
Kerin Millar
Re: command substitution when timing grouped commands fails
On Thu, 07 Sep 2023 05:50:49 -0700 hacke...@member.fsf.org wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -g -O2 -fstack-protector-strong -Wformat > -Werror=format-security -Wall > uname output: Linux abyssal 6.4.0-3-amd64 #1 SMP PREEMPT_DYNAMIC Debian > 6.4.11-1 (2023-08-17) x86_64 GNU/Linux > Machine Type: x86_64-pc-linux-gnu > > Bash Version: 5.2 > Patch Level: 15 > Release Status: release > > Description: > > Bash gives a syntax error when using the $(...) form of > command substitution and timing grouped commands. > > However, Bash works correctly when using the `...` form of > command substitution. > > > Repeat-By: > > The 'time' built-in command can measure a group of commands > run in a subshell, for example: > > $ time (date; sleep 1) > Thu Sep 7 05:19:21 AM PDT 2023 > > real0m1.005s > user0m0.003s > sys 0m0.001s > > Attempting to save the output of time to a variable fails when > using $(...) command substitution. For example, > > $ x=$( time ( date; sleep 1 ) 2>&1 ) > -bash: syntax error near unexpected token `date' > > However, old versions of bash (~2016) used to work correctly. > And, indeed, even the current version of bash works if one > uses backticks for command substitution. > > $ x=` time ( date; sleep 1 ) 2>&1 ` > $ # no error > > There should be no difference between $(...) and `...`. This issue, which affects 5.2, was previously reported here: https://lists.gnu.org/archive/html/bug-bash/2023-08/msg00124.html It was fixed in the devel branch in the fashion described by: https://lists.gnu.org/archive/html/bug-bash/2023-09/msg00013.html The issue has not yet been addressed by any available 5.2 patchlevel. Should you wish to patch 5.2 yourself - as I did - apply the above-mentioned change while ignoring the addition of "case DOLBRACE:". -- Kerin Millar
Re: Prompt messed up if PS1 contains ANSI escape sequences
On Thu, 7 Sep 2023 17:33:45 +0200
alex xmb ratchev wrote:
> On Thu, Sep 7, 2023, 16:51 Kerin Millar wrote:
>
> > On Thu, 7 Sep 2023 15:53:03 +0200
> > alex xmb ratchev wrote:
> >
> > > On Thu, Sep 7, 2023, 15:46 Gioele Barabucci wrote:
> > >
> > > > On 07/09/23 15:00, alex xmb ratchev wrote:
> > > > > u have to \[ esc-seq \]
> > > > > eg inside \[ and \]
> > > > >
> > > > > PS1=$'\u\[\e[1m\]\h\[\e[0m- '
> > > > >
> > > > > should display hostname bold
> > > >
> > > > Thanks for the suggestion, but adding \] does not really fix the
> > > > problem, it just masks it in many cases (better than nothing).
> > > >
> > > > Try:
> > > >
> > > > $ long_name="$(printf 'abcdef0123456789/%.0s' {0..20})"
> > > > $ mkdir -p /tmp/$long_name
> > > > $ cd /tmp/$long_name
> > > > $ PS1='\n\[\e[1m\]\w\[\e[m\] \$ '
> > > >
> > >
> > > foo=$' .. '
> > > not
> > > foo=' .. '
> >
> > $'' quoting is not required for that particular definition of PS1.
> >
>
> o cause \e gets expanded .. ? didnt know ..
Yes. \n also.
--
Kerin Millar
Re: math operations with base#prefix
Hi Victor, On Sun, 17 Sep 2023, at 8:59 PM, Victor Pasko wrote: > Hi, > > Could you please take a look at attached bug.bash. > > Maybe, not all math combinations were presented there or the test has > duplications somehow. > Here are results of several runs with test# as argument > > > *% bash --version*GNU bash, version 5.2.15(3)-release (x86_64-pc-cygwin) > > Good test without argument but others with errors :( > *% ./bug.bash* > > res1=010 good 010 base8 > res2=03 good 03 base8 > res=17 good result 17 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > res1=8 good result 8 base10 > res1=10 good result 10 > res1=10 good result 10 > res1=010 good result 010 base8 > base10-res1=10 good result 10 > res1=16 good result 16 > > > *% ./bug.bash 1* > TESTCASE=1 > res1=010 good 010 base8 > res2=03 good 03 base8 > res=17 good result 17 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > ./bug.bash: line 29: let: res = base10#010 + base10#03 * 3: syntax error: > invalid arithmetic operator (error token is "#010 + base10#03 * 3") This seems like a misinterpretation of the manual. The manual states that numbers "take the form [base#]n, where the optional base is a decimal number between 2 and 64 representing the arithmetic base". As such, "base10" is not a decimal number between 2 and 64, whereas "10" would be. > base10-res=19 good result 19 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > res1=8 good result 8 base10 > res1=10 good result 10 > res1=10 good result 10 > res1=010 good result 010 base8 > base10-res1=10 good result 10 > res1=16 good result 16 > > > *% ./bug.bash 2* > TESTCASE=2 > res1=010 good 010 base8 > res2=03 good 03 base8 > res=17 good result 17 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > base10-res=19 good result 19 base10 (res1+3*res2) > ./bug.bash: line 35: let: res = 10#res1: value too great for base (error > token is "10#res1") For numbers in the form "[base#]n", it isn't practically possible for n to be specified using a variable without prefixing it with a sigil (so that it is treated as a parameter expansion and injected). There is a very good reason for this: numbers in a base higher than 10 can require alphabetical letters to be expressed. Consider the following example. $ echo $(( 16#ff )) 255 This is the appropriate outcome. It would be undesirable for "ff" to be treated as a variable name identifier there. In your case, the error is that the letters "r", "e" and "s" have ordinal values that are too high to be valid for base 10, but they could have been valid for a higher base. $ echo $(( 29#res1 )) 671090 -- Kerin Millar
Re: math operations with base#prefix
On Mon, 18 Sep 2023 04:56:18 +0200 alex xmb ratchev wrote: > On Mon, Sep 18, 2023, 04:03 Kerin Millar wrote: > > > Hi Victor, > > > > On Sun, 17 Sep 2023, at 8:59 PM, Victor Pasko wrote: > > > Hi, > > > > > > Could you please take a look at attached bug.bash. > > > > > > Maybe, not all math combinations were presented there or the test has > > > duplications somehow. > > > Here are results of several runs with test# as argument > > > > > > > > > *% bash --version*GNU bash, version 5.2.15(3)-release (x86_64-pc-cygwin) > > > > > > Good test without argument but others with errors :( > > > *% ./bug.bash* > > > > > > res1=010 good 010 base8 > > > res2=03 good 03 base8 > > > res=17 good result 17 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > res1=8 good result 8 base10 > > > res1=10 good result 10 > > > res1=10 good result 10 > > > res1=010 good result 010 base8 > > > base10-res1=10 good result 10 > > > res1=16 good result 16 > > > > > > > > > *% ./bug.bash 1* > > > TESTCASE=1 > > > res1=010 good 010 base8 > > > res2=03 good 03 base8 > > > res=17 good result 17 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > ./bug.bash: line 29: let: res = base10#010 + base10#03 * 3: syntax error: > > > invalid arithmetic operator (error token is "#010 + base10#03 * 3") > > > > This seems like a misinterpretation of the manual. The manual states that > > numbers "take the form [base#]n, where the optional base is a decimal > > number between 2 and 64 representing the arithmetic base". As such, > > "base10" is not a decimal number between 2 and 64, whereas "10" would be. > > > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > res1=8 good result 8 base10 > > > res1=10 good result 10 > > > res1=10 good result 10 > > > res1=010 good result 010 base8 > > > base10-res1=10 good result 10 > > > res1=16 good result 16 > > > > > > > > > *% ./bug.bash 2* > > > TESTCASE=2 > > > res1=010 good 010 base8 > > > res2=03 good 03 base8 > > > res=17 good result 17 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > base10-res=19 good result 19 base10 (res1+3*res2) > > > ./bug.bash: line 35: let: res = 10#res1: value too great for base (error > > > token is "10#res1") > > > > For numbers in the form "[base#]n", it isn't practically possible for n to > > be specified using a variable without prefixing it with a sigil (so that it > > is treated as a parameter expansion and injected). There is a very good > > reason for this: numbers in a base higher than 10 can require alphabetical > > letters to be expressed. Consider the following example. > > > > $ echo $(( 16#ff )) > > 255 > > > > some were prefixed with 0 which make those also not work You're probably thinking of the 'September' issue but it isn't true. Consider the declarations of res1 and res2. res1=010 # base8 number res2=03 # base8 number The comments make it clear that Victor is expecting for those to be treated as base 8 (octal). At least, without qualifying the base. There is no issue there; both are perfectly valid octal numbers. $ echo $(( 010 )) $(( 03 )) 8 3 On the other hand, the base number in "base#n" notation may not have leading zeroes. $ echo $(( 064#1 )) bash: 064#1: invalid number (error token is "064#1") However, that particular mistake is never made in the program. -- Kerin Millar
Re: math operations with base#prefix
On Tue, 19 Sep 2023, at 8:40 AM, Victor Pasko wrote: > Thanks for your response. > In my opinion, in let "<>" and $((<>)) constructs all variables should be > evaluated, so that $-sign for them is to be just optional You haven't thought this through. It would amount to an egregious break of backward compatibility, even by the standards of bash. How would you expect for digits above 9 to be expressed for bases above 10? You might, perhaps, argue that the n in "base#n" could be quoted. $ ff='fe' $ echo $(( 16#ff )) # imaginarily prints 254 (instead of 255) $ echo $(( 16#'ff' )) # imaginarily prints 255 (instead of being an error of syntax) Keep in mind that this would render bash less consistent - and harder to reason with - than it already is. For example, $(( 'ff' )) is already an error of syntax in the Shell Command Language. Not to mention that it would further distance bash from other shells, such as ksh93 and zsh. Additionally, you imply that let and $(( should be treated specially. At least, it seems that way because you speak of those two arithmetic contexts, yet not of the others [1]. Why? [1] ((, for, array subscripts, string-slicing parameter expansions etc -- Kerin Millar
Re: math operations with base#prefix
On Tue, 19 Sep 2023 20:00:13 +0700 Robert Elz wrote: > Date:Tue, 19 Sep 2023 09:52:21 +0100 > From: "Kerin Millar" > Message-ID: <4c2e3d39-0392-41ae-b73c-3e17296a9...@app.fastmail.com> > > | On Tue, 19 Sep 2023, at 8:40 AM, Victor Pasko wrote: > | > Thanks for your response. > | > In my opinion, in let "<>" and $((<>)) constructs all variables should > be > | > evaluated, so that $-sign for them is to be just optional > | > | You haven't thought this through. > > You didn't think that through. This assertion would be more compelling had you explained at some point during the ensuing treatise how to potentially square the request being made with the base#n notation, as presently implemented by bash. > > | It would amount to an egregious break of backward compatibility, > > That's correct, but > > | How would you expect for digits above 9 to be expressed for bases above > 10? > > that's completely missing the point Victor was making. > > But Victor, they are all evaluated (in many shells, just once), if you > have > > a=10 > > then > $(( a + 5 )) and $(( $a + 5 )) > > achieve exactly the same thing.To be postable however if we also > have > > b=a > > then > $(( b + 5 )) is quite likely to fail, whereas $(( $b + 5 )) will > work fine (after expansions it becomes $(( a + 5 )) which is evaluated as > above. > > However, where the expansions are really needed is with > > o1=+ > o2=* > > used in > > $(( 10 $o1 5 $o2 6 )) > > If we were to write it instead as > > $(( 10 o1 5 o2 6 )) > > then the expression parser would have no idea at all how to > interpret that, as var names are not evaluated until needed in > the expression - what's there is just meaningless nonsense. But > when the variables have the $ ahead of them, the expression parser > sees > $(( 10 + 5 * 6 )) > > and can build the correct evaluation tree to generate 40 as the answer. > (The "10" could have been any of 'a' '$a' or '$b' given the values assigned > earlier, that would make no difference, the ' chars there are just for > clarity in this e-mail, not to be included). > > Much more complex expressions, which rely upon vars being expanded before > the expression is parsed, can also be created, but not really needed here. So, how would you accommodate Victor's request, theoretically speaking? Permitting n as an identifier for bases between 11 and 64 seems out of the question; there would be no reasonable way to disambiguate them from legitimate integer constants. Could they be tolerated for bases between 2 and 10? Perhaps. At that point they would no longer be integer constants and the interface fundamentally changes. To my mind, it would make for yet another exception to the rule to be borne in mind, and to potentially confound the expectations of those acquainted with the notation from having used it in other shells. However one might go about it, it requires for more thought to be invested than to simply state that $ be "optional" for the let builtin and for arithmetic expansion. -- Kerin Millar
Re: math operations with base#prefix
On Wed, 20 Sep 2023 01:41:30 +0700
Robert Elz wrote:
> Date:Tue, 19 Sep 2023 18:09:13 +0100
> From: Kerin Millar
> Message-ID: <20230919180913.bd90c16b908ab7966888f...@plushkava.net>
>
> | > | On Tue, 19 Sep 2023, at 8:40 AM, Victor Pasko wrote:
> | > | > in let "<>" and $((<>)) constructs all variables should be
> | > | > evaluated
>
> | This assertion would be more compelling had you explained at some point
> | during the ensuing treatise how to potentially square the request being
> | made with the base#n notation, as presently implemented by bash.
>
> I didn't even consider that case plausible, or what was intended, but now
> I can see that maybe it was - but that could never work. Not (quite) for
> the reason that you gave (or even Chet's explanation, though if he had
> explained why it is like it is, the explanation might have been like I
> am about to give), but because the syntax simply doesn't work out like that.
>
> Given a token of x#y that's not a variable, variables have no #'s in their
> names, so one cannot be expecting that (this would mean something entirely
> different if actually written) ${x#y} to be evaluated in that case.
>
> So, the only way to get variables out of that would be to split it into
> two (or three) tokens, x and #y or x # and y. One might parse it like
> that, and then evaluate x and y as variables, but if that were done, now
> we'd have 3 tokens, not the one (representing a number in some other base)
> to deal with, say 11 # 97 (where the 11 and 97 are now integers, not strings).
>
> That's not what was desired, which was 11#97 as one token (106 decimal, if
> my mental arithmetic is correct), and the only way to get it back would be
> to invent a new (very high priority, must be higher than unary '-' for
> example) # operator, which takes a base as its left operand, and a value
> as its right, and somehow reinterprets the value in that base - but that's
> essentially impossible, as we now have binary 97, which might have originally
> been 0141 or 0x61 - 11#0141 is an entirely different thing from 11#97
> and at this stage we simply wouldn't know which was intended.
>
> So that method can't work either.
>
> The $x#$y form works, as that (everything in $(( )) or other similar
> contexts) is being treated just like inside a double quoted string.
> Those get expanded first before being used, in this case as 11#97 (just
> as strings, variable expansion has no idea of the context, nor does it
> generally care what characters it produces) as a char sequence in the
> effectively double quoted string. The arith parser can then parse that,
> and see it has a specific base, and value - if y had been 0141 it would have
> been parsing 11#0141 instead, unlike a simple reference of 'y' in the
> expression, where all of 0x61 97 and 0141 turn into the binary value "97"
> for arithmetic to operate on).
>
> That's why I never even considered that might have been what was being
> requested, it can't work as hoped.
It is exactly the nature of the request. I don't know whether you looked at
Victor's "bug.bash" script. To recap, it contains a number of arithmetic
expressions, beginning with "res = res1 + res2 * 3" (clearly understanding it
to be fine). Ultimately, the script solicited a response concerning two
particular situations. Firstly, this.
let "res = base10#$res1 + base10#$res2 * 3"
Rightly dismissed as invalid syntax so there is nothing more to be said for
that.
Secondly, this.
# without $-signs before both res
let "res = 10#res1 + 3 * 10#res2" # let: res = 10#res1: value too great for
base (error token is "10#res1")
I explained in my initial post that the sigils are required because "res1" and
"res2" will otherwise be treated as integer values that (in this case) happen
not to be within the scope of the specified base. Victor did not respond to
this post but responded to a subsequent post by Chet, opining that $ should be
"optional" regardless. That is the point at which I concluded that the matter
had not been thought through.
Anyway, thank you for the additional commentary.
--
Kerin Millar
Re: math operations with base#prefix
On Tue, 19 Sep 2023 21:29:32 +0200
alex xmb ratchev wrote:
> On Tue, Sep 19, 2023, 21:14 Kerin Millar wrote:
>
> > On Wed, 20 Sep 2023 01:41:30 +0700
> > Robert Elz wrote:
> >
> > > Date:Tue, 19 Sep 2023 18:09:13 +0100
> > > From:Kerin Millar
> > > Message-ID: <20230919180913.bd90c16b908ab7966888f...@plushkava.net>
> > >
> > > | > | On Tue, 19 Sep 2023, at 8:40 AM, Victor Pasko wrote:
> > > | > | > in let "<>" and $((<>)) constructs all variables should be
> > > | > | > evaluated
> > >
> > > | This assertion would be more compelling had you explained at some
> > point
> > > | during the ensuing treatise how to potentially square the request
> > being
> > > | made with the base#n notation, as presently implemented by bash.
> > >
> > > I didn't even consider that case plausible, or what was intended, but now
> > > I can see that maybe it was - but that could never work. Not (quite) for
> > > the reason that you gave (or even Chet's explanation, though if he had
> > > explained why it is like it is, the explanation might have been like I
> > > am about to give), but because the syntax simply doesn't work out like
> > that.
> > >
> > > Given a token of x#y that's not a variable, variables have no #'s in
> > their
> > > names, so one cannot be expecting that (this would mean something
> > entirely
> > > different if actually written) ${x#y} to be evaluated in that case.
> > >
> > > So, the only way to get variables out of that would be to split it into
> > > two (or three) tokens, x and #y or x # and y. One might parse it like
> > > that, and then evaluate x and y as variables, but if that were done, now
> > > we'd have 3 tokens, not the one (representing a number in some other
> > base)
> > > to deal with, say 11 # 97 (where the 11 and 97 are now integers, not
> > strings).
> > >
> > > That's not what was desired, which was 11#97 as one token (106 decimal,
> > if
> > > my mental arithmetic is correct), and the only way to get it back would
> > be
> > > to invent a new (very high priority, must be higher than unary '-' for
> > > example) # operator, which takes a base as its left operand, and a value
> > > as its right, and somehow reinterprets the value in that base - but
> > that's
> > > essentially impossible, as we now have binary 97, which might have
> > originally
> > > been 0141 or 0x61 - 11#0141 is an entirely different thing from 11#97
> > > and at this stage we simply wouldn't know which was intended.
> > >
> > > So that method can't work either.
> > >
> > > The $x#$y form works, as that (everything in $(( )) or other similar
> > > contexts) is being treated just like inside a double quoted string.
> > > Those get expanded first before being used, in this case as 11#97 (just
> > > as strings, variable expansion has no idea of the context, nor does it
> > > generally care what characters it produces) as a char sequence in the
> > > effectively double quoted string. The arith parser can then parse that,
> > > and see it has a specific base, and value - if y had been 0141 it would
> > have
> > > been parsing 11#0141 instead, unlike a simple reference of 'y' in the
> > > expression, where all of 0x61 97 and 0141 turn into the binary value "97"
> > > for arithmetic to operate on).
> > >
> > > That's why I never even considered that might have been what was being
> > > requested, it can't work as hoped.
> >
> > It is exactly the nature of the request. I don't know whether you looked
> > at Victor's "bug.bash" script. To recap, it contains a number of arithmetic
> > expressions, beginning with "res = res1 + res2 * 3" (clearly understanding
> > it to be fine). Ultimately, the script solicited a response concerning two
> > particular situations. Firstly, this.
> >
> > let "res = base10#$res1 + base10#$res2 * 3"
> >
>
> me to get into the mails topic ..
> .. what are res1 and res2 values
It really doesn't matter (see below).
>
> Rightly dismissed as invalid syntax so there is nothing more to be said for
> > that.
> >
> > Secondly, this.
> >
> > # without $-signs before both res
> > let "res = 10#res1 + 3 * 10#res2" # let: res = 10#res1: value too great
> > for base (error token is "10#res1")
This is what matters. The implied complaint is that the $ symbols have to be
there and that they should somehow be optional. In other words, Victor wants
for "res = 10#res1 + 10#res2" to be able to consider res1 and res2 as
(variable) identifiers instead of integer constants. Both "res1" and "res2" are
perfectly valid integer constants for bases between 29 and 64.
$ echo $(( 29#res1 )) $(( 29#res2 ))
671090 671091
That is why bash correctly complains that the value is too great for the base
of 10. It doesn't matter whether res1 or res2 exist as variables, whether they
are set or what their values are. The n in base#n notation is always taken as a
number, so the only way to have n be the value of a variable is to expand it.
--
Kerin Millar
Re: math operations with base#prefix
On Tue, 19 Sep 2023 22:08:25 +0200 alex xmb ratchev wrote: > so u mean a $ sign requirement ? For dereferencing variable identifiers in base#n notation, yes. > i didnt get the base values , i tried simple one > i faced the ' without $ it doesnt work ' I don't fully understand this sentence. Anyway, it works as described in the manual. > > This is what matters. The implied complaint is that the $ symbols have to > > be there and that they should somehow be optional. In other words, Victor > > wants for "res = 10#res1 + 10#res2" to be able to consider res1 and res2 > > > optional as in make default values in if empty ? No. Victor wants for, say, "10#res1" to consider res1 as an identifier, instead of a number. I can't know how it could be stated any more clearly. -- Kerin Millar
Re: error message lacks useful debugging information
On Wed, 4 Oct 2023 20:05:41 -0400 Dave Cigna via Bug reports for the GNU Bourne Again SHell wrote: > Description: > > Attempting to tun an executable file (not a bash script) with the following > command: > > ./Candle > > the following error message is reported by bash: > > bash: ./Candle: cannot execute: required file not found > > The executable file 'Candle' does exist in the current directory; > if it didn't then bash would report a different error. > > The problem may be a missing dependency. However, the BUG is in bash in that That is, indeed, the problem. https://lists.gnu.org/archive/html/bug-bash/2022-05/msg00048.html https://lists.gnu.org/archive/html/bug-bash/2022-05/msg00050.html > it doesn't offer any useful debugging information. Debugging the issue could > go from "nearly hopeless" to "I think I can handle this" if bash simply > reported what required file was missing. i.e. path and filename. > > Repeat-By: > > Here's how I encountered the problem. You might not be able to reproduce > it on your machine, but that doesn't mean that it's not a bug with bash: While it can be considered as a matter of usability, it is not a bug in bash. The real problem (in so far as there is one) is that the kernel reports the error in a manner that is coarse enough to confuse users as to the matter of exactly which file is missing. That is, there is no distinct error code that exists to indicate that the missing file just happens to be one that the dynamic loader implemented by your OS was looking for. Moreover, bash is only the messenger. It is not realistic to expect for bash to have any genuine insight into (precisely) why a given kernel and userland - for a given distribution of a given OS - decides to raise ENOENT after being charged with executing a binary. The information that would be necessary to print any accurate debugging information is simply beyond the purview of the shell. Ironically, bash does already supply its own diagnostic message for ENOENT (the one you saw and reported), rather than print the native error string which would simply have been "No such file or directory". In principle, you could propose yet another improvement in its wording. In my view, the value in replacing or decorating native error messages is dubious. There is only so much that can be said without making too many assumptions about the underlying platform or, worse, implementing dodgy heuristics. Although, there is a heuristic that tries to determine whether ENOENT was caused by a script containing a shebang specifying an invalid interpreter, which does not help in this case. -- Kerin Millar
Re: Some minor notes on manual chapter 4 "Shell Builtin Commands"
On Mon, 9 Oct 2023 01:16:41 +0200 Martin Schulte wrote: > Hello, > > I took a closer look on the online manual chapter 4 "Shell Builtin Commands" > and found some inconsistencies: > > - true and false seem to be missing (in 4.1 Bourne Shell Builtins). > > The following has been tested with bash 5.2.15: > > - 3.7.5 Exit Status says: "All builtins return an exit status of 2 to > indicate incorrect usage, generally invalid options or missing arguments." > but cd with two or more non-optional arguments returns an exit status of 1. > > - The same is true if exit is called with two or more argument where the > first is numeric. This exit doesn't terminate bash. > > - When exit is invoked with a non-numeric first argument it terminates bash. > That seems to be inconsistent with the behaviour described before, while the > exit status of the shell is 2 and consistent in some way. Just to add that, while POSIX does not specify the behaviour of the exit builtin where its operand "is not an unsigned decimal integer or is greater than 255", it does forbid all special built-in utilities (including exit) from causing an interactive shell to exit on account of an error. So, for an interactive instance of bash to raise a "numeric argument required" error then proceed to exit seems contrary to the specification. Taking dash as a counter-example, its behaviour fulfils the requirements laid out by 2.8.1 Consequences of Shell Errors for both interactive and non-interactive instances. $ exec dash $ exit foo dash: 1: exit: Illegal number: foo $ echo $? 2 $ printf %s\\n 'exit foo' 'echo done' | dash dash: 1: exit: Illegal number: foo $ echo $? 2 -- Kerin Millar
Re: Some minor notes on manual chapter 4 "Shell Builtin Commands"
On Mon, 9 Oct 2023 10:35:20 -0400 Chet Ramey wrote: > On 10/8/23 7:16 PM, Martin Schulte wrote: > > > The following has been tested with bash 5.2.15: > > > > - 3.7.5 Exit Status says: "All builtins return an exit status of 2 to > > indicate incorrect usage, generally invalid options or missing arguments." > > but cd with two or more non-optional arguments returns an exit status of 1. > > There is surprising variance in behavior here, from a status of 2 to 1 > to 0 (dash), plus the silly ksh "substitute old for new in $PWD," which > the NetBSD sh (!) also performs. I agree that a status of 2 is reasonable. > > The historical sh behavior is to ignore additional arguments. > > > - The same is true if exit is called with two or more argument where the > > first is numeric. This exit doesn't terminate bash. > > More varying behavior. ash-based shells (dash, BSD sh, etc.) ignore any > additional arguments -- the historical sh behavior. bash and yash treat > it as a non-fatal error. mksh treats it as a fatal error, which I suppose > it can because `exit' is a posix special builtin. Posix makes it all > explicitly unspecified, even whether the return status is non-zero. > > > - When exit is invoked with a non-numeric first argument it terminates > > bash. That seems to be inconsistent with the behaviour described before, > > while the exit status of the shell is 2 and consistent in some way. > > Everyone does this (including the exit status of 2) except ksh93, which > simply ignores any error and exits with a 0 status. Posix makes the > behavior unspecfied. Although, not everyone does it in the case that the shell is interactive. There is scarcely any opportunity to read the ensuing diagnostic message before the terminal potentially closes. Here is how it looks in Apple's Terminal, which defaults to "Don't close the window". $ exit foo logout -bash: exit: foo: numeric argument required Saving session... ...copying shared history... ...saving history...truncating history files... ...completed. That is, bash chooses to treat it as an error (which is perfectly sensible) but exits the interactive instance as a consequence. I think that the latter behaviour goes against the spirit of section 2.8.1 and that it is neither helpful nor useful. Instead, I think that it should continue to print the diagnostic message and set $? to 2, but not exit the interactive instance. -- Kerin Millar
Re: Some minor notes on manual chapter 4 "Shell Builtin Commands"
On Mon, 9 Oct 2023 14:56:24 -0400 Chet Ramey wrote: > On 10/9/23 1:57 AM, Kerin Millar wrote: > > > Just to add that, while POSIX does not specify the behaviour of the exit > > builtin where its operand "is not an unsigned decimal integer or is greater > > than 255", it does forbid all special built-in utilities (including exit) > > from causing an interactive shell to exit on account of an error. > > Again, it's a mixed bag. The ash-derived shells and mksh exit, others Very mixed indeed. > (bash, yash) do not. I can see not exiting if the interactive shell is > in posix mode. I don't mean to nitpick over a minor issue but bash does also report that a numeric argument is "required", implying (at least, to my mind) that the built-in requires a valid argument to be able to act at all. Should bash remain able to exit in the interactive context, it might make sense to avoid that particular word. -- Kerin Millar
Re: variable set in exec'ing shell cannot be unset by child shell
On Fri, 13 Oct 2023 13:02:30 -0400 Ti Strga wrote: > First off, I have a feeling that GMail is going to garble the line > wrapping in this message; I cannot get it to stop being "helpful". > Apologies if that happens. > > I've encountered some behavior that I cannot find described anywhere in > the man page, and I'm hoping to learn whether it's a bug (it seems like > unintended behavior) or just a quirk for hysterical raisins. If it's > the latter then I'm also hoping there's a BASH_COMPAT level that might > adjust the behavior, although I'll state right now that I have no idea > whether previous versions behaved any differently. > > The summary is that if a parameter is set specifically for a '.'/'source' > command, and the source'd file calls 'exec' to run another script, then > that exec'd script cannot unset the parameter; if we want that parameter > to not be present in the exec'd script, then the source'd file must do > the unsetting prior to exec. > > We're running this... > > $ declare -p BASH_VERSINFO > declare -ar BASH_VERSINFO=([0]="5" [1]="2" [2]="15" [3]="3" > [4]="release" [5]="x86_64-pc-cygwin") > > ...although the platform [5] doesn't seem to matter; the same behavior > was reported to me on Linux as well as what I'm observing on Cygwin. I > did not have a chance to verify the Linux behavior firsthand. > > > === Background (or, I Promise This Isn't Just Code Golf) > > The example reproduction here is a calling script "outer" sourcing > "inner.sh". The real world situation is that "inner.sh" is a small > library of shell functions and environment variable setup for our workflow, > and multiple top-level scripts each '.' that library. > > The games here with exec are to support scripts that might be running for > a long time. For those we want the script to make a temporary copy of > itself and exec the temp copy, so that potential updates to the installed > scripts don't hose up the long-running shell when it suddenly reads from > a different point in the script.[*] The way it's implemented, the author > of the top-level script can simply set a parameter when sourcing the > library; the library makes the copy and performs the exec. When the copy > sets the same parameter and sources the library, the library detects the > cloning and will not keep doing it. (The library also fixes up what gets > reported as "name of current script" for error messages and whatnot, but > none of that is shown here as it doesn't change the weird behavior.) > > [*] Alternatively, there's the trick about putting the entire script > contents inside a compound statement to force the parser to read it all, > but that just makes the script harder for a human to read. Copy-and-exec > makes the top-level scripts cleaner IMHO. > > The kicker is that the parameters that trigger all this behavior must be > unset before going on with the remainder of the library and back to the > calling script. If not, then anytime a "cloned" script might call any > other script, that will be cloned as well even if its author did not write > anything saying to do that. (And in a couple cases, the scripts actually > start an interactive subshell; if the parameters get exported to there, > then "CLONE ALL THE THINGS" behavior just keeps propagating through the > scripts. Hilarity ensues.) > > > === Reproduction Bash employs dynamic scoping and the behaviour of unset can be confusing in some cases (with local variables in particular). However, given the code that you presented, I am unable to reproduce the claimed behaviour in any of 5.1.16, 5.2.15 and the devel branch. In the absence of a minimal reproducible example, it is difficult to comment. Incidentally, echo is not ideal as a tool for determining the state of a given variable. Consider "declare -p INSIDE OUTSIDE" instead. -- Kerin Millar
Re: Simple use of HOME with ${} instead of single $ sometimes doen't work... Ubuntu 22.04.3 LTS
On Fri, 20 Oct 2023 09:41:26 + (UTC)
Etienne Lorrain via Bug reports for the GNU Bourne Again SHell
wrote:
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -g -O2 -flto=auto -ffat-lto-objects -flto=auto
> -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
> -Wall
> uname output: Linux etienne-7950x 6.2.0-34-generic #34~22.04.1-Ubuntu SMP
> PREEMPT_DYNAMIC Thu Sep 7 13:12:03 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Bash Version: 5.1
> Patch Level: 16
> Release Status: release
>
> Description:
> simple unmodified cut&paste of a session:
>
> etienne@etienne-7950x:~$ /bin/bash --version
> GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
> Copyright (C) 2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>
> This is free software; you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> etienne@etienne-7950x:~$ /bin/bash
> etienne@etienne-7950x:~$ echo
> ${HOME}:/home/${USER}
> bash: ${HOME}:/home/${USER}: bad substitution
> etienne@etienne-7950x:~$ echo ${HOME}
> bash: ${HOME}: bad substitution
The above commands are chock full of ZERO-WIDTH SPACE characters, encoded as
UTF-8.
65 63 68 6f 20 24 7b e2 80 8b e2 80 8b e2 80 8b |echo ${.|
0010 e2 80 8b e2 80 8b e2 80 8b e2 80 8b 48 4f 4d 45 |HOME|
0020 7d e2 80 8b e2 80 8b e2 80 8b e2 80 8b e2 80 8b |}...|
0030 e2 80 8b e2 80 8b 0a |...|
0037
Bash prints these non-printing characters within the diagnostic message exactly
as they are, making the fact harder to diagnose. Nevertheless, it is quite
correct in pointing out that it is a bad substitution.
--
Kerin Millar
Re: Fwd: Strange results
On Fri, 27 Oct 2023 02:00:01 +0700
Victor Pasko wrote:
> -- Forwarded message -
> From: Victor Pasko
> Date: Fri, Oct 27, 2023 at 1:57 AM
> Subject: Re: Strange results
> To: Dennis Williamson
>
>
>
> Also
>
> echo10 ${ASCII_SET:$((-10)):1}
This is the "Substring Expansion" kind of parameter expansion.
>
> and
>
> echo11 ${ASCII_SET:-10:1}
This is the "Use Default Values" kind of parameter expansion.
>
> have different behaviour:(
Substring expansions already imply a numeric context. A single pair of
enclosing brackets is enough to avoid this pitfall.
${ASCII_SET:(-10):1}
Another method is to have a leading space.
${ASCII_SET: -10:1}
>
> Both of these say "output the character that's 10th from the end" which is
> > "u". What did you expect it to output?
> >
> > echo "echo11 ${ASCII_SET:-10:1}"
> >
>
> Firstly, expected the only one symbol from ASCII_SET string
>
> This says, according to the man page:
> >
> >${parameter:-word}
> > Use Default Values. If parameter is unset or null, the
> > expansion of word is substituted. Otherwise, the value of parameter is
> > substituted
> >
> > which means output "10:1" if ASCII_SET is unset or null. Since it isn't,
> > the contents of that variable are output giving you a long sequence of
> > ASCII characters.
> >
>
> But ASCII_SET is not unset so -word must not be used
It behaves precisely as the manual states. The parameter, ASCII_SET, is neither
unset nor null (empty). Therefore, the value of the parameter is substituted,
rather than the given word of "10:1".
--
Kerin Millar
Re: Fwd: Strange results
On Fri, 27 Oct 2023 19:28:15 +0700
Victor Pasko wrote:
> See my comments below inline
>
> On Fri, Oct 27, 2023 at 2:50 AM Kerin Millar wrote:
>
> > On Fri, 27 Oct 2023 02:00:01 +0700
> > Victor Pasko wrote:
> >
> > > -- Forwarded message -
> > > From: Victor Pasko
> > > Date: Fri, Oct 27, 2023 at 1:57 AM
> > > Subject: Re: Strange results
> > > To: Dennis Williamson
> > >
> > >
> > >
> > > Also
> > >
> > > echo10 ${ASCII_SET:$((-10)):1}
> >
> > This is the "Substring Expansion" kind of parameter expansion.
> > >
> > > and
> > >
> > > echo11 ${ASCII_SET:-10:1}
> >
> > This is the "Use Default Values" kind of parameter expansion.
> >
> > >
> > > have different behaviour:(
> >
> > Substring expansions already imply a numeric context. A single pair of
> > enclosing brackets is enough to avoid this pitfall.
> >
> > ${ASCII_SET:(-10):1}
> >
> > Another method is to have a leading space.
> >
> > ${ASCII_SET: -10:1}
> >
>
> Well, it's kind of a workaround to use brackets or extra space, but how to
> recognize such expectations according to string operation with -10:1 ?
That much is easy to explain.
${ASCII_SET:-
^ At this point, it looks like a "Default Value" parameter expansion
Thefore, it will be treated as one. If your intention is to use a negative
offset with a Substring Expansion then you must write your code in such a way
that it can be disambiguated by the parser. Here are some other ways of going
about it.
echo "${ASCII_SET:0-10:1}"
i=-10; echo "${ASCII_SET:i:1}"
Keep in mind that the Shell Command Language specification requires that
"Default Value" parameter expansion be implemented in the way that it is, and
that there are countless scripts that depend on the status quo. However, the
Shell Command Language also doesn't suffer from this ambiguity because it
doesn't specify any means of performing "Substring Expansion" to begin with
(it's a bash extension). Since there is no way for bash to otherwise know that
"-10:1" wasn't intended as the word in ${parameter:-word}, you'll have to
choose a preferred workaround for negative offsets and live with it, as irksome
as it may be.
As concerns your other questions, I shall respond to them in a separate message.
--
Kerin Millar
Re: Fwd: Strange results
On Fri, 27 Oct 2023 15:34:16 +0100
Kerin Millar wrote:
> Keep in mind that the Shell Command Language specification requires that
> "Default Value" parameter expansion be implemented in the way that it is, and
> that there are countless scripts that depend on the status quo. However, the
> Shell Command Language also doesn't suffer from this ambiguity because it
> doesn't specify any means of performing "Substring Expansion" to begin with
> (it's a bash extension). Since there is no way for bash to otherwise know
> that "-10:1" wasn't intended as the word in ${parameter:-word}, you'll have
> to choose a preferred workaround for negative offsets and live with it, as
> irksome as it may be.
Of course, I meant to write "10:1" there, not "-10:1".
--
Kerin Millar
Re: Fwd: Strange results
On Fri, 27 Oct 2023 19:28:15 +0700
Victor Pasko wrote:
> Let me ask more questions presented in the sent bug2.bash
> Look at the following lines:
>
> *printf -v a_int '%d' "'a" *# very strange syntax here to use char as
> integer
> echo "echo4 $a_int" # should be 97 at the end
All implementations of the standard printf utility are required to support
this, as expressed by the third-from-last paragraph of its EXTENDED
DESCRIPTION, and the bullet points that follow it.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html
>
>
> echo "echo5 *${ASCII_SET:$((a_int-12)):1}" *# you can see letter u at the
> end as at 97-12=85 place
> echo "echo5.1 ${ASCII_SET:a_int-12:1}"# you can see letter u at the end
> as at 97-12=85 place
>
> 1) Could you please suggest elegant way to cast string symbol to
> integer instead of printf for * "'a"*
Your request implies that printf is ill-suited to the task but I do not see
why. It allows for you to convert a character to its ordinal value according to
the system's character type (LC_CTYPE). What is your objection to using it? If
it simply that you don't consider it to be elegant, there are many other
languages to choose from.
> 2) Here are some unsuccessful examples to use bitwise operations with
> symbols:
>
> % echo $(("'a" >> 4))
> -bash: 'a >> 4: syntax error: operand expected (error token is "'a >> 4")
You are in an arithmetic context there, and must abide by its rules of syntax
(which are mostly those of ANSI C, not the printf utility). Obtain the integer
value first.
$ char=a; printf -v ord %d "'${char}"; echo $(( ord >> 4 ))
6
--
Kerin Millar
Re: TAB completion bug
On Tue, 5 Dec 2023 23:46:51 +
Ole Tange via Bug reports for the GNU Bourne Again SHell
wrote:
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -g -O2
> uname output: Linux aspire 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
> 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Bash Version: 5.2
> Patch Level: 21
> Release Status: release
>
> Description:
> Tested on git (2023-12-06).
>
> For the (admitedly weirdly named) dirs below TAB completion does not work
> correctly.
>
> Repeat-By:
> #!/bin/bash
>
> # TAB works
> # $ ls -l ta
> #
> # Tab completes but is escaped wrongly:
> # $ ls -l ta
> #
I can confirm this for both 5.2.21 and the development branch. The backticks
are not quoted as they ought to be, resulting in a command substitution.
>
> mkdir -p 'tab/
> `/tmp/trip`>/tmp/tripwire;
> '"'"'@
> # These give the same
> # $ ls -l tw
> # $ ls -l tw
> # But the last should include tmp
I was not able to reproduce this, however.
$ cd ta
$ echo "${PWD@Q}"
$'/home/kerin/tange/tab/\n`/tmp/trip`>'
$ cd ../../../..
$ cd ta
$ echo "${PWD@Q}"
$'/home/kerin/tange/tab/\n`/tmp/trip`>/tmp'
That was with programmable completion disabled (shopt -u progcomp) and the
following directory structure in place.
$ LC_ALL=C find . -mindepth 1 -exec ls -1d --quoting-style=c {} +
"./tab"
"./tab/\n`"
"./tab/\n`/tmp"
"./tab/\n`/tmp/trip`>"
"./tab/\n`/tmp/trip`>/tmp"
"./tab/\n`/tmp/trip`>/tmp/tripwire;\n'@/tmp/tripwire;\n'@
