Bash parameter transforamtion on empty array triggers unset variable.
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-musl
Compiler: gcc
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux 28e237a5e16f 5.5.7-200.fc31.x86_64 #1 SMP Fri Feb 28
17:18:37 UTC 2020 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-musl
Bash Version: 5.1
Patch Level: 0
Release Status: alpha
Description:
I do not know if this is related to bash 5.1 erroneously being
"a little aggressive about skipping over empty strings" mentioned
in "Declaring arrays with empty string in one line is bugged", but using
parameter transformation on an empty array, throws an error if "set -u" is
turned on. This was not how bash 4.4 and 5.0 worked. This bug makes it
impossible to check if an empty variable is an array using parameter
transformation while "set -u" is turned on
Repeat-By:
# Indirection
docker run -it --rm bash:5.1-alpha bash -uc 'foo(){ echo "${!1@a}";
}; bar=(); foo bar'
# Empty array
docker run -it --rm bash:5.1-alpha bash -uc 'bar=(); echo "${bar@a}"'
# Declared unset array
docker run -it --rm bash:5.1-alpha bash -uc 'declare -a bar; echo
"${bar@a}"'
# Empty associative array
docker run -it --rm bash:5.1-alpha bash -uc 'declare -A Bar=();
echo "${Bar@a}"'
# Declared unset associative array
docker run -it --rm bash:5.1-alpha bash -uc 'declare -A Bar; echo
"${Bar@a}"'
# I also tested on bash:devel-20200805, with the same results
Fix:
# All five of the above examples work in bash 4.4 and 5.0, and I
would expect the same behavior in bash 5.1
# Expected behavior that is already there in 5.1 alpha
docker run -it --rm bash:5.1-alpha bash -uc 'bar=(); echo
"${bar[@]}' # succeeds, but this is already consistent with bash 4.4 and
5.0. So this is expected to succeed.
docker run -it --rm bash:5.1-alpha bash -uc 'bar=(); echo
"${bar[@]+set}' # echos nothing, but this is already consistent with bash
3.2 and 5.0. So this is expected behavior
docker run -it --rm bash:5.1-alpha bash -uc 'bar=(); echo "${bar}'
# fails, but this is already consistent with bash 4.4 and 5.0. So this is
expected to fail, there is no 0th element to the array.
docker run -it --rm bash:5.1-alpha bash -uc 'bar=(); echo
"${bar+set}' # echos nothing, but this is already consistent with bash 3.2
and 5.0. So this is expected behavior
Thanks
Re: Bash parameter transforamtion on empty array triggers unset variable.
Ah, I see the confusion.
The issue you pointed out, "@Q breaks set -o nounset" refers to quote
parameter expansion, as does the line in CHANGES-5.1, 1.q, which makes
sense to call this a bug that was allowed in bash 4.4 and 5.0.
I should have specified, the focus of this issue is the "@a" expansion. It
makes sense that @Q/E/P/A expansion should not work on unset variables with
nounset enabled. However, @a is uniquely different, in that it does not
have to do with the value of the variable, but rather the variable type. My
"set -eu" compatible library uses the @a expansion (for bash 4.4 and newer)
because it had previously worked on unset values with set -u enabled, which
was a very useful feature.
Here are 3 specific details I would like to address:
1. @a expansion should work on unset variables with "set -u" in bash 5.1.
It seems like the correct thing to do. Only @a expansion. This has been a
very useful feature in bash 4.4 and 5.0.
Should fail: (set -eu; declare -a x; echo "${x@Q}")
Should not fail: (set -eu; declare -a x; echo "${x@a}")
2. With "set -u", the following works in bash 4.4 or newer (and makes sense
that it works): (set -eu; x=(); echo "${x[@]}")
Here x is not unset, it is set to an empty array. This expansion make sense
with nounset turned on because x is not unset, it is set to ()
However, this fails: (set -eu; x=(); echo "${x@a}")
This is an inconsistent behavior, and it seems to me the ${x@a} should
definitely work here, with nounset turns on
3. The same as #2, but for associative arrays
Works: (set -eu; declare -A x=(); echo "${x[@]}")
Does not work, but should: (set -eu; declare -A x=(); echo "${x@a}")
Thanks
On Tue, Aug 11, 2020 at 9:51 AM Chet Ramey wrote:
> On 8/10/20 5:52 PM, Andrew Neff wrote:
>
> > Bash Version: 5.1
> > Patch Level: 0
> > Release Status: alpha
> >
> > Description:
> > I do not know if this is related to bash 5.1 erroneously being
> > "a little aggressive about skipping over empty strings" mentioned
> > in "Declaring arrays with empty string in one line is bugged", but using
> > parameter transformation on an empty array, throws an error if "set -u"
> is
> > turned on. This was not how bash 4.4 and 5.0 worked. This bug makes it
> > impossible to check if an empty variable is an array using parameter
> > transformation while "set -u" is turned on
>
> This was a bug in bash-5.0 (and 4.4) and was fixed in early March 2019 as
> the result of
>
> https://lists.gnu.org/archive/html/bug-bash/2019-03/msg00010.html
>
> It's in CHANGES.
>
> There are some other changes in how bash displays attributes of unset
> variables when `nounset' is not enabled, but unset variables used in word
> expansions should trigger an error -- with the usual @/* exceptions -- when
> set -u is enabled.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
caller returns wrong line number in command substitution
Machine: x86_64
OS: linux-musl
Compiler: gcc
Compilation CFLAGS: -g -O2
uname output: Linux cfa1574b05c7 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed
Mar 2 00:30:59 UTC 2022 x86_64 GNU/Linux
uname output: Linux 3beae0f31cdf 5.18.18-100.fc35.x86_64 #1 SMP PREEMPT_DYNAMIC
Wed Aug 17 16:09:22 UTC 2022 x86_64 Linux
Machine Type: x86_64-pc-linux-musl
Docker: docker run -it --rm bash:5.2
Description:
Using the "caller" command on a line of bash code in a process substitution
has been incorrect from bash 3.2 through 5.1, but I could write my code in such
a way to work around it. Some of these workarounds no longer function in bash
5.2. I saw that you made some changes to this code [see below], however, I
think they introduced another regression to how caller calculates the line
offset. In the following tests, caller should always return the last line in a
multi-line bash call, but that is not the case now in a process substitution in
bash 5.2 (test 5)
[quote]
c. Rewrote the command substitution parsing code to call the parser
recursively and rebuild the command string from the parsed command. This allows
better syntax checking and catches errors much earlier. Along with this, if
command substitution parsing completes with here-documents remaining to be
read, the shell prints a warning message and reads the here-document bodies
from the current input stream
Repeat-By:
Here's a small script to repeat the problem:
#!/usr/bin/env bash
function foobar()
{
caller >&2
}
foobar "test0
...
bar"
true <(foobar "test1
...
bar")
true <(
foobar "test2
...
bar")
true <(\
foobar "test3
...
bar")
read -rd '' foo < <(foobar "test4
...
bar") || :
read -rd '' foo < <(
foobar "test5
...
bar") || :
read -rd '' foo < <(\
foobar "test6
...
bar")
read -rd '' foo < \
<(foobar "test7
...
bar")
Results:
Test Ans 5.1 5.2
08 8 8
111 13 13
215 18 17
319 21 21
422 22 22
526 26 25
630 29 29
734 33 33
Summary:
Test 0: no command substitution, it works.
Test 1: the answer is off by the number of lines. So 1 line is right, 2 lines
if off by one line, 3 lines (as show) is off by 2. If it was 10 lines, the
answer would be off by 9.
Test 2: Same offset as Test 1. On bash 3.2-5.1 off by one additional line
Test 3: Same as test 1
Test 4: Actually right
Test 5: Off by -1 lines in bash 5.2, right on bash 3.2-5.1
Test 6: Always off by -1 lines
Test 7: Same as Test 6
Fix/Workarounds:
Only the Test0 and Test4 consistently give the right answer. However for
readability and other reasons, I don't always want those syntaxes.
Re: caller returns wrong line number in command substitution
Oops, my mistake. Got some terms mixed up a little there. Yes, every time I command substitution, I meant process substitution. So that release note for "Rewrote the command substitution" most likely nothing to do with this. From: Robert Elz Sent: Tuesday, October 18, 2022 7:09 PM To: Andrew Neff Cc: bug-bash@gnu.org Subject: Re: caller returns wrong line number in command substitution There are no command substitutions in any of your examples. A command substitution is what you get with $( ) (or if you really like obsolete syntax for some reason, ``). What you're showing is process substitution. This has nothing to do with whether or not there is an issue here that's worth fixing however - but bash's management of line numbers has always been "close enough" rather than "accurate". kre
Exit trap changes return value of subshell for uncaught trap signals
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
uname output: Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2
00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
uname output: Linux 17d50c74abf4 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed
Mar 2 00:30:59 UTC 2022 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu
Environment Bugged: docker run -it --rm bash:5.1
Environment Working: docker run -it --rm bash:5.0
Bash Version: 5.1
Patch Level: 12
Release Status: release
Description:
I discovered an inconsistent bug starting in bash 5.1.12. Setting the
EXIT trap in the grandparent of a subshell that exits via an unset trap signal
causes different return values. If the EXIT trap is unset, the subshell returns
128+signal number (e.g. 138 , for SIGUSR1 (10)). However when the EXIT trap is
set in the grandparent, the subshell returns 0 instead.
Repeat-By:
Run this code in bash 5.1.12 or newer. You’ll see “Error: 138” both
before and after the EXIT signal is set on the older bashes, but not on 5.1.12
or newer.
#!/usr/bin/env bash
set -eu
child_signal()
(
trap "echo you will not see this" USR1
(
kill -USR1 "${BASHPID}"
# echo "Uncomment this to mysteriously fix"
) || echo "Error: ${rv=${?}}"
[ "${rv-0}" != "0" ]
echo
)
echo "No exit trap"
child_signal
echo "Exit trap set"
trap 'echo atexiting...' EXIT
child_signal
echo "The end"
Observed Erroneous Result:
No exit trap
User defined signal 1
Error: 138
Exit trap set
atexiting...
Expected Result (e.g., as in 5.1.11):
No exit trap
User defined signal 1
Error: 138
Exit trap set
User defined signal 1
Error: 138
The end
atexiting...
Fix:
The bug started in the 5.1.12 patch, and it does not happen in 5.1.11.
The bug was still present in the 2022-10-15 devel branch. I am only able to
manifest it with the EXIT signal. I could not reproduce it with other signals
(e.g. “set -T” + the RETURN signal)
One workaround: uncomment the “Uncomment this to mysteriously fix” line
and the bug disappears.
