Error message garbage when parameter expansion used inside (()) and variable unset
Section 6.5 Shell Arithmetic says, "Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax. A shell variable that is null or unset evaluates to 0 when referenced by name without using the parameter expansion syntax." - http://www.gnu.org/software/bash/manual/bash.html#Shell-Arithmetic The above tells us what happens to an unset variable if not using parameter expansion. But if a shell variable uses parameter expansion and is null or unset, what does it evaluate to inside (()) syntax? It evaluates to 0 inside [[]], but gives an error inside (()). The error message is different between bash 4.3.11 and 4.4.19. The later contains garbage. See test 3 and 4 below. ### Test 1: # No parameter expansion. var2 not set. Should evaluate to 0 thus when compared # to 0, 1A and 1B should echo yes. They do. echo;echo 1A ( set -x;var=0;var1=var; [[ var1 -eq var2 ]] && echo yes || echo no ) echo 1B ( set -x;var=0;var1=var; (( var1 == var2 )) && echo yes || echo no ) ### Test 2: # No parameter expansion. var2 not set. Should evaluate to 0 thus when compared # to 5, 2A and 2B should echo no. They do echo;echo 2A ( set -x;var=5;var1=var; [[ var1 -eq var2 ]] && echo yes || echo no ) echo 2B ( set -x;var=5;var1=var; (( var1 == var2 )) && echo yes || echo no ) ### Test 3: # Parameter expansion. var2 not set. echo;echo 3A ( set -x;var=0;var1=var; [[ var1 -eq $var2 ]] && echo yes || echo no ) echo 3B ( set -x;var=0;var1=var; (( var1 == $var2 )) && echo yes || echo no ) ### Test 4: echo;echo 4A ( set -x;var=5;var1=var; [[ var1 -eq $var2 ]] && echo yes || echo no ) echo 4B ( set -x;var=5;var1=var; (( var1 == $var2 )) && echo yes || echo no ) It appears that 3A and 4A evaluate to 0 because of the arithmetic context. 3A echo's yes; 4A echo's no. The problem is what is happening with 3B and 4B. I tested on bash 4.3.11 and bash 4.4.19 and got a slightly different error message. Bash version 4.3.11: ./tt: line 18: var1: var1 == : syntax error: operand expected (error token is "== ") Bash version 4.4.19 ( was garabage): ./tt: line 18: : var1 == : syntax error: operand expected (error token is "== ") Peggy Russell
Re: Error message garbage when parameter expansion used inside (()) and variable unset
Chet, is the output on opensuse running bash 4.4.19, correct?
The specific output:
./t.sh: line 9: ���#V: var1 == : syntax error: operand expected (error
token is "== ")
archlinux has the same version of bash and I got the same results as on
opensuse.
Below are the details of running ./t.sh &>t.log on multiple versions of
bash.
When I run this script (t.sh),
..
#!/bin/bash
# system info
lsb_release -d
printf -- 'Bash Version: %s\n\n' "${BASH_VERSION}"
# test example
echo 3B
( set -x; var=0;var1=var; (( var1 == $var2 )) && echo yes || echo no )
..
I get,
..
Description: openSUSE Tumbleweed
Bash Version: 4.4.19(1)-release
3B
+ t.sh line 9 : var=0
+ t.sh line 9 : var1=var
+ t.sh line 9 : (( var1 == ))
./t.sh: line 9: ���#V: var1 == : syntax error: operand expected (error
token is "== ")
+ t.sh line 9 : echo no
no
..
..
Description:Linux Mint 17.3 Rosa
Bash Version: 4.3.11(1)-release
3B
+ t.sh line 9 : var=0
+ t.sh line 9 : var1=var
+ t.sh line 9 : (( var1 == ))
./t.sh: line 9: var1 == : var1 == : syntax error: operand expected
(error token is "== ")
+ t.sh line 9 : echo no
no
..
If I remove the "set -x" I get,
..
Description: openSUSE Tumbleweed
Bash Version: 4.4.19(1)-release
3B
./t.sh: line 9: ((: var1 == : syntax error: operand expected (error
token is "== ")
no
..
..
Description:Linux Mint 17.3 Rosa
Bash Version: 4.3.11(1)-release
3B
./t.sh: line 9: ((: var1 == : syntax error: operand expected (error
token is "== ")
no
..
--
Peggy Russell
On 04/03/2018 08:49 AM, Chet Ramey wrote:
On 4/2/18 5:16 PM, PRussell wrote:
Section 6.5 Shell Arithmetic says,
"Within an expression, shell variables may also be referenced by name without
using the parameter expansion syntax. A shell variable that is null or unset
evaluates to 0 when referenced by name without using the parameter expansion
syntax." - http://www.gnu.org/software/bash/manual/bash.html#Shell-Arithmetic
The above tells us what happens to an unset variable if not using parameter
expansion.
But if a shell variable uses parameter expansion and is null or unset, what
does it evaluate to inside (()) syntax?
Since ((...)) is equivalent to let "...", as noted in the description of
`((' in the man page and info doc, it expands to the same thing that it
would when you perform a double-quoted word expansion. If you expand a
shell variable that's null or unset, you get the usual: it disappears.
The problem is what is happening with 3B and 4B. I tested on bash 4.3.11 and
bash 4.4.19 and got a slightly different error message.
Bash version 4.3.11:
./tt: line 18: var1: var1 == : syntax error: operand expected (error token is "==
")
Bash version 4.4.19 ( was garabage):
./tt: line 18: : var1 == : syntax error: operand expected (error token is "==
")
I don't get this. I get `((' as the command name in the error message for
both bash-4.3.46 and bash-4.4.19:
./x18: line 1: ((: var1 == : syntax error: operand expected (error token
is "== ")
Re: Error message garbage when parameter expansion used inside (()) and variable unset
Hi,
The error seems to be localized to the expansion of PS4 when "set -x" is active.
Please see sample script below.
I am aware of the unusual parameter expansion for FUNCNAME. There might be a
local historical reason. :-)
It does not happen outside of the PS4 expansion. It also behaves differently on
4.3 vs 4.4.
On 4.4, valgrind shows "Invalid Reads".
On 4.3, valgrind shows no errors.
I included below a small part of valgrind's output. If you need more let me
know.
Sample script:
.
#!/bin/bash
function main() {
# Shows different successful parameter expansions of FUNCNAME
echo -n 'declare -p FUNCNAME:'
declare -p FUNCNAME
# parentheses surrounding expansion
echo '(${FUNCNAME:+${FUNCNAME[0]##*/}})':"(${FUNCNAME:+${FUNCNAME[0]##*/}})"
# parentheses inside expansion
echo '${FUNCNAME:+(${FUNCNAME[0]##*/})}':"${FUNCNAME:+(${FUNCNAME[0]##*/})}"
echo
# PS4: Use FUNCNAME in PS4 - parentheses surrounding expansion *No ERROR*
declare -x PS4='+ ${BASH_SOURCE[0]##*/} line ${LINENO}
(${FUNCNAME:+${FUNCNAME[0]}}):'
declare -p PS4
echo "PS4=${PS4}"
set -x; var=0;var1=var; (( var1 == $var2 )) && echo yes || echo no
echo
# PS4: Use FUNCNAME in PS4 - parentheses inside expansion **ERROR HERE**
declare -x PS4='+ ${BASH_SOURCE[0]##*/} line ${LINENO}
${FUNCNAME:+(${FUNCNAME[0]})}:'
declare -p PS4
echo "PS4=${PS4}"
set -x; var=0;var1=var; (( var1 == $var2 )) && echo yes || echo no
}
main
.
Partial valgrind output:
.
==759== Memcheck, a memory error detector
==759== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==759== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==759== Command: ./x19
==759==
declare -p FUNCNAME:declare -a FUNCNAME=([0]="main" [1]="main")
${FUNCNAME:+(${FUNCNAME[0]##*/})}:(main)
(${FUNCNAME:+${FUNCNAME[0]##*/}}):(main)
${FUNCNAME[0]##*/}:(main)
${FUNCNAME[0]}:(main)
${FUNCNAME}:(main)
declare -x PS4="+ \${BASH_SOURCE[0]##*/} line \${LINENO}
(\${FUNCNAME:+\${FUNCNAME[0]}}):"
PS4=+ ${BASH_SOURCE[0]##*/} line ${LINENO} (${FUNCNAME:+${FUNCNAME[0]}}):
==759== Invalid read of size 16
==759==at 0x533D488: __wcsnlen_sse4_1 (in /usr/lib/libc-2.26.so)
==759==by 0x532D5C2: wcsrtombs (in /usr/lib/libc-2.26.so)
==759==by 0x128D4C: ??? (in /usr/bin/bash)
==759==by 0x160243: ??? (in /usr/bin/bash)
==759==by 0x160B96: ??? (in /usr/bin/bash)
==759==by 0x161D0D: ??? (in /usr/bin/bash)
==759==by 0x1637FB: expand_prompt_string (in /usr/bin/bash)
==759==by 0x12D530: decode_prompt_string (in /usr/bin/bash)
==759==by 0x13BD92: indirection_level_string (in /usr/bin/bash)
==759==by 0x13C078: xtrace_print_assignment (in /usr/bin/bash)
==759==by 0x15CD12: ??? (in /usr/bin/bash)
==759==by 0x165835: ??? (in /usr/bin/bash)
==759== Address 0x5935610 is 0 bytes after a block of size 16 alloc'd
==759==at 0x4C2CEDF: malloc (vg_replace_malloc.c:299)
==759==by 0x532C5EF: wcsdup (in /usr/lib/libc-2.26.so)
==759==by 0x128BB9: ??? (in /usr/bin/bash)
==759==by 0x160243: ??? (in /usr/bin/bash)
==759==by 0x160B96: ??? (in /usr/bin/bash)
==759==by 0x161D0D: ??? (in /usr/bin/bash)
==759==by 0x1637FB: expand_prompt_string (in /usr/bin/bash)
==759==by 0x12D530: decode_prompt_string (in /usr/bin/bash)
==759==by 0x13BD92: indirection_level_string (in /usr/bin/bash)
==759==by 0x13C078: xtrace_print_assignment (in /usr/bin/bash)
==759==by 0x15CD12: ??? (in /usr/bin/bash)
==759==by 0x165835: ??? (in /usr/bin/bash)
==759==
==759== Invalid read of size 16
==759==at 0x533D48D: __wcsnlen_sse4_1 (in /usr/lib/libc-2.26.so)
==759==by 0x532D5C2: wcsrtombs (in /usr/lib/libc-2.26.so)
.
--
Peggy Russell
On 04/03/2018 01:15 PM, Chet Ramey wrote:
> I don't see the same type of memory corruption. I get:
>
> chet-mail(1)$ lsb_release -d
> Description: Red Hat Enterprise Linux Server release 6.9 (Santiago)
> chet-mail(1)$ cat ./x18
> ( set -x;var=0;var1=var; (( var1 == $var2 )) && echo yes || echo no )
> chet-mail(1)$ ./bash -c 'echo $BASH_VERSION'
> 4.4.19(4)-release
> chet-mail(1)$ ./bash ./x18
> + var=0
> + var1=var
> + (( var1 == ))
> ./x18: line 1: ((: var1 == : syntax error: operand expected (error token
> is "== ")
> + echo no
> no
>
> But otherwise the results are correct.
>
> If you'd like, take a look at running your version under valgrind or a
> similar tool to see if bash is touching freed memory. (I don't happen to
> see that running on RHEL, but your results may vary with a distribution-
> compiled version.)
