Fixing coproc race condition bug
#!/usr/bin/env bash
set -e -u
coproc date
sleep 1
# In bash 5.0.17, this aborts with "COPROC[0]: unbound variable"
# if the coproc exits before the main process gets here.
read -u ${COPROC[0]} line
# Discussion:
#
# To prevent this race condition, Bash must not close the coproc output pipe
# when the coprocess exits, but keep it open until the main process has had
# a chance to read buffered data.
#
# However bash must prevent unbounded leaking file descriptors/pipes
# -- see ideas below.
#
# (The coproc *input* pipe can be closed when the coprocess exits because
# it can no longer be used -- a SIGPIPE error would occur if written to.)
#
# SUGGESTED STRATEGY:
#
# Close the coproc output pipe after the main process reads EOF from it.
# --This handles all cases where the coproc writes but does not read.
#
# Close both pipes before executing another 'coproc' command with the
# same effective NAME, i.e. which will overwrite the "COPROC" array.
# --This prevents leaking fds in loops or repetitive code.
#
# Close both pipes and undef COPROC if the main process reaps a coproc
# child with 'wait'.
# --This allows scripts to explicitly release coproc resources
#
# The above is probably good enough.
#
# Another tweak would be define a "no-op" coproc command to be a special
# case which does not create new pipes, but only closes any previous pipes
# associated with the same NAME (as described above).
# e.g.
# coproc; # implied NAME is "COPROC"
# coproc NAME {}; # explicit NAME
#
# However this would be unnecessary (i.e. redundant) if doing 'wait' on
# a coproc child implicitly closes it's pipes.
#
# Note: Ignore errors when closing pipes, as the user might have
manually closed
# the fds using 'exec {varname}>&-' or similar.
#
#(END)
Re: Fixing coproc race condition bug
On 10/24/21 1:44 PM, Jim Avera wrote:
# Note: Ignore errors when closing pipes, as the user might have
manually closed
# the fds using 'exec {varname}>&-' or similar.
Actually this isn't a good idea because the fd might have been re-used
for something else; so an error should be reported to make that
bug-condition noticeable so it can be fixed.
Instead, bash should search all active CPROC variables for any file
descriptors being closed explicitly, and undef the corresponding COPROC
slot; that will prevent bash from calling close() on a previously-closed
file descriptor.
-Jim
set -e in (subshells) does not work independently of outer context
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 -Wall uname output: Linux lxjima 3.0.0-15-generic #25-Ubuntu SMP Mon Jan 2 17:44:42 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.2 Patch Level: 10 Release Status: release Description: set -e in (subshells) should be independent of surrounding context. The man page says "[set -e] applies to the shell environment and each subshell environment separately", but actually set -e is prevented from working in a (subshell) if it is disabled in the surrounding context. Repeat-By: set +e ( set -e cat /non/existent/path echo "Did not abort. Isn't this a bug?" ) || true Discussion: Set -e rightly ignores errors in the outer shell because of the || expression, but it is not clear why that should have any effect inside the (subshell) expression. For example, if you say set -e set -o errtrace trap 'echo -e "Command failed unexpectedly at \c"; caller 0; exit 1' ERR then you might hope to find out the location of any unexpected command failures. However, commands which fail in (subshell code) like above will not cause the trap because they will not abort the subshell. In summary, shouldn't set -e in (subshell code) operate independently of the surrounding context?
printf -v array[$i] "%b" "" poisons array causing segfault later
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib
-g -O2 -Wall
uname output: Linux lxjima 3.0.0-15-generic #26-Ubuntu SMP Fri Jan 20
17:23:00 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu
Bash Version: 4.2
Patch Level: 10
Release Status: release
Description:
If printf is used to set an array element, and the format
contains %b and the corresponding arg is empty, then the
array is somehow poisoned such that later use of array[*] or [@]
causes a segfault.
No segfault occurs if the string interpolated by %b is not empty,
or if %s is used instead of %b (in those cases, setting an array
element with printf seems to work fine).
Repeat-By:
bash -c 'declare -a ary; printf -v ary[0] "%b" ""; x="${ary[*]}"'
#!/bin/bash
declare -a ary
printf -v ary[0] "%b" ""
echo "after printf"
x="${ary[*]}" # segfaults here
echo "after use"
backslash at end of input causes eval parse error
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' -DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -g -O2 uname output: Linux lxjima 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-unknown-linux-gnu Bash Version: 4.3 Patch Level: 0 Release Status: release Description: A \ is supposed to be ignored, but if it occurs at the very end of the input then it causes a prior eval containing an array assignment to be mis-parsed. However if the eval contains something other than an array assignment then no error occurs. Repeat-By: #!/bin/bash PATH=/path/to/bash-4.3:$PATH; export PATH type bash # This one fails with the following error: # bash: eval: line 1: syntax error near unexpected token `foo' # echo "*** eval containing array assignment; backslash-newline (FAILS)" bash -ex <<'EOF' eval "array=(foo bar)" ; echo AAA\ EOF # But these work fine... # echo "*** eval containing something else; backslash-newline (works)" bash -ex <<'EOF' eval "scalar='(foo bar)'" ; echo AAA\ EOF echo "*** eval containing array assignment; backslash-newline; text (works)" bash -ex <<'EOF' eval "array=(foo bar)" ; echo AAA\ BBB EOF
Final backslash causes array assignment mis-parse in previous eval
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' -DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -g -O2 uname output: Linux lxjima 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-unknown-linux-gnu Bash Version: 4.3 Patch Level: 0 Release Status: release Description: NOTE: This repeats (approximately) a bug I sent a few minutes ago, but with an invalid From: address. Most likely my ISP dropped the message as spam, but if not, please accept my appoligies for the dup. \ is supposed to be effectively ignored, but if it appears at the end of the script then parse errors sometimes occur with previous statements. In particular eval "array=(values...)" is mis-parsed if a subsequent statement ends with and there is nothing further in the script. However, no error occurs if the eval contains something other than an array assignment. Or if something follows the \. Repeat-By: #!/bin/bash PATH=/home/jima/ptmp/downloads/bash-4.3:$PATH; export PATH ; type bash # This one shows the problem # An error occurs: # bash: eval: line 1: syntax error near unexpected token `foo' # echo "*** eval containing array assignment; backslash-newline (FAILS)" bash -ex <<'EOF' eval "array=(foo bar)" ; echo AAA\ EOF # But these work fine... echo "*** eval containing something else; backslash-newline (WORKS)" bash -ex <<'EOF' eval "scalar='(foo bar)'" ; echo AAA\ EOF echo "*** eval containing array assignment; backslash-newline; text (works)" bash -ex <<'EOF' eval "array=(foo bar)" ; echo AAA\ BBB EOF
"source" cmd creates entry in BASH_SOURCE etc. only if within function
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu'
-DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -g -O2
uname output: Linux lxjima 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu
Bash Version: 4.3
Patch Level: 0
Release Status: release
Description:
The "source" (or ".") command, if invoked from within a function,
creates a call frame, setting $FUNCNAME[0] to "source" and
$BASH_SOURCE[1] and $BASH_LINENO[1] to the location of the "source"
statement.
However this does not occur if the "source" statement occurs in
mainline code, i.e. outside of any function.
This means that code in "sourced" files can not reliably
display the location of their call, i.e., the location of
the "source" (or ".") statement which invoked them.
Repeat-By:
cat <<'EOF' >/tmp/sourced
echo "\$@=$@ \$0=$0 #FUNCNAME=${#FUNCNAME[@]}"
for ((i=0 ; i < ${#FUNCNAME[@]}; i=i+1)) ; do
echo "(i=$i) FUNCNAME ${FUNCNAME[$i]} called from line ${BASH_LINENO[$i]} in
${BASH_SOURCE[$i+1]}"
done
EOF
cat <<'EOF' >/tmp/test
#!/bin/bash
myfunc() {
echo "- Sourcing from inside a function:"
source /tmp/sourced
}
myfunc
echo "- Sourcing from mainline code:"
source /tmp/sourced
EOF
bash /tmp/test
Actual Results:
- Sourcing from inside function:
$@= $0=/tmp/test #FUNCNAME=3
(i=0) FUNCNAME source called from line 4 in /tmp/test
(i=1) FUNCNAME myfunc called from line 6 in /tmp/test
(i=2) FUNCNAME main called from line 0 in
- Sourcing directly from mainline code:
$@= $0=/tmp/test #FUNCNAME=0
Expected Results:
- Sourcing from inside function:
$@= $0=/tmp/test #FUNCNAME=3
(i=0) FUNCNAME source called from line 4 in /tmp/test
(i=1) FUNCNAME myfunc called from line 6 in /tmp/test
(i=2) FUNCNAME main called from line 0 in
- Sourcing directly from mainline code:
$@= $0=/tmp/test #FUNCNAME=1
(i=0) FUNCNAME source called from line 8 in /tmp/test
[(i=1) FUNCNAME main called from line 0 in]
