For the record, here is my bash workaround to convert a heredoc fd into a
tmpfile.
Code golfers - please feel free to simplify it
Just prefix the command with "herefile" and the fd that needs converting to
a file,
e.g.
herefile 3 command ... /dev/fd/3 .... 3<<<blah
# $1 is the herefoc filedescriptor to convert to a file
# ${@:2} is the command and args to run
# @ and _ are abused as lexical scope variables
herefile() {
set -- "$(mktemp)" "$@"
test -n "$1" || return $?
{ set -- "$_" "$@" # save write fd _ before it gets trashed
{ set -- "$_" "$@" # save read fd _ before it gets trashed
# $1 is fd to read tmp filename
# $2 is fd to write tmp filename
# $3 is now tmp filename
# $4 is fd of data to copy to $1
# ${@:5} is command to run
# now it's gone and can't leak data
rm -f "$3" &&
# populate the tmp file from the fd
eval 'cat <&'"$4"' >&'"$2" &&
# run command reading from file via specific fd but closed tmp file
eval '"${@:5}"' "$4<&$1" "$1<&-" "$2<&-"
# remember exit code and fd's to close
set -- "$1" "$2" "$?"
# finally close the tmp files cos bash doesn't do it for you if you
name them
eval exec "$1<&-" "$2<&-"
# return saved exit code
return "$3"
} {_}<"$2" # open tmp file into _ and save at the top of the block
} {_}>"$1" # open tmp file into _ and save at the top of the block
}
# tests without the herefile fix
$ python2 -c 'import os; print(os.path.isfile("/dev/fd/3"))' 3<<<x
False
$ stat -L -c '%F' /dev/fd/3 3<<<x
fifo
$ ls -l /dev/fd/ 3<<<x
total 0
lrwx------ 1 sliddicott sliddicott 64 Apr 22 16:12 0 -> /dev/pts/1
lrwx------ 1 sliddicott sliddicott 64 Apr 22 16:12 1 -> /dev/pts/1
lrwx------ 1 sliddicott sliddicott 64 Apr 22 16:12 2 -> /dev/pts/1
lr-x------ 1 sliddicott sliddicott 64 Apr 22 16:12 3 -> 'pipe:[170069]'
lr-x------ 1 sliddicott sliddicott 64 Apr 22 16:12 4 -> /proc/17483/fd
# tests with
$ herefile 3 python2 -c 'import os; print(os.path.isfile("/dev/fd/3"))'
3<<<x
True
$ herefile 3 stat -L -c '%F' /dev/fd/3 3<<<x
regular file
$ herefile 3 ls -l /dev/fd/ 3<<<x
total 0
lrwx------ 1 sliddicott sliddicott 64 Apr 22 16:05 0 -> /dev/pts/1
lrwx------ 1 sliddicott sliddicott 64 Apr 22 16:05 1 -> /dev/pts/1
lrwx------ 1 sliddicott sliddicott 64 Apr 22 16:05 2 -> /dev/pts/1
lr-x------ 1 sliddicott sliddicott 64 Apr 22 16:05 3 ->
'/tmp/tmp.rkDXYvp4hn (deleted)'
lr-x------ 1 sliddicott sliddicott 64 Apr 22 16:05 4 -> /proc/16519/fd
On Fri, 22 Apr 2022 at 14:51, Sam Liddicott <s...@liddicott.com> wrote:
>
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -g -O2 -flto=auto -ffat-lto-objects -flto=auto
-ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
-Wall
> uname output: Linux junior 5.15.0-25-generic #25-Ubuntu SMP Wed Mar 30
15:54:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Bash Version: 5.1
> Patch Level: 16
> Release Status: release
>
> Description:
> Listed in the changes:
> c. Here documents and here strings now use pipes for the expanded
> document if it's smaller than the pipe buffer size, reverting
> to temporary files if it's larger.
>
> This causes problems with many programs suffering from the TOCTOU
> bug of checking whether or not the input is actually a file
> instead of just using it as one.
>
> e.g. "repo" tool performs in manifest_xml.py:
>
> if not os.path.isfile(path):
> raise ManifestParseError('manifest %s not found' % name)
>
> This bug is clearly in repo and (these other tools) and certainly
is
> not bash's fault but there is going to be a lot of breakage with
the
> short and medium term remedy to downgrade to bash 5.0
>
> I *like* that files aren't needed any more but there are decades
> of scripts integrating with tools than make such checks, and which
> work on the unknown accidental assumption that heredocs are files.
>
> Repeat-By:
> python2 -c 'import os; print(os.path.isfile("/dev/fd/3"))' 3<<<x
>
> emits True for bash 5.0 and before but emits False for bash 5.1
>
> Fix:
> Please could we at least have a shopt to maintain the old
behaviour?
>
>
>
