Re: Assignment-like word shouldn't be subjected to tilde expansion in POSIX mode
Date:Sun, 19 Jul 2020 15:21:07 -0400 From:Chet Ramey Message-ID: <5f066ef0-1fed-ad5b-d564-490268d32...@case.edu> | Maybe, but Posix says this isn't a variable assignment context, as POSIX | defines variable assignments, and technically the shell shouldn't perform | tilde expansions after `=' or `:' at all. You could argue that bash has a | bug here in that it expands the tilde after the `='. It may also be that this is a bug in POSIX, as all ksh shells seem to act the way that bash does, expanding the first ~ but not the secpod. Most other shells expand neither, except the NetBSD shell which treats this as a var assignment (which it is really, however explained away in POSIX) and expands both. It is likely that if someone were to file a posix bug report about this, it would turn into "unspecified" whether ~ expansion is done in that context. kre
Re: 'foo > >(bar)' doesn't set $! for external foo not invoked via 'command'
Oğuz:
> > For context - I'm filtering a program's stdout and stderr
> > (separately), requiring successful exit statuses for the program and
> > both filters:
> >
> > set -u -o pipefail
> > { program 2> >(stderr_filter >&2) && wait $!; } | stdout_filter &&
> > ...
> Not sure if process substitution is really necessary here,
>
> set -u -o pipefail
> { program 2>&3 | stdout_filter; } 3>&1 | stderr_filter && ...
>
> does the same thing.
That one filters program's stdout twice - first through stdout_filter
and then through stderr_filter - with program's stdout and stderr both
finally arriving at stdout. But tweaked like this, it seems to cover
all the bases:
set -u -o pipefail
{
program 2>&1 >&"$out" {out}>&- |
stderr_filter >&2 {out}>&-
} {out}>&1 | stdout_filter
And it even waits for stderr_filter if program failed. My original
snippet neglected that case, otherwise it would have looked more like
set -u -o pipefail
(
trap 'wait $! || exit $?' EXIT
command program 2> >(stderr_filter >&2)
) | stdout_filter
which isn't exactly pretty either, even if the bug(?) requiring
'command' is fixed.
Rusty
signature.asc
Description: PGP signature
Re: 'foo > >(bar)' doesn't set $! for external foo not invoked via 'command'
20 Temmuz 2020 Pazartesi tarihinde Rusty Bird yazdı:
> Oğuz:
> > > For context - I'm filtering a program's stdout and stderr
> > > (separately), requiring successful exit statuses for the program and
> > > both filters:
> > >
> > > set -u -o pipefail
> > > { program 2> >(stderr_filter >&2) && wait $!; } |
> stdout_filter &&
> > > ...
>
> > Not sure if process substitution is really necessary here,
> >
> > set -u -o pipefail
> > { program 2>&3 | stdout_filter; } 3>&1 | stderr_filter && ...
> >
> > does the same thing.
>
> That one filters program's stdout twice - first through stdout_filter
> and then through stderr_filter - with program's stdout and stderr both
> finally arriving at stdout. But tweaked like this, it seems to cover
> all the bases:
>
> set -u -o pipefail
> {
> program 2>&1 >&"$out" {out}>&- |
> stderr_filter >&2 {out}>&-
> } {out}>&1 | stdout_filter
>
>
Yes, I thought stdout_filter and stderr_filter didn't produce output, silly
me. `>&"$out" is very ugly though, it'd be nice if `{var}' thing worked at
the RHS of redirection operator, like `>&{var}`, which, on bash 5.0.11,
ignores `&' -another bug?- and redirects stdout to a file named `{var}'.
> And it even waits for stderr_filter if program failed. My original
> snippet neglected that case, otherwise it would have looked more like
>
>set -u -o pipefail
>(
>trap 'wait $! || exit $?' EXIT
>command program 2> >(stderr_filter >&2)
>) | stdout_filter
>
> which isn't exactly pretty either, even if the bug(?) requiring
> 'command' is fixed.
>
> Rusty
>
--
Oğuz
Re: 'foo > >(bar)' doesn't set $! for external foo not invoked via 'command'
Oğuz:
> `>&"$out" is very ugly though, it'd be nice if `{var}' thing worked at
> the RHS of redirection operator, like `>&{var}`, which, on bash 5.0.11,
> ignores `&' -another bug?- and redirects stdout to a file named `{var}'.
Probably more backwards compatibility than a bug - '>&word' (with word
neither a number nor a dash) is an alternative form of '&>word':
https://www.gnu.org/software/bash/manual/html_node/Redirections.html#Redirecting-Standard-Output-and-Standard-Error
Rusty
signature.asc
Description: PGP signature
Re: 'foo > >(bar)' doesn't set $! for external foo not invoked via 'command'
On 7/20/20 1:10 AM, Oğuz wrote: >> Bash Version: 5.0 >> Patch Level: 17 >> Release Status: release >> >> Description: >> $! isn't set to the PID of 'bar' for >> >> foo > >(bar) >> >> if 'foo' is an external program (as opposed to a builtin or >> function) - unless it's invoked via 'command'. Redirections are performed in the subshell created to execute `foo', so the process substitution is invoked there and can't affect the parent's $!. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Issue with Bash
On 7/20/20 2:32 AM, Rishita Saha16 wrote: > Hi All, > > From what we have found out, it does not seem like the signal is SIGTTOU. > We are working to find out more about it. Meanwhile, any insight would be > helpful. If the process isn't an interactive shell, it would be helpful to know why it's calling readline and whether that's intended. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: 'foo > >(bar)' doesn't set $! for external foo not invoked via 'command'
Chet Ramey:
> >> Description:
> >> $! isn't set to the PID of 'bar' for
> >>
> >> foo > >(bar)
> >>
> >> if 'foo' is an external program (as opposed to a builtin or
> >> function) - unless it's invoked via 'command'.
>
> Redirections are performed in the subshell created to execute `foo', so the
> process substitution is invoked there and can't affect the parent's $!.
I guess my question is: Would it not make sense to always perform the
redirection before the subshell is created?
OTOH, looks like it's trivial to work around the issue by using
'{ foo; } > >(bar)'.
Rusty
signature.asc
Description: PGP signature
Re: Assignment-like word shouldn't be subjected to tilde expansion in POSIX mode
On 7/20/20 4:28 AM, Robert Elz wrote:
> Date:Sun, 19 Jul 2020 15:21:07 -0400
> From:Chet Ramey
> Message-ID: <5f066ef0-1fed-ad5b-d564-490268d32...@case.edu>
>
> | Maybe, but Posix says this isn't a variable assignment context, as POSIX
> | defines variable assignments, and technically the shell shouldn't perform
> | tilde expansions after `=' or `:' at all. You could argue that bash has a
> | bug here in that it expands the tilde after the `='.
>
> It may also be that this is a bug in POSIX, as all ksh shells seem to
> act the way that bash does, expanding the first ~ but not the secpod.
> Most other shells expand neither, except the NetBSD shell which treats
> this as a var assignment (which it is really, however explained away in
> POSIX) and expands both.
>
> It is likely that if someone were to file a posix bug report about this,
> it would turn into "unspecified" whether ~ expansion is done in that context.
You can make a case for the bash/ksh tilde expansion: the word
expansion is ${PARAM:=WORD}, and the WORD is subject to tilde expansion
according to the enumerated list in 2.6.2. Since the first character of
WORD is a tilde, if you say the tilde-prefix stops at the `:', the tilde
gets expanded, and since it's not an assignment, the `:~' doesn't trigger
subsequent tilde expansion. Bash has done this since at least the early 1.x
days (at least 1.10), and there is code to handle that case in bash to this
day.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: 'foo > >(bar)' doesn't set $! for external foo not invoked via 'command'
On 7/20/20 10:11 AM, Rusty Bird wrote: > Chet Ramey: Description: $! isn't set to the PID of 'bar' for foo > >(bar) if 'foo' is an external program (as opposed to a builtin or function) - unless it's invoked via 'command'. >> >> Redirections are performed in the subshell created to execute `foo', so the >> process substitution is invoked there and can't affect the parent's $!. > > I guess my question is: Would it not make sense to always perform the > redirection before the subshell is created? You could make a case for it, but I'm not inclined to invalidate 30+ years of bash backward compatibility unless the reason is a lot more compelling. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
