Re: bash completion after a multiline string
May be posting a link is not appropriate so I cut/paste it here I bumped into this problem regarding bash completion, can't find reference to it. When doing $ echo foo "bar" /tm I got /tm expanded to /tmp/ that is indeed correct. But if I do $ echo foo "bar more bar" /tm No completion is done. My question is doest this behavior a feature or a bug? My bash version is GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu) Thanx in advance.
'wait' in command group in pipeline fails to recognize child process
Hello!
I ran into something that looks like a bug to me, although I'm not super
familiar curly-brace command groups.
Bash version: latest from GitHub mirror (commit ce23728: Bash-5.1 patch 7)
Minimal repro:
$ sleep 1 & { wait $!; } | cat
[1] 665454
bash: wait: pid 665454 is not a child of this shell
I was expecting a success, just like we get without the pipeline:
$ sleep 1 & { wait $!; }
[1] 665591
[1]+ Donesleep 1
The following scripts are similar but print PIDs along the way:
failing script
$ cat wait-for-child
echo "main pid: $$"
/usr/bin/sleep 1 &
job_pid=$!
echo "job pid: $job_pid"
{
echo "command group pid: $$"
wait "$job_pid" || echo "wait failed!"
} | cat
Output:
$ ./bash wait-for-child
main pid: 664755
job pid: 664756
wait-for-child: line 7: wait: pid 664756 is not a child of this shell
command group pid: 664755
wait failed!
^ We see that the current PID in the command group is the same as in the
rest of the shell, as it should be. It is unclear to me why 'wait'
believes that the background job's PID is not of a child.
successful script
Compare to the same script without having the command group in a
pipeline (I removed '| cat'):
$ cat wait-for-child-ok
echo "main pid: $$"
/usr/bin/sleep 1 &
job_pid=$!
echo "job pid: $job_pid"
{
echo "command group pid: $$"
wait "$job_pid" || echo "wait failed!"
}
$ ./bash wait-for-child-ok
main pid: 664705
job pid: 664706
command group pid: 664705
^ OK
Martin
Re: 'wait' in command group in pipeline fails to recognize child process
On Tue, Jun 22, 2021 at 02:42:40AM -0700, Martin Jambon wrote:
> I ran into something that looks like a bug to me, although I'm not super
> familiar curly-brace command groups.
>
> Bash version: latest from GitHub mirror (commit ce23728: Bash-5.1 patch 7)
>
> Minimal repro:
>
> $ sleep 1 & { wait $!; } | cat
> [1] 665454
> bash: wait: pid 665454 is not a child of this shell
>
> I was expecting a success, just like we get without the pipeline:
A pipeline creates two or more subshells, one for each command in the
pipeline. Therefore, your wait command is running in a different
process than the one which created the sleep background job.
The curly braces are irrelevant here.
unicorn:~$ sleep 1 & wait "$!"|cat
[1] 1290127
bash: wait: pid 1290127 is not a child of this shell
Re: 'wait' in command group in pipeline fails to recognize child process
On 6/22/21 5:42 AM, Martin Jambon wrote:
Hello!
I ran into something that looks like a bug to me, although I'm not super
familiar curly-brace command groups.
It's not the brace command; it's the pipeline.
Minimal repro:
$ sleep 1 & { wait $!; } | cat
[1] 665454
bash: wait: pid 665454 is not a child of this shell
I was expecting a success, just like we get without the pipeline:
The pipeline does, in fact, make the difference. Each process in a pipeline
is run in a subshell, and that subshell isn't the parent of the previously-
created asynchronous job. You can't wait for a process that isn't one of
your children.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Feature request: index of index
Meanwhile you may use functions to setup your variables something along
those lines.
PW$ function first-index { echo $1; }
PW$ function last-index { shift $(($#-1)) ; echo $1; }
PW$ declare -a array=([5]="hello" [11]="world" [42]="here")
PW$ declare -i first_index=$(first-index ${!array[@]})
PW$ declare -i last_index=$(last-index ${!array[@]})
PW$ echo $first_index $last_index
5 42
On Thu, May 6, 2021 at 1:24 PM Léa Gris wrote:
> Currently, to know the index of an array's indexes (example: first or
> last index), it needs an intermediary index array:
>
> > #!/usr/bin/env bash
> >
> > declare -a array=([5]=hello [11]=world [42]=here)
> > declare -ai indexes=("${!array[@]}")
> > declare -i first_index=${indexes[*]:0:1}
> > declare -i last_index=${indexes[*]: -1:1}
> > declare -p array indexes first_index last_index
>
> Which prints:
> > declare -a array=([5]="hello" [11]="world" [42]="here")
> > declare -ai indexes=([0]="5" [1]="11" [2]="42")
> > declare -i first_index="5"
> > declare -i last_index="42"
>
> It would be convenient to be able to index directly with this syntax:
>
> declare -i first_index=${!array[@]:0:1}
> declare -i last_index=${!array{@}: -1:1}
>
>
> --
> Léa Gris
>
>
>
Re: 'wait' in command group in pipeline fails to recognize child process
On 6/22/21 6:15 AM, Chet Ramey wrote: On 6/22/21 5:42 AM, Martin Jambon wrote: Hello! I ran into something that looks like a bug to me, although I'm not super familiar curly-brace command groups. It's not the brace command; it's the pipeline. Thank you! It's the $$ that confused me since I thought incorrectly that it would print the current process ID. I was also misled by the term "subshell" which is not a proper shell like a subprocess is just another process.
Re: 'wait' in command group in pipeline fails to recognize child process
On 6/22/21 4:31 AM, Greg Wooledge wrote:
On Tue, Jun 22, 2021 at 02:42:40AM -0700, Martin Jambon wrote:
I ran into something that looks like a bug to me, although I'm not super
familiar curly-brace command groups.
Bash version: latest from GitHub mirror (commit ce23728: Bash-5.1 patch 7)
Minimal repro:
$ sleep 1 & { wait $!; } | cat
[1] 665454
bash: wait: pid 665454 is not a child of this shell
I was expecting a success, just like we get without the pipeline:
A pipeline creates two or more subshells, one for each command in the
pipeline. Therefore, your wait command is running in a different
process than the one which created the sleep background job.
The curly braces are irrelevant here.
unicorn:~$ sleep 1 & wait "$!"|cat
[1] 1290127
bash: wait: pid 1290127 is not a child of this shell
Thank you! Now I know that a subshell is not a shell, $$ being the
process ID of the shell, not of the subshell.
Re: 'wait' in command group in pipeline fails to recognize child process
On Tue, Jun 22, 2021 at 01:12:10PM -0700, Martin Jambon wrote: > On 6/22/21 4:31 AM, Greg Wooledge wrote: > > A pipeline creates two or more subshells, one for each command in the > > pipeline. Therefore, your wait command is running in a different > > process than the one which created the sleep background job. > > > > The curly braces are irrelevant here. > > > > unicorn:~$ sleep 1 & wait "$!"|cat > > [1] 1290127 > > bash: wait: pid 1290127 is not a child of this shell > > Thank you! Now I know that a subshell is not a shell, $$ being the process > ID of the shell, not of the subshell. It's a forked child process of bash, and as such, it's still a shell. Specifically, it's bash. There's some formal definition that you can ignore. The real definition of a subshell is "the direct result of a fork()". It inherits all of the shell variables, functions, aliases, and so on that the parent shell process had at the time of the fork. This is different from a direct call to "bash", which would not inherit shell variables and so on -- only environment variables. In the pipeline, there are two subshells created. One of them runs the command wait "$!", and the other runs the command cat. The special parameter $! is inherited from the parent shell, and contains the PID of the sleep process that the parent shell created. But the child subshell can't wait for that process, because it belongs to someone else. Hence the error message.
Latest push for command substitutions
The latest devel branch push has a rewrite of parse_comsub() to recursively call the parser (yyparse()). I was finally able to get all the state working correctly, though it will probably only work with bison. Please run it through any tests or scripts you have so we can shake out any bugs early. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: 'wait' in command group in pipeline fails to recognize child process
On 6/22/21 1:37 PM, Greg Wooledge wrote: On Tue, Jun 22, 2021 at 01:12:10PM -0700, Martin Jambon wrote: On 6/22/21 4:31 AM, Greg Wooledge wrote: A pipeline creates two or more subshells, one for each command in the pipeline. Therefore, your wait command is running in a different process than the one which created the sleep background job. The curly braces are irrelevant here. unicorn:~$ sleep 1 & wait "$!"|cat [1] 1290127 bash: wait: pid 1290127 is not a child of this shell Thank you! Now I know that a subshell is not a shell, $$ being the process ID of the shell, not of the subshell. It's a forked child process of bash, and as such, it's still a shell. Specifically, it's bash. There's some formal definition that you can ignore. The real definition of a subshell is "the direct result of a fork()". It inherits all of the shell variables, functions, aliases, and so on that the parent shell process had at the time of the fork. This is different from a direct call to "bash", which would not inherit shell variables and so on -- only environment variables. Great, then I would change the documentation of '$$'. Currently it says this: ($$) Expands to the process ID of the shell. In a () subshell, it expands to the process ID of the invoking shell, not the subshell. This first mentions just "the shell". I suggest calling it the "root shell" instead, i.e. the process where "exec bash" took place. I would also mention pipelines here, since these are more commonly used than () subshells. I don't know if there are other ways of creating subshells. If that's all, I think it would be valuable to mention those two cases rather than just one. Well, I suppose "root shell" could be misunderstood as a shell run by the 'root' user but this doesn't make much sense in this context. I can't find a better name. Here's how we might change the description of '$$': ($$) Expands to the process ID of the root shell. In a pipeline or in a () subshell, it expands to the process ID of the root shell, not the subshell. I also changed "invoking shell" to "root shell" because the invoking shell or parent shell is not necessarily the same as the root shell e.g. $ echo $$; (echo $$; (echo $$)) 688918 688918 688918 In the pipeline, there are two subshells created. One of them runs the command wait "$!", and the other runs the command cat. The special parameter $! is inherited from the parent shell, and contains the PID of the sleep process that the parent shell created. But the child subshell can't wait for that process, because it belongs to someone else. Hence the error message. 👍 Yup. I think the error message is clear. No complaint about that.
Re: 'wait' in command group in pipeline fails to recognize child process
On Tue, Jun 22, 2021, at 6:32 PM, Martin Jambon wrote: > I would also mention pipelines here, since these are more commonly > used than () subshells. I don't know if there are other ways of > creating subshells. If that's all, I think it would be valuable > to mention those two cases rather than just one. There are also command substitutions and any commands run asynchronously. If the description applies to all subshells, not just ones inside (...), then perhaps the documentation should just omit "()" instead of spelling out every possible way a subshell can be created. > Well, I suppose "root shell" could be misunderstood as a shell run by > the 'root' user but this doesn't make much sense in this context. Yes, but then the reader is left to reverse-engineer the meaning of this term you just made up. If the documentation is to be changed, it should be more clear than before. "Root shell" makes it less clear and should not be used. There wouldn't be anything wrong with adding a more verbose but more precise explanation. > I also changed "invoking shell" to "root shell" because the invoking > shell or parent shell is not necessarily the same as the root shell e.g. > > $ echo $$; (echo $$; (echo $$)) > 688918 > 688918 > 688918 Maybe something like this would get the point across: ($$) Expands to the process ID of the shell. In a subshell, it expands to the value that $$ has in the invoking shell. -- vq
Re: 'wait' in command group in pipeline fails to recognize child process
On 6/22/21 5:00 PM, Lawrence Velázquez wrote: Maybe something like this would get the point across: ($$) Expands to the process ID of the shell. In a subshell, it expands to the value that $$ has in the invoking shell. It's better. However, the reader is still left wondering what "the shell" is referring to in first sentence. Here's my next suggestion, which borrows some language from the documentation for BASHPID: ($$) Expands to the ID of the process that initialized Bash. The value of $$ is inherited in subshells, such as pipelines, regardless of their own process ID. See also the Bash variable BASHPID. For reference, the documentation for 'BASHPID' is this: Expands to the process ID of the current Bash process. This differs from $$ under certain circumstances, such as subshells that do not require Bash to be re-initialized. Assignments to BASHPID have no effect. If BASHPID is unset, it loses its special properties, even if it is subsequently reset.
Re: 'wait' in command group in pipeline fails to recognize child process
On Tue, Jun 22, 2021, at 8:52 PM, Martin Jambon wrote: > It's better. However, the reader is still left wondering what "the > shell" is referring to in first sentence. Subshells aside, I have a hard time believing that "the process ID of the shell" confuses anybody in practice. Even POSIX doesn't overcomplicate this: $ Expands to the decimal process ID of the invoked shell. In a subshell (see Shell Execution Environment), '$' shall expand to the same value as that of the current shell. (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_05_02) -- vq
Re: 'wait' in command group in pipeline fails to recognize child process
> It's not the brace command; it's the pipeline.
>
>> Minimal repro:
>>
>>$ sleep 1 & { wait $!; } | cat
>>[1] 665454
>>bash: wait: pid 665454 is not a child of this shell
Interestingly, this is almost trivial to clairfy:
$ sleep 5 & { wait $!; }
[1] 19069
[1]+ Donesleep 5
$
> I was also misled by the term "subshell" which is not a proper shell
> like a subprocess is just another process.
The criticial point is that "being a subshell" is a relationship between
the child bash process and its parent. Intuitively, a subshell is
created whenever a bash child process is created "implicitly". The
manual page et al. enumerate all the ways a subshell can be created;
search for the word "subshell".
Dale
Re: 'wait' in command group in pipeline fails to recognize child process
On 6/22/21 6:11 PM, Lawrence Velázquez wrote: On Tue, Jun 22, 2021, at 8:52 PM, Martin Jambon wrote: It's better. However, the reader is still left wondering what "the shell" is referring to in first sentence. Subshells aside, I have a hard time believing that "the process ID of the shell" confuses anybody in practice. Even POSIX doesn't overcomplicate this: $ Expands to the decimal process ID of the invoked shell. In a subshell (see Shell Execution Environment), '$' shall expand to the same value as that of the current shell. (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_05_02) This language implies that a subshell is part of the shell but not a shell in itself. This is what I pointed out earlier but I received a conflicting answer. This is confirmed by this definition from posix: A subshell environment shall be created as a duplicate of the shell environment, except that signal traps that are not being ignored shall be set to the default action. Changes made to the subshell environment shall not affect the shell environment. Command substitution, commands that are grouped with parentheses, and asynchronous lists shall be executed in a subshell environment. Additionally, each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment. All other commands shall be executed in the current shell environment. (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_12) In the posix definition, a subshell - is not necessarily implemented as a separate process - belongs to a unique shell - is not a shell This makes the term "the shell" unambiguous to a reader who's aware of this - but it's obvious to me that "subshell" should be clarified in the context of '$$'. A subset is a set, a subgroup is a group, a subtype is a type, a subtree is a tree, a subprocess is a process. However a submarine is not a marine, and a subshell is not a shell (sea stuff!). Anyway, I left a suggestion to revise the documentation for '$$' in my previous email. That's the best I can produce at this time. Martin
Re: 'wait' in command group in pipeline fails to recognize child process
On Tue, Jun 22, 2021 at 06:54:18PM -0700, Martin Jambon wrote: > This is confirmed by this definition from posix: > > A subshell environment shall be created as a duplicate of the shell > environment, except [...] > In the posix definition, a subshell > - is not necessarily implemented as a separate process > - belongs to a unique shell > - is not a shell I have no idea how you came to the conclusion that it's "not a shell". It's a DUPLICATE OF A SHELL, and therefore a shell. The point is that it's a DIFFERENT shell. Not the original one. I mentioned earlier that there's a formal definition you can largely ignore, and just go with "a subshell is the direct result of a fork()".[1] That's all you really need to know. The formal language exists to allow for implementations in some theoretical non-Unix-like environment where fork() is not a thing. It's simply describing what a fork() does, without using that term. [1] Except for that bit about resetting signal handlers.
