Re: Examples of concurrent coproc usage?

2024-04-03 Thread Chet Ramey

On 4/2/24 12:22 PM, Carl Edquist wrote:

This seems like it would be pretty easy to fix if a coproc closed the fds 
corresponding to an existing coproc in the child after the fork


the forked coproc has to close its fds to/from _all_ other existing coprocs 
(as there can be several).


And there is the issue. Without multi-coproc support, the shell only keeps
track of one coproc at a time, so there's only one set of pipe file
descriptors to close.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/




Re: Document ulimit's "unlimited" limit in manual page

2024-04-03 Thread Martin Guy

Il 03/04/24 15:10, Martin Guy ha scritto:
The "unlimited" keyword is not mentioned in the manual page, and 
should be.



Oops. Yes it is. Only dash lacks a mention of it. Sorry for the noise.


    M



OpenPGP_signature.asc
Description: OpenPGP digital signature


Document ulimit's "unlimited" limit in manual page

2024-04-03 Thread Martin Guy

Hi.

    ulimit -c unlimited


The "unlimited" keyword is not mentioned in the manual page, and should be.

(nor is it for dash(1) so it looks like someone copied some else's 
manual page!)



With thanks


    M



OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: Examples of concurrent coproc usage?

2024-04-03 Thread Chet Ramey

On 3/14/24 5:58 AM, Carl Edquist wrote:


HOWEVER!!!

Unexpectedly, the new multi-coproc code seems to close the user shell's end 
of a coprocess's pipes, once the coprocess has terminated.  When compiled 
with MULTIPLE_COPROCS=1, this is true even if there is only a single coproc:


 $ coproc WC { wc; }
 $ exec {WC[1]}>&-
 [1]+  Done    coproc WC { wc; }

 # WC var gets cleared!!
 # shell's ${WC[0]} is also closed!

 # now, can't do:

 $ read -u ${WC[0]} X
 $ echo $X

I'm attaching a "bad-coproc-log.txt" with more detailed ps & ls output 
examining the open fds at each step, to make it clear what's happening.


It's straightforward: the coproc process terminates, the shell reaps it,
marks it as dead, notifies the user that the process exited, and reaps it
before printing the next prompt. I don't observe any different behavior
between the default and when compiled for multiple coprocs.

It depends on when the process terminates as to whether you get a prompt
back and need to run an additional command before reaping the coproc
(macOS, RHEL), which gives you the opportunity to run the `read' command:

$ coproc WC { wc; }
[1] 48057
$ exec {WC[1]}>&-
$ read -u ${WC[0]} X
[1]+  Donecoproc WC { wc; }
bash: DEBUG warning: cpl_reap: deleting 48057
$ echo $X
0 0 0

(I put in a trace statement to show exactly when the coproc gets reaped and
deallocated.)

I can't reproduce your results with non-interactive shells, either, with
job control enabled or disabled.


This is a bug.  The shell should not automatically close its read pipe to a 
coprocess that has terminated -- it should stay open to read the final 
output, and the user should be responsible for closing the read end 
explicitly.


How long should the shell defer deallocating the coproc after the process
terminates? What should it do to make sure that the variables don't hang
around with invalid file descriptors? Or should the user be responsible for
unsetting the array variable too? (That's never been a requirement,
obviously.)


It also invites trouble if the shell variable that holds the fds gets 
removed unexpectedly when the coprocess terminates.  (Suddenly the variable 
expands to an empty string.)  It seems to me that the proper time to clear 
the coproc variable (if at all) is after the user has explicitly closed 
both of the fds. 


That requires adding more plumbing than I want to, especially since the
user can always save the file descriptors of interest into another
variable if they want to use them after the coproc terminates.

*Or* else add an option to the coproc keyword to 
explicitly close the coproc - which will close both fds and clear the 
variable.


Not going to add any more options to reserved words; that does more
violence to the grammar than I want.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/




Re: Examples of concurrent coproc usage?

2024-04-03 Thread Zachary Santer
On Wed, Apr 3, 2024 at 10:32 AM Chet Ramey  wrote:
>
> How long should the shell defer deallocating the coproc after the process
> terminates? What should it do to make sure that the variables don't hang
> around with invalid file descriptors? Or should the user be responsible for
> unsetting the array variable too? (That's never been a requirement,
> obviously.)

For sake of comparison, and because I don't know the answer, what does
bash do behind the scenes in this situation?

exec {fd}< <( some command )
while IFS='' read -r line <&"${fd}"; do
  # do stuff
done
{fd}<&-

Because the command in the process substitution isn't waiting for
input, (I think) it could've exited at any point before all of its
output has been consumed. Even so, bash appears to handle this
seamlessly.

As the programmer, I know ${fd} contains an fd that's no longer valid
after this point, despite it not being unset.