On Tue, 16 Apr 2024, Chet Ramey wrote:
The bigger concern was how to synchronize between the processes, but
that's something that the script writer has to do on their own.
Right. It can be tricky and depends entirely on what the user's up to.
My concern was always coproc fds leaking into other processes,
especially pipelines. If someone has a coproc now and is `messy' about
cleaning it up, I feel like there's the possibility of deadlock.
I get where you're coming from with the concern. I would welcome being
shown otherwise, but as far as I can tell, deadlock is a ghost of a
concern once the coproc is dead.
Maybe it helps to step through it ...
- First, where does deadlock start? (In the context of pipes)
I think the answer is: When there is a read or write attempted on a pipe
that blocks (indefinitely).
- What causes a read or a write on a pipe to block?
A pipe read blocks when a corresponding write-end is open,
but there is no data available to read.
A pipe write blocks when a corresponding read-end is open,
but the pipe is full.
- Are the coproc's corresponding ends of the shell's pipe fds open?
Well, not if the coproc is really dead.
- Will a read or write ever be attempted?
If the shell's stray coproc fds are left open, sure they will leak into
pipelines too - but since they're forgotten, in theory no command will
actually attempt to use them.
- What if a command attempts to use these stray fds anyway, by mistake?
If the coproc is really dead, then its side of the pipe fds will have been
closed. Thus read/write attempts on the fds on the shell's side (either
from the shell itself, or from commands / pipelines that the fds leaked
into) WILL NOT BLOCK, and thus will not result in deadlock.
(A read attempt will hit EOF, a write attempt will get SIGPIPE/EPIPE.)
HOPEFULLY that is enough to put any reasonable fears of deadlock to bed -
at least in terms of the shell's leaked fds leading to deadlock.
- But what if the _coproc_ leaked its pipe fds before it died?
At this point I think perhaps we get into what you called a "my arm hurts
when I do this" situation. It kind of breaks the whole coproc model: if
the stdin/stdout of a coproc are still open by one of the coproc's
children, then I might say the coproc is not really dead.
But anyway I want to be a good sport, for completeness.
An existing use case that would lead to trouble would perhaps have to look
something like this:
The shell sends a quit command to a coproc, without closing the shell's
coproc fds.
The coproc has a child, then exits. The coproc (parent) is dead. The
coproc's child has inherited the coproc's pipe fds. The script author
_expects_ that the coproc parent will exit, and expects that this will
trigger the old behavior, that the shell will automatically close its fds
to the coproc parent. Thus the author _expects_ that the coproc exiting
will, indirectly but automatically, cause any blocked reads/writes on
stdin/stdout in the coproc's child to stop blocking. Thus the author
_expects_ the coproc's child to promptly complete, even though its output
_will not be consumable_ (because the author _expects_ that its stdout
will be attached to a broken pipe).
But [here's where the potential problem starts] with the new deferring
behavior, the shell's coproc fds are not automatically closed, and thus
the coproc's _child_ does not stop blocking, and thus the author's
short-lived expectations for this coproc's useless child are dashed to the
ground, while that child is left standing idle until the cows come home.
(That is, until the shell exits.)
It really seems like a contrived and senseless scenario, doesn't it?
(Even to me!)
[And an even more far-fetched scenario: a coproc transmits copies of its
pipe fds to another process over a unix socket ancillary message
(SCM_RIGHTS), instead of to a child by inheritance. The rest of the story
is the same, and equally senseless.]
But I don't know how extensively they're used, or all the use cases, so
I'm not sure how likely it is. I've learned there are users who do
things with shell features I never imagined. (People wanting to use
coprocs without the shell as the arbiter, for instance. :-) )
Hehe...
Well, yeah, once you gift-wrap yourself a friendly, reliable interface and
have the freedom to play with it to your heart's content - you find some
fun things to do with coprocesses. (Much like regular shell pipelines.)
I get your meaning though - without knowing all the potential uses, it's
hard to say with absolute certainty that no user will be negatively
affected by a new improvement or bug fix.
[This is a common model for using coprocs, by the way, where an
auxiliary coprocess is left open for the lifetime of the shell session
and never explicitly closed. When the shell session exits, the fds are
closed implicitly by the OS, and the coprocess sees EOF and exits on
its own.]
That's
