On 4/12/24 12:49 PM, Carl Edquist wrote:
Where with a coproc
coproc X { potentially short lived command with output; }
exec {xr}<&${X[0]} {xw}>&${X[1]}
there is technically the possibility that the coproc can finish and be
reaped before the exec command gets a chance to run and duplicate the fds.
But, I also get what you said, that your design intent with coprocs was for
them to be longer-lived, so immediate termination was not a concern.
The bigger concern was how to synchronize between the processes, but that's
something that the script writer has to do on their own.
Personally I like the idea of 'closing' a coproc explicitly, but if it's
a bother to add options to the coproc keyword, then I would say just let
the user be responsible for closing the fds. Once the coproc has
terminated _and_ the coproc's fds are closed, then the coproc can be
deallocated.
This is not backwards compatible. coprocs may be a little-used feature,
but you're adding a burden on the shell programmer that wasn't there
previously.
Ok, so, I'm trying to imagine a case where this would cause any problems or
extra work for such an existing user. Maybe you can provide an example
from your own uses? (Where it would cause trouble or require adding code
if the coproc deallocation were deferred until the fds are closed explicitly.)
My concern was always coproc fds leaking into other processes, especially
pipelines. If someone has a coproc now and is `messy' about cleaning it up,
I feel like there's the possibility of deadlock. But I don't know how
extensively they're used, or all the use cases, so I'm not sure how likely
it is. I've learned there are users who do things with shell features I
never imagined. (People wanting to use coprocs without the shell as the
arbiter, for instance. :-) )
My first thought is that in the general case, the user doesn't really need
to worry much about closing the fds for a terminated coproc anyway, as they
will all be closed implicitly when the shell exits (either an interactive
session or a script).
Yes.
[This is a common model for using coprocs, by the way, where an auxiliary
coprocess is left open for the lifetime of the shell session and never
explicitly closed. When the shell session exits, the fds are closed
implicitly by the OS, and the coprocess sees EOF and exits on its own.]
That's one common model, yes. Another is that the shell process explicitly
sends a close or shutdown command to the coproc, so termination is
expected.
If a user expects the coproc variable to go away automatically, that user
won't be accessing a still-open fd from that variable for anything.
I'm more concerned about a pipe with unread data that would potentially
cause problems. I suppose we just need more testing.
As for the forgotten-about half-closed pipe fds to the reaped coproc, I
don't see how they could lead to deadlock, nor do I see how a shell
programmer expecting the existing behavior would even attempt to access
them at all, apart from programming error.
Probably not.
The only potential issue I can imagine is if a script (or a user at an
interactive prompt) would start _so_ many of these longer-lived coprocs
(more than 500??), one at a time in succession, in a single shell session,
that all the available fds would be exhausted. (That is, if the shell is
not closing them automatically upon coproc termination.) Is that the
backwards compatibility concern?
That's more of a "my arm hurts when I do this" situation. If a script
opened 500 fds using exec redirection, resource exhaustion would be their
own responsibility.
Meanwhile, the bash man page does not specify the shell's behavior for when
a coproc terminates, so you might say there's room for interpretation and
the new deferring behavior would not break any promises.
I could always enable it in the devel branch and see what happens with the
folks who use that. It would be three years after any release when distros
would put it into production anyway.
And as it strikes me anyway, the real "burden" on the programmer with the
existing behavior is having to make a copy of the coproc fds every time
coproc X { cmd; }
exec {xr}<&${X[0]} {xw}>&${X[1]}
and use the copies instead of the originals in order to reliably read the
final output from the coproc.
Maybe, though it's easy enough to wrap that in a shell function.
First, just to be clear, the fds to/from the coproc pipes are not
invalid when the coproc terminates (you can still read from them); they
are only invalid after they are closed.
That's only sort of true; writing to a pipe for which there is no reader
generates SIGPIPE, which is a fatal signal.
Eh, when I talk about an fd being "invalid" here I mean "fd is not a valid
file descriptor" (to use the language for EBADF from the man page for
various system calls like read(2), write(2), close(2)). That's why I say
the fds only become invalid after they
