Hi bug-bash,
The Linux kernel supports creating pipes in a mode (enabled by passing
the O_DIRECT flag, and known sometimes as a "packetized pipe") that
respects message boundaries, to an extent.
An excerpt from the Linux manpage pipe(2):
O_DIRECT (since Linux 3.4)
Create a pipe that performs I/O in "packet" mode. Each
write(2) to the pipe is dealt with as a separate
packet, and read(2)s from the pipe will read one packet
at a time. Note the following points:
* Writes of greater than PIPE_BUF bytes (see pipe(7))
will be split into multiple packets. The constant
PIPE_BUF is defined in <limits.h>.
* If a read(2) specifies a buffer size that is smaller
than the next packet, then the requested number of
bytes are read, and the excess bytes in the packet
are discarded. Specifying a buffer size of PIPE_BUF
will be sufficient to read the largest possible
packets (see the previous point).
* Zero-length packets are not supported. (A read(2)
that specifies a buffer size of zero is a no-op, and
returns 0.)
Older kernels that do not support this flag will
indicate this via an EINVAL error.
I would like to use a packetized pipe in a conventional shell pipeline
to support parallelism. Multiple processes can read from the same pipe
at the same time and not receive any partial lines, as long as the pipe
is being written to one-line-at-a-time, with lines under PIPE_BUF size.
(Parallelism in shell pipelines is an interest of mine; I wrote about
one technique here: https://catern.com/posts/pipes.html but bash support
of packetized pipes would make this much less painful)
So, it would be nice if it was possible in bash to somehow explicitly
request that a pipe between two processes be packetized (on systems that
support this behavior).
I can go into more detail about my use cases, and I can work on adding
such support. Before I do so, though, does this sound like a good
feature to add to bash?
Thanks!
