I think that when programmers first learn shell programming, this is a hard piece of information to effectively convey. The Bash documentation provides the important facts:
- Subshells are quietly and automatically constructed for a variety of shell programming constructs, including pipelines - Code run in a subshell can't affect the parent shell's execution environment But what's important for a new shell programmer, and so easy to miss, is the implication that comes from these facts: piping a command's output into "read" seems like a perfectly reasonable thing to do if you haven't wrapped your head around those two facts and their implications. And if you do try "cmd | read x", it's not considered to be any kind of error or anything, the side-effects of "read" are just carried out and then quietly discarded. So I look at this not just as a RTFM issue, it's a pitfall built-in to the design of the language, and programmers need to understand a bit about the implementation of the language to understand what's going on. As such I think it may be worth spelling it out a bit more directly in terms of the implications here. For instance, stick it in the help for 'read' and 'mapfile': "Care must be taken when using 'read' in a pipeline or another form of subshell environment, as this may cause the data that's read to be lost. See BASH SCRIPTING BASICS (section whatever) for more information." It's the same basic information, but it's better targeted: It lives in the help for the command that's apparently "failing", it says your data may be "lost", and it points at a relevant section of a newbie FAQ that explains why. I'd also wonder if it's maybe worth trying to detect cases of this kind of thing and flagging them as errors... Like if "read" or "mapfile" wind up as the only command in a subshell, the command's side-effects are going to be lost, so it's an error. Of course that does nothing for the wide variety of other cases impacted by this issue, but maybe it's still worth it... If someone does "cmd | read x", and "lastpipe" isn't on AND in effect, it's almost certainly a mistake... (Of course, there's ALWAYS the possibility that someone relies on the current behavior - for instance to see if "read" would succeed, or trigger a SIGPIPE in "cmd" after a certain amount of data is read...) The whole issue of sub-shells is kind of a mess IMO - there's all these cases where one gets created automatically, and the (cmd) syntax which exists specifically to run a command in a subshell, looks to the uninitiated like a simple command-grouping syntax, because that's how parentheses work in C and many other languages... And if something winds up in a subshell that shouldn't be, its side-effects on the shell environment are simply lost without warning. Ideally, forking the shell shouldn't be so baked-in to the language that people wind up tripping over it. Instead these cases that require parallelism should use threading and synchronized access to a shared environment. (So a pipeline could contain *multiple* built-ins or shell functions with side-effects for the shell's environment) But I think that's a difficult direction to pursue, unfortunately, and I'm guessing it's not one that will happen in Bash... (On the bright side it sounds as though POSIX allows it...) ----- Original Message ----- From: chet.ra...@case.edu To:"Keith Thompson" <keith.s.thomp...@gmail.com>, "Eduardo_A._Bustamante_López" <dual...@gmail.com> Cc:<bug-bash@gnu.org>, <chet.ra...@case.edu> Sent:Thu, 29 Jun 2017 14:07:32 -0400 Subject:Re: mapfile doesn't accept input from a pipe On 6/29/17 12:38 PM, Keith Thompson wrote: > I suggest that it would be worthwhile to mention this issue in the > documentation. "Each command in a pipeline is executed as a separate process (i.e., in a subshell). See COMMAND EXECUTION ENVIRONMENT for a description of a subshell environment. If the lastpipe option is enabled using the shopt builtin (see the description of shopt below), the last element of a pipeline may be run by the shell process." or maybe "Builtin commands that are invoked as part of a pipeline are also executed in a subshell environment." -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU c...@case.edu http://cnswww.cns.cwru.edu/~chet/
