Re: Proposed new feature for bash: unbuffered pipes
See stdbuf(1). Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: nameref pointing to array element
On 4/22/20 6:05 PM, Daniel Molina wrote: > Hello, > > Is the following an intended behavior? Undefined? A bug? > > $ a[1]=2; declare -n ref=a[1]; echo $ref $((ref)) > 2 0 Thanks for the report. It looks like a bug, even though the rules for evaluating strings in the expression evaluator are slightly different from the ones for nameref resolution. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Proposed new feature for bash: unbuffered pipes
Andreas Schwab writes: > See stdbuf(1). The limitation(s) of stdbuf are: It can only be applied to the standard I/O streams. It doesn't affect statically-linked executables. It only applies to a single executable, so if the command is a shell function or script, or creates subprocesses, it doesn't affect them. Dale
Re: Proposed new feature for bash: unbuffered pipes
On 4/23/20 4:39 PM, Dale R. Worley wrote: Andreas Schwab writes: See stdbuf(1). The limitation(s) of stdbuf are: It can only be applied to the standard I/O streams. It doesn't affect statically-linked executables. It only applies to a single executable, so if the command is a shell function or script, or creates subprocesses, it doesn't affect them. If you want to define an environment variable for affecting stdout buffering, you can. (Good luck getting any libc to implement the support you're imagining: libc authors are *very* conservative about everything.) But whether or not you have this environment variable, you don't need special bash syntax to set it: you can just prefix the command you want with an environment variable assignment. I don't think the need you're discussing is widespread enough to warrant further complicating shell syntax. A much simpler option might be just convincing libc people to make line-buffering the default whether isatty or not. Machines are very fast these days, and any program that _knows_ it's emitting non-line-oriented data can opt into full buffering.
Unbuffered pipes: cases to be supported
The cases I've found where bash allocates a pipe, and thus an unbuffered pipe may be wanted, are: 1. pipelines: command1 | command2 By analogy with the cases below, I think the demo should be amended to use this syntax: command1 >>| command2 and the parallel for redirecting both stdout and stderr: command1 >>|& command2 This adds these productions to parser.y: pipeline: pipeline GREATER_GREATER_BAR newline_list pipeline | pipeline GREATER_GREATER_BAR_AND newline_list pipeline and parallel changes to read_token(). This notation has the problem that it looks like ">|", which has a completely different meaning (it's OK to write over an existing file). But I think the risks to users are low, as trying to use ">>|" as an append-redirection would cause the next word to be interpreted as a command name, and it would be unlikely to be executable. 2. command substitution: command1 arg arg <( command2 ) arg command1 arg arg >( command2 ) arg In the first case, you may want output from command2 to be unbuffered. In the second case, you may want output from command1 to the stream it opens based on the substituted argument to be unbuffered. A reasonably evocative syntax for these is: command1 arg arg <<( command2 ) arg command1 arg arg >>( command2 ) arg Implementing this is messier because the reading of "words" is all hand coded. But the initial parsing is in read_token_word(), and the processing starts with extract_process_subst() in subst.c. This case also shows that we need to be able to pass to a command indications to not buffer multiple dev/inos, since one can write: command1 >>( command2 ) >>( commnd3 ) and command1 needs to be told to not buffer both pipes. 3. coprocesses: coproc [NAME] command This creates a pipe from bash to the stdin of command and a pipe from the stdout of command to bash, either or both of which could be unbuffered. I was thinking that an intuitive syntax would be: coproc >> [NAME] command coproc << [NAME] command coproc >> << [NAME] command but that doesn't work at all -- command can start with a redirection. Perhaps the indicators can be put at the end of the statement: coproc [NAME] command >> coproc [NAME] command << coproc [NAME] command >> << since a command can't end with a redirection operator and can't be followed by a word. Dale
Cannot shadow local -r when assigning local to array in declaration
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: darwin19.3.0
Compiler: clang
Compilation CFLAGS: -DSSH_SOURCE_BASHRC -Wno-parentheses
-Wno-format-security
uname output: Darwin Thrawn.local 19.4.0 Darwin Kernel Version 19.4.0: Wed
Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64
Machine Type: x86_64-apple-darwin19.3.0
Bash Version: 5.0
Patch Level: 16
Release Status: release
Description:
If a function, f, declares a local readonly variable, v, using local -r, an
error occurs if f calls another function, g, that declares its own local
variable v (therefore shadowing v from f) and that assigns v to an array in
the same statement as the shadowing declaration.
g can, however, assign v to a scalar in a shadowing declaration.
g can also assign a shadowing v to an array, as long as it isn't assigned
in the same statement as the declaration.
Repeat-By:
function aaa() {
local x='1 2 3'
echo "aaa ${x}"
}
function bbb() {
local x
x=(4 5 6)
echo "bbb ${x[*]}"
}
function ccc() {
local x=(7 8 9)
echo "ccc ${x[*]}"
}
function ddd() {
local -r x='0'
echo "ddd ${x}"
aaa
bbb
ccc
}
ddd
# the above outputs the following:
ddd 0
aaa 1 2 3
bbb 4 5 6
bash: x: readonly variable
Re: Proposed new feature for bash: unbuffered pipes
On April 22, 2020 6:32:07 PM wor...@alum.mit.edu (Dale R. Worley) wrote: The crux of the problem, IMHO, is to look at it from the right angle: Occasionally, the user desires that I/O through certain pipes should be unbuffered, that is, the stdio stream(s) that write into the pipe should be unbuffered, rather than the default block-buffered. These are situations where the user can tell that it is desirable to sacrifice efficiency to gain low latency, whereas the writer of the program that the user is running could not predict the needs of this particular invocation. (This is where Ulrich Drepper was incorrect; the writer of the program doesn't know when this is desirable.) This facility is best activated by a feature in bash, as the need arises from the relationship between two processes. The ideal way to implement this would be for the kernel to provide two flavors of pipe, one designated buffered and one designated unbuffered, and according to the user's instructions, bash would allocate an unbuffered pipe when it was needed. The kernel itself would treat the two types of pipe in the same way, but stdio would create buffered streams for fd's that were opened on buffered pipes and it would create unbuffered streams for fd's that were opened on unbuffered pipes. However, getting a coordinated change into the kernel seems like it would be quite difficult. So an alternative implementation is to have a way for bash to notify processes it creates that certain pipes (identified by their dev/ino numbers) should have streams that are unbuffered. Pretty much the only way to do this is to have a conventional environment variable for communicating this information. Of course, either of these solutions requires changes to glibc, but then, if you're going to modify buffering behavior of streams, you are going to have to modify glibc. Given the current political realities in the relevant FOSS ecosystem, it might be easier to add a new pipe flavor or attribute to the kernel and then get libc implementations to support it as a fait accompli than to do something purely in libc. Also, an environment variable is semantically wrong: its scope is too broad, well beyond the specific pipe you care about. Imagine the variable leaking into a daemon that lives forever.
