Re: nofork command substitution
On 5/23/23 6:52 AM, Koichi Murase wrote:
I really appreciate that the feature ${ command; } is finally
implemented. I have a function that mimics the behavior of the nofork
command substitution in my project. I changed the function to switch
to use the new nofork command substitutions in new Bash versions [1]
and tested it in the devel branch of Bash. The initial push of the
nofork command substitution (commit e44e3d50) did not work at all
because of the same error as reported by Oguz [2]. Now with the
latest push (commit 782af562), it seems to work perfectly so far.
Thanks for testing it.
1. Question about the grammar
If the first character following the open brace is a '(', COMMAND
is executed in a subshell, and COMMAND must be terminated by a ')'.
I guess this is just a minor wording issue, but if I literally read
the above description, the nofork command substitution starting with
`${(' needs to have the form « ${(COMMAND);} ». However, as far as I
test it, « ${(COMMAND)} » is also allowed.
Both are valid, but don't get too fond of ${(command)} -- I'm going to
remove it in favor of people using ${ (command); } if they want a subshell.
The parser already did that translation internally, so it was just
syntactic sugar anyway.
I guess `${(' is actually not really
different from the other `${', `${', and `${',
but is just a version where the COMMAND starts with a subshell (...).
Exactly.
Then, can I understand the grammar in the following way? First, there
are two types of nofork command substitutions:
${ compound_list }
${| compound_list }
where `compound_list' is what is defined by EBNF in POSIX XCU 2.10.2.
You could always look at the shell grammar:
comsub: DOLPAREN compound_list ')'
{
$$ = $2;
}
| DOLPAREN newline_list ')'
{
$$ = (COMMAND *)NULL;
}
;
funsub: DOLBRACE compound_list '}'
{
$$ = $2;
}
| DOLBRACE newline_list '}'
{
$$ = (COMMAND *)NULL;
}
;
If we understand it in this way, it is natural to include as an
introducer to the nofork command substitutions in addition to
and because it is the case in the brace grouping. The
opening paren `(' is also the same. There seems to be a suggestion to
exclude , but I think it is strange and inconsistent to exclude
. By the way, if we would be more strictly consistent with the
grammar in the brace grouping, the delimiters `<' and `>' should also
introduce nofork command substitutions, such as `${< file.txt;}'
(which would be a synonym of `$(< file.txt)', I guess) or `${<
file.txt sed s/a/b/g;}'.
I don't think either of those is necessary.
2. About the ending brace
There seems to be a suggestion to allow « } » in an arbitrary position
to terminate the nofork command substitution, but I'm actually opposed
to the suggestion even if it is different from the undocumented
behaviors of ksh and mksh.
I'm not going to do that. I'm surprised ksh93 and mksh allow it.
3. ${(...)} vs $(...)
There seems to be a doubt in introducing `${( compound_list )}' as a
construct distinct from the normal command substitution `$(
compound_list )', but I do need `${( compound_list )}' because the
normal command substitution doesn't create a process group while the
subshell (...) in the nofork command substitution creates it.
Then use ${ (compound_list); }. It's equivalent and doesn't need special
handling.
4. Use cases of ${| ... }
[...]
Having a substitution through a variable `${| compound_list }' along
with the one through stdout `${ compound_list }' is very reasonable
from my perspective as a user of Bash as a scripting language. In
addition, as Chet pointed out, return-via-variable can be done without
calling any syscalls and is much more efficient than constructing a
pipe, writing data, reading data, and tearing down the pipe.
It's not a pipe, right now it's an anonymous file. pipes have size limits
and blocking behavior, and you don't want that for something that's
performed without forking and has to be able to handle arbitrary amounts
of data.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Unset during expansion w/ nofork sub
There are a lot of code paths that (reasonably) do not expect a variable to
disappear while being expanded/assigned to, and usually cause a segfault if
that happens (as is now possible with the new nofork command substitution).
I don't think there is any legitimate use case to be supported here, and I
can't really think of a sensible way to avoid this without adding a bunch
of overhead.
Hypothetically, bash could perform another variable lookup to confirm after
expanding the other words in the PE, but this might have to deal with
issues like array types changing when a new variable is brought into scope,
etc.
A similar situation is already ignored in mapfile's callback (which can
unset the array being populated) and it seems fine to me to ignore these
too, but posting in case others have better ideas.
a=(x); a=(${ unset a; })
heap-use-after-free arrayfunc.c:667 in assign_compound_array_list
a=(x); : ${a[${ unset a; }]}
heap-use-after-free arrayfunc.c:1499 in array_value_internal
a=(x); : ${#a[${ unset a; }]}
heap-use-after-free subst.c:7378 in array_length_reference
a=(x); : ${a[@]/${ unset a; }}
heap-use-after-free subst.c:9331 in parameter_brace_patsub
a=(x); : ${a[@]:${ unset a; }}
heap-use-after-free subst.c:8305:11 in verify_substring_values
a=(x); : ${a[@]:0:${ unset a; }}
heap-use-after-free subst.c:8964:16 in parameter_brace_substring
declare -A A; A=(x ${ unset A; })
heap-use-after-free variables.c:2859:36 in make_variable_value
Re: nofork command substitution
On 5/23/23 4:17 PM, Grisha Levit wrote:
Things don't seem to work right with an empty funsub:
bash --pretty-print <<<'${ }'
${ ; }
It should allow the empty command substitution, like $(), and not add the
`;' when it reconstitutes the text.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: nofork command substitution
On 5/23/23 4:17 PM, Grisha Levit wrote:
An interactive shell gets confused (PS1 shown instead of PS2) when using
newline as the first character of the substitution:
$ bash --norc -i <<<$'${\n:;}'
bash-5.2$ ${
bash-5.2$ :;}
Thanks for the report.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: parsing command substitution inside parameter expansion in interactive shell
On 5/23/23 2:50 PM, Grisha Levit wrote:
One more similar case when the parameter expansion is quoted:
bash --norc -in <<<$'"${_+$("\n")}"'
bash-5.2$ "${_+$("
bash: command substitution: line 2: unexpected EOF while looking for
matching `"'
> ")}"
bash --norc -in <<<$'"${_+$(:\n)}"'
bash-5.2$ "${_+$(:
bash: command substitution: line 3: unexpected EOF while looking for
matching `)'
> )}"
Thanks for the report. These are the same issue.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Searches in bash don't works
Hi. It's the first time I send a message about a bug using bashbug. I don't known if the message was sent properly. So, I repeat it that way. Machine: i686 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall uname output: Linux equipo 4.4.0-210-generic #242~14.04.1-Ubuntu SMP Fri Apr 16 15:02:48 UTC 2021 i686 i686 i686 GNU/Linux Machine Type: i686-pc-linux-gnu Bash Version: 4.3 Patch Level: 11 Release Status: release Description: [Detailed description of the problem, suggestion, or complaint.] *Simply, searching only shows coincidences in the visible text of the window.* Repeat-By: [Describe the sequence of events that causes the problem to occur.] *You search anything; if the term is not visible in the window, you don't obtain results. If you scroll, you can obtain a result if visible.** **Fix:* [Description of how to fix the problem. If you don't know a fix for the problem, don't include this section.]
Re: Searches in bash don't works
On Wed, May 24, 2023, 5:21 PM William via Bug reports for the GNU Bourne Again SHell wrote: > Hi. It's the first time I send a message about a bug using bashbug. I > don't known if the message was sent properly. So, I repeat it that way. > > > Machine: i686 > > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu' > -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' > -DSHELL -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib > -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 > -Wformat -Werror=format-security -Wall > uname output: Linux equipo 4.4.0-210-generic #242~14.04.1-Ubuntu SMP Fri > Apr 16 15:02:48 UTC 2021 i686 i686 i686 GNU/Linux > Machine Type: i686-pc-linux-gnu > > Bash Version: 4.3 > Patch Level: 11 > Release Status: release > > Description: > [Detailed description of the problem, suggestion, or complaint.] > *Simply, searching only shows coincidences in the visible text of the > window.* > Repeat-By: > [Describe the sequence of events that causes the problem > to occur.] > *You search anything; if the term is not visible in the window, you > don't obtain results. If you scroll, you can obtain a result if visible.** > **Fix:* > [Description of how to fix the problem. If you don't know a > fix for the problem, don't include this section.] > I think you're probably describing an issue with your terminal rather than with Bash. How do you perform the search (what key do you press or menu do you select)? Specific and detailed answers to questions like this are what should be provided in the "Describe the sequence" section rather than general conceptual descriptions. If the problem is with your terminal you will need to contact support for it or for the distribution or operating system that provides it. >
text similar to JSON can cause memory of bash to become abnormally large
hi
I accidentally copied a section of JSON text to the terminal and executed it,
which resulted in a system crash.Bash version is 5.1.8.
Later, I reproduced it and found that when I executed the following command,
the memory usage of bash became very large.
$
[{"key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1"},{"key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1"},{"key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1"},{"key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1","key":"1"}]
Through tracking the code, it was found that in the brace_expand_word_list
function, there is a continuous recursive execution of the brace_expand
function, which causes the len2 variable of function array_concat to become
abnormally large, and then a large amount of memory will be malloced based on
the value of len2
This may be a normal manifestation of bash rather than a problem. But in the
case of user misoperation, it can cause system crashes, so I still hope that
this situation can be improved
