Shell Grammar man page

[Mike Jonkmans]

Hi,

It seems some things are missing in the bash manual.
Notably definition of command and placements of coproc- and function-definition.

The section 'SHELL GRAMMAR' describes:
- simple-command
- pipeline
- list
- compound-command
- coproc
- function-definition

Simplified, a pipeline is:
[ ! ] command1 [ | command2 ]

A list is a sequence of pipelines separated by ;, &, &&, or ||
and optionally terminated by ;, & or NL.

Within the list description, the and-list and or-list are described as:
and-list: command1 && command2
or-list: command1 || command2
Though the text says that and/or-lists are sequences of pipelines
(separated by && or ||).

That is slightly inconsistent.
A pipeline can be reduced to a command.
I don't think a command can be a pipeline nor a list.
In that case, the and/or list should read:
and-list: pipeline1 && pipeline2
or-list: pipeline1 || pipeline2

These are valid:
: | { :; }
list: pipeline: command1 s|' command2
command1: simple-command
command2: compound-command

{ :; } || :
list: or-list: pipeline1 || pipeline2
pipeline1: command : compound-command
pipeline2: command: simple-command

:() if [ x = x ]; then echo; fi 

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/22/21 12:59 AM, Oğuz wrote:



21 Şubat 2021 Pazar tarihinde Chet Ramey > yazdı:


On 2/21/21 3:05 PM, Oğuz wrote:

With the latest push to devel bash now accepts `if ! [[ ... ]] then
:; fi' too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time
[[ ... ]]') a valid compound command or not?


Technically not; they're pipelines.


So, if the last component of a pipeline is a compound command, a separator 
between the pipeline and `then', `fi', `do', `done', or `esac' is optional,


It's really a reserved word that terminates a command that makes the
closing `esac' optional.



but a pipeline alone can not be a function body no matter what.


Correct. The POSIX grammar makes this fairly clear, its issues with
`rule 4' aside.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

Okay, thank you


22 Şubat 2021 Pazartesi tarihinde Chet Ramey  yazdı:

> On 2/22/21 12:59 AM, Oğuz wrote:
>
>>
>>
>> 21 Şubat 2021 Pazar tarihinde Chet Ramey > chet.ra...@case.edu>> yazdı:
>>
>> On 2/21/21 3:05 PM, Oğuz wrote:
>>
>> With the latest push to devel bash now accepts `if ! [[ ... ]]
>> then
>> :; fi' too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time
>> [[ ... ]]') a valid compound command or not?
>>
>>
>> Technically not; they're pipelines.
>>
>>
>> So, if the last component of a pipeline is a compound command, a
>> separator between the pipeline and `then', `fi', `do', `done', or `esac' is
>> optional,
>>
>
> It's really a reserved word that terminates a command that makes the
> closing `esac' optional.
>
>
> but a pipeline alone can not be a function body no matter what.
>>
>
> Correct. The POSIX grammar makes this fairly clear, its issues with
> `rule 4' aside.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>


-- 
Oğuz

Re: Behaviour of test -v with assoc array and quote character in key

[Chet Ramey]

On 2/15/21 1:28 PM, Daniel Gröber wrote:


On Mon, Feb 15, 2021 at 09:11:48AM -0500, Chet Ramey wrote:

`test' is always going to be problematic here because, as a shell builtin,
its arguments undergo a round of word expansions before it's invoked. It's
difficult to reliably detect the closing bracket in an array subscript as a
result, even if the array subscript expansion didn't perform any expansions
on its own. (Consider what would happen if $1 were `]').


You're absolutely right, I didn't consider the ']' case at all! That would
obviously break. So indeed this just user-error then and not really a bug.


Well, it's less than ideal. This is why I introduced the assoc_expand_once
option, but as many folks on here will tell you, it's not a perfect 
solution. This is one of the cases where it's not.


I think I have a better way to do it, one that doesn't require using an
extra level of quoting or assoc_expand_once, but I'm not ready to commit
anything yet, and builtins like test/[ and let will always cause problems.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

building 5.1.3 -- some probs...

[L A Walsh]

I'm trying to build bash 5.1.3, and at first
I tried w/bash-malloc, but got:
/bash-5.1> ./bash

malloc: subst.c:4751: assertion botched
free: called with unallocated block argument
Aborting...Aborted (core dumped)
---

Another prob which seems a bit odd -- more than once, on the
first time after a make clean+ rerun config, then
doing a 'make -j 6', I've gotten a:
/usr/bin/gcc  -DPROGRAM='"bash"' -DCONF_HOSTTYPE='"x86_64"' 
-DCONF_OSTYPE='"linux-gnu"' -DCONF_MACHTYPE='"x86_64-pc-linux-gnu"' 
-DCONF_VENDOR='"pc"' -DLOCALEDIR='"//share/locale"' -DPACKAGE='"bash"' 
-DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib-fpic 
-march=native -pipe -fpic -march=native -pipe -fpic -march=native -pipe 
-fpic -march=native -pipe -fpic -march=native -pipe -fpic -march=native 
-pipe -fpic -march=native -pipe -fpic -march=native -pipe -flto 
-Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb 
-flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 
-ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed 
-Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto 
-Wl,--no-as-needed -Og -g3 -ggdb  -c stringlib.c
bashline.c:65:10: fatal error: builtins/builtext.h: No such file or 
directory

#include "builtins/builtext.h"  /* for read_builtin */
 ^
compilation terminated.
make: *** [Makefile:101: bashline.o] Error 1
make: *** Waiting for unfinished jobs
/usr/lib64/gcc/x86_64-suse-linux/8/../../../../x86_64-suse-linux/bin/ld: 
total time in link: 0.020174

./mkbuiltins -externfile builtext.h -structfile builtins.c \
   -noproduction -D .   ./alias.def ./bind.def ./break.def 
./builtin.def ./caller.def ./cd.def ./colon.def ./command.def 
./declare.def ./echo.def ./enable.def ./eval.def ./getopts.def 
./exec.def ./exit.def ./fc.def ./fg_bg.def ./hash.def ./help.def 
./history.def ./jobs.def ./kill.def ./let.def ./read.def ./return.def 
./set.def ./setattr.def ./shift.def ./source.def ./suspend.def 
./test.def ./times.def ./trap.def ./type.def ./ulimit.def ./umask.def 
./wait.def ./reserved.def ./pushd.def ./shopt.def ./printf.def 
./complete.def ./mapfile.def

make[1]: Leaving directory '/home/tools/bash/bash-5.1/builtins'

However, then I find if I do a 'make' and no -j, it finally ends w/no 
error, but running I still get a core dump:

bash-5.1> ./bash

malloc: subst.c:4751: assertion botched
free: called with unallocated block argument
Aborting...Aborted (core dumped)

The config I tried:
declare -a enable=(
   alias arith-for-command array-variables
   brace-expansion
   casemod-attributes casemod-expansions command-timing
   cond-command cond-regexp coprocesses
   debugger
   directory-stack disabled-builtins
   dparen-arithmetic
   extended-glob extended-glob-default function-import
   glob-asciiranges-default
   help-builtin history
   job-control
   multibyte
   net-redirections
   process-substitution progcomp prompt-string-decoding
   readline select single-help-strings
)

declare -a disable=( nls rpath )
declare -a with=( gnu-ld installed-readline bash-malloc)
declare -a without=()


FWIW, I just tried a make clean followed by a plain make (no
parallel, and it did build, but same core dump.  Also,
my final 'link' step for bash:
make[1]: Leaving directory '/home/tools/bash/bash-5.1/lib/malloc'
rm -f bash
/usr/bin/gcc -L./builtins -L./lib/readline -L./lib/readline -L./lib/glob 
-L./lib/tilde -L./lib/malloc -L./lib/sh  -fpic -march=native -pipe -fpic 
-march=native -pipe -fpic -march=native -pipe -fpic -march=native -pipe 
-fpic -march=native -pipe -fpic -march=native -pipe -fpic -march=native 
-pipe -fpic -march=native -pipe -flto -Wl,--no-as-needed -Og -g3 -ggdb 
-flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 
-ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed 
-Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto 
-Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb 
-Wl,--default-imported-symver -Wl,--default-symver -Wl,--stats -fpic 
-march=native -pipe -fpic -march=native -pipe -fpic -march=native -pipe 
-fpic -march=native -pipe -fpic -march=native -pipe -fpic -march=native 
-pipe -fpic -march=native -pipe -flto -Wl,--no-as-needed -Og -g3 -ggdb 
-flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 
-ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed 
-Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto 
-Wl,--no-as-needed -Og -g3 -ggdb -Wl,--default-imported-symver 
-Wl,--default-symver -Wl,--stats -fpic -march=native -pipe -fpic 
-march=native -pipe -fpic -march=native -pipe -fpic -march=native -pipe 
-fpic -march=native -pipe -fpic -march=native -pipe -flto 
-Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb 
-flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed -Og -g3 
-ggdb -flto -Wl,--no-as-needed -Og -g3 -ggdb -flto -Wl,--no-as-needed 
-Og -g3 -ggdb -Wl,--default-imported-s

Re: building 5.1.3 -- some probs...

[Chet Ramey]

On 2/22/21 8:08 PM, L A Walsh wrote:

I'm trying to build bash 5.1.3, and at first
I tried w/bash-malloc, but got:
/bash-5.1> ./bash

malloc: subst.c:4751: assertion botched
free: called with unallocated block argument
Aborting...Aborted (core dumped)


You could start with the actual command that's dumping core (something in
one of your startup files) and your locale. A backtrace from the core dump
would help, too, but not as much.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: building 5.1.3 -- some probs...

[L A Walsh]

(Doi!) Forgot it was executing initial rc scripts.
Turned on -x since last statement seems pretty mundane.
Also 6 statements after where it claimed it crashed, is
a custom function for printing pwd for the prompt.

I've tried with different compile ops (optim vs. dbg).
with builtin readline and included readline
with bash-malloc set, and unset..

This isn't blocking anything, so no time pressure.
Thanks! & let me know if you want anything else...


locale has:

LANG=
LC_CTYPE=C.UTF-8
LC_NUMERIC=C
LC_TIME=C
LC_COLLATE=C
LC_MONETARY=C
LC_MESSAGES=C
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
---

On 2021/02/22 18:22, Chet Ramey wrote:

You could start with the actual command that's dumping core (something in
one of your startup files) and your locale. A backtrace from the core dump
would help, too, but not as much.

attach is output from bash -x .. last line displayed in that
was line#288.
Starting from line 288, about 4 statements down are
the defines for a "short"-pwd (spwd)
for my prompt. 


It might have been in that function -- as it seems it is the
only thing of any complexity. I skipped blank lines and some
comments.  Line number is line after the line# comment.

#line 285:
export LC_COLLATE=C
export MOUNT_PRINT_SOURCE=1
# line 288:
if [[ ! ${qUSER:-""} ]]; then
 printf -v qUSER "%q" $USER
 export qUSER
fi
# line 293:
export _home_prefix=${HOME%/*}/

# return a shortened path when displayed path would
# take up > 50% width of the screen
# line 306:
declare -a _als=( "_e=echo -En"  "ret=return" )
alias "${_als[@]}"
export __dpf__='local -a PF=(
   "/$1/$2/$3/…/\${$[$#-1]}/\${$#}"
   "/$1/$2/…/\${$[$#-1]}/\${$#}"
   "/$1/…/\${$[$#-1]}/\${$#}"
   "/$1/…/\${$#}"
   "…/\${$#}"
   "…" )'
line 314:
spwd () {  my _f_=""  ;\
 [[ ${_f_:=${-//[^x]/}} ]] && set +$_f_  ;\
 (($#))|| { set "${PWD:=$(echo -En $( \
 eval "{,{,/usr}/bin/}pwd 2>&-||:" ))}"  ;\
 (($#)) || ret 1; }  ;\
 int w=COLUMNS/2 ;\
 ( printf -v _p "%s" "$1" ; export IFS=/ ;\
   set $_p; shift; unset IFS ;\
   t="${_p#$_home_prefix}"   ;\
   int tl=${#t}  ;\
   if (($#<=6 && tl gdb ./bash core
Reading symbols from ./bash...done.
[New LWP 42448]
Core was generated by `./bash -x'.
Program terminated with signal SIGABRT, Aborted.
#0  0x003000238d8b in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: zypper install 
glibc-debuginfo-2.29-1.2.x86_64

(gdb) where
#0  0x003000238d8b in raise () from /lib64/libc.so.6
#1  0x003000222612 in abort () from /lib64/libc.so.6
#2  0x00451933 in programming_error ()
#3  0x004e8d93 in xbotch (mem=mem@entry=0x54e460, e=e@entry=2,
   s=s@entry=0x507308 "free: called with unallocated block argument",
   file=file@entry=0x4ef0b0 "subst.c", line=line@entry=4751) at 
malloc.c:376

#4  0x004e8f1a in internal_free (mem=0x54e460,
   file=0x4ef0b0 "subst.c", line=4751, flags=flags@entry=1) at malloc.c:974
#5  0x004e9b58 in sh_free (mem=, file=out>,

   line=) at malloc.c:1380
#6  0x0049e3d5 in sh_xfree ()
#7  0x00467829 in remove_pattern ()
#8  0x00468c8a in parameter_brace_remove_pattern ()
#9  0x00471b4e in parameter_brace_expand ()
#10 0x004726fa in param_expand ()
#11 0x00473a71 in expand_word_internal ()
#12 0x00476fa2 in shell_expand_word_list ()
#13 0x00477292 in expand_word_list_internal ()
#14 0x004760d5 in expand_words ()
#15 0x00445d15 in execute_simple_command ()
#16 0x0043f850 in execute_command_internal ()
#17 0x004a4f54 in parse_and_execute (string=,
   from_file=from_file@entry=0x5f7390 "/etc/local/aliases.sh",
   flags=flags@entry=20) at evalstring.c:489
#18 0x004a4306 in _evalfile (
   filename=0x5f7390 "/etc/local/aliases.sh", flags=14) at evalfile.c:285
#19 0x004a441d in source_file (
   filename=filename@entry=0x5f7390 "/etc/local/aliases.sh",
   sflags=sflags@entry=0) at evalfile.c:380
#20 0x004aebf7 in source_builtin (list=0x5f80b0) at ./source.def:195
#21 0x00446b1e in execute_builtin ()
#22 0x00447a1e in execute_builtin_or_function ()
#23 0x00446420 in execute_simple_command ()
#24 0x0043f850 in execute_command_internal ()
#25 0x0043ecda in execute_command ()
#26 0x00442aeb in execute_connection ()
#27 0x0043fc92 in execute_command_internal ()
#28 0x0043ecda in execute_command ()
#29 0x00444791 in execute_if_command ()
---Type  to continue, or q  to quit---
#30 0x0043fb8f in execute_command_internal ()
#31 0x004a4f54 in parse_and_execute (string=,
   from_file=from_file@entry=0x5fd150 "/etc/local/bashrc.sh",
   flags=flags@entry=20) at evalstring.c:489
#32 0x004a