Heredoc leading tab/space behaviour

[Ing. Gerold Broser]

Hello,

Configuration Information 1:

Machine: x86_64
OS: linux-gnu
Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.3 
-L/home/abuild/rpmbuild/BUILD/bash-4.3/../readline-6.3
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' 
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H -I. -I. -I./include -I./lib -fmessage-length=0 
-grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 
-funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g 
-D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g -Wuninitialized -Wextra 
-Wno-unprototyped-calls -Wno-switch-enum -Wno-unused-variable 
-Wno-unused-parameter -Wno-parentheses -ftree-loop-linear -pipe -DBNC382214=0 
-DIMPORT_FUNCTIONS_DEF=0 -fprofile-use
uname output: Linux <...host name...> 4.12.14-122.116-default #1 SMP Tue Apr 5 
13:16:39 UTC 2022 (b42b08b) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-suse-linux-gnu

Bash Version: 4.3
Patch Level: 48
Release Status: release

Configuration Information 2 (Win 10 Enterprise, Git Bash v2.34.1):

Machine: x86_64
OS: msys
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash.exe' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='msys' -DCONF_MACHTYPE='x86_64-pc-msys' -DCONF_VENDOR='pc' 
-DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H 
-DRECYCLES_PIDS -I. -I. -I./include -I./lib -DWORDEXP_OPTION 
-Wno-discarded-qualifiers -march=x86-64 -mtune=generic -O2 -pipe 
-Wno-parentheses -Wno-format-security -D_STATIC_BUILD -g
uname output: MINGW64_NT-10.0-19042 <...host name...> 3.1.7-340.x86_64 
2021-10-12 16:29 UTC x86_64 Msys
Machine Type: x86_64-pc-msys

Bash Version: 4.4
Patch Level: 23
Release Status: release


Description:

The Bash Reference Manual says about Heredoc's redirection operator '<<-':

"If the redirection operator is ‘<<-’, then all leading tab characters are 
stripped from input lines and the line containing delimiter."
– https://www.gnu.org/software/bash/manual/bash.html#Here-Documents

This is only half true since apparently also leading spaces are stripped from 
input lines (while they are not from the delimiter line, and lead to an error 
there).


Repeat-By:

 Note 
Unicode characters:
- blank (U+2423) for space (U+20)
and
- tab (U+2B73) + 7 spaces for Tab (U+9)
used for clarity in the following.
--

heredoc-error.sh:

#!/bin/bash
sed -f - heredoc-error.sh <<-SCRIPT
␣␣s/something/else/
⭲   s/more/else/
␣␣SCRIPT

$ . heredoc.sh
bash: warning: here-document at line 2 delimited by end-of-file (wanted 
`SCRIPT')
sed: file - line 3: unknown command: `S'


$ . heredoc-ok.sh
#!/bin/bash
sed -f - heredoc-ok.sh <<-SCRIPT
␣␣s/else/else/
⭲   s/else/else/
⭲   SCRIPT


Fix:

I think it'd be convenient if Heredoc supports indentation with spaces at the 
delimiter line, too.

And, BTW, there's a mixture of (leading) backtick (U+60) and (trailing) 
apostrophe (U+27) at `SCRIPT' and `S' in the error message.

Kind regards

Gerold 'Geri' Broser

Re: Heredoc leading tab/space behaviour

[Kerin Millar]

Hi Gerold,

On Fri, 24 Jun 2022 13:53:07 +0200 (CEST)
"Ing. Gerold Broser" via Bug reports for the GNU Bourne Again SHell 
 wrote:

> Hello,
> 
> Configuration Information 1:
> 
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.3 
> -L/home/abuild/rpmbuild/BUILD/bash-4.3/../readline-6.3
> Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' 
> -DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
> -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -fmessage-length=0 
> -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 
> -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g 
> -D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g -Wuninitialized -Wextra 
> -Wno-unprototyped-calls -Wno-switch-enum -Wno-unused-variable 
> -Wno-unused-parameter -Wno-parentheses -ftree-loop-linear -pipe -DBNC382214=0 
> -DIMPORT_FUNCTIONS_DEF=0 -fprofile-use
> uname output: Linux <...host name...> 4.12.14-122.116-default #1 SMP Tue Apr 
> 5 13:16:39 UTC 2022 (b42b08b) x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-suse-linux-gnu
> 
> Bash Version: 4.3
> Patch Level: 48
> Release Status: release
> 
> Configuration Information 2 (Win 10 Enterprise, Git Bash v2.34.1):
> 
> Machine: x86_64
> OS: msys
> Compiler: gcc
> Compilation CFLAGS: -DPROGRAM='bash.exe' -DCONF_HOSTTYPE='x86_64' 
> -DCONF_OSTYPE='msys' -DCONF_MACHTYPE='x86_64-pc-msys' -DCONF_VENDOR='pc' 
> -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H 
> -DRECYCLES_PIDS -I. -I. -I./include -I./lib -DWORDEXP_OPTION 
> -Wno-discarded-qualifiers -march=x86-64 -mtune=generic -O2 -pipe 
> -Wno-parentheses -Wno-format-security -D_STATIC_BUILD -g
> uname output: MINGW64_NT-10.0-19042 <...host name...> 3.1.7-340.x86_64 
> 2021-10-12 16:29 UTC x86_64 Msys
> Machine Type: x86_64-pc-msys
> 
> Bash Version: 4.4
> Patch Level: 23
> Release Status: release
> 
> 
> Description:
> 
> The Bash Reference Manual says about Heredoc's redirection operator '<<-':
> 
> "If the redirection operator is ‘<<-’, then all leading tab characters are 
> stripped from input lines and the line containing delimiter."
> – https://www.gnu.org/software/bash/manual/bash.html#Here-Documents
> 
> This is only half true since apparently also leading spaces are stripped from 
> input lines (while they are not from the delimiter line, and lead to an error 
> there).

They're not, though.

$ printf '%s\n' 'cat <<-EOF' $'\x20\x20foo' $'\t\tbar' EOF | bash | od -a
000  sp  sp   f   o   o  nl   b   a   r  nl
012

Note that the two space characters preceding foo are preserved, whereas the two 
leading tab characters preceding bar are not.

> 
> 
> Repeat-By:
> 
>  Note 
> Unicode characters:
> - blank (U+2423) for space (U+20)
> and
> - tab (U+2B73) + 7 spaces for Tab (U+9)
> used for clarity in the following.
> --
> 
> heredoc-error.sh:
> 
> #!/bin/bash
> sed -f - heredoc-error.sh <<-SCRIPT
> ␣␣s/something/else/
> ⭲   s/more/else/
> ␣␣SCRIPT
> 
> $ . heredoc.sh
> bash: warning: here-document at line 2 delimited by end-of-file (wanted 
> `SCRIPT')
> sed: file - line 3: unknown command: `S'
> 
> 
> $ . heredoc-ok.sh
> #!/bin/bash
> sed -f - heredoc-ok.sh <<-SCRIPT
> ␣␣s/else/else/
> ⭲   s/else/else/
> ⭲   SCRIPT
> 
> 
> Fix:
> 
> I think it'd be convenient if Heredoc supports indentation with spaces at the 
> delimiter line, too.

One issue with this proposal is that it would violate the rules of the Shell 
Command Language, per 
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04.

-- 
Kerin Millar

Re: Heredoc leading tab/space behaviour

[Chet Ramey]

On 6/24/22 7:53 AM, Ing. Gerold Broser via Bug reports for the GNU Bourne 
Again SHell wrote:



Bash Version: 4.3
Patch Level: 48
Release Status: release


I know you're kind of at the mercy of your distro here, but this is almost
six years old, and three versions out of date. You might consider an
update.



Description:

The Bash Reference Manual says about Heredoc's redirection operator '<<-':

"If the redirection operator is ‘<<-’, then all leading tab characters are stripped 
from input lines and the line containing delimiter."
– https://www.gnu.org/software/bash/manual/bash.html#Here-Documents

This is only half true since apparently also leading spaces are stripped from 
input lines (while they are not from the delimiter line, and lead to an error 
there).


They are not, as this example shows:

$ cat x3
cat <<-SCRIPT
 one
  two
   three
four
SCRIPT
$ ../bash-4.4-patched/bash ./x3
 one
  two
   three
four
$ ../bash-4.4-patched/bash --version
GNU bash, version 4.4.23(8)-release (x86_64-apple-darwin15.6.0)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [ ! ! hey ] -> [: too many arguments

[Steffen Nurpmeso]

Greg Wooledge wrote in
 :
 |On Thu, Jun 23, 2022 at 05:58:23PM -0400, Chet Ramey wrote:
 |> On 6/23/22 5:08 PM, Steffen Nurpmeso wrote:
 |>>bash -c 'if [ ! ! hey = hey ]; then echo du; fi'
 |
 |> Sure. This one isn't a common idiom for shell programmers, apparently.
 |
 |I've seen it used in a math context, inside PS1.  Like so:
 |
 |color=(
 |  "$(tput setaf 2)"  # 0 = green
 |  "$(tput setaf 1)"  # 1 = red
 |)
 |normal="$(tput sgr0)"
 |PS1='\[${color[!!$?]}\]$?\[$normal\] \h:\w\$ '
 |
 |This writes the previous command's exit status in either green or red.
 |It "smashes" all nonzero exit codes to 1, using a double negation in
 |a C-style arithmetic context, in order to generate an array index for
 |the color.
 |
 |I don't recall ever seeing it used in the way Steffen showed, though.

Because the devil is in your car!
That always makes a subdivision.

Other than that just dance this mess around.

 --End of 

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Heredoc leading tab/space behaviour

[Lawrence Velázquez]

On Fri, Jun 24, 2022, at 9:57 AM, Kerin Millar wrote:
> On Fri, 24 Jun 2022 13:53:07 +0200 (CEST)
> "Ing. Gerold Broser" via Bug reports for the GNU Bourne Again SHell 
>  wrote:
>
>> heredoc-error.sh:
>> 
>> #!/bin/bash
>> sed -f - heredoc-error.sh <<-SCRIPT
>> ␣␣s/something/else/
>> ⭲   s/more/else/
>> ␣␣SCRIPT
>> 
>> $ . heredoc.sh
>> bash: warning: here-document at line 2 delimited by end-of-file (wanted 
>> `SCRIPT')
>> sed: file - line 3: unknown command: `S'
>> 
>> 
>> $ . heredoc-ok.sh
>> #!/bin/bash
>> sed -f - heredoc-ok.sh <<-SCRIPT
>> ␣␣s/else/else/
>> ⭲   s/else/else/
>> ⭲   SCRIPT

It has already been explained that you're mistaken, but I don't
understand how these examples are supposed to demonstrate that
leading spaces have been removed from the input.  sed(1) ignores
blank characters before commands.

>> Fix:
>> 
>> I think it'd be convenient if Heredoc supports indentation with spaces at 
>> the delimiter line, too.
>
> One issue with this proposal is that it would violate the rules of the 
> Shell Command Language, per 
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04.

Coincidentally, a request to allow <<- to behave this way was
recently submitted.  It was rejected within two days, which feels
like it should be some sort of record.

https://austingroupbugs.net/view.php?id=1588

-- 
vq