Re: Follow-up to Non-expanding here-documents inside command substitution

2017-02-24 Thread Eduardo Bustamante 
On Thu, Feb 23, 2017 at 11:14 PM, Kevin Grigorenko
 wrote:
> On February 11th 2017, there was a discussion on the topic of "Non-expanding
> here-documents inside command substitution are subject to newline joining"
> where it was confirmed that Bash contains a bug. Two questions:
>
> 1. Is there a plan to fix this or should a patch be submitted?

The fix is already in the "devel" branch:
http://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel&id=ac495185aab17ef6030a02ddc0be33380a27a94d

Re: Follow-up to Non-expanding here-documents inside command substitution

2017-02-24 Thread Chet Ramey 
On 2/24/17 12:14 AM, Kevin Grigorenko wrote:
> On February 11th 2017, there was a discussion on the topic of
> "Non-expanding here-documents inside command substitution are subject to
> newline joining" where it was confirmed that Bash contains a bug. Two
> questions:
> 
> 1. Is there a plan to fix this or should a patch be submitted?
> 2. Regarding the response:
> 
>> [POSIX] is not necessarily ambiguous, but it does require close
>> reading.
> 
> Can you please explain how it was concluded that it's a bug based on the
> specification?

OK.  The command substitution is a red herring; it's relevant only in that
it pointed out where the bug was.

The delimiter to the here-document is quoted, so the lines are not
expanded. In this case, the shell reads lines from the input as if they
were quoted. If a backslash appears in a context where it is quoted, it
does not act as an escape character (see below), and the special handling
of backslash-newline does not take place. In fact, if any part of the
delimiter is quoted, the here-document lines are read as if single-quoted.

The text in Posix 2.2.1 is written awkwardly, but means that the backslash
is only treated specially when it's not quoted.  You can quote a backslash
and inhibit all all expansion only with single quotes or another backslash.

The close reading part is the "not expanded" text implying the single
quotes. The standard says in 2.2 that here documents are "another form of
quoting," but the only form of quoting in which words are not expanded at
all is single quotes.  So it's a form of quoting that is just about
exactly like single quotes, but not single quotes.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: [BUG] With null IFS, ${1+$*}, ${var-$*}, etc. don't generate fields

2017-02-24 Thread Grisha Levit 
On Sat, Feb 18, 2017 at 8:35 PM, Martijn Dekker  wrote:
>
> When IFS is null, unquoted $* within an unquoted substitution (e.g.
> ${1+$*} or ${unset_var-$*}) joins the PPs together into one field, as if
> either "$*" or the whole substitution were quoted.
>
> POSIX says the unquoted substitution is supposed to generate one field
> for each positional parameter, with further field splitting inactive due
> to empty IFS.


XRAT C.2.5 : Special Parameters [1] specifically addresses the case of
${unset_var-$*}.  The following example is provided:

set "abc" "def ghi" "jkl"
IFS='' # null
unset var
printf '%s\n' ${var-$*}

abcdef ghijkl

..which seems to contradict your proposed expected output.  It doesn't
specifically address ${1+$*}, but having ${1+$*} mirror the behavior of
${unset_var-$*} seems appropriate?

   [1]
http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_05_02

Re: [BUG] With null IFS, ${1+$*}, ${var-$*}, etc. don't generate fields

2017-02-24 Thread Chet Ramey 
On 2/24/17 3:20 PM, Grisha Levit wrote:
> 
> 
> On Sat, Feb 18, 2017 at 8:35 PM, Martijn Dekker  > wrote:
>>
>> When IFS is null, unquoted $* within an unquoted substitution (e.g.
>> ${1+$*} or ${unset_var-$*}) joins the PPs together into one field, as if
>> either "$*" or the whole substitution were quoted.
>>
>> POSIX says the unquoted substitution is supposed to generate one field
>> for each positional parameter, with further field splitting inactive due
>> to empty IFS.

If you want the expansion to be aware of the value of IFS and whether or
field splitting will take place as a result, and I think that's the intent
of the example, then the part of the description of $* that details what
will happen when field splitting does not take place is active.  That's
where it describes joining the positional parameters into a single string.

The question is whether it's the null IFS that's supposed to dictate
whether  field splitting will be performed, or the context of the
expansion (e.g., whether it's on the RHS of an assignment statement).  The
description of $@ makes the distinction between context and effect, but
the description of $* does not.

This was the result of a long discussion in October, 2014 that resulted in
Posix interpretation 888:

http://austingroupbugs.net/view.php?id=888

You might be able to find the discussion in a mailing list archive somewhere.

You might also want to send a message to the austin group and ask for a
discussion concerning that specific case.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: [BUG] With null IFS, ${1+$*}, ${var-$*}, etc. don't generate fields

2017-02-24 Thread Chet Ramey 
On 2/24/17 4:07 PM, Chet Ramey wrote:
> On 2/24/17 3:20 PM, Grisha Levit wrote:
>>
>>
>> On Sat, Feb 18, 2017 at 8:35 PM, Martijn Dekker > > wrote:
>>>
>>> When IFS is null, unquoted $* within an unquoted substitution (e.g.
>>> ${1+$*} or ${unset_var-$*}) joins the PPs together into one field, as if
>>> either "$*" or the whole substitution were quoted.
>>>
>>> POSIX says the unquoted substitution is supposed to generate one field
>>> for each positional parameter, with further field splitting inactive due
>>> to empty IFS.
> 
> If you want the expansion to be aware of the value of IFS and whether or
> field splitting will take place as a result, and I think that's the intent
> of the example, then the part of the description of $* that details what
> will happen when field splitting does not take place is active.  That's
> where it describes joining the positional parameters into a single string.

It might also be the case that, as I think I said in my original reply (?),
field splitting is just not one of the expansions that is performed on the
`word' in a ${paramOPword} expansion, and that is what determines the
expansion of $*.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

$* vs ${ARRAY[*]} differences in some parameter expansions

2017-02-24 Thread Grisha Levit 
bash currently handles all parameter expansions with $* and $@ as described
in interpretation 888, and the behavior when using an array subscripted
with @ matches identically the behavior of the examples concerning $@.
However, the behavior when using an array subscripted with * does not
always match the behavior of the examples concerning $*.

This difference is surprising since otherwise the behaviors are identical
(though I'm not sure if would be proper to call this a "bug" since the
isomorphism is not explicitly defined in the documentation).


$ set -- "abc" "def ghi" "jkl"
$ A=("$@")



$ IFS=':'

$ unset var; printf '%s\n' ${var-$*}
abc
def ghi
jkl

$ unset var; printf '%s\n' ${var-${A[*]}}
abc def ghi jkl



$ unset  var; printf '%s\n' ${var=$*}
abc
def ghi
jkl
$ printf 'var=%s\n' "$var"
var=abc:def ghi:jkl

$ unset var; printf '%s\n' ${var=${A[*]}}
abc def ghi jkl
$ printf 'var=%s\n' "$var"
var=abc def ghi jkl



$ IFS=''

$ unset var; printf '%s\n' ${var-$*}
abcdef ghijkl

$ unset var; printf '%s\n' ${var-${A[*]}}
abc
def ghi
jkl



$ unset var; printf '%s\n' ${var=$*}
abcdef ghijkl
$ printf 'var=%s\n' "$var"
var=abcdef ghijkl

$ unset var; printf '%s\n' ${var=${A[*]}}
abc
def
ghi
jkl
$ printf 'var=%s\n' "$var"
var=abc def ghi jkl

COMP_WORDS, namerefs and readonly variables

2017-02-24 Thread Grisha Levit 
In bind_comp_words, the readonly attribute is unconditionally removed from
COMP_WORDS, even if that variable is a nameref.  This allows unsetting the
readonly attribute on any arbitrary variable.

v = find_variable ("COMP_WORDS");
if (v == 0)
  v = make_new_array_variable ("COMP_WORDS");
if (readonly_p (v))
  VUNSETATTR (v, att_readonly);

For example:

$ readonly var=foo
$ declare -n COMP_WORDS=var
$ f() { :; }
$ compgen -F f .
bash: compgen: warning: -F option may not work as you expect
$ declare -p var
declare -a var=([0]="foo")

I'm not sure why shooting yourself in the foot by making COMP_WORDS
readonly should be forbidden, since the readonly attribute is not removed
from any of the other COMP_* variables.

COMPREPLY as a nameref to a readonly associative array

2017-02-24 Thread Grisha Levit 
It's possible to change any readonly associative array to the indexed
array (''):

$ declare -rA var=([foo]=bar)
$ f() { declare -ng COMPREPLY=var; }
$ compgen -F f .
bash: compgen: warning: -F option may not work as you expect

$ declare -p var
declare -ar var=([0]="")