Re: Changing the way bash expands associative array subscripts

2021-04-15 Thread Chet Ramey

On 4/13/21 11:11 PM, Koichi Murase wrote:

2021年4月14日(水) 0:24 Chet Ramey :

On 4/13/21 5:01 AM, Koichi Murase wrote:

But I expected some design consideration enabling a[$key] for an
arbitrary key in the indirect expansions and namerefs.


Why? Why should the shell carry around (and expect the user to remember)
information about the original value assigned to a variable pre-expansion?


I didn't mean we should preserve the syntactic information, which I
don't either think is a good idea.  Maybe we could always treat
`assoc[@]' as a single element reference.


Do you mean, for example, in assignment statements? I can see it in that
case, and for  `unset', and maybe for `test', because of the semantics of
builtin commands.

But if you take it farther than that, then there is no way to get the
elements of an associative array without introducing yet more syntax. My
guess is there are many, many more cases where ${assoc[@]} is intended to
mean "all the array elements" instead of "the element '@'", and I'm not
interested in that kind of incompatibility.



If you are talking about the new behavior as I expect, why would
«iref='a[\@]'» work?


Why would it not? The rationale for the change is to eliminate the double
expansion of array indices that happens under certain circumstances, not
eliminate expansion of array indices entirely.


I think now I understand the proposed change.  So, indirect expansions
and name references are not planned to be changed because it already
undergoes only a single set of expansions at reference time?


That's the thinking, yes.



Under what circumstances should they not be expanded? Because the
indirect expansions of array subscripts still undergo a single set
of word expansions.


 From the users' point of view, indirect expansions and name references
currently undergo "double expansions" in assigning time and in
reference time; I mean naive users will write as « iref=a[$key] »
instead of « iref='a[$key]' » and run « echo "${!iref}" » to find that
the original right-hand side of the assignment is finally
double-expanded until the reference time.


That's the point of indirect expansions! Even in the most basic use case:

foo=bar
bar=qux
v=$foo
echo ${!v}

nobody should be surprised to see a `double expansion'. I suppose you
can't protect people from their bad assumptions, but for the life of me I
don't see why anyone would not expect

key=one
iref=a[$key]
echo $iref

to display `a[one]', and it's completely straightforward to explain that
${!iref} and ${a[one]} have the same behavior.



But, at the same time, I think it is also a valid discussion that we
want to keep « ${!iref} » and « eval "\${$iref}" » equivalent to each
other.  There is no perfect solution.  If one takes a rational
semantics, I think the current behavior of ${!iref} should be kept.


I think I'd come out in favor of rational semantics.


If one takes the semantics that naive users won't be surprised, I
think the subscripts shouldn't be expanded at the reference time.  In
any way, I would like to see consistency with « ref=a[$key]; unset
"$ref" ».  If one takes the rational semantics, I believe the `unset'
builtin should expand the array subscripts by itself so that « unset
'a[$key]' » works (as with the current default, shopt -u
assoc_expand_once).  If one takes the semantics that is friendly to
naive users, the `unset' builtin can stop its own expansion (shopt -s
assoc_expand_once).


Don't limit yourself to `unset'. The same thing happens with test and
other builtin commands because of the set of word expansions they undergo
before the builtin even sees its arguments.



I actually agree with konsolebox that assoc_expand_once for unset
shouldn't be defaulted.  The option `assoc_expand_once' is incomplete
in the sense that the behavior of `a[@]' and `a[*]' are subtle.  I see
the current default behavior (with `assoc_expand_once' turned off)
more consistent and clean. 


Yeah, maybe. But explaining the requirements for quoting things in multiple
ways is confusing, even to experienced users, and leads to knee-jerk
overreactions like "don't use associative arrays ever" that don't help
anyone. That's one of the motivations for this entire discussion.

 It seems to me that the option

`assoc_expand_once' is just for naive users who fail to properly quote > the 
arguments of `unset' but is inconsistent and ambiguous.


No. The `assoc_expand_once' option wasn't intended for `unset' at all. It
was to solve the double-expansion problem in other array contexts, like
arithmetic, as I said in my original message.


and we can tell users to always write « unset 'a[$key]' », «
iref='a[$key]'; echo "${!iref}" » and « declare -n nref='a[$key]';
echo "$nref" ».


That's what I thought people would do with arrays in the first place. It
obviously hasn't worked out the way I anticipated.

So the current state has people doing

declare -A assoc
key='x]'

assoc[$key]=hello
declare -p assoc

unset assoc["$key"]
declare -p assoc

Re: Changing the way bash expands associative array subscripts

2021-04-15 Thread konsolebox
On Fri, Apr 16, 2021 at 12:10 AM Chet Ramey  wrote:
> > I actually agree with konsolebox that assoc_expand_once for unset
> > shouldn't be defaulted.  The option `assoc_expand_once' is incomplete
> > in the sense that the behavior of `a[@]' and `a[*]' are subtle.  I see
> > the current default behavior (with `assoc_expand_once' turned off)
> > more consistent and clean.
>
> Yeah, maybe. But explaining the requirements for quoting things in multiple
> ways is confusing, even to experienced users, and leads to knee-jerk
> overreactions like "don't use associative arrays ever" that don't help
> anyone. That's one of the motivations for this entire discussion.

I think the best compromise here is to have unset converted to a
special builtin like local so tokens like `a[$key]`, `a["$key"]`, and
`a[@]` are directly parsed but keep the old re-evaluating behavior on
string arguments like `'a[$key]'`.  This will make everyone happy and
avoid scripts from breaking.

This behavior would entirely be independent from assoc_expand_once and
would simply be a subject for compat.  I don't see compat-ing it
necessary though.

-- 
konsolebox