Re: Changing the way bash expands associative array subscripts
2021年4月13日(火) 0:16 Chet Ramey :
> On 4/6/21 12:46 PM, Koichi Murase wrote:
> > Looking at another thread
> > https://lists.gnu.org/archive/html/bug-bash/2021-04/threads.html#00051,
> > I'm now also interested in how we can handle the indirect expansions
> > for 'a[@]' and the namerefs for 'a[@]':
> >
> > $ declare -A a=(['@']=x [1]=y)
> > $
> > $ # indirect expansions
> > $ iref1=a[@]; printf '<%s>' "${!iref1}"; echo
> >
> > $ key=@; iref2=a[$key]; printf '<%s>' "${!iref2}"; echo
> > # <-- unexpected
>
> But these cases are identical. After the assignment statement, iref2 has
> value 'a[@]', just like iref1. I don't see why one would be more unexpected
> than the other, and they're both equivalent to ${a[@]}. Unless that's your
> point?
That's my point. Sorry for my insufficient explanation. I know why
these examples don't work in the *current* Bash. I also think it
wouldn't work with a naive design of *new* assoc_expand_once either.
But I expected some design consideration enabling a[$key] for an
arbitrary key in the indirect expansions and namerefs. (By *new*
assoc_expand_once in the previous and this reply, I meant the planned
change on assoc_expand_once but not the current behavior of
assoc_expand_once.)
> > Currently, we need to write as «iref='a[$key]'; echo "${!iref}"» so
> > that $key is not expanded until the referencing (just like «unset
> > 'a[$key]'»). If the new assoc_expand_once enables the indirect
> > expansions of the form «iref="d[$key]"; echo "${!iref}"» like «unset
> > "a[$key]"» and also if we don't change the current behavior of
> > «iref=a[@]; echo "${!iref}"» (being expanded to all the elements of
> > the array) as Greg's suggestion, we need to work around the case key=@
> > for «iref="d[$key]"; echo "${!iref}"». However, it seems to me that
> > there is no workaround. What string should we assign to `iref' to
> > create a reference to the element associated with key=@ with Greg's
> > suggestion?
>
> Ah, ok. I see. You can assign '\@' to iref to get the behavior you want.
> It's not perfect, but it's possible:
Sorry, I'm a bit confused. May I confirm that we are talking about the
behavior after the planned change on the subscript expansions? If you
are talking about the solution with the current Bash, I know that. If
you are talking about the new behavior as I expect, why would
«iref='a[\@]'» work?
(a) Is it because indirect expansions and name references are not
planned to be modified this time (unlike `unset') and will still
require careful quoting of the key strings after the change?
(b) Or, is it because the associative array subscripts will still be
subject to quote removal after the change even though the command
substitutions in the subscripts will not be evaluated (as shown in
your initial example «unset -v a["$key"]» in
https://lists.gnu.org/archive/html/bug-bash/2021-03/msg00056.html )?
I thought the purpose of the planned change of the associative array
subscripts is to solve the inconsistency between the usages of ((
a[\$key] )), unset 'a[$key]', etc. and the usages of a[$key]=xxx,
${a[$key]} so that we can write (( a[$key] )), unset "a[$key]",
iref=a[$key], declare -n nref=a[$key], etc. With both cases of the
above (a) and (b), we still cannot write «iref=a[$key]» and «declare
-n nref=a[$key]» for arbitrary keys but need to write
«iref=a[$quoted_key]» with a non-trivial conversion of the key to
`quoted_key'. If so, I feel that the change of this time wouldn't
reduce the inconsistency but just changes the boundary where the
inconsistency happens?
--
Koichi
Re: Changing the way bash expands associative array subscripts
2021年4月13日(火) 18:01 Koichi Murase :
> 2021年4月13日(火) 0:16 Chet Ramey :
> > On 4/6/21 12:46 PM, Koichi Murase wrote:
> > > Looking at another thread
> > > https://lists.gnu.org/archive/html/bug-bash/2021-04/threads.html#00051,
> > > I'm now also interested in how we can handle the indirect expansions
> > > for 'a[@]' and the namerefs for 'a[@]':
I actually doubt whether there are real use cases for the behavior
that «iref=a[@]; ${!iref}» or «declare -n nref=a[@]; $nref» expands to
all the elements of *associative* arrays. Maybe it is useful to
expand the indexed arrays in this way because indexed arrays contain a
sequence of values. But associative arrays usually contain a set of
unordered key-value pairs, so I guess there are not so many chances to
expand only the values without specifying keys.
It's just a naive idea and may be imperfect, but how about changing
the behavior of assoc[@] (except for "${assoc[@]}") only for
associative arrays while keeping the current behavior for indexed
arrays? In the following list, the items marked with (*) are the
behavior different from the current one. I listed examples for key=@,
but the same for key=*.
unset 'indexed' # remove the indexed array
unset 'assoc' # remove the associative array
unset 'indexed[@]' # empty the array (*)
unset 'assoc[@]' # remove the element of key=@ (*)
iref=indexed[@]; ${!iref} # all the elements
iref=assoc[@]; ${!iref} # the element of key=@ (*)
declare -n nref=indexed[@] # reference all the elements
declare -n nref=assoc[@] # reference the element of key=@ (*)
test -v 'indexed[@]'
[ -v 'indexed[@]' ]
[[ -v indexed[@] ]] # test if the indexed array is non-empty
test -v 'assoc[@]'
[ -v 'assoc[@]' ]
[[ -v assoc[@] ]] # test if the associative array has the key '@' (*)
printf -v 'indexed[@]' xxx # error
printf -v 'assoc[@]' xxx # assign to the element of key=@ (*)
We don't change the following behavior of associative arrays where the
syntactic information may distinguish a[$key] from a[@]:
indexed[@]=xxx, assoc[@]=xxx # error
${indexed[@]}, ${assoc[@]} # all the values
(( indexed[@] )), (( assoc[@] )) # all the values (error with
multiple non-empty values)
--
Koichi
Re: Changing the way bash expands associative array subscripts
On 4/13/21 5:01 AM, Koichi Murase wrote:
2021年4月13日(火) 0:16 Chet Ramey :
On 4/6/21 12:46 PM, Koichi Murase wrote:
Looking at another thread
https://lists.gnu.org/archive/html/bug-bash/2021-04/threads.html#00051,
I'm now also interested in how we can handle the indirect expansions
for 'a[@]' and the namerefs for 'a[@]':
$ declare -A a=(['@']=x [1]=y)
$
$ # indirect expansions
$ iref1=a[@]; printf '<%s>' "${!iref1}"; echo
$ key=@; iref2=a[$key]; printf '<%s>' "${!iref2}"; echo
# <-- unexpected
But these cases are identical. After the assignment statement, iref2 has
value 'a[@]', just like iref1. I don't see why one would be more unexpected
than the other, and they're both equivalent to ${a[@]}. Unless that's your
point?
That's my point. Sorry for my insufficient explanation. I know why
these examples don't work in the *current* Bash. I also think it
wouldn't work with a naive design of *new* assoc_expand_once either.
But I expected some design consideration enabling a[$key] for an
arbitrary key in the indirect expansions and namerefs.
Why? Why should the shell carry around (and expect the user to remember)
information about the original value assigned to a variable pre-expansion?
And to let that value affect future expansions? That is not a simplifying
change.
What string should we assign to `iref' to
create a reference to the element associated with key=@ with Greg's
suggestion?
Ah, ok. I see. You can assign '\@' to iref to get the behavior you want.
It's not perfect, but it's possible:
Sorry, I'm a bit confused. May I confirm that we are talking about the
behavior after the planned change on the subscript expansions? If you
are talking about the solution with the current Bash, I know that. If
you are talking about the new behavior as I expect, why would
«iref='a[\@]'» work?
Why would it not? The rationale for the change is to eliminate the double
expansion of array indices that happens under certain circumstances, not
eliminate expansion of array indices entirely. If that makes the semantics
more regular and easier to explain, that's a benefit.
(a) Is it because indirect expansions and name references are not
planned to be modified this time (unlike `unset') and will still
require careful quoting of the key strings after the change?
Do you think that array indices should not be expanded at all? Under what
circumstances should they not be expanded? Because the indirect expansions
of array subscripts still undergo a single set of word expansions.
(b) Or, is it because the associative array subscripts will still be
subject to quote removal after the change even though the command
substitutions in the subscripts will not be evaluated (as shown in
your initial example «unset -v a["$key"]» in
https://lists.gnu.org/archive/html/bug-bash/2021-03/msg00056.html )?
These examples are not the same. In each case, the array subscript would
undergo a single round of word expansion. In the `unset' case, the
expansion would happen as part of the standard word expansions performed
before a builtin is invoked, so there would have to be a way to both find
the end of the subscript and avoid expanding it again. In the indirect
expansion case, the subscript is '\@', and after expansion it is `@'.
I thought the purpose of the planned change of the associative array
subscripts is to solve the inconsistency between the usages of ((
a[\$key] )), unset 'a[$key]', etc. and the usages of a[$key]=xxx,
${a[$key]} so that we can write (( a[$key] )), unset "a[$key]",
iref=a[$key], declare -n nref=a[$key], etc.
The purpose of the change is as I wrote above, and as I wrote originally:
"basically force the equivalent of `assoc_expand_once' on all the
time, with additional changes to prevent unwanted double expansion in an
arithmetic expression context."
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Bash 5.1: Completion bug fix
Fix a bug in attempt_shell_completion() since 4.3-beta that allows the
programmable completion code to override a prior decision made by
attempt_shell_completion() that command completion is >not< to be used.
A test case (the simplest of two possible) follows.
Please Reply-To-All.
Thanks.
Marc.
--
In an empty directory, do ...
touch a
Then (where [tab] means the TAB key) ...
echo >& [tab]# correctly gives the above touch'ed filename
echo >& a[tab][tab] # gives a list of commands that start with "a"
echo >& aa[tab][tab] # gives a list of commands that start with "aa"
.. and so on, until the aa... string is long enough to not match any
command, at which point the touch'ed filename is correctly displayed.
--
--- bash-5.1/bashline.c
+++ devel-5.1/bashline.c
@@ -1560,7 +1560,7 @@ attempt_shell_completion (text, start, end)
in_command_position++;
if (check_redir (ti) == 1)
- in_command_position = 0;
+ in_command_position = -1; /* No override */
}
else
{
@@ -1569,7 +1569,7 @@ attempt_shell_completion (text, start, end)
assignments. */
}
- if (in_command_position && invalid_completion (text, ti))
+ if (in_command_position > 0 && invalid_completion (text, ti))
{
rl_attempted_completion_over = 1;
return ((char **)NULL);
@@ -1577,9 +1577,9 @@ attempt_shell_completion (text, start, end)
/* Check that we haven't incorrectly flagged a closed command substitution
as indicating we're in a command position. */
- if (in_command_position && ti >= 0 && rl_line_buffer[ti] == '`' &&
+ if (in_command_position > 0 && ti >= 0 && rl_line_buffer[ti] == '`' &&
*text != '`' && unclosed_pair (rl_line_buffer, end, "`") == 0)
-in_command_position = 0;
+in_command_position = -1; /* No override */
/* Special handling for command substitution. If *TEXT is a backquote,
it can be the start or end of an old-style command substitution, or
@@ -1587,8 +1587,9 @@ attempt_shell_completion (text, start, end)
succeed. Don't bother if readline found a single quote and we are
completing on the substring. */
if (*text == '`' && rl_completion_quote_character != '\'' &&
- (in_command_position || (unclosed_pair (rl_line_buffer, start, "`") &&
-unclosed_pair (rl_line_buffer, end, "`"
+ (in_command_position > 0 ||
+ (unclosed_pair (rl_line_buffer, start, "`") &&
+unclosed_pair (rl_line_buffer, end, "`"
matches = rl_completion_matches (text, command_subst_completion_function);
#if defined (PROGRAMMABLE_COMPLETION)
@@ -1596,7 +1597,8 @@ attempt_shell_completion (text, start, end)
have_progcomps = prog_completion_enabled && (progcomp_size () > 0);
iw_compspec = progcomp_search (INITIALWORD);
if (matches == 0 &&
- (in_command_position == 0 || text[0] == '\0' || (in_command_position &&
iw_compspec)) &&
+ (in_command_position == 0 || text[0] == '\0' ||
+ (in_command_position > 0 && iw_compspec)) &&
current_prompt_string == ps1_prompt)
{
int s, e, s1, e1, os, foundcs;
@@ -1718,7 +1720,7 @@ attempt_shell_completion (text, start, end)
if (matches == 0)
{
dflags = 0;
- if (in_command_position)
+ if (in_command_position > 0)
dflags |= DEFCOMP_CMDPOS;
matches = bash_default_completion (text, start, end, qc, dflags);
}
Re: Changing the way bash expands associative array subscripts
On 2021/04/06 08:52, Greg Wooledge wrote:
In that case, I have no qualms about proposing that unset 'a[@]' and
unset 'a[*]' be changed to remove only the array element whose key is
'@' or '*', respectively, and screw backward compatibility. The current
behavior is pointless and is nothing but a surprise landmine. Anyone
relying on the current behavior will just have to change their script.
So echo ${a[@]} = expansion of all, but
unset a[@] would only delete 1 element w/key '@'
how do I echo 1 element with key '@'
Creating arbitrary definitions of behavior for the similar syntax
seems like a sign of random feature-ism.
Re: Changing the way bash expands associative array subscripts
On Wed, Apr 14, 2021 at 6:37 AM L A Walsh wrote:
> So echo ${a[@]} = expansion of all, but
> unset a[@] would only delete 1 element w/key '@'
> how do I echo 1 element with key '@'
Indeed we can only quote:
a['@']=1234
echo "${a['@']}"
unset "a['@']"
Or have it interpreted as a value of a variable:
key=@
a[$key]=1234
echo "${a[$key]}"
unset 'a[$key]'
The current behavior of unset already does it right because unset is
not a keyword and not a special builtin like local. Quoting and
re-evaluation are necessary so it gets evaluated well like how it is
evaluated during expansion and assignment.
I think I finally can say I'm against unset being defaulted to the
behavior of assoc_expand_once. It's oversimplified, adds
inconsistencies, and will majorly break scripts.
The only change I'd want is have `unset 'a[@]'` only unset the
elements and not the array itself.
--
konsolebox
Re: Changing the way bash expands associative array subscripts
On 4/13/21 6:36 PM, L A Walsh wrote:
So echo ${a[@]} = expansion of all, but
unset a[@] would only delete 1 element w/key '@'
how do I echo 1 element with key '@'
Creating arbitrary definitions of behavior for the similar syntax
seems like a sign of random feature-ism.
The behavior of the `@' and `*' subscripts meaning all elements of the
array long predates associative arrays.
This is one reason for the ability to quote subscripts to defer evaluation.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Changing the way bash expands associative array subscripts
2021年4月14日(水) 0:24 Chet Ramey :
> On 4/13/21 5:01 AM, Koichi Murase wrote:
> > But I expected some design consideration enabling a[$key] for an
> > arbitrary key in the indirect expansions and namerefs.
>
> Why? Why should the shell carry around (and expect the user to remember)
> information about the original value assigned to a variable pre-expansion?
I didn't mean we should preserve the syntactic information, which I
don't either think is a good idea. Maybe we could always treat
`assoc[@]' as a single element reference. Maybe there is another
better idea. Maybe there is no good way, but I thought it's worth
taking some consideration. (But these discussions are all based on my
false assumption that indirect expansions and namerefs would be
changed not to undergo expansions on the reference time.)
> > If you are talking about the new behavior as I expect, why would
> > «iref='a[\@]'» work?
>
> Why would it not? The rationale for the change is to eliminate the double
> expansion of array indices that happens under certain circumstances, not
> eliminate expansion of array indices entirely.
I think now I understand the proposed change. So, indirect expansions
and name references are not planned to be changed because it already
undergoes only a single set of expansions at reference time?
> > (a) Is it because indirect expansions and name references are not
> > planned to be modified this time (unlike `unset') and will still
> > require careful quoting of the key strings after the change?
>
> Do you think that array indices should not be expanded at all?
No, I don't think so. I agree that the expansions should be performed once.
> Under what circumstances should they not be expanded? Because the
> indirect expansions of array subscripts still undergo a single set
> of word expansions.
>From the users' point of view, indirect expansions and name references
currently undergo "double expansions" in assigning time and in
reference time; I mean naive users will write as « iref=a[$key] »
instead of « iref='a[$key]' » and run « echo "${!iref}" » to find that
the original right-hand side of the assignment is finally
double-expanded until the reference time.
But, at the same time, I think it is also a valid discussion that we
want to keep « ${!iref} » and « eval "\${$iref}" » equivalent to each
other. There is no perfect solution. If one takes a rational
semantics, I think the current behavior of ${!iref} should be kept.
If one takes the semantics that naive users won't be surprised, I
think the subscripts shouldn't be expanded at the reference time. In
any way, I would like to see consistency with « ref=a[$key]; unset
"$ref" ». If one takes the rational semantics, I believe the `unset'
builtin should expand the array subscripts by itself so that « unset
'a[$key]' » works (as with the current default, shopt -u
assoc_expand_once). If one takes the semantics that is friendly to
naive users, the `unset' builtin can stop its own expansion (shopt -s
assoc_expand_once).
I actually agree with konsolebox that assoc_expand_once for unset
shouldn't be defaulted. The option `assoc_expand_once' is incomplete
in the sense that the behavior of `a[@]' and `a[*]' are subtle. I see
the current default behavior (with `assoc_expand_once' turned off)
more consistent and clean. It seems to me that the option
`assoc_expand_once' is just for naive users who fail to properly quote
the arguments of `unset' but is inconsistent and ambiguous. I even
think the effect of `assoc_expand_once' on `unset' could be removed,
and we can tell users to always write « unset 'a[$key]' », «
iref='a[$key]'; echo "${!iref}" » and « declare -n nref='a[$key]';
echo "$nref" ».
> > I thought the purpose of the planned change of the associative array
> > subscripts is to solve the inconsistency between the usages of ((
> > a[\$key] )), unset 'a[$key]', etc. and the usages of a[$key]=xxx,
> > ${a[$key]} so that we can write (( a[$key] )), unset "a[$key]",
> > iref=a[$key], declare -n nref=a[$key], etc.
>
> The purpose of the change is as I wrote above, and as I wrote originally:
>
> "basically force the equivalent of `assoc_expand_once' on all the
> time, with additional changes to prevent unwanted double expansion in an
> arithmetic expression context."
I completely agree on the change on the arithmetic expression ((
assoc[$key] )). I don't like the behavior of `unset' under
`assoc_expand_once' because it seems be inconsistent to me, but there
may be different perspective. But, If `assoc_expand_once' is needed
to make the behavior more friendly to naive users, I think we should
also take care of naive users who write « iref='a[$key]'; echo
${!iref} ».
--
Koichi
Re: Changing the way bash expands associative array subscripts
> But, If `assoc_expand_once' is needed
> to make the behavior more friendly to naive users, I think we should
> also take care of naive users who write « iref='a[$key]'; echo
> ${!iref} ».
Sorry, typo of "naive users who write « iref=a[$key]; echo ${!iref} »".
