Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On both Bash 4.4.12(1)-release and 5.0.0(1)-alpha, a subarray slice like
${a[@]:0} expands to just one word if unquoted (and if IFS doesn't
contain a space):
$ a=(aa bb); IFS=x; printf ":%s:\n" ${a[@]:0}
:aa bb:
I expected it would expand to separate words, as it does without the
slice, and just like $@ does, sliced or not:
$ a=(aa bb); IFS=x; printf ":%s:\n" ${a[@]}
:aa:
:bb:
$ set -- aa bb; IFS=x; printf ":%s:\n" $@
:aa:
:bb:
$ set -- aa bb; IFS=x; printf ":%s:\n" ${@:1}
:aa:
:bb:
It's as if it first joins the picked elements with spaces, and then
splits using IFS, instead of producing multiple words and word-splitting
them individually.
The same thing happens with ${a[*]:0} (but not with ${*:1}):
the array elements get joined with spaces to a single word. If IFS is
empty, unset, or contains a space the result is multiple words as
expected with both [@] and [*].
An expansion like that should in most cases be quoted,
but the current behaviour still seems a bit inconsistent.
--
Ilkka Virta / itvi...@iki.fi
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On Wed, Aug 01, 2018 at 02:43:27PM +0300, Ilkka Virta wrote:
> On both Bash 4.4.12(1)-release and 5.0.0(1)-alpha, a subarray slice like
> ${a[@]:0} expands to just one word if unquoted (and if IFS doesn't
> contain a space):
This just reinforces the point that unquoted $@ or $* (or the array
equivalent) is a bug in the script. It gives unpredictable results.
Don't do it.
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On 08/01/2018 07:12 PM, Greg Wooledge wrote: > This just reinforces the point that unquoted $@ or $* (or the array > equivalent) is a bug in the script. It gives unpredictable results. If the results are unpredictable, isn't that a bug that wants fixing? That would be a very noble goal, should it not be possible? Peter
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On Wed, Aug 01, 2018 at 07:42:30PM +0700, pepa65 wrote: > On 08/01/2018 07:12 PM, Greg Wooledge wrote: > > This just reinforces the point that unquoted $@ or $* (or the array > > equivalent) is a bug in the script. It gives unpredictable results. > > If the results are unpredictable, isn't that a bug that wants fixing? > That would be a very noble goal, should it not be possible? Maybe, but for a script writer, it's already too late. Unless you fast forward 20-30 years into the future when EVERY shell has had these bugs fixed for long enough that you can be reasonably confident that your script will not be executed by one of the older shell versions. Isn't it simpler just not to write buggy scripts in the first place? "$@" expands to a list of words, and "$*" expands to a single string with an optional single-character delimiter inserted between substrings. Those are the only two options you've got. What's the intent of a script that uses unquoted $@ or $*? What did the script's author think was going to happen? If it's one of the two cases above, then the missing quotes are simply a bug in the script. If it's something else, then I'd be interested in hearing it. I don't speak for Chet on any of these issues. He may feel an obligation to fix bash if he perceives the behavior as a bug. That's independent of whether a script writer should actually USE such syntax.
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On 1.8. 15:12, Greg Wooledge wrote:
On Wed, Aug 01, 2018 at 02:43:27PM +0300, Ilkka Virta wrote:
On both Bash 4.4.12(1)-release and 5.0.0(1)-alpha, a subarray slice like
${a[@]:0} expands to just one word if unquoted (and if IFS doesn't
contain a space):
This just reinforces the point that unquoted $@ or $* (or the array
equivalent) is a bug in the script. It gives unpredictable results.
Unquoted $* seems well-defined in Bash's reference manual:
($*) Expands to the positional parameters, starting from one. When the
expansion is not within double quotes, each positional parameter
expands to a separate word.
The reference doesn't really say anything about an unquoted $@, but then
there's the POSIX definition which should be well-defined in this case,
since clearly field-splitting should be performed here.
@: Expands to the positional parameters, starting from one, initially
producing one field for each positional parameter that is set. When
the expansion occurs in a context where field splitting will be
performed, any empty fields may be discarded and each of the non-empty
fields shall be further split as described in Field Splitting.
Now, of course POSIX doesn't say anything about arrays or the
subarray/slice notation, but then Bash's reference mentions that
[@] and [*] are supposed to be analoguous to $@ and $*, and the
description of ${parameter:offset:length} doesn't say that
${array[@]:n:m} would act differently from ${array[@]} let alone
differently from ${@:n:m}.
Instead, the wording of the subarray/slice expansion is similar for both
${@:n:m} and ${array[@]:n:m}:
${parameter:offset:length}
If parameter is ‘@’, the result is length positional parameters
beginning at offset.
If parameter is an indexed array name subscripted by ‘@’ or ‘*’, the
result is the length members of the array beginning with
${parameter[offset]}.
It doesn't say what's done with those parameters or array members, but
if the behaviour is supposed to be different between these two cases,
it's not documented.
--
Ilkka Virta / itvi...@iki.fi
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On Wed, Aug 01, 2018 at 04:06:03PM +0300, Ilkka Virta wrote: > Unquoted $* seems well-defined in Bash's reference manual: > > ($*) Expands to the positional parameters, starting from one. When the > expansion is not within double quotes, each positional parameter > expands to a separate word. But not in reality. https://lists.gnu.org/archive/html/bug-bash/2017-06/msg00283.html https://lists.gnu.org/archive/html/bug-bash/2017-09/msg00058.html https://lists.gnu.org/archive/html/bug-bash/2018-01/msg00035.html Unquoted $* or $@ is just a disaster. Don't do it.
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
Date:Wed, 1 Aug 2018 09:03:20 -0400 From:Greg Wooledge Message-ID: <20180801130320.wwmnw6ixxztih...@eeg.ccf.org> | "$@" expands to a list of words, and "$*" expands to a single string | with an optional single-character delimiter inserted between substrings. | Those are the only two options you've got. No they aren't, the third is the unquoted form (where $@ and $* are the same, so we don't get 4 cases). | What's the intent of a script that uses unquoted $@ or $*? Either (or both) of field splitting of the args, or filename expansion. As in set -- '*.c' '*.h' where echo $@ is quite different to echo "$@" which is (a little) different to echo "$*" (in that the latter separates $1 from $2 by the first char of IFS whereas the previous one uses a space). Of course if the command were not "echo" they would be more different. Please get off your "always quote" hobbyhorse. In many situations that is good advice, but not always. | What did the script's author think was going to happen? [jinx]$ set -- 'a b' c [jinx]$ echo $# 2 [jinx]$ set -- $@ [jinx]$ echo $# 3 That is what I expect, and want, to happen. In this case, field splitting, just as if I had written set -- $1 $2 There's no special case, or magic, involved when $@ or $* is used instead of listing the args separately (except the shell works out how many args need be listed, instead of me...) | If it's one of the two cases above, It isn't. | then the missing quotes are simply a bug in the script. Indeed, it would be.But as it isn't... | If it's something else, then I'd be interested in hearing it. Is your interest satisfied now? | I don't speak for Chet on any of these issues. He may feel an obligation | to fix bash if he perceives the behavior as a bug. That's independent | of whether a script writer should actually USE such syntax. It is indeed, and as the case in question is a private bash (ksh too??) extension, bash (aka Chet) gets to decide what the behaviour should be. But for the simple cases ($@ and $*) posix says what should happen, users are entitled to expect that (and as far as I can tell, with bash, they get that) and also to use it, if that's what they need to happen. kre
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On Wed, Aug 01, 2018 at 09:05:20PM +0700, Robert Elz wrote: > Date:Wed, 1 Aug 2018 09:03:20 -0400 > From:Greg Wooledge > | What's the intent of a script that uses unquoted $@ or $*? > > Either (or both) of field splitting of the args, or filename expansion. > > As in > set -- '*.c' '*.h' OK, let's run with that. wooledg:/tmp/x$ touch "file one.txt" file2.txt wooledg:/tmp/x$ set -- '*.txt' wooledg:/tmp/x$ args $* 2 args: At this point one may think, "Hey, this is actually working. That Greg guy doesn't know what he's talking about!" Now, what if we want to glob for the files that have spaces? Here's the glob that matches them: wooledg:/tmp/x$ args *\ * 1 args: So we should be able to put that glob in the positional parameter list and use your unquoted $* trick, right? wooledg:/tmp/x$ set -- '* *' wooledg:/tmp/x$ args $* 4 args: It's a disaster, as I said. It works ONLY IN THE STUPIDLY SIMPLE CASES and then falls apart as soon as you do anything more. Like putting spaces in the glob. Or setting IFS to the empty string. This is not a "hobby horse". This is real. And people wonder why I'm cranky all the time.
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
Date:Wed, 1 Aug 2018 09:40:12 -0400 From:Greg Wooledge Message-ID: <20180801134012.r6ojiewtx3npa...@eeg.ccf.org> | https://lists.gnu.org/archive/html/bug-bash/2017-06/msg00283.html A bug in dash which as far as I can tell has since been fixed. | https://lists.gnu.org/archive/html/bug-bash/2017-09/msg00058.html A bug in bash, but I don't have a recent enough version to know if it has been fixed or not. | https://lists.gnu.org/archive/html/bug-bash/2018-01/msg00035.html This one is fixed as well (seems to have been a transitory bug as I don't see it in versions of bash I have on different systems than are newer, or older, than the one shown as affected.) | Unquoted $* or $@ is just a disaster. Don't do it. There are bugs in some shells (and just about everything else) in some edge cases (the two above related to IFS='') which it is good to be aware of. The solution is not to run away and hide, but to report the bugs and get them fixed, which is what seems to have happed with dash and the first of those three (and bash and the third) and perhaps for bash and the middle one (it certainly should get fixed if it hasn't already.) Then from the second message ... wooledg:/tmp/x$ set -- '* *' wooledg:/tmp/x$ args $* 4 args: exactly as it should be. Field splitting comes first, so the unquoted $* expands to two words, each containing a single character, '*', and then filename expansion happens. Each '*' expands to the two filenames that exist. This has nothing at all to do with $* the same happens ... (in the same environment you set up, except here I'm using the NetBSD sh rather than bash - I'm sure bash does the same) [jinx]$ v='* *' [jinx]$ args $v 4 args: There are ways to code almost everything that one wants to do, some of them hit bugs in some shells. kre
Re: Unquoted array slice ${a[@]:0} expands to just one word if IFS doesn't have a space
On 8/1/18 10:52 AM, Robert Elz wrote: > Date:Wed, 1 Aug 2018 09:40:12 -0400 > From:Greg Wooledge > Message-ID: <20180801134012.r6ojiewtx3npa...@eeg.ccf.org> > > | https://lists.gnu.org/archive/html/bug-bash/2017-06/msg00283.html > > A bug in dash which as far as I can tell has since been fixed. > > | https://lists.gnu.org/archive/html/bug-bash/2017-09/msg00058.html > > A bug in bash, but I don't have a recent enough version to know > if it has been fixed or not. It was fixed a couple of weeks after being reported. > > | https://lists.gnu.org/archive/html/bug-bash/2018-01/msg00035.html > > This one is fixed as well (seems to have been a transitory > bug as I don't see it in versions of bash I have on different > systems than are newer, or older, than the one shown as > affected.) It's fixed in bash-5.0-alpha. It was fixed a couple of months before this report. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
