test '-v' - associative vs. normal array discrepancy - a bug ?
Good day - Please could anyone explain why the first command below produces no output: $ ( declare -A a=([a]=1); if [ -v a ]; then echo yes; fi ) $ ( declare -a a=([0]=1); if [ -v a ]; then echo yes; fi ) yes $ There does not appear to be any documentation about different behaviour of -v for associative vs. normal arrays: -v varname True if the shell variable varname is set (has been assigned a value). Should that be ammended to : -v varname True if the shell variable varname is not an associative array and is set (has been assigned a value). or is this a bug with -v ? I'm using bash-4.3.30(1) , but it appears also to be an issue with bash-4.3.11(1) . Regards, Jason
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 04:20:51PM +, Jason Vas Dias wrote: > Good day - > Please could anyone explain why the first command below produces no output: > $ ( declare -A a=([a]=1); if [ -v a ]; then echo yes; fi ) > $ ( declare -a a=([0]=1); if [ -v a ]; then echo yes; fi ) > yes In a lot of places, when you use the name of an array without an index, Bash assumes you mean index 0. imadev:~$ unset a; declare -a a=([1]=1); test -v a && echo yes imadev:~$ unset a; declare -A a=([0]=1); test -v a && echo yes yes So it's not indexed vs. associative arrays. It's simply the fact that you used index a instead of index 0, plus the fact that -v a is really -v 'a[0]'. Does this make -v unusable for arrays? Possibly.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
Hi, it is here: test.c: 642t = assoc_reference (assoc_cell (v), "0"); mostly what Greg said. do you think we should return true there for non-empty associative arrays? cheers, pg On Wed, Nov 19, 2014 at 5:20 PM, Jason Vas Dias wrote: > Good day - > Please could anyone explain why the first command below produces no output: > $ ( declare -A a=([a]=1); if [ -v a ]; then echo yes; fi ) > $ ( declare -a a=([0]=1); if [ -v a ]; then echo yes; fi ) > yes > $ > There does not appear to be any documentation about > different behaviour of -v for associative vs. normal arrays: > >-v varname > True if the shell variable varname is set (has been > assigned a value). > > Should that be ammended to : > -v varname > True if the shell variable varname is not an associative > array and is set > (has been assigned a value). > or is this a bug with -v ? > > I'm using bash-4.3.30(1) , but it appears also to be an issue with > bash-4.3.11(1) . > > Regards, > Jason >
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
hi, the following makes -v return true for non-empty associative arrays, what do you think? diff --git a/test.c b/test.c index ab7bec7..8a91d1e 100644 --- a/test.c +++ b/test.c @@ -638,9 +638,13 @@ unary_test (op, arg) } else if (v && invisible_p (v) == 0 && assoc_p (v)) { - char *t; - t = assoc_reference (assoc_cell (v), "0"); - return (t ? TRUE : FALSE); + HASH_TABLE *h=assoc_cell (v); + if (h && h->nentries>0) +{ + return (TRUE); +} else { + return (FALSE); +} } #endif return (v && invisible_p (v) == 0 && var_isset (v) ? TRUE : FALSE); cheers, pg On Wed, Nov 19, 2014 at 5:39 PM, Greg Wooledge wrote: > On Wed, Nov 19, 2014 at 04:20:51PM +, Jason Vas Dias wrote: >> Good day - >> Please could anyone explain why the first command below produces no output: >> $ ( declare -A a=([a]=1); if [ -v a ]; then echo yes; fi ) >> $ ( declare -a a=([0]=1); if [ -v a ]; then echo yes; fi ) >> yes > > In a lot of places, when you use the name of an array without an index, > Bash assumes you mean index 0. > > imadev:~$ unset a; declare -a a=([1]=1); test -v a && echo yes > imadev:~$ unset a; declare -A a=([0]=1); test -v a && echo yes > yes > > So it's not indexed vs. associative arrays. It's simply the fact that you > used index a instead of index 0, plus the fact that -v a is really -v 'a[0]'. > > Does this make -v unusable for arrays? Possibly. >
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On 11/19/14, 11:20 AM, Jason Vas Dias wrote: > Good day - > Please could anyone explain why the first command below produces no output: > $ ( declare -A a=([a]=1); if [ -v a ]; then echo yes; fi ) > $ ( declare -a a=([0]=1); if [ -v a ]; then echo yes; fi ) > yes > $ Dereferencing an array without a subscript is equivalent to referencing element 0 (integer or string, for indexed and associated arrays, respectively). Had you assigned the value to subscript 1 in the indexed array, you wouldn't have gotten `yes' either. If you want to test whether any element in an associative array is set, you can do that: if [ ${#a[@]} -gt 0 ]; then echo yes; fi -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On 11/19/14, 2:14 PM, Piotr Grzybowski wrote: > hi, > > the following makes -v return true for non-empty associative arrays, > what do you think? No. There's a way to test for a non-zero number of elements directly. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 8:25 PM, Chet Ramey wrote: > > No. There's a way to test for a non-zero number of elements directly. > Sure thing, I just had a feeling that Jason wants to use -v instead. I can turn it into -E VAR True if the shell variable VAR is an empty array ;-) cheers, pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
Thanks to all who replied. I would really like -v to do as it documented to do : " -v True if the shell variable varname is set (has been assigned a value) " To me, the fact that -v does not return true if the variable is an array and does not have element 0 - or element '0' in the case of assocs - means it does not behave as documented. Either its behaviour should be changed to return true if an array is non-empty (contains ANY non-empty element) or the documentation should be changed to document '-v's behaviour for both normal and associative arrays. This is the function I was using to replace -v if bash version < 4.3 is detected - it does more than -v in that it actually returns the value of the variable, but it works for any array or array member, and does not care if the array does not have element 0 or member '0' : function is_defined { if (( $# == 0 )); then return 1; fi local v="$1"; if [[ "$v" =~ \[ ]]; then local val; eval 'val="${'$v'}"'; if [ x"$val" = x ]; then return 1; fi echo -n "${val//[\'\"]/}"; return 0; else local val="$(declare -p $v 2>/dev/null)"; if [[ "$val" =~ ^declare[\ \ xaAgri\-]+[\ \ ]*[^\ \ \=]+[\=]([^\=].*)[\ \ ]*$ ]]; then val="${BASH_REMATCH[1]/()/}"; else return 1; fi echo -n "${val//[\'\"]/}"; fi } I thought the intent of -v was to free us of such code . I think there is a need for some built-in or test option that can tell us if a variable has been set or not, regardless if it is an array or if its 0th element or member '0' is set or not. To get this currently, we'd have to test if the variable is an array, if it is an associative or normal array, and have different methods for each case . [ ${#v[@] -gt 0 ] does not work if $v is a normal variable. It would be nice if -v could mean 'has any non-empty member' for both associative and normal arrays - I think Piotr's suggestion to make it do this is a good one. Thanks & Regards, Jason On 11/19/14, Piotr Grzybowski wrote: > On Wed, Nov 19, 2014 at 8:25 PM, Chet Ramey wrote: >> >> No. There's a way to test for a non-zero number of elements directly. >> > > Sure thing, I just had a feeling that Jason wants to use -v instead. > I can turn it into > > -E VAR True if the shell variable VAR is an empty array > > ;-) > > cheers, > pg >
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 07:59:42PM +, Jason Vas Dias wrote: > [ ${#v[@] -gt 0 ] does not work > if $v is a normal variable. Doesn't it? It seems to work for me (once you fix the typo): imadev:~$ unset v; v=''; test "${#v[@]}" -gt 0 && echo yes yes imadev:~$ unset v; v='x'; test "${#v[@]}" -gt 0 && echo yes yes imadev:~$ unset v; test "${#v[@]}" -gt 0 && echo yes imadev:~$ Granted, it's an awkward expression. I'm not arguing against fixing -v.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 8:59 PM, Jason Vas Dias wrote: > [..] > It would be nice if -v could mean 'has any non-empty member' for > both associative and normal arrays - I think Piotr's suggestion > to make it do this is a good one. just as you were writing this, I kind of agreed that using ${#a[*]} is a better idea :) Maybe someone will find some solution, without a 20-mails argument ;-) cheers, pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
People, are we forgetting that this is supposed to work in a function, by passing the name of a variable? i.e. it has to look like this: a= b=x is_defined a -> yes is_defined b -> yes is_defined c -> no The 'length of array' expansion doesn't work when you're given a name. So far, the best I could think of is 'is_defined3': dualbus@hp ~ % ./test2 functiona b c A B C D AA BB CC DD is_defined1 1 0 1 1 0 1 0 1 0 1 1 is_defined2 1 0 1 1 0 1 0 1 0 1 1 is_defined3 0 0 1 0 0 1 0 0 0 1 0 note: 0 : yes, 1 : no Greg, close your eyes, you will *not* like this :) dualbus@hp ~ % cat ./test2 #!/bin/bash # note for is_defined[12]: if you expand an associative array with "${aa[@]}"; # it will expand to its values. is_defined1() { local an="$1[@]" local -a a=("${!an}") [[ ${!1+x} = x ]] || [[ "${#a}" -gt 0 ]] } is_defined2() { local an="$1[@]" local -a a=("${!an}") [[ -v "$1" ]] || [[ "${#a}" -gt 0 ]] } is_defined3() { { declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2 } a= b=x A=() B=(X) D=('') declare -A AA=() declare -A BB=(['x']=x) declare -A DD=(['x']=) declare -A is_defined{1..3} vars=(a b c A B C D AA BB CC DD) for var in "${vars[@]}"; do is_defined1 "$var"; is_defined1["$var"]=$? is_defined2 "$var"; is_defined2["$var"]=$? is_defined3 "$var"; is_defined3["$var"]=$? done printf '%s' function for var in "${vars[@]}"; do printf '\t%s' "$var" done; printf \\n for result in is_defined{1..3}; do printf '%s' "$result" for var in "${vars[@]}"; do element="$result[\"$var\"]" printf '\t%s' "${!element}" done; printf \\n done
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 02:41:13PM -0600, Eduardo A. Bustamante López wrote: > People, are we forgetting that this is supposed to work in a function, by > passing the name of a variable? Passing the name of a variable has NEVER worked in ANY function for ANY purpose in bash. It only works in ksh93, which has proper namerefs. Every time you try to do it in bash, you introduce a security hole, or you ride the eval bus to hell. This is precisely why I tell people not to try to write "libraries" of reusable code in bash. You just can't do it. It's pointless to try.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
now I wonder, maybe this: $#a should return number of elements in array, if a is an array, and strlen($a) for scalars? cheers, pg On Wed, Nov 19, 2014 at 9:47 PM, Greg Wooledge wrote: > On Wed, Nov 19, 2014 at 02:41:13PM -0600, Eduardo A. Bustamante López wrote: >> People, are we forgetting that this is supposed to work in a function, by >> passing the name of a variable? > > Passing the name of a variable has NEVER worked in ANY function for ANY > purpose in bash. It only works in ksh93, which has proper namerefs. > Every time you try to do it in bash, you introduce a security hole, or > you ride the eval bus to hell. > > This is precisely why I tell people not to try to write "libraries" of > reusable code in bash. You just can't do it. It's pointless to try. >
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 09:52:39PM +0100, Piotr Grzybowski wrote: > now I wonder, maybe this: > > $#a > > should return number of elements in array, if a is an array, and > strlen($a) for scalars? That would break existing scripts. $#a already means "argc followed by the letter a". If you meant ${#a}, that would *still* break existing scripts, because ${#a} when a is an array currently means "the length of a[0]".
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 9:56 PM, Greg Wooledge wrote: > > That would break existing scripts. $#a already means "argc followed by > the letter a". > > If you meant ${#a}, that would *still* break existing scripts, because > ${#a} when a is an array currently means "the length of a[0]". yes, right of course. but dont you think that there should be something that returns length of an object, in the above sense? pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 09:59:38PM +0100, Piotr Grzybowski wrote: > On Wed, Nov 19, 2014 at 9:56 PM, Greg Wooledge wrote: > > > > That would break existing scripts. $#a already means "argc followed by > > the letter a". > > > > If you meant ${#a}, that would *still* break existing scripts, because > > ${#a} when a is an array currently means "the length of a[0]". > > yes, right of course. but dont you think that there should be > something that returns length of an object, in the above sense? Honestly, I believe a script writer should know, in every situation, whether a variable is an array or an associative array or a regular variable. This is because a script writer does not write libraries of code that accept unknown variable name references from other people's scripts. This is because you DO NOT WRITE LIBRARIES IN BASH. You can't. This issue is just one of so very many pitfalls. It's an intractable problem. You can't get the length of an "object", as you call it. You can't get its value. You can't assign to it. You can't even determine WHAT IT IS. Nothing works correctly! You know what else you can't do in bash? You can't return a value from a function. Mathematically speaking, bash has no functions. It just has recursive procedure calls with local variables. The only way to get information out of one is to use the file system, or an open file descriptor, or an outer-scope (possibly global) variable. If you go with a variable, the caller AND the function both have to agree on its name, because there's no way to pass that name TO the function. It has to be hard-coded INSIDE the function. So no, I do not agree with you. Every variable in your script should be YOURS, and you should know what it is, because you put it there yourself.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
Greg, I swear to you, to anyone, this was not about writing libraries in bash. I would never dare to write to you about reusable code. From the code perspective, the information about what kind of object the identifier points to is present. And I use words "object" and "identifier" not to make you mad ;-) but in an abstract way, to refer to those "things". It is possible, to add either a builtin, or another test argument, or some wicked $@a syntax to the dereference, that will return number of elements if given "object" is currently an array or number of "characters" if it currently is a "scalar". I too consider variables in scripts mine, and I would never try to convince you to think otherwise. Love the spirit in your mails though :) I do understand your point. Dont you understand the need to check in a simple (I know, nothing works, nothing is simple ;-)) way that given variable has value? no matter what it is? cheers, pg On Wed, Nov 19, 2014 at 10:14 PM, Greg Wooledge wrote: > On Wed, Nov 19, 2014 at 09:59:38PM +0100, Piotr Grzybowski wrote: >> On Wed, Nov 19, 2014 at 9:56 PM, Greg Wooledge wrote: > > Honestly, I believe a script writer should know, in every situation, > whether a variable is an array or an associative array or a regular > variable. > > This is because a script writer does not write libraries of code that > accept unknown variable name references from other people's scripts. > > This is because you DO NOT WRITE LIBRARIES IN BASH. You can't. This > issue is just one of so very many pitfalls. It's an intractable problem. > > You can't get the length of an "object", as you call it. You can't get > its value. You can't assign to it. You can't even determine WHAT IT IS. > Nothing works correctly! > > You know what else you can't do in bash? You can't return a value from > a function. Mathematically speaking, bash has no functions. It just > has recursive procedure calls with local variables. The only way to > get information out of one is to use the file system, or an open file > descriptor, or an outer-scope (possibly global) variable. If you go > with a variable, the caller AND the function both have to agree on its > name, because there's no way to pass that name TO the function. It has > to be hard-coded INSIDE the function. > > So no, I do not agree with you. Every variable in your script should be > YOURS, and you should know what it is, because you put it there yourself.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On 11/19/14, 2:59 PM, Jason Vas Dias wrote: > Thanks to all who replied. > > I would really like -v to do as it documented to do : > " -v >True if the shell variable varname is set (has been assigned a value) > " > To me, the fact that -v does not return true if the variable is an array > and does not have element 0 - or element '0' in the case of assocs - > means it does not behave as documented. > Either its behaviour should be changed to return true if an array is > non-empty (contains ANY non-empty element) or the documentation should > be changed to document '-v's behaviour for both normal and associative > arrays. The problem is that you can only put this stuff in so many places. The section on arrays says: "Referencing an array variable without a subscript is equivalent to referencing element zero." If you want to use -v to check whether or not an arbitray variable has a value, use var[@]. Scalars accept the @ subscript, and it works for unset variables as well. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 10:32:52PM +0100, Piotr Grzybowski wrote: > I do understand your point. Dont you understand the need to check in > a simple (I know, nothing works, nothing is simple ;-)) way that given > variable has value? no matter what it is? No, I don't. If I'm writing in C, and I want to know whether one of my variables is "empty", I have to know what type of variable it is. The code to check whether a string (char x[10]) is "empty" is very different from the code to check whether an array of integers (int x[10]) is "empty". Hell, the meaning of "empty" would have to be defined by the application. Would it mean every element is 0? Or something else? And what if x is a float, or a double? Or a pointer? Or a struct? The same thing happens in bash. It's simply not the kind of language where you have one syntax that works for multiple situations.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 10:32:52PM +0100, Piotr Grzybowski wrote: > I do understand your point. Dont you understand the need to check in > a simple (I know, nothing works, nothing is simple ;-)) way that given > variable has value? no matter what it is? there: is_defined3() { { declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2 } works for strings, arrays and associative arrays. No need for hacks like eval, ${!var}, or a need to add a new operator to bash's test.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 10:48 PM, Greg Wooledge wrote: > > No, I don't. I completely respect that. > > If I'm writing in C, and I want to know whether one of my variables is > "empty", I have to know what type of variable it is. The code to check > whether a string (char x[10]) is "empty" is very different from the code > to check whether an array of integers (int x[10]) is "empty". Hell, > the meaning of "empty" would have to be defined by the application. > Would it mean every element is 0? Or something else? And what if x is > a float, or a double? Or a pointer? Or a struct? exactly. why is that? you touched the most important thing: types. thats why in script languages, where you do not define variables with types as such, you have the meaning of "empty", a="" makes a empty, declare -A b makes b empty. > > The same thing happens in bash. It's simply not the kind of language > where you have one syntax that works for multiple situations. well, I could argue:) but I wont. I see your point. cheers, pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On 11/19/14, 5:03 PM, Piotr Grzybowski wrote: > exactly. why is that? you touched the most important thing: types. > thats why in script languages, where you do not define variables with > types as such, you have the meaning of "empty", a="" makes a empty, > declare -A b makes b empty. You're using `empty' imprecisely. In your example, a is set, while b is not. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 10:51 PM, Eduardo A. Bustamante López wrote: > On Wed, Nov 19, 2014 at 10:32:52PM +0100, Piotr Grzybowski wrote: >> I do understand your point. Dont you understand the need to check in >> a simple (I know, nothing works, nothing is simple ;-)) way that given >> variable has value? no matter what it is? > > there: > > is_defined3() { > { declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2 > } I dont know, but looks like a reusable code to me. cheers, pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 11:05 PM, Chet Ramey wrote: > > You're using `empty' imprecisely. In your example, a is set, while b > is not. one could say that if it is not set, it is probably empty? yes, but it was on purpose, not to enter the distinction between hash table with no keys vs. uninitialized hash. cheers, pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 11:05:51PM +0100, Piotr Grzybowski wrote: > On Wed, Nov 19, 2014 at 10:51 PM, Eduardo A. Bustamante López > wrote: > > is_defined3() { > > { declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2 > > } > > I dont know, but looks like a reusable code to me. No, it fails if the argument is both a function AND a variable. imadev:~$ is_defined3() { { declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2; } imadev:~$ declare -p foo bash: declare: foo: not found imadev:~$ is_defined3 foo ; echo $? 1 So far, so good. But imadev:~$ foo() { echo foo; } imadev:~$ foo=bar imadev:~$ is_defined3 foo ; echo $? 1
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 11:16 PM, Greg Wooledge wrote: > So far, so good. But > > imadev:~$ foo() { echo foo; } > imadev:~$ foo=bar > imadev:~$ is_defined3 foo ; echo $? > 1 Greg, does that mean, that "nothing works correctly"? pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 11:22:48PM +0100, Piotr Grzybowski wrote: > On Wed, Nov 19, 2014 at 11:16 PM, Greg Wooledge wrote: > > So far, so good. But > > > > imadev:~$ foo() { echo foo; } > > imadev:~$ foo=bar > > imadev:~$ is_defined3 foo ; echo $? > > 1 > > Greg, does that mean, that "nothing works correctly"? It demonstrates how easy it is to do things wrong in bash. Eduardo is an experienced bash programmer, and even after several tries, he still got it wrong.
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
On Wed, Nov 19, 2014 at 11:30 PM, Greg Wooledge wrote: > > It demonstrates how easy it is to do things wrong in bash. Eduardo is > an experienced bash programmer, and even after several tries, he still > got it wrong. actually he said that it works for arrays, and variables, so basically you are using his library in an unsupported manner. pg
Re: test '-v' - associative vs. normal array discrepancy - a bug ?
> So far, so good. But > > imadev:~$ foo() { echo foo; } > imadev:~$ foo=bar > imadev:~$ is_defined3 foo ; echo $? > 1 Ouch! Last try: is_defined4() { declare -p -- "$1" 2>/dev/null >&2 } But I agree with you, the shell is tricky.