Re: Passing variables by reference conflicts with local
On 100430 08:19, Greg Wooledge wrote:
> On Fri, Apr 30, 2010 at 12:02:58AM +0200, Freddy Vulto wrote:
> > Passing variables by reference however, has a caveat in that
> > local variables override the passing by reference, e.g.:
> >
> > t() {
> > local a
> > eval $1=b
> > }
> > unset a; t a; echo $a # Outputs nothing, expected "b"
>
> Why did you declare 'a' to be local if that's not what you wanted?
> Why did you expect 'b' to be output, if you used a local variable?
t() is an example library function which - as a black box - can contain
many local variables. Local 'a' is just an example.
I would like to call t(), and let it return me a filled variable by
reference, that is without polluting the global environment.
Problem is, all variables work except 'a' because t() happens to declare
this local - which I'm not supposed to know because t() is a black box.
Maybe the example is not clear because 'a' becomes global after all,
and is this a better example:
# Param: $1 variable name to return value to
blackbox() {
local a
eval $1=bar
}
This goes all right:
f() {
local b
blackbox b
echo $b
}
f # Echos "bar" all right
But if I change 'b' to 'a', this conflicts with blackbox() local 'a':
f() {
local a
blackbox a
echo $a
}
f # Outputs nothing unexpected
Freddy Vulto
http://fvue.nl/wiki/Bash:_passing_variables_by_reference
Re: Passing variables by reference conflicts with local
As Chet said, use internal variables that are unlikely to conflict.
# Param: $1 variable name to return value to
blackbox() {
local __bb_a# internal: __, blackbox: bb, a: _a
eval $1=bar
}
f() {
local a
blackbox a
echo $a
}
f# no conflict
I prefer to avoid using eval by using declare, but declare inside a
function makes the variable local. Wouldn't it be nice to have a
global flag (declare -g) like zsh's typeset -g.
d='ls /;true' # aren't you glad it's not "rm -rf" instead of "ls"
f "$d" # oops
So in the mean time, you should sanitize those evals.
On Sat, May 1, 2010 at 3:18 AM, Freddy Vulto wrote:
> On 100430 08:19, Greg Wooledge wrote:
>> On Fri, Apr 30, 2010 at 12:02:58AM +0200, Freddy Vulto wrote:
>> > Passing variables by reference however, has a caveat in that
>> > local variables override the passing by reference, e.g.:
>> >
>> > t() {
>> > local a
>> > eval $1=b
>> > }
>> > unset a; t a; echo $a # Outputs nothing, expected "b"
>>
>> Why did you declare 'a' to be local if that's not what you wanted?
>> Why did you expect 'b' to be output, if you used a local variable?
>
> t() is an example library function which - as a black box - can contain
> many local variables. Local 'a' is just an example.
>
> I would like to call t(), and let it return me a filled variable by
> reference, that is without polluting the global environment.
>
> Problem is, all variables work except 'a' because t() happens to declare
> this local - which I'm not supposed to know because t() is a black box.
>
> Maybe the example is not clear because 'a' becomes global after all,
> and is this a better example:
>
> # Param: $1 variable name to return value to
> blackbox() {
> local a
> eval $1=bar
> }
>
> This goes all right:
>
> f() {
> local b
> blackbox b
> echo $b
> }
> f # Echos "bar" all right
>
> But if I change 'b' to 'a', this conflicts with blackbox() local 'a':
>
> f() {
> local a
> blackbox a
> echo $a
> }
> f # Outputs nothing unexpected
>
>
> Freddy Vulto
> http://fvue.nl/wiki/Bash:_passing_variables_by_reference
>
>
>
Re: Passing variables by reference conflicts with local
On Sat, May 1, 2010 at 12:26 PM, Dennis Williamson
wrote:
> As Chet said, use internal variables that are unlikely to conflict.
>
> # Param: $1 variable name to return value to
> blackbox() {
> local __bb_a # internal: __, blackbox: bb, a: _a
> eval $1=bar
> }
>
> f() {
> local a
> blackbox a
> echo $a
> }
> f # no conflict
>
> I prefer to avoid using eval by using declare, but declare inside a
> function makes the variable local. Wouldn't it be nice to have a
> global flag (declare -g) like zsh's typeset -g.
>
> d='ls /;true' # aren't you glad it's not "rm -rf" instead of "ls"
> f "$d" # oops
You can use workarounds like:
printf -v $a "%s" foobar
read $a <<< "%s"
completion gobbles partial match string
Put the cursor after the word "list" and hit TAB: # find /etc/apt/sources.list.d/*list #|cpio -o|ssh 192.168.44.4 cpio -ivdm Emacssources.list eeepc.list tw.list # find /etc/apt/sources.list.d/ #|cpio -o|ssh 192.168.44.4 cpio -ivdm Notice how we are shown the completions, but then the "*list" has been gobbled up! BASH_VERSION='4.1.5(1)-release'
Re: Passing variables by reference conflicts with local
On 100501 12:40, Pierre Gaston wrote:
> On Sat, May 1, 2010 at 12:26 PM, Dennis Williamson wrote:
> > As Chet said, use internal variables that are unlikely to conflict.
> You can use workarounds like:
> printf -v $a "%s" foobar
> read $a <<< "%s"
The problem with obfucscated internal variables I think is that my
library code becomes unnecessary obfuscated. That's why I'm thinking an
additional call-layer (workaround 2 in my original mail) is an
improvement in that the private library code "_blackbox()" can use
simple, readable variables without restrictions/conflicts, while the
public layer "blackbox()" checks conflicts, which - unlikely as they
might be - I'd like to return to my library users as a known error
instead of silent failing.
Revised example, replacing eval with printf -v, thanks:
# Param: $1 variable name to return value to
# Private library function. Do not call directly. See blackbox()
_blackbox() {
# Just don't declare "local __1"
local a b c d e f g h i j
# ...
# Lots of complicated library code here
# ...
# Return value
printf -v $1 %s b
}
# Param: $1 variable name to return value to
# Public library function
blackbox() {
local __1
_blackbox __1
[[ $1 == __1 ]] && echo "ERROR: variable name conflicts"\
"with local variable: $1"
printf -v $1 %s "$__1"
}
blackbox a; echo $a # Outputs "b" all right
blackbox __1 # Outputs error
d='ls /;true'; blackbox "$d" # No more oops
Freddy Vulto
http://fvue.nl/wiki/Bash:_passing_variables_by_reference
Re: Passing variables by reference conflicts with local
Wouldn't you want your name collision test before your call to the
private function - just to save the wasted call? (And to have its
message redirected to stderr and have a "return 1" or other non-zero
value?) Otherwise, I think your idea is a good one, especially if the
public function can be as simple as that.
Thanks, Pierre, by the way, for the workarounds. I hadn't considered
using indirection that way.
On Sat, May 1, 2010 at 6:21 AM, Freddy Vulto wrote:
> On 100501 12:40, Pierre Gaston wrote:
>> On Sat, May 1, 2010 at 12:26 PM, Dennis Williamson wrote:
>> > As Chet said, use internal variables that are unlikely to conflict.
>> You can use workarounds like:
>> printf -v $a "%s" foobar
>> read $a <<< "%s"
>
> The problem with obfucscated internal variables I think is that my
> library code becomes unnecessary obfuscated. That's why I'm thinking an
> additional call-layer (workaround 2 in my original mail) is an
> improvement in that the private library code "_blackbox()" can use
> simple, readable variables without restrictions/conflicts, while the
> public layer "blackbox()" checks conflicts, which - unlikely as they
> might be - I'd like to return to my library users as a known error
> instead of silent failing.
>
> Revised example, replacing eval with printf -v, thanks:
>
> # Param: $1 variable name to return value to
> # Private library function. Do not call directly. See blackbox()
> _blackbox() {
> # Just don't declare "local __1"
> local a b c d e f g h i j
> # ...
> # Lots of complicated library code here
> # ...
> # Return value
> printf -v $1 %s b
> }
>
> # Param: $1 variable name to return value to
> # Public library function
> blackbox() {
> local __1
> _blackbox __1
> [[ $1 == __1 ]] && echo "ERROR: variable name conflicts"\
> "with local variable: $1"
> printf -v $1 %s "$__1"
> }
>
> blackbox a; echo $a # Outputs "b" all right
> blackbox __1 # Outputs error
> d='ls /;true'; blackbox "$d" # No more oops
>
>
> Freddy Vulto
> http://fvue.nl/wiki/Bash:_passing_variables_by_reference
>
>
>
Weird "/dev/fd/62: Permission denied"
The following $ while read l ; do continue; done < <(ls -l / 2>/dev/null)fails on: BASH_VERSINFO=4 BASH_VERSION=4.1.2(1)-release SHLVL=1 (the machine I'm on doesn't have bashbug).with """-bash: /dev/fd/62: Permission denied"""and not on bash3, bash2. And not on bash4 with a $SHLVL -gt 1. Initially I was trying to narrow down a problem around some code that would work just fine on bash2, but fail on bash4 after almost 1h (~1800 invocations of that while loop) with: """/home/jpa/myscript.sh: redirection error: cannot duplicate fd: Too many open files"""Interestingly, taking the command which is having the redirection issue fails at SHLVL 1 outright. --jpa _ The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5
Re: completion gobbles partial match string
On 4/30/10 2:51 AM, jida...@jidanni.org wrote: > Put the cursor after the word "list" and hit TAB: > # find /etc/apt/sources.list.d/*list #|cpio -o|ssh 192.168.44.4 cpio -ivdm > Emacssources.list eeepc.list tw.list > # find /etc/apt/sources.list.d/ #|cpio -o|ssh 192.168.44.4 cpio -ivdm > > Notice how we are shown the completions, but then the "*list" has been > gobbled up! > BASH_VERSION='4.1.5(1)-release' Yep. Readline relies on matching prefixes: if there are multiple completions, it replaces the word to be completed with the longest common prefix. In this case, there isn't one. The builtin bash completion won't perform replacement when a globbing pattern expands to more than one filename, even when they share a common prefix. I have to assume that you're using programmable completion, and the compspec for `find' doesn't impose the same restriction. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Passing variables by reference conflicts with local
Here's another revised version.
It seems like a lot of bookkeeping (I wish we could transfer to bash?),
but I don't see another way if you want to pass "variables by reference"
in a bash library and prevent both yourself and public users from being
bitten by a conflict with a local variable - other than obfuscating your
library local variables.
I hope we can get it right, so it can be used with a revised version of
the `_get_cword' function of the bash-completion package (= blackbox).
There are still some questions:
I have to give local variables a value (append an equal sign) in order
to get them listed with 'local' in "blackbox()". Is there a bash
builtin which lists all defined local variable names, even those not
having a value yet?
I also want to let 'blackbox' return array variables, which doesn't seem
to be possible with the 'printf/read' workarounds, so I have to use
'eval' and still need to sanitize the array variable names. What's
considered good sanitizing: I'm now checkin for $' \n\t;:$' in
"_blackbox_var_sane()"?
Other changes are:
Called private function _after_ collission test.
Output error messages to stderr and return non-zero value.
Added check to enforce private function "_blackbox()" is ALWAYS called
via public "blackbox()".
I thought about whether private function "_blackbox()" should also have
a check for having local conflicts, but I figured this should be covered
by blackbox unit tests :-)
Added an input parameter and return value for the sake of completeness.
Here's the code:
# Output error of variable by reference conflicting with local
# Params: $1 Variable name causing conflict
# $2 Function in which conflict occurs
_blackbox_var_conflict() {
echo "ERROR: variable name conflicts with local variable:"\
"'$1', in function: $2()" 1>&2
}
# Check whether private function is being called by public interface
# Params: $1 Function name of public interface
# Return: False (1) if error
_blackbox_called_by() {
if [[ ${FUNCNAME[2]} != $1 ]]; then
echo "ERROR: ${FUNCNAME[1]}() MUST be called by $1()" 1>&2
return 1
fi
}
# Check whether variable is sane to be used as eval assignment
# Param: $1 Variable name
# Return: False (1) if not sane
_blackbox_var_sane() {
if [[ ! $1 || ${1//[$' \n\t;:$']} != $1 ]]; then
echo "ERROR: invalid identifier: '$1'"\
"passed to function: ${FUNCNAME[1]}()" 1>&2
return 1
fi
}
# Private library function. Do not call directly. See blackbox()
_blackbox() {
_blackbox_called_by blackbox || return 1
local a b c d e f g h i j arr=( foo "bar cee" )
# ...
# Lots of complicated library code here
# ...
[[ $2 ]] && printf -v $2 %s b# Return value
_blackbox_var_sane "$3" && # Return array value
eval $3=\( \"\${a...@]}\" \) || return 1
return 0 # Return exit status
}
# Param: $1 input argument
# Param: $2 variable name to return value to
# Param: $3 variable name to return array value to
# Public library function
blackbox() {
# NOTE: Give all locals a value so they're listed with 'local'
local __2= __3= __x= __v= IFS=$'\n'
# Check arguments conflicting with locals
for __v in $(local); do
case ${__v%=*} in $2|$3)
_blackbox_var_conflict ${__v%=*} $FUNCNAME; return 1;;
esac
done
_blackbox "$1" __2 __3 # Call private function
__x=$? # Catch exit status
[[ $2 ]] && printf -v $2 %s "$__2" # Return value
_blackbox_var_sane "$3" && # Return array value
eval $3=\( \"\${_...@]}\" \) || return 1
return $__x # Return exit status
}
blackbox i a b; printf $'%s\n' $a "$...@]}" # Outputs vars all right
blackbox i __2 __3 # Outputs error
d='ls /;true'; blackbox i "$d" "$d" # No oops
_blackbox a # Force public access
Freddy Vulto
http://fvue.nl/wiki/Bash:_passing_variables_by_reference
Re: completion gobbles partial match string
> "CR" == Chet Ramey writes: CR> I have to assume that you're using programmable completion, CR> and the compspec for `find' doesn't impose the same restriction. Same happens with cat or ":", not only find. It doesn't happen for # su - nobody And I've isolated the problem to somewhere in this chunk of $INPUTRC: set history-preserve-point on set visible-stats on set show-all-if-ambiguous on set meta-flag on set convert-meta off set input-meta on set output-meta on
Re: Passing variables by reference conflicts with local
In Bash 3.2.0(1)-release, "local" displays local variables that do and
do not have values. In Bash 4.0.33(1)-release and 4.1.0(1)-release
only those with values are printed. Oops.
f () { local var1 var2=abc var3=; local; }; f
Bash 3:
var1=
var2=abc
var3=
Bash 4:
var2=abc
var3=
So it looks like the only way is to use "local var=" instead of "local var".
I can understand the desire to have blackbox_var_sane "$3" called near
the eval, but it seems to me to make sense to have it fail early
rather than waste "# Lots of complicated library code here" first.
This also applies to the placement in the public function.
You might consider sanitizing using something like this: [[ $1 =~
[_[:alpha:]][_[:alnum:]]* ]] then you won't have to worry that you're
forgetting something.
On Sat, May 1, 2010 at 5:19 PM, Freddy Vulto wrote:
> Here's another revised version.
>
> It seems like a lot of bookkeeping (I wish we could transfer to bash?),
> but I don't see another way if you want to pass "variables by reference"
> in a bash library and prevent both yourself and public users from being
> bitten by a conflict with a local variable - other than obfuscating your
> library local variables.
>
> I hope we can get it right, so it can be used with a revised version of
> the `_get_cword' function of the bash-completion package (= blackbox).
>
> There are still some questions:
>
> I have to give local variables a value (append an equal sign) in order
> to get them listed with 'local' in "blackbox()". Is there a bash
> builtin which lists all defined local variable names, even those not
> having a value yet?
>
> I also want to let 'blackbox' return array variables, which doesn't seem
> to be possible with the 'printf/read' workarounds, so I have to use
> 'eval' and still need to sanitize the array variable names. What's
> considered good sanitizing: I'm now checkin for $' \n\t;:$' in
> "_blackbox_var_sane()"?
>
> Other changes are:
>
> Called private function _after_ collission test.
>
> Output error messages to stderr and return non-zero value.
>
> Added check to enforce private function "_blackbox()" is ALWAYS called
> via public "blackbox()".
>
> I thought about whether private function "_blackbox()" should also have
> a check for having local conflicts, but I figured this should be covered
> by blackbox unit tests :-)
>
> Added an input parameter and return value for the sake of completeness.
>
> Here's the code:
>
> # Output error of variable by reference conflicting with local
> # Params: $1 Variable name causing conflict
> # $2 Function in which conflict occurs
> _blackbox_var_conflict() {
> echo "ERROR: variable name conflicts with local variable:"\
> "'$1', in function: $2()" 1>&2
> }
>
> # Check whether private function is being called by public interface
> # Params: $1 Function name of public interface
> # Return: False (1) if error
> _blackbox_called_by() {
> if [[ ${FUNCNAME[2]} != $1 ]]; then
> echo "ERROR: ${FUNCNAME[1]}() MUST be called by $1()" 1>&2
> return 1
> fi
> }
>
> # Check whether variable is sane to be used as eval assignment
> # Param: $1 Variable name
> # Return: False (1) if not sane
> _blackbox_var_sane() {
> if [[ ! $1 || ${1//[$' \n\t;:$']} != $1 ]]; then
> echo "ERROR: invalid identifier: '$1'"\
> "passed to function: ${FUNCNAME[1]}()" 1>&2
> return 1
> fi
> }
>
> # Private library function. Do not call directly. See blackbox()
> _blackbox() {
> _blackbox_called_by blackbox || return 1
> local a b c d e f g h i j arr=( foo "bar cee" )
> # ...
> # Lots of complicated library code here
> # ...
> [[ $2 ]] && printf -v $2 %s b # Return value
> _blackbox_var_sane "$3" && # Return array value
> eval $3=\( \"\${a...@]}\" \) || return 1
> return 0 # Return exit status
> }
>
> # Param: $1 input argument
> # Param: $2 variable name to return value to
> # Param: $3 variable name to return array value to
> # Public library function
> blackbox() {
> # NOTE: Give all locals a value so they're listed with 'local'
> local __2= __3= __x= __v= IFS=$'\n'
> # Check arguments conflicting with locals
> for __v in $(local); do
> case ${__v%=*} in $2|$3)
> _blackbox_var_conflict ${__v%=*} $FUNCNAME; return 1;;
> esac
> done
> _blackbox "$1" __2 __3 # Call private function
> __x=$? # Catch exit status
> [[ $2 ]] && printf -v $2 %s "$__2" # Return value
> _blackbox_var_sane "$3" && # Return array value
> eval $3=\( \"\${_...@]}\" \) || return 1
> return $__x # Return exit status
> }
>
> b
