Re: currently doable? Indirect notation used w/a hash

[Linda Walsh]

Pierre Gaston wrote:


bash4 has associative arrays:

declare -A array
array[foobar]=baz
echo "${array[foobar]}"

---

Right, and bash's namespace is also an associative array -- names & values.

In the main namespace you can use '!' to introduce indirection,
but not in a generalized way.

i.e. a=foo ${!a}=1; echo $foo => '1'

What I found myself wanting was having several 'sets' of
the same parameters of info.  so I could have
multiple hashes or associative arrays,

eth0=([ip]=1.2.3.4/24  [mtu]=1500  [startmode]=auto)
eth1=([ip]=192.168.0.1/24  [mtu]=9000 [startmode]=onboot)

Then wanted to be able to pass which interface to work on, into
a function i.e. pass in 'eth0' or 'eth1'... etc.

In the function, the parameter would be in a var maybe "IF".

Now I want to access the value for IP for the current "IF" (IF holding
eth0 or eth1 or some other InterFace name).

i.e.
IF=eth0;  echo "${!IF[IP]}"

but that doesn't work.  You need secondary dereferencing, like:

eval echo \${$IF[IP]}

I ended up using a sub, since after reading in some config files
99% of my usage is reading, so like:

sub read_n_apply_cfg () {
my file="${1:?}"
my this="cfg_$file"
	eval "hash -g $this"  <- creates per item hash/assoc. array (assigned somewhere 
later)


#accessor
sub this () { eval echo "\${$this[$1]:-""}" ;} <- hide indirecton in sub

usage:
local ip=$(this ip)  #(with '$this' set at top of routine)

--
A bit contorted, but anyone doing shell programming has to be
used to that.  ;-)

Re: currently doable? Indirect notation used w/a hash

[Pierre Gaston]

On Mon, Jun 10, 2013 at 12:43 PM, Linda Walsh  wrote:
>
>
> Pierre Gaston wrote:
>
>> bash4 has associative arrays:
>>
>> declare -A array
>> array[foobar]=baz
>> echo "${array[foobar]}"
>
> ---
>
> Right, and bash's namespace is also an associative array -- names & values.
>
> In the main namespace you can use '!' to introduce indirection,
> but not in a generalized way.
>
> i.e. a=foo ${!a}=1; echo $foo => '1'
>
> What I found myself wanting was having several 'sets' of
> the same parameters of info.  so I could have
> multiple hashes or associative arrays,
>
> eth0=([ip]=1.2.3.4/24  [mtu]=1500  [startmode]=auto)
> eth1=([ip]=192.168.0.1/24  [mtu]=9000 [startmode]=onboot)
>
> Then wanted to be able to pass which interface to work on, into
> a function i.e. pass in 'eth0' or 'eth1'... etc.
>
> In the function, the parameter would be in a var maybe "IF".
>
> Now I want to access the value for IP for the current "IF" (IF holding
> eth0 or eth1 or some other InterFace name).
>
> i.e.
> IF=eth0;  echo "${!IF[IP]}"
>
> but that doesn't work.  You need secondary dereferencing, like:
>
> eval echo \${$IF[IP]}
>
> I ended up using a sub, since after reading in some config files
> 99% of my usage is reading, so like:
>
> sub read_n_apply_cfg () {
> my file="${1:?}"
> my this="cfg_$file"
> eval "hash -g $this"  <- creates per item hash/assoc. array
> (assigned somewhere later)
>
> #accessor
> sub this () { eval echo "\${$this[$1]:-""}" ;} <- hide indirecton in
> sub
>
> usage:
> local ip=$(this ip)  #(with '$this' set at top of routine)
>
> --
> A bit contorted, but anyone doing shell programming has to be
> used to that.  ;-)
>

Well, it's easier to ask for what you really want first.
For your use case, it seems to me that you can use the same trick that
is common for simulating arrays of arrays in awk:

declare -A ip
ip['eth0 ip']=1.2.3.4/24
ip['eth1 mtu']=1200

apply () {
  echo  "${ip["$1 ip"]}"
}
apply eth0

One drawback is that you can list the interfaces in the array with
this method, but you can use an helper array if this is needed.

eg:
declare -A ip
declare -a interfaces
addInterface () {
   interfaces+=( "$1")
   ip["$1 ip"]=$2
}

addInterface eth0 1.2.3.4/24

Re: noclobber vs command not found

[Greg Wooledge]

On Sun, Jun 09, 2013 at 08:39:10AM +0800, jida...@jidanni.org wrote:
> # gdk-pixbuf-query-loaders >| 
> /usr/lib/i386-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders.cache
> bash: gdk-pixbuf-query-loaders: command not found

I know it doesn't directly address your issue, but in cases like these,
I find tab completion to be a primary safeguard.  If I type gdk-
and there are no completion results, then I won't bother typing (or
pasting) the rest of the command.

Many programs assume that you have a full KDE or GNOME suite installed,
but lots of us have no such thing.

Re: currently doable? Indirect notation used w/a hash

[Greg Wooledge]

On Sun, Jun 09, 2013 at 02:02:02PM -0700, Linda Walsh wrote:
> I was wondering if I was missing some syntax somewhere...
> but I wanted to be able to pass the name of a hash in
> and store stuff in it and later retrieve it... but it
> looks like it's only possible with an eval or such?

Passing arrays "by reference" (by name) to a function will be possible
in bash 4.3.  It is not possible in current versions of bash, without
using eval trickery, as you have already noted.  And doing it with
eval is so difficult that it's easier to switch languages entirely.

Re: currently doable? Indirect notation used w/a hash

[Chris F.A. Johnson]

On Mon, 10 Jun 2013, Greg Wooledge wrote:


On Sun, Jun 09, 2013 at 02:02:02PM -0700, Linda Walsh wrote:

I was wondering if I was missing some syntax somewhere...
but I wanted to be able to pass the name of a hash in
and store stuff in it and later retrieve it... but it
looks like it's only possible with an eval or such?


Passing arrays "by reference" (by name) to a function will be possible
in bash 4.3.  It is not possible in current versions of bash, without
using eval trickery, as you have already noted.  And doing it with
eval is so difficult that it's easier to switch languages entirely.


   It is not the least bit difficult with eval:

eval "array=( \"\${$1[@]}\" )"

--
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: currently doable? Indirect notation used w/a hash

[Chris Down]

Enjoy your arbitrary command execution.
On 10 Jun 2013 14:15, "Chris F.A. Johnson"  wrote:

> On Mon, 10 Jun 2013, Greg Wooledge wrote:
>
>  On Sun, Jun 09, 2013 at 02:02:02PM -0700, Linda Walsh wrote:
>>
>>> I was wondering if I was missing some syntax somewhere...
>>> but I wanted to be able to pass the name of a hash in
>>> and store stuff in it and later retrieve it... but it
>>> looks like it's only possible with an eval or such?
>>>
>>
>> Passing arrays "by reference" (by name) to a function will be possible
>> in bash 4.3.  It is not possible in current versions of bash, without
>> using eval trickery, as you have already noted.  And doing it with
>> eval is so difficult that it's easier to switch languages entirely.
>>
>
>It is not the least bit difficult with eval:
>
> eval "array=( \"\${$1[@]}\" )"
>
> --
>Chris F.A. Johnson, 
>Author:
>Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
>Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
>
>

Re: currently doable? Indirect notation used w/a hash

[Chet Ramey]

On 6/10/13 5:43 AM, Linda Walsh wrote:

> What I found myself wanting was having several 'sets' of
> the same parameters of info.  so I could have
> multiple hashes or associative arrays,
> 
> eth0=([ip]=1.2.3.4/24  [mtu]=1500  [startmode]=auto)
> eth1=([ip]=192.168.0.1/24  [mtu]=9000 [startmode]=onboot)
> 
> Then wanted to be able to pass which interface to work on, into
> a function i.e. pass in 'eth0' or 'eth1'... etc.
> 
> In the function, the parameter would be in a var maybe "IF".
> 
> Now I want to access the value for IP for the current "IF" (IF holding
> eth0 or eth1 or some other InterFace name).

This is an excellent statement of the rationale for nameref variables,
which will be implemented in bash-4.3.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: currently doable? Indirect notation used w/a hash

[Greg Wooledge]

> On 10 Jun 2013 14:15, "Chris F.A. Johnson"  wrote:
> >It is not the least bit difficult with eval:
> >
> > eval "array=( \"\${$1[@]}\" )"

On Mon, Jun 10, 2013 at 09:17:23PM +0800, Chris Down wrote:
> Enjoy your arbitrary command execution.

To be fair, Chris Johnson was probably assuming a programming environment
where the function is only callable from within the same script, and
thus has only trustworthy arguments provided by the caller.  Sometimes,
this is the case.  Sometimes, it isn't.

In a more general sense, using eval SAFELY requires undergoing several
tedious steps, all of which were brushed aside as "not the least bit
difficult", which is certainly an exaggeration.

1) Figure out what command you would want to execute if you had the actual
   array name in front of you right now:

   arrayname[i++]="$foo $bar"

2) Escape all the metacharacters in such a way that you can stick it inside
   double quotes without mangling anything:

   arrayname[i++]=\"\$foo \$bar\"

3) Figure out where the arrayname is going to come from, and whether it is
   trustworthy (or "untainted" if you prefer perl's wording).  If it is
   untrustworthy (tainted) then you'll either need to run it through
   printf %q, or check it for invalid characters and abort if found,
   depending on the needs of the program.

   local aref
   printf -v aref %q "$1"

4) Replace the "arrayname" placeholder in the escaped command:

   $aref[i++]=\"\$foo \$bar\"

5) Wrap eval "..." around it.

All together:

   local aref
   printf -v aref %q "$1"
   eval "$aref[i++]=\"\$foo \$bar\""

To call this "not the least bit difficult" strikes me as somewhat
disingenuous.  But I'll concede that it is certainly *possible*, albeit
only with the use of bash's printf %q extension.  It is not possible
to do this safely in strict POSIX shells, other than by searching the
input for invalid characters and aborting.

Re: currently doable? Indirect notation used w/a hash

[Chris F.A. Johnson]

On Mon, 10 Jun 2013, Chris Down wrote:


Enjoy your arbitrary command execution.


   Can you give me an example, using the code I posted, where that would happen?


On 10 Jun 2013 14:15, "Chris F.A. Johnson"  wrote:


On Mon, 10 Jun 2013, Greg Wooledge wrote:

 On Sun, Jun 09, 2013 at 02:02:02PM -0700, Linda Walsh wrote:



I was wondering if I was missing some syntax somewhere...
but I wanted to be able to pass the name of a hash in
and store stuff in it and later retrieve it... but it
looks like it's only possible with an eval or such?



Passing arrays "by reference" (by name) to a function will be possible
in bash 4.3.  It is not possible in current versions of bash, without
using eval trickery, as you have already noted.  And doing it with
eval is so difficult that it's easier to switch languages entirely.



   It is not the least bit difficult with eval:

eval "array=( \"\${$1[@]}\" )"


--
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: currently doable? Indirect notation used w/a hash

[Greg Wooledge]

On Mon, Jun 10, 2013 at 10:23:10AM -0400, Chris F.A. Johnson wrote:
> On Mon, 10 Jun 2013, Chris Down wrote:
> 
> >Enjoy your arbitrary command execution.
> 
>Can you give me an example, using the code I posted, where that would 
>happen?

> >On 10 Jun 2013 14:15, "Chris F.A. Johnson"  wrote:
> >>eval "array=( \"\${$1[@]}\" )"

imadev:~$ foobar() { set -x; eval "array=( \"\${$1[@]}\" )"; }
imadev:~$ foobar 'a}"); date; b=("${q'
+ foobar 'a}"); date; b=("${q'
+ set -x
+ eval 'array=( "${a}"); date; b=("${q[@]}" )'
++ array=("${a}")
++ date
Mon Jun 10 10:31:41 EDT 2013
++ b=("${q[@]}")

A really clever attack wouldn't leave those extra variables lying around,
either.  I stopped at "working" and didn't spend the extra time for
"clever".

Re: currently doable? Indirect notation used w/a hash

[Chris F.A. Johnson]

On Mon, 10 Jun 2013, Greg Wooledge wrote:


On 10 Jun 2013 14:15, "Chris F.A. Johnson"  wrote:

   It is not the least bit difficult with eval:

eval "array=( \"\${$1[@]}\" )"


On Mon, Jun 10, 2013 at 09:17:23PM +0800, Chris Down wrote:

Enjoy your arbitrary command execution.


To be fair, Chris Johnson was probably assuming a programming environment
where the function is only callable from within the same script, and
thus has only trustworthy arguments provided by the caller.  Sometimes,
this is the case.  Sometimes, it isn't.


   The example I gave is always safe; it expands, given 'q' as the argument, to:

array=( "${q[@]}" )

   What's unsafe about that? Why would it need any preprocessing?

--
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: currently doable? Indirect notation used w/a hash

[Pierre Gaston]

On Mon, Jun 10, 2013 at 1:37 PM, Pierre Gaston  wrote:
> On Mon, Jun 10, 2013 at 12:43 PM, Linda Walsh  wrote:
>>
>>
>> Pierre Gaston wrote:
>>
>>> bash4 has associative arrays:
>>>
>>> declare -A array
>>> array[foobar]=baz
>>> echo "${array[foobar]}"
>>
>> ---
>>
>> Right, and bash's namespace is also an associative array -- names & values.
>>
>> In the main namespace you can use '!' to introduce indirection,
>> but not in a generalized way.
>>
>> i.e. a=foo ${!a}=1; echo $foo => '1'
>>
>> What I found myself wanting was having several 'sets' of
>> the same parameters of info.  so I could have
>> multiple hashes or associative arrays,
>>
>> eth0=([ip]=1.2.3.4/24  [mtu]=1500  [startmode]=auto)
>> eth1=([ip]=192.168.0.1/24  [mtu]=9000 [startmode]=onboot)
>>
>> Then wanted to be able to pass which interface to work on, into
>> a function i.e. pass in 'eth0' or 'eth1'... etc.
>>
>> In the function, the parameter would be in a var maybe "IF".
>>
>> Now I want to access the value for IP for the current "IF" (IF holding
>> eth0 or eth1 or some other InterFace name).
>>
>> i.e.
>> IF=eth0;  echo "${!IF[IP]}"
>>
>> but that doesn't work.  You need secondary dereferencing, like:
>>
>> eval echo \${$IF[IP]}
>>
>> I ended up using a sub, since after reading in some config files
>> 99% of my usage is reading, so like:
>>
>> sub read_n_apply_cfg () {
>> my file="${1:?}"
>> my this="cfg_$file"
>> eval "hash -g $this"  <- creates per item hash/assoc. array
>> (assigned somewhere later)
>>
>> #accessor
>> sub this () { eval echo "\${$this[$1]:-""}" ;} <- hide indirecton in
>> sub
>>
>> usage:
>> local ip=$(this ip)  #(with '$this' set at top of routine)
>>
>> --
>> A bit contorted, but anyone doing shell programming has to be
>> used to that.  ;-)
>>
>
> Well, it's easier to ask for what you really want first.

my apologies, I misread "name of a hash" and read pass a hash and
thought you were just after an associative array

Re: currently doable? Indirect notation used w/a hash

[Chris F.A. Johnson]

On Mon, 10 Jun 2013, Greg Wooledge wrote:


On Mon, Jun 10, 2013 at 10:23:10AM -0400, Chris F.A. Johnson wrote:

On Mon, 10 Jun 2013, Chris Down wrote:


Enjoy your arbitrary command execution.


   Can you give me an example, using the code I posted, where that would
   happen?



On 10 Jun 2013 14:15, "Chris F.A. Johnson"  wrote:

eval "array=( \"\${$1[@]}\" )"


imadev:~$ foobar() { set -x; eval "array=( \"\${$1[@]}\" )"; }
imadev:~$ foobar 'a}"); date; b=("${q'
+ foobar 'a}"); date; b=("${q'
+ set -x
+ eval 'array=( "${a}"); date; b=("${q[@]}" )'
++ array=("${a}")
++ date
Mon Jun 10 10:31:41 EDT 2013
++ b=("${q[@]}")

A really clever attack wouldn't leave those extra variables lying around,
either.  I stopped at "working" and didn't spend the extra time for
"clever".


  Point taken, but the only way such a string would be passed as a
  variable name is if it was given as user input -- which would,
  presumably, be sanitized before being used. Programming it literally
  makes as much sense as 'rm -rf /'.

--
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: don't just seek to the next line if the script has been edited

[Mike Frysinger]

On Sunday 09 June 2013 16:59:15 Linda Walsh wrote:
> jida...@jidanni.org wrote:
> > All I know is there I am in emacs seeing things in the output of a
> > running bash script that I want to tweak and get busy tweaking and saving
> > changes before the script finishes, thinking that all this stuff will be
> > effective on the next run of it, when lo and behold now it begins
> > executing random bytes... Yes one can say that these are not C programs
> 
> 
> If they were C programs, and you were to edit binary in place, while it was
> executing, using memory-mapped I/O, the machine might very well start
> executing garbage and your C program would dump core or similar.

pretty sure the linux kernel (and others?) would return ETXTBSY and not even 
allow the write
-mike


signature.asc
Description: This is a digitally signed message part.

Re: don't just seek to the next line if the script has been edited

[Linda Walsh]

Mike Frysinger wrote:
pretty sure the linux kernel (and others?) would return ETXTBSY and not even 
allow the write


I think that's a relatively new innovation -- i.e. since
the ability to setup read-only code segments was implemented,
though FWIW, you are right.

I think it's a text editor thing --
for Example, on windows, I can't write to a running
bash-script (says it is read-only for the duration of execution.
Though I can on linux.

I suppose if bash were to mmap the script file and mark
it as "executable", it would get the same read-only protection as
binaries..??

Re: currently doable? Indirect notation used w/a hash

[Linda Walsh]

  Point taken, but the only way such a string would be passed as a
  variable name is if it was given as user input -- which would,
  presumably, be sanitized before being used. Programming it literally
  makes as much sense as 'rm -rf /'.

---
That still didn't POSIX-Gnu rm from disabling that ability.

Though the one that really causes a pain is them removing
the ability to safely delete all files in a directory with the 'rm' command.

Now, many contortions are necessary.

(i.e.:   "cd testing/output/ && rm --one-file-system  -fr ."
used to safely deleted everything in output -- except the "." --
but it was 'last' (recursive 'rm' has to be depth first), and
the -f would suppress the error you got about not being able to remove ".".

Now they put in a special check to check the starting arguments first,
before doing the depth-first remove and abort any processing for files
in "."   You need  to use 'find' with alot more typing to do something similar.

All done in the name of the new posix spec.   I was told that the reason
why posix has changed from descriptive to prescriptive is because there are
no more companies representing the SYSV flavors of the utils -- so the BSD
reps decided to use POSIX to force all the flavors to become like BSD by
changing POSIX to describe BSD and forcing others to comply or be branded
not-posix compliant.

Of course it's a little more complicated than that description, but that seems
to a large part of what is going on.

Re: currently doable? Indirect notation used w/a hash

[Chris F.A. Johnson]

On Mon, 10 Jun 2013, Linda Walsh wrote:




  Point taken, but the only way such a string would be passed as a
  variable name is if it was given as user input -- which would,
  presumably, be sanitized before being used. Programming it literally
  makes as much sense as 'rm -rf /'.

---
That still didn't POSIX-Gnu rm from disabling that ability.


   Did they? I'm not going to test it :(


Though the one that really causes a pain is them removing
the ability to safely delete all files in a directory with the 'rm' command.


  Since when?


Now, many contortions are necessary.

(i.e.:   "cd testing/output/ && rm --one-file-system  -fr ."
used to safely deleted everything in output -- except the "." --
but it was 'last' (recursive 'rm' has to be depth first), and
the -f would suppress the error you got about not being able to remove ".".


   Contortions???


Now they put in a special check to check the starting arguments first,
before doing the depth-first remove and abort any processing for files
in "."   You need  to use 'find' with alot more typing to do something 
similar.


  What's wrong with:

rm -rf *

--
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: currently doable? Indirect notation used w/a hash

[Linda Walsh]

Chris F.A. Johnson wrote:


  What's wrong with:

rm -rf *


1) it may or may not ignore hidden files depending on shell settings.
2) it crosses into mounted files systems

Re: currently doable? Indirect notation used w/a hash

[Linda Walsh]

Linda Walsh wrote:



Chris F.A. Johnson wrote:


  What's wrong with:

rm -rf *


1) it may or may not ignore hidden files depending on shell settings.
2) it crosses into mounted files systems


Forgot an important one:

  3) Follows symlinks in the directory you are deleting in. (so if
They point outside the tree, no telling what you are deleting.

Re: currently doable? Indirect notation used w/a hash

[Mike Frysinger]

On Monday 10 June 2013 18:20:44 Chris F.A. Johnson wrote:
> On Mon, 10 Jun 2013, Linda Walsh wrote:
> >>   Point taken, but the only way such a string would be passed as a
> >>   variable name is if it was given as user input -- which would,
> >>   presumably, be sanitized before being used. Programming it literally
> >>   makes as much sense as 'rm -rf /'.
> > 
> > ---
> > 
> > That still didn't POSIX-Gnu rm from disabling that ability.
> 
> Did they? I'm not going to test it :(

do it as non-root:
$ rm -rf /
rm: it is dangerous to operate recursively on `/'
rm: use --no-preserve-root to override this failsafe
-mike


signature.asc
Description: This is a digitally signed message part.

Re: noclobber vs command not found

[jidanni]

> "GW" == Greg Wooledge  writes:
GW> I find tab completion to be a primary safeguard
Yes but in this case I am merely obediently copying the whole line with
my mouse and stuffing it into the shell. Thanks anyway though.