Re: simple prob?
On Tue, Jun 29, 2021 at 10:23 PM L A Walsh wrote:
> I hope a basic question isn't too offtopic.
> Say I have some number of jobs running:
>
> > jobs|wc -l
> 3
> ---
>
> Would like to pass a varname to njobs to store the answer in, like:
>
> So I can run:
>
> > njobs n
> echo "$n"
> 3
>
>
This a a double question 'how to', and I see no bash bugs here.
The 2 questions are
- how do I pass a variable name as an output argument to a function ('n' in
your 'jobs n' example)
- how to set a variable in a sub command? that is no doable, a sub command
can't return variable to its parent, so you obviously have to do things
diffrently.
A simple 2 liners, solve all this, with no code injection blah
PW$ jobs
[1] Running sleep 111 &
[2] Running sleep 111 &
[3]- Running sleep 111 &
[4]+ Running sleep 111 &
PW$
PW$ function njobs
> { [ "$1" != "n" ] && typeset -n n="$1"
>typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> }
PW$ njobs n ; echo $n
4
# explanations (you may skip here)
#===
[ "$1" != "n" ] && typeset -n n="$1"
This make sure the given output variable name is a valid SHELL identifier,
providing anything not valid in "$1" will break there.
This also enforce that the given $1 output variable name doesn't match our
own local nameref name, if it match we don't do our nameref, and re-use the
upper scope output variable name, that by definition is a valid variable
name if we got that far.
typeset -a t
define a local array that we will fill, being local mean the booking is
done at function return
readarray t <<<"$(jobs)" ;
Fill the array with your command you want to count lines for.
n=${#t[@]}
Fill the output variable
All is safe, all is clean, no 'apparent' temp file, no sub command :)
Shell programing is fun :)
Re: simple prob?
jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
On Fri, Jul 2, 2021, 09:10 Phi Debian wrote:
> On Tue, Jun 29, 2021 at 10:23 PM L A Walsh wrote:
>
> > I hope a basic question isn't too offtopic.
> > Say I have some number of jobs running:
> >
> > > jobs|wc -l
> > 3
> > ---
> >
> > Would like to pass a varname to njobs to store the answer in, like:
> >
> > So I can run:
> >
> > > njobs n
> > echo "$n"
> > 3
> >
> >
> This a a double question 'how to', and I see no bash bugs here.
>
> The 2 questions are
> - how do I pass a variable name as an output argument to a function ('n' in
> your 'jobs n' example)
> - how to set a variable in a sub command? that is no doable, a sub command
> can't return variable to its parent, so you obviously have to do things
> diffrently.
>
> A simple 2 liners, solve all this, with no code injection blah
>
>
> PW$ jobs
> [1] Running sleep 111 &
> [2] Running sleep 111 &
> [3]- Running sleep 111 &
> [4]+ Running sleep 111 &
>
> PW$
>
> PW$ function njobs
> > { [ "$1" != "n" ] && typeset -n n="$1"
> >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > }
>
> PW$ njobs n ; echo $n
> 4
>
> # explanations (you may skip here)
> #===
> [ "$1" != "n" ] && typeset -n n="$1"
> This make sure the given output variable name is a valid SHELL identifier,
> providing anything not valid in "$1" will break there.
> This also enforce that the given $1 output variable name doesn't match our
> own local nameref name, if it match we don't do our nameref, and re-use the
> upper scope output variable name, that by definition is a valid variable
> name if we got that far.
>
> typeset -a t
> define a local array that we will fill, being local mean the booking is
> done at function return
>
> readarray t <<<"$(jobs)" ;
> Fill the array with your command you want to count lines for.
>
> n=${#t[@]}
> Fill the output variable
>
> All is safe, all is clean, no 'apparent' temp file, no sub command :)
>
> Shell programing is fun :)
>
Re: simple prob?
On Fri, Jul 2, 2021 at 9:24 AM Alex fxmbsw7 Ratchev
wrote:
> jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
>
This give
PW$ jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
PW$ echo $jobsn
3644 3645 3646 3647
I guess you meant jobsn=${#jobsn[@]}
^ You missed the '#'
Yet there are some left over
PW$ jobsn=( $( jobs -p ) ) jobsn=${#jobsn[@]}
PW$ echo ${jobsn[@]}
4 3645 3646 3647
So it is not clean, and the OP want a function with a named output arg.
Yet to elaborate on your technic may be this one is a little cleaner
PW$ jobsn=$(printf '%c' $(jobs -p)) jobsn=${#jobsn} ; echo $jobsn
4
Re: simple prob?
good debugging, yea i missed the n # count char, i type stupidly from a
cell foun, ..
and yea leftover arr ekements exceots when u just use the first one as n
i just wanted to show shorter code
i also made somewhen a which_pid_exited resolver etc
peace
On Fri, Jul 2, 2021, 11:10 Phi Debian wrote:
>
>
> On Fri, Jul 2, 2021 at 9:24 AM Alex fxmbsw7 Ratchev
> wrote:
>
>> jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
>>
>
> This give
> PW$ jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
> PW$ echo $jobsn
> 3644 3645 3646 3647
>
> I guess you meant jobsn=${#jobsn[@]}
> ^ You missed the '#'
>
> Yet there are some left over
> PW$ jobsn=( $( jobs -p ) ) jobsn=${#jobsn[@]}
> PW$ echo ${jobsn[@]}
> 4 3645 3646 3647
>
> So it is not clean, and the OP want a function with a named output arg.
>
> Yet to elaborate on your technic may be this one is a little cleaner
>
> PW$ jobsn=$(printf '%c' $(jobs -p)) jobsn=${#jobsn} ; echo $jobsn
> 4
>
Re: simple prob?
On Fri, Jul 2, 2021 at 11:15 AM Alex fxmbsw7 Ratchev wrote: > good debugging, yea i missed the n # count char, i type stupidly from a > cell foun, .. > and yea leftover arr ekements exceots when u just use the first one as n > i just wanted to show shorter code > Yes I like those little one liners when other goes long way with multi fork/exec etc :), just fun challenges :)
Re: simple prob?
On Fri, Jul 02, 2021 at 09:09:34AM +0200, Phi Debian wrote:
> PW$ function njobs
> > { [ "$1" != "n" ] && typeset -n n="$1"
> >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > }
<<<$() is a poor imitation of < <() which is what you really want.
readarray -t t < <(jobs); n=${#t[@]}
Combining <<< and $() only gives an approximation of the same result
(the command substitution strips off all trailing newline characters,
and then the here-string syntax appends one newline), and it's less
efficient, because it involves slurping the entire input into memory,
then writing it out to a temporary file, then opening the temporary
file for input and unlinking it, then reading it back in a second time.
Using < <() avoids the newline alterations and the temporary file.
(Note: in some sufficiently new versions of bash, there may not be a
temporary file in some cases. But it's still not the best solution,
as it still involves storing and reading the whole output multiple times.)
Stepping back a moment, you're using the name reference version of the
"pass an output variable by reference" strategy. This requires bash 4.3,
which is reasonable, and it requires some additional sanity checks which
you did not show.
What's really interesting to me is that you did a *partial* sanity check,
refusing to create a circular name reference if the user passed "n"
as their output variable name. But you forgot to check for "t", which is
another local variable you're using. Also, in Linda's original example,
the output variable was literally named "n", so choosing that as your
name reference and explicitly disallowing it is a really spiteful choice.
Finally, you didn't do any sanity checking of the output variable name
(beyond comparing it to one of your two local variable names), so your
function is susceptible to the same code injection attacks we discussed
earlier in the thread.
unicorn:~$ njobs_ref() { typeset -n n="$1"; n=42; }
unicorn:~$ njobs_ref 'x[0$(date>&2)]'
Fri Jul 2 07:54:49 EDT 2021
As I mentioned a few days ago, all variants of the "pass a variable name
by reference" method are basically equivalent to each other, and all
of them need input sanity checking in order to avoid code injections.
(Some of the variants avoid *some* flavors of code injection, but none of
them avoid this one.)
Re: simple prob?
Ha yes I lost sight of <(jobs -t), I think it is a good improvement to the
challenge :)
Regarding the local t, you right, I missed to precise that any function
that do output parameters via nameref should namespace all their locals and
check the namespace that is indeed a bit combersome, that's why I tend to
stay away from that :) yet it was asked..
Here I typed to fast (challenged by providing the shortest answer :) ) and
throwed away my basics principles :)
Regarding the code injection I am not sure I got it.
If you are sitting at a prompt, why would you trick
unicorn:~$ njobs_ref 'x[0$(date>&2)]'
when you could simply type
unicorn:~$ date
I assume protected script/source (the ones you can't write into), are wise
enough not to run command based on user input, in short I guess no
protected script are doing thing like read in; eval $in :) that is the
simplest code injection :) and then would never let you have a chance to
enter 'x[0$(date>&2)]' at any time.
In all case since doing output parameter require some kind of name spacing
check it would reject input of the form 'x[0$(date>&2)]'
I guess a typical output parameter function should ressemble something like
this
function foo
{ [[ ! "$1" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]] &&
echo "Invalid parameter name '$1'" >&2 &&
return 1
[ "${1:0:4}" = "foo_" ] && echo "Namespace collision '$1' " >&2 &&
return 1
typeset -n foo_out="$1"
foo_out="value"
}
This one reject the bad 'x[0$(date>&2)]' only accept scalar variable names
as output parameter and reject scalar in the foo() namespace.
On Fri, Jul 2, 2021 at 2:01 PM Greg Wooledge wrote:
> On Fri, Jul 02, 2021 at 09:09:34AM +0200, Phi Debian wrote:
> > PW$ function njobs
> > > { [ "$1" != "n" ] && typeset -n n="$1"
> > >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > > }
>
> <<<$() is a poor imitation of < <() which is what you really want.
>
> readarray -t t < <(jobs); n=${#t[@]}
>
> Combining <<< and $() only gives an approximation of the same result
> (the command substitution strips off all trailing newline characters,
> and then the here-string syntax appends one newline), and it's less
> efficient, because it involves slurping the entire input into memory,
> then writing it out to a temporary file, then opening the temporary
> file for input and unlinking it, then reading it back in a second time.
>
> Using < <() avoids the newline alterations and the temporary file.
>
> (Note: in some sufficiently new versions of bash, there may not be a
> temporary file in some cases. But it's still not the best solution,
> as it still involves storing and reading the whole output multiple times.)
>
>
> Stepping back a moment, you're using the name reference version of the
> "pass an output variable by reference" strategy. This requires bash 4.3,
> which is reasonable, and it requires some additional sanity checks which
> you did not show.
>
> What's really interesting to me is that you did a *partial* sanity check,
> refusing to create a circular name reference if the user passed "n"
> as their output variable name. But you forgot to check for "t", which is
> another local variable you're using. Also, in Linda's original example,
> the output variable was literally named "n", so choosing that as your
> name reference and explicitly disallowing it is a really spiteful choice.
>
> Finally, you didn't do any sanity checking of the output variable name
> (beyond comparing it to one of your two local variable names), so your
> function is susceptible to the same code injection attacks we discussed
> earlier in the thread.
>
> unicorn:~$ njobs_ref() { typeset -n n="$1"; n=42; }
> unicorn:~$ njobs_ref 'x[0$(date>&2)]'
> Fri Jul 2 07:54:49 EDT 2021
>
> As I mentioned a few days ago, all variants of the "pass a variable name
> by reference" method are basically equivalent to each other, and all
> of them need input sanity checking in order to avoid code injections.
> (Some of the variants avoid *some* flavors of code injection, but none of
> them avoid this one.)
>
>
Re: simple prob?
On Fri, Jul 02, 2021 at 05:45:23PM +0200, Phi Debian wrote: > Regarding the code injection I am not sure I got it. > > If you are sitting at a prompt, why would you trick > > unicorn:~$ njobs_ref 'x[0$(date>&2)]' > > when you could simply type > unicorn:~$ date > > I assume protected script/source (the ones you can't write into), are wise > enough not to run command based on user input, in short I guess no > protected script are doing thing like read in; eval $in :) that is the > simplest code injection :) and then would never let you have a chance to > enter 'x[0$(date>&2)]' at any time. For functions that you've written exclusively for personal use, it's not an immediate concern. It's more of a thing that you want to be aware of for the future. Where it becomes important is when you're writing scripts for other people to use, or which run as different user accounts, or with different privileges. The classic example of this is a script that's run by a web server in a CGI environment, which accepts query parameters from the end user. If one of those query parameters is used in an unsafe way, it can execute undesired commands on the web server. Of course, there are *many* other places that shell scripts are used, such as booting an operating system, starting various services, and so on. In some of these cases, there is no external input being read, or the external inputs are "trusted" files owned and edited only by the system admin (root). But in other cases, untrusted input may be read. So, there's merit in adopting a proactive strategy to shell script security. Maintaining a slightly paranoid mindset can help you spot potential security holes and possibly avoid disasters.
Re: simple prob?
Ha ok, I got it, I was focused on our interactive session. Surely a script running on a UID behalf would be ill-advised to interpret (in the sense of shell evaluation) an arbitrary shell expression. On Fri, Jul 2, 2021 at 6:06 PM Greg Wooledge wrote: > On Fri, Jul 02, 2021 at 05:45:23PM +0200, Phi Debian wrote: > > Regarding the code injection I am not sure I got it. > > > > If you are sitting at a prompt, why would you trick > > > > unicorn:~$ njobs_ref 'x[0$(date>&2)]' > > > > when you could simply type > > unicorn:~$ date > > > > I assume protected script/source (the ones you can't write into), are > wise > > enough not to run command based on user input, in short I guess no > > protected script are doing thing like read in; eval $in :) that is the > > simplest code injection :) and then would never let you have a chance to > > enter 'x[0$(date>&2)]' at any time. > > For functions that you've written exclusively for personal use, it's > not an immediate concern. It's more of a thing that you want to be > aware of for the future. > > Where it becomes important is when you're writing scripts for other > people to use, or which run as different user accounts, or with > different privileges. > > The classic example of this is a script that's run by a web server in a > CGI environment, which accepts query parameters from the end user. If > one of those query parameters is used in an unsafe way, it can execute > undesired commands on the web server. > > Of course, there are *many* other places that shell scripts are used, > such as booting an operating system, starting various services, and > so on. In some of these cases, there is no external input being read, > or the external inputs are "trusted" files owned and edited only by > the system admin (root). But in other cases, untrusted input may be > read. > > So, there's merit in adopting a proactive strategy to shell script > security. Maintaining a slightly paranoid mindset can help you spot > potential security holes and possibly avoid disasters. > >
