Please advise on bash programming tactics/strategy
I was wondering if there is any way I can convince netstat to return
its output to bash variables for additional processing.
Pretty simple logic:
Do forever:
Call netstat to obtain RX & TX byte counts for eth0
Print delta {current .. minus previous .. byte counts}
Save current byte counts
Wait for a second or so ..
I initially thought I could just pipe the "netstat -in" command to the
invocation of a bash function.
The function would have taken care of the gory details of parsing &
formatting the output of the netstat command and then stored the current
byte counts where they would be available for the next time the function
is invoked.
The trouble is that I haven't been able to find anything like a "static"
local variable -- ie. variables that are not reinitialized every time
the function is invoked.
Is there such a thing?
Or is there any way the function could save the current byte counts to
global variables?
Naturally, I thought of using an array but, at least as I understand it,
the bash doc seems to indicate that even with arrays, function calls
result in the array's contents being copied to a local copy that only
lives for the the duration of the current invocation of the function.
Or is there an altogether better .. more natural way to do the above in
bash?
Hope the above makes sense .. Will clarify if necessary.
:-)
Please advise on programming tactics/strategy
I was wondering if there is any way I can convince netstat to return
its output to bash variables for processing.
Pretty simple logic:
Do forever:
Call netstat to obtain RX & TX byte counts
Print delta {current .. minus previous .. byte counts}
Save current byte counts
Wait for a second or so ..
I initially thought I could just pipe the "netstat -in" command to the
invocation of a bash function.
The function would have taken care of the gory details of parsing &
formatting the output of the netstat command and would then have stored
the current byte counts where they would be available for the next time
the function is invoked.
The trouble is that I haven't been able to find anything like a "static"
local variable that is not reinitialized every time the function is
invoked.
Is there such a thing?
Or is there any way the function could save the current byte counts to
global variables?
Naturally, I thought of using an array but, at least as I understand it,
the bash doc seems to indicate that even with arrays, function calls
result in the array's contents being copied to a local copy that only
lives for the the duration of the current invocation of the function.
Or is there an altogether better .. more natural way to do the above in
bash?
Hope the above makes sense .. Will clarify if necessary.
:-)
Re: Please advise on bash programming tactics/strategy
On Wed, Dec 12, 2007 at 12:31:47AM EST, Bob Proulx wrote:
> cga2000 wrote:
> > I was wondering if there is any way I can convince netstat to return
> > its output to bash variables for additional processing.
>
> Do you mean like this?
>
> rxcnt=$(netstat -ni | awk '/^eth0/{print$4}')
Precisely. Funny I keep forgetting this very powerful feature of bash.
>
> > Pretty simple logic:
> >
> > Do forever:
> >
> > Call netstat to obtain RX & TX byte counts for eth0 Print delta
> > {current .. minus previous .. byte counts} Save current byte
> > counts Wait for a second or so ..
>
> Such as like this?
>
> #!/bin/sh
[..]
> exit 0
Excellent!
I adapted the above and came up with this:
#!/bin/sh
interface=eth0
get_data()
{
netstat -ni | awk '/^'"$interface"'/{print$4,$8}'
}
init_data()
{
netdata=$(get_data)
prevrxcnt=$(echo $netdata | awk '{print$1}')
prevtxcnt=$(echo $netdata | awk '{print$2}')
}
save_data()
{
netdata=$(get_data)
rxcnt=$(echo $netdata | awk '{print$1}')
txcnt=$(echo $netdata | awk '{print$2}')
diffrxcnt=$(($rxcnt - $prevrxcnt))
difftxcnt=$(($txcnt - $prevtxcnt))
prevrxcnt=$rxcnt
prevtxcnt=$txcnt
}
init_data
while sleep 1;
do
save_data
echo $diffrxcnt $difftxcnt |
awk '{printf "%4.1f k/s %4.1f k/s\n",$1*576/1024,$2*567/1024}'
done
exit 0
I provides exactly the output I need .. although bash must provide a
more elegant (and less expensive) way to split a variable that contains
two fields separated by a space than invoking awk.
;-(
I would have liked to take the packets-to-kilobytes conversion out of
the last awk invocation, but then it looks like bash only does integer
arithmetic.
In any event any suggestions how I could improve the above is very
welcome.
> I would not normally use global variables like this but it was
> specifically what you were asking so I used them. Normally I prefer
> to avoid global variables except for global configuration data. Being
> able to trace the flow through the code as you read it is very
> important to the long term maintainability IMNHO. So generally I
> advise to write the code such as to avoid using globals.
Yes, but I don't see how this could be done in bash if you'd rather
have something modular.
Naturally, one way to get rid of global variables would be to rewrite
the above like so:
#!/bin/sh
interface=eth0
get_data()
{
netstat -ni | awk '/^'"$interface"'/{print$4,$8}'
}
netdata=$(get_data)
prevrxcnt=$(echo $netdata | awk '{print$1}')
prevtxcnt=$(echo $netdata | awk '{print$2}')
while sleep 1; do
netdata=$(get_data)
rxcnt=$(echo $netdata | awk '{print$1}')
txcnt=$(echo $netdata | awk '{print$2}')
diffrxcnt=$(($rxcnt - $prevrxcnt))
difftxcnt=$(($txcnt - $prevtxcnt))
prevrxcnt=$rxcnt
prevtxcnt=$txcnt
echo $diffrxcnt $difftxcnt |
awk '{printf "%4.1f k/s %4.1f k/s\n",$1*576/1024,$2*567/1024}'
done
exit 0
But then, this is beginning to look as cryptic as my initial one-liner.
> > I initially thought I could just pipe the "netstat -in" command to
> > the invocation of a bash function.
>
> Yes?
>
> filter_data() { awk '/^'"$interface"'/{print$4}' ;}
>
> get_data() { netstat -ni | filter_data ;}
>
> > The function would have taken care of the gory details of parsing &
> > formatting the output of the netstat command and then stored the
> > current byte counts where they would be available for the next time
> > the function is invoked.
>
> Oh, stored, global variables, blech, but okay.
>
> grab_data() { prevrxcnt=$(awk '/^'"$interface"'/{print$4}') ;}
>
> > The trouble is that I haven't been able to find anything like a
> > "static" local variable -- ie. variables that are not reinitialized
> > every time the function is invoked.
>
> In C static variables are global to the compilation unit. In bash the
> entire script is the compilation unit. In C static variables are not
> visible across compilation units. In bash there is no individual
> compilation with a separate link step.
>
> Traditionally a naming convention such as applying the name of the
> function to the variable name is used to make sure that name
> collisions are avoided.
>
> > Is there such a thing? Or is there any way the function could save
> > the current byte counts to global variables?
>
> See my examples. Or did I miss the point of the question entirely?
Oh, no .. spot on!
After you'd shown me how I could assign the parsed output of netstat to
variables, and since you kindly provided a sample script that worked out
of the box, there was little left for me to do..
Thanks much for your help.
Re: Please advise on bash programming tactics/strategy
On Thu, Dec 13, 2007 at 12:38:06PM EST, Mike Stroyan wrote:
> On Wed, Dec 12, 2007 at 06:49:25PM -0500, cga2000 wrote:
[..]
> > I provides exactly the output I need .. although bash must provide a
> > more elegant (and less expensive) way to split a variable that contains
> > two fields separated by a space than invoking awk.
>
> You can use
> read rxcnt txcnt <<< $netdata
> to split out the two values.
Very elegant indeed.
And I would never have figured it out from the man page, which states:
"Here Strings
A variant of here documents, the format is:
<< You don't need awk for the splitting of netstat output into words.
:-)
Just kidding ..
.. and sincerely hoping someone knowledgeable would show me the right
way to do this.
Needless to say, the solution provided goes way beyond what I hoped.
> The bash read command can do that.
> This will split the lines into array a and examine each line.
>
> get_data()
> {
> local a
> netstat -ni |
> while read -a a
> do
> if [[ ${a[0]} == $interface ]]
> then
> echo ${a[3]} ${a[4]}
> fi
> done
> }
This is very nice!
So I could write in "main" ..
read netdata[0] netdata[1] <<< $(get_data)
and retrieve my RX/TX packet counts in the netdata array for further
processing!
... not that I need an array in this particular case .. plain
one-dimensional variables would do just as well.
read rxcnt txcnt <<< $(get_data)
> Or this next version will split the lines into variables and examine
> them for matches. The "d" variable is a dummy placeholder for unused
> fields.
nice ..
> The last use of "d" variable gets the entire remainder of each line.
That I figured, when, while testing, I noticed that my last variable was
polluted by all the orphans/leftovers ..
> get_data()
> {
> local int d rxcnt txcnt
> netstat -ni |
> while read int d d rxcnt txcnt d
> do
> if [[ $int == $interface ]]
> then
> echo $rxcnt $txcnt
> fi
> done
> }
... bash feels like such a "jungle" with its countless features (as
compared with C, which has so few it only takes a couple of days to
learn them all..) it's so difficult to figure out a decent way to do
stuff ..
... or in other words for a beginner like myself, jumping from pseudo
code to actual scripts feels like another quantum leap.
> It would be more modular to use an argument to get_data to pass
> the interface instead of using the $interface global variable.
>
> get_data()
> {
> local int d rxcnt txcnt target
> target=$1
> netstat -ni |
> while read int d d rxcnt txcnt d
> do
> if [[ $int == $target ]]
> then
> echo $rxcnt $txcnt
> fi
> done
> }
>
> Then you would invoke it as netdata=$(get_data $interface)
Thank you very much for this mini-tutorial.
Re: Please advise on programming tactics/strategy
On Tue, Dec 18, 2007 at 09:57:44PM EST, Matthew Woehlke wrote:
> cga2000 wrote:
> >I was wondering if there is any way I can convince netstat to return
> >its output to bash variables for processing.
> >
> >Pretty simple logic:
> >
> >Do forever:
> >
> > Call netstat to obtain RX & TX byte counts Print delta {current ..
> > minus previous .. byte counts} Save current byte counts Wait for a
> > second or so ..
> >
> >I initially thought I could just pipe the "netstat -in" command to
> >the invocation of a bash function.
> >
> >The function would have taken care of the gory details of parsing &
> >formatting the output of the netstat command and would then have
> >stored the current byte counts where they would be available for the
> >next time the function is invoked.
> >
> >The trouble is that I haven't been able to find anything like a
> >"static" local variable that is not reinitialized every time the
> >function is invoked.
>
> Pipes run as seperate processes, e.g. in 'somecommand | function',
> 'function' is run as a subshell, so anything done in 'function' does
> not propagate up to the parent.
Never does. That's why I was looking for a way to do it via global
variables, static local variables, or .. more natural where I'm
concerned, a pointer.
> You might want to look at bash's command redirection instead.
?
> Alternatively, something like 'foo=$(netstat | extract)' where
> 'extract' filters the netstat output and condenses it down to
> something more easily parsed in the parent, e.g. 'TX:RX'. Then you can
> do something like 'tx=${foo%:*}; rx=${foo#*:}'.
Pretty much what I found.
Here's what I eventually came up with:
#!/bin/bash
interface=eth0
get_data()
{
netstat -ni | awk '/^'"$interface"'/{print$4,$8}'
}
read rxp txp <<< $(get_data)
while sleep 1; do
read rx tx <<< $(get_data)
echo $((rx - rxp)) $((tx - txp)) |
awk '{printf "%4.1f k/s %4.1f k/s\n",$1*576/1024,$2*567/1024}'
rxp=$rx txp=$tx
done
exit 0
Someone on another mailing list showed me how to use the "<<<" Here
String technique to split the output of function "get_data" into two
variables. Kinda like it better than the classic shell foo%:* etc.
Mainly because the syntax is a lot easier to remember, I guess.. the
downside is that <<< appears to be a bash-only extension and not
portable to regular Bourne sh. Not that this matters much in this
particular case.
All in all, I'm fairly happy with this current version .. Fairly compact
and yet quite readable.
I'm no sure about the logic of it .. how accurate its results are.
Also, I'm definitely not too happy about the overhead .. there are idle
times on my system when this short script seems to eat up more cpu
cycles than everything else .. :-)
Maybe the next step would be to rewrite it in C.
Thanks much for your comments.
