RE: Bug/limitation in 'time'
The man page is clear that it is displaying the results of wait3(). However
it doesn't mention that this means that sub-process startup time is not
accounted for. That's what I feel should be clarified. Otherwise a CPU bound
task may appear to not be CPU bound.
My expectation is that the sum of 'user' and 'sys' time should equal the
elapsed time because the overall task is 100% CPU bound (I've confirmed
this). It is unfortunate that the sub-process startup time is not accounted
for in 'user' or 'sys' time, and I think it would be appropriate to document
this.
Anyway, if the docs can be updated I think that would be great. If not, no
worries.
-Original Message-
From: Pierre Gaston [mailto:pierre.gas...@gmail.com]
Sent: Saturday, March 16, 2013 11:31 PM
To: Bruce Dawson
Cc: bug-bash@gnu.org
Subject: Re: Bug/limitation in 'time'
On Sun, Mar 17, 2013 at 4:33 AM, Bruce Dawson
wrote:
> Thanks -- good to know that there is a fast and POSIX compliant method
> of doing this. I should have included my optimized counting loop --
> it's what we switched to when we realized that $(expr) was a problem. Here
it is now:
>
> # This code performs quite well
> function BashCount() {
> i=$1
> while [ $i -gt 0 ]; do
> (( i-- ))
> done
> echo Just did $1 iterations using bash math } time BashCount
> 15
>
> It's a *lot* faster, of course. BTW, I've poked around in the 'time'
> source code enough to know that it is just displaying the results of
> wait3(), so the misleading CPU consumption information is ultimately a
> wait3()/kernel issue. However showing this in the documentation would be
great.
At least the man page of time on my ubuntu system is pretty much clear about
what it does.
The result is not striking me as impossible though, I can imagine a lot of
real time spent waiting for the scheduler to run expr and then to run bash
again.
I tried a little experiment that I think shows the importance of the
scheduler on the real time result:
I run at the same time this little loop with different "niceness"
i=0;time while ((i++<1));do /bin/echo -n;done
sudo nice -n 19 bash -c 'i=0;time while ((i++<1));do /bin/echo -n;done'
2>&1| sed s/^/19:\ / & sudo nice -n -20 bash -c 'i=0;time while
((i++<1));do /bin/echo -n;done' 2>&1| sed s/^/-20:\ /
I get:
-20: real 0m9.331s
-20: user 0m0.468s
-20: sys0m1.504s
19: real0m14.004s
19: user0m0.532s
19: sys 0m1.660s
so the nicer loop takes twice as much real time indicating that much real
time is spent waiting for the process to run.
Re: Bug/limitation in 'time'
This version is actually slightly faster... And the sys time goes down
very close to zero.
Now, what were you actually looking to test??
My version might be utterly irrelevant.
function BashCount() {
i=$1
while (( i-- )) ; do
true
done
echo Just did $1 iterations using bash math
}
time BashCount 15
On 2013-03-17 07:31, Pierre Gaston wrote:
On Sun, Mar 17, 2013 at 4:33 AM, Bruce Dawson
wrote:
Thanks -- good to know that there is a fast and POSIX compliant method of
doing this. I should have included my optimized counting loop -- it's what
we switched to when we realized that $(expr) was a problem. Here it is now:
# This code performs quite well
function BashCount() {
i=$1
while [ $i -gt 0 ]; do
(( i-- ))
done
echo Just did $1 iterations using bash math
}
time BashCount 15
It's a *lot* faster, of course. BTW, I've poked around in the 'time' source
code enough to know that it is just displaying the results of wait3(), so
the misleading CPU consumption information is ultimately a wait3()/kernel
issue. However showing this in the documentation would be great.
At least the man page of time on my ubuntu system is pretty much clear
about what it does.
The result is not striking me as impossible though, I can imagine a
lot of real time spent waiting for the scheduler to run expr and then
to run bash again.
I tried a little experiment that I think shows the importance of the
scheduler on the real time result:
I run at the same time this little loop with different "niceness"
i=0;time while ((i++<1));do /bin/echo -n;done
sudo nice -n 19 bash -c 'i=0;time while ((i++<1));do /bin/echo
-n;done' 2>&1| sed s/^/19:\ / &
sudo nice -n -20 bash -c 'i=0;time while ((i++<1));do /bin/echo
-n;done' 2>&1| sed s/^/-20:\ /
I get:
-20: real 0m9.331s
-20: user 0m0.468s
-20: sys0m1.504s
19: real0m14.004s
19: user0m0.532s
19: sys 0m1.660s
so the nicer loop takes twice as much real time indicating that much
real time is spent waiting for the process to run.
Re: Bug/limitation in 'time'
On Sun, Mar 17, 2013 at 5:58 PM, Bruce Dawson wrote: > The man page is clear that it is displaying the results of wait3(). However > it doesn't mention that this means that sub-process startup time is not > accounted for. That's what I feel should be clarified. Otherwise a CPU bound > task may appear to not be CPU bound. > > My expectation is that the sum of 'user' and 'sys' time should equal the > elapsed time because the overall task is 100% CPU bound (I've confirmed > this). It is unfortunate that the sub-process startup time is not accounted > for in 'user' or 'sys' time, and I think it would be appropriate to document > this. I'm not sure these are not taken into account, my guess is that the difference between real and sys+user can well be due to the other processes of your system trying to run
Re: Bug/limitation in 'time'
Bruce Dawson wrote: > The man page is clear that it is displaying the results of wait3(). Man page for time? You mean the time section of the man page for bash. If you are looking at the time man page then you are looking at the standalone /usr/bin/time command and not the bash builtin time command. There is also the 'help time' bash builtin help. If you are simply using "time foo" then you will be using the builtin version and should be looking at the bash man page for information about it. To be clear there are two time commands. $ type -a time time is a shell keyword time is /usr/bin/time If you type in 'time' then you get the shell keyword and the bash builtin version. If you type in 'command time' or '/usr/bin/time' then you get the external standalone version. If you try them you will see that the default behavior of each is quite different. Using the "-p" option will force the portable format so that they will produce similar formats. The overlap of external standalone programs and shell builtin functionality is an often confused item. Same thing for 'test' and '[' for example. Bob
Re: Bug/limitation in 'time'
On Sun, Mar 17, 2013 at 9:07 PM, Bob Proulx wrote: > Bruce Dawson wrote: >> The man page is clear that it is displaying the results of wait3(). > > Man page for time? You mean the time section of the man page for > bash. no > If you are looking at the time man page then you are looking at > the standalone /usr/bin/time command and not the bash builtin time > command. The OP in his original email showed that he was aware of that.
Re: Bug/limitation in 'time'
On Sunday, March 17, 2013 01:09:47 AM William Park wrote:
> On Sat, Mar 16, 2013 at 10:15:50PM -0400, Chris F.A. Johnson wrote:
> > On Sun, 17 Mar 2013, Chris Down wrote:
> > > ExprCount() {
> > > for (( i = $1 ; i > 0 ; i-- )); do
> > > :
> > > done
> > > echo "$1 iterations"
> > > }
> >
> >Or, in a POSIX-compliant manner:
> >
> > ExprCount() {
> > i=$1
> > while [ $(( i -= 1 )) -ge 0 ]; do
> > :
> > done
> > echo Just did $1 iterations using expr math
> > }
>
> Are you saying that
>
> for (( ; ; ))
>
> is not POSIX?
Not only is it not POSIX, but it's rather uncommon (bash, zsh, ksh93 only),
which is unfortunate because writing the exact equivalent using ''while'' and
(()) alone is quite ugly. Usually I put my loops within functions, so the
variable initialization part is handled by a typeset that's needed anyway.
Certain other aspects are not so easy to emulate cleanly, for instance,
preventing a redundant increment on the last iteration, and avoiding an
increment on the first iteration. All the workarounds kind of suck.
The very best alternative to for ((;;)) is to try and work in a $(())
somewhere in the loop body and do an increment at the same time.
function f {
typeset n=$1 # localize + initialize
while (( n )); do # end condition
# cmds...
cmd $((n--)) # Hope that there's a convienient spot for $(())
done
}
This construct at least extends the portability to pdksh and probably a few
others. I usually draw the line at shells that lack typeset and inform people
to upgrade to something modern.
--
Dan Douglas
Re: Bug/limitation in 'time'
Bruce Dawson wrote: > Yep, we've changed our loops to use roughly that syntax. Unfortunately a lot > of online resources recommend the $(expr) technique. My understanding is > that using $(expr) is more portable, because i-- is bash specific, but I > don't really know. but $((i-=1)) isn't Bash specific and does the same thing...
RE: Bug/limitation in 'time'
Chris Down pointed that out. My loop now looks like this -- portable (I
believe) and fast:
BashCount() {
for (( i = $1 ; i > 0 ; i-- )); do
:
done
echo Just did $1 iterations using bash math
}
-Original Message-
From: Linda Walsh [mailto:b...@tlinx.org]
Sent: Sunday, March 17, 2013 8:36 PM
To: Bruce Dawson
Cc: 'Chris Down'; bug-bash@gnu.org; b...@packages.debian.org
Subject: Re: Bug/limitation in 'time'
Bruce Dawson wrote:
> Yep, we've changed our loops to use roughly that syntax. Unfortunately
> a lot of online resources recommend the $(expr) technique. My
> understanding is that using $(expr) is more portable, because i-- is
> bash specific, but I don't really know.
but $((i-=1)) isn't Bash specific and does the same thing...
RE: Bug/limitation in 'time'
On Sun, 17 Mar 2013, Bruce Dawson wrote:
Chris Down pointed that out. My loop now looks like this -- portable (I
believe) and fast:
BashCount() {
for (( i = $1 ; i > 0 ; i-- )); do
No, that is not portable.
The only portable use of (( ... )) is $(( ... ))
--
Chris F.A. Johnson,
Re: Bug/limitation in 'time'
-bug-bash, +help-bash
Bruce Dawson wrote:
> Chris Down pointed that out. My loop now looks like this -- portable (I
> believe) and fast:
>
> BashCount() {
> for (( i = $1 ; i > 0 ; i-- )); do
Not actually portable to non ksh-based shells. See
