Protect Loop Execution with Traps

[Roger]

I've always had a problem with Bash script (eg. for/while) loops creating havoc 
upon a ctrl-c keypress.

One good idea, is not to put statements (eg. rm) within the loop that could 
possibly create problems upon partial execution.

Another idea, addressing the monkey within the room, should a trap statement 
always be used within for/while loops?

Or would a trap even help at all?

I've seen some examples on the Internet where a subshell, eg. "( statements )", 
is included within the loop, however this seems to cause more problems as a 
value of a counter variable (i=i+1) cannot be easily incremented due to 
subshells not inherienting previously defined variables of the parent shell.

Example Script: 


#!/bin/bash

declare -i i=0

while (( $i >= 0 )); do
# Protect loop
trap "printf 'Caught SIGINT\n'; exit" SIGINT

# Statements here
printf "Pres CTRL+C to stop... %s\n" ${i}

sleep 1

let "i++"

done




-- 
Roger
http://rogerx.sdf.org/


signature.asc
Description: Digital signature

Re: Protect Loop Execution with Traps

[Greg Wooledge]

On Mon, Jan 27, 2020 at 09:03:22PM -0500, Roger wrote:
> I've always had a problem with Bash script (eg. for/while) loops creating 
> havoc 
> upon a ctrl-c keypress.

What's *in* the loop?  It matters.

Consider the following two scripts:

==
#!/bin/bash
while true; do
  echo "Zzz"
  sleep 1
done
==

==
#!/bin/bash
while true; do
  # Assuming Linux
  ping -c 1 8.8.8.8
done
==

If you run these, and try to kill them with Ctrl-C, you may find that
the first one behaves perfectly (stops when you ask), and the second
one does not.  It may take several tries to kill the second one.  You
might have better luck suspending it with Ctrl-Z first, then killing
the shell only, then fg'ing the job.

See  for details.

In a nutshell, ping(8) is a misbehaving program.  It sets a SIGINT
trap (so that it can report statistics when you interrupt it), but
it doesn't exit as a result of the signal.  The shell sees that the
program didn't exit due to SIGINT, and decides that it (the shell)
should not exit either.

Whatever program you're using inside your loop may have the same
issues.  It's extremely common.

Re: Protect Loop Execution with Traps

[Roger]

>If you run these, and try to kill them with Ctrl-C, you may find that
>the first one behaves perfectly (stops when you ask), and the second
>one does not.  It may take several tries to kill the second one.  You
>might have better luck suspending it with Ctrl-Z first, then killing
>the shell only, then fg'ing the job.
>
>See  for details.
>
>In a nutshell, ping(8) is a misbehaving program.  It sets a SIGINT
>trap (so that it can report statistics when you interrupt it), but
>it doesn't exit as a result of the signal.  The shell sees that the
>program didn't exit due to SIGINT, and decides that it (the shell)
>should not exit either.

As I slept on this, I realized the likeliness some programs are also trapping 
CTRL-C as you just explained.

The programs I'm using within a loop were ffmpeg && mv (rename) after verifying 
ffmpeg created a file >0 bytes.  As I speculated, I was likely doing too much 
within one loop and should likely isolate each task to it's own specific loop, 
mitigating such behaviour.  (The check should have also likely checked for a 
ffmpeg created file >= to the original file before renaming/moving.)

In other words, it's safer to

1) Have ffmpeg process all files within it's own loop, then

2) Perform a second loop renaming files using mv.

But as I re-think think this, should be good Bash scripting practice to 
integrate a trap within each loop? 


>Whatever program you're using inside your loop may have the same
>issues.  It's extremely common.

Another thought occurred to me last night, no wonder they call it Bash. When 
you execute a Bash loop, you'll likely be sure to find yourself pounding 
"CTRL-C" on your keyboard!

As far as the design/intigration of the trap within a loop, I'm guessing my 
initial post contained the most simplistic, readable, proper form of a trap for 
creating a safe quick exiting loop?


--
Roger
http://rogerx.sdf.org/


signature.asc
Description: Digital signature

Preventing Bash Variable Confusion

[Roger]

I've used Bash for quite some time now and have heard lots about how variables 
should be named or styled.

1) Bash internal reserved words cannot be used a variables.  (OK. All of us who 
have programmed code get this and of course abide whole heartedly, else we fail 
quickly!)

2) Operating System (Bash or other) reserved variables are all defined as 
capitol letters, and all capitol letter variables should be avoided within Bash 
scripts.  (I've heard of this, however, I enjoy defining my variables in all 
capitol letters due to the increased readability of the script ... or in 
essence, all capitol letters readily distinguishes variables from other Bash 
words and operators.)

Some say variables should be prefixed (or suffixed) to further distinguish Bash 
variables from possible collisions with operating system Bash or other related 
variables.

I've thought about using "_VARIABLE" and have seen similar, but also requires 
an extra odd stressed finger combination prior to typing all capitol letters.  
Typing all capitol letters can be stressful, but the task improves readability 
in my point of view that the time and effort are well worth the effort, until I 
get to the additional underscore.

When and if I program in C, I tend to use all lower case style, my_variable.

Anybody have any further insight concerning variable naming styles aside from 
what's already written within the documentation?

I could do something like MY_VARIABLE, but then prefixing with "MY_" eats up 
three more chars I could have used for describing my variable better.  
Shrugs...

-- 
Roger
http://rogerx.sdf.org/


signature.asc
Description: Digital signature

Re: Protect Loop Execution with Traps

[Greg Wooledge]

On Tue, Jan 28, 2020 at 03:49:32PM -0500, Roger wrote:
> As I slept on this, I realized the likeliness some programs are also trapping 
> CTRL-C as you just explained.
> 
> The programs I'm using within a loop were ffmpeg && mv (rename) after 
> verifying 
> ffmpeg created a file >0 bytes.

I'm not familiar with ffmpeg in detail.  An ffmpeg mailing list might
be able to offer more focused advice.

> But as I re-think think this, should be good Bash scripting practice to 
> integrate a trap within each loop? 

That doesn't sound right.  Let me go back to my previous example using
ping.

Let's say that for reasons outside the scope of bug-bash, you're forced
to perform a ping in a loop.  And that you will almost certainly want
to abandon the loop prematurely using Ctrl-C.  And that you aren't allowed
to fix the misbehaving ping command at the source code level.

Here's a simple fix, that involves setting up ONE trap within the
shell script, to override the shell's default SIGINT handling heuristic.


#!/bin/bash
trap exit INT
while true; do
  ping -c 3 8.8.8.8
done


There.  Now, when I hit Ctrl-C, the whole script exits, not just one
instance of ping.

(Switching from -c 1 to -c 3 made it a *lot* less spammy, and also much
harder to kill using the "press Ctrl-C twice really fast" approach.
The INT trap works around it beautifully for me, though.)

Whether this same stuff applies to ffmpeg, I have no idea.  I also
don't know why you're Ctrl-C'ing out of an ffmpeg loop often enough
that this has become a concern.

But, the "simple fix" that I used here has the same issue that ping
itself has -- we're catching SIGINT and handling it and exiting,
instead of letting SIGINT kill us.  If something runs *us* in a loop,
it'll have the same problem that *we* had when we tried to run ping
in a loop.

So, in the interest of not causing problems for other programs, here's
the more correct fix:


#!/bin/bash
trap 'trap INT; kill -INT $$' INT
while true; do
  ping -c 3 8.8.8.8
done

Re: Protect Loop Execution with Traps

[Roger]

>Here's a simple fix, that involves setting up ONE trap within the
>shell script, to override the shell's default SIGINT handling heuristic.
>
>
>#!/bin/bash
>trap exit INT
>while true; do
>  ping -c 3 8.8.8.8
>done
>
>
>There.  Now, when I hit Ctrl-C, the whole script exits, not just one
>instance of ping.
>
>(Switching from -c 1 to -c 3 made it a *lot* less spammy, and also much
>harder to kill using the "press Ctrl-C twice really fast" approach.
>The INT trap works around it beautifully for me, though.)
>
>Whether this same stuff applies to ffmpeg, I have no idea.  I also
>don't know why you're Ctrl-C'ing out of an ffmpeg loop often enough
>that this has become a concern.
>
>But, the "simple fix" that I used here has the same issue that ping
>itself has -- we're catching SIGINT and handling it and exiting,
>instead of letting SIGINT kill us.  If something runs *us* in a loop,
>it'll have the same problem that *we* had when we tried to run ping
>in a loop.

Yes. That's not desirable then, as sweet and short as it is.

>So, in the interest of not causing problems for other programs, here's
>the more correct fix:
>
>
>#!/bin/bash
>trap 'trap INT; kill -INT $$' INT
>while true; do
>  ping -c 3 8.8.8.8
>done

Wow, " trap 'trap INT; kill -INT $$' INT "  not easily readable for me.

A trap calling kill inside of a trap.

I'm thinking, put "trap INT; kill -INT $$" inside of it's own function, named 
something like exit_recurs_loop, so the first line reads "trap exit_recurs_loop 
INT"

I'll put these additional trap ideas into practice here whenever I'm doing 
loops and see what happens.

This trap stuff has me pulling out my hair, as I likely do not know the details 
of how programs use signals and how Bash interacts with such activity.

Thanks for your time!


--
Roger
http://rogerx.sdf.org/

Re: Preventing Bash Variable Confusion

[Robert Elz]

Date:Tue, 28 Jan 2020 16:02:25 -0500
From:Roger 
Message-ID:  <20200128210225.GC12574@localhost4.local>

  | 1) Bash internal reserved words cannot be used a variables.

No such rule.   Vars are always assigned using xxx= (no reserved words
contain an '=') and accessed using $...  (in one form or another) or
occasionally (assign or access, or attribute modification) as an arg
to a command.   Reserved words only occur as command names, and can be
used for any other purpose anywhere else.   (Consider "echo if" - you
can also do "if=3; if [ $if = 3 ]; then then=7; fi=2; fi" (etc).
Probably not a great idea however.)

  | 2) Operating System (Bash or other) reserved variables are all defined as
  | capitol letters,

There are no OS reserved variables to consider, only some that are defined
and used by the shell (eg PATH, OPTIND, ...) which you need to avoid using,
unless you're using them as intended.   BASH defines a whole set of such
things, though many of them are safe to reuse if you like (you simply lose
the bash defined attribute of that variable, and it becomes ordinary.)
Not all however.   There's a list in the man page.

  | and all capitol letter variables should be avoided within Bash

That's one way to avoid issues, but quite conservative.   You can certainly
use one char (alphabetic - but not '_') var names in upper case.

  | Some say variables should be prefixed (or suffixed) to further distinguish

Whatever method you choose, simply avoid those in the list.  Sticking
"_VAR" or something on the end of everything looks like it would make the
script unreadable quite quickly, and in any case isn't guaranteed to never
conflict with a bash provided variable.

Incidentally, since sh (all implementations, including bash) has a global
variable namespace, the biggest source of variable name conflicts is
usually with other sh scripts (functions, startup files,
environment variables...) that are incorporated into your script.   This one
is hard to deal with, as anything at all is possible, depending upon
the origin of such a thing - and while re-using such a name is unlikely to
harm your script (providing you always initialize vars, and never just assume
they will be null valued/unset before their first assignment), but you can
break other functionality by stealing one of its variables.

Note that no convention you adopt can avoid this problem, as the other
function might have adopted the exact same convention.

  | Anybody have any further insight concerning variable naming styles
  | aside from what's already written within the documentation?

Use whatever you like, avoiding those listed by the doc as being defined
by bash.

  | I could do something like MY_VARIABLE, but then prefixing with "MY_"
  | eats up three more chars I could have used for describing my variable
  | better.

Not really, it just makes your names longer (there is no length limit for
var names - I have written test cases that use var names thousands of
characters long, omit the spaces and any punctuation, and you could easily
use one of Shakespeare's sonnets as a var name if you wanted.   I don't
recommend it.

But it doesn't help for the real problem.

The best way to avoid problems is to avoid making huge scripts (split
different functionality into different scripts) and when you do need
something that is going to be a bit complex, embed it in a function,
and make the variables all be local (which at least protects them from
interfering with other code your function does not call - you need to
handle non-local var names in functions that you do call yourself, as
the global namespace means that you cannot protect those).

kre

Re: Protect Loop Execution with Traps

[Robert Elz]

Date:Tue, 28 Jan 2020 16:25:51 -0500
From:Roger 
Message-ID:  <20200128212551.GD12574@localhost4.local>

  | Wow, " trap 'trap INT; kill -INT $$' INT "  not easily readable for me.

You can often help with things like that by reformatting

trap '
trap - INT
kill -s INT $$
' INT

and (with all respect to Gred) please avoid archaic uses, and use the
commands as they're currently specified, while "trap - INT" and "trap INT"
do the same thing, the former is the standard way, similarly for
"kill -INT ..." and "kill -s INT ..." the latter is the modern version.

  | A trap calling kill inside of a trap.

Quite common.   Try to avoid recursive traps however, those will blow
your stack and result in a core dump from bash if you're not careful.

  | I'm thinking, put "trap INT; kill -INT $$" inside of it's own function,

That's acceptable, sometimes perferable, sometimes just more complexity for
no real benefit (makes code harder to understand).

  | This trap stuff has me pulling out my hair,

traps can do that.   In general, any programming dealing with async
events is much trickier to get right than simple synchronous programming.
At least traps are only ever executed between commands, which avoids
lots of otherwise potentially impossible to deal with situations (the
signal that invokes the trap might have occurred anywhere).

kre