Re: initialisation bash variables
On 17.08.2011 20:53, Stephane CHAZELAS wrote: 2011-08-17, 08:32(+02), Francky Leyn: On 8/16/2011 10:53 PM, Stephane CHAZELAS wrote: 2) If VAR coincides with an environment variable, and in the script I change it value, is this then propagated to outside the script? Is the environment variable affected? The environment is a list of strings (by convention of the format [...] In no circumstance are variable definitions in one process going to affect the environment of other processes (an exception to that is the "fish" shell) Could it be that the two of you are not talking about the same thing? Just for clarity: environment variables (henceforth "the environment") of a process are (is) inherited by its children. Therefore, what *does* happen, is that if Stephane, as in 2), changes VAR in script, the change gets propagated to the scripts *child* processes. $ X=1 bash 1*) $ echo $X 1 $ X=2 $ bash $ echo $X 2 But what does of course not happen, is that the change would get propagated to the *parent* process. $ export X=12*) $ echo $X 1 $ bash $ X=2 $ exit $ echo $X 1 1*) X will exist only in the environment of the child process being started 2*) X will exist in the environment of the current process and will therefore be inherited and exist in the environment of its children as well (What is the "fish" shell ???)
Re: Syntax Question...
On 18.08.2011 12:44, Stephane CHAZELAS wrote:
2011-08-17, 08:24(-04), Greg Wooledge:
On Tue, Aug 16, 2011 at 03:41:19PM -0700, Linda Walsh wrote:
Ken Irving wrote:
Maybe this?
today_snaps=( ${snap_prefix} )
but as you mention, that will put them into an arraysorry "imprecise
terminology" list for me is some number of objects in a string
separated by some
separator.
This is an extremely bad idea. Legacy Bourne shell code from the
1980s kind of bad -- from the horrible days before we *had* arrays
in shells. How are you going to handle filenames with spaces in them?
With newlines in them? With commas in them? With colons in them? Tabs?
DEL characters? Those are all valid in filenames. Any delimiter you
can *put* in a shell string is also a valid character in a filename (or
at least in a pathname, which eliminates the possibility of using slash).
In this code:
today_snaps=( ${snap_prefix} )
With the default value of IFS in bash and without globbing
disabled, the problematic characters are SPC, TAB, NL, *, ?, [
and potentially more if you have extended globbing enabled.
I think this works! You're missing the context or something.
"snap_prefix" is a shell glob pattern here, ending in *.
$ touch ' ' $'\t' $'\n' '*' '?' '['
$ snap_prefix=* # no glob happening here!
$ today_snaps=( ${snap_prefix} ) # glob happening here and
# the results are assigned to the
# elements of the today_snaps array
$ for e in "${l[@]}"; do stat -c%N "$e"; done
`\t'
`\n'
` '
`*'
`?'
`['
I don't see a problem...
Variable passed to system contains garbage characters
Configuration Information [Automatically generated, do not change]: Machine: i686 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-redhat-linux-gnu' -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables uname output: Linux DL.second 2.6.23.1 #1 Sun Oct 21 17:49:04 CEST 2007 i686 athlon i386 GNU/Linux Machine Type: i686-redhat-linux-gnu Bash Version: 3.2 Patch Level: 33 Release Status: release Description: The script below fails on my box with the following output: 1197676800 date: invalid date `1970-01-01 \033[?1034h1197676800 sec' So for some reason the value passed to date got a nasty prefix. I guess this could be charset-related, so here's my locale LANG=en_US LC_CTYPE="en_US" LC_NUMERIC="en_US" LC_TIME="en_US" LC_COLLATE="en_US" LC_MONETARY="en_US" LC_MESSAGES="en_US" LC_PAPER="en_US" LC_NAME="en_US" LC_ADDRESS="en_US" LC_TELEPHONE="en_US" LC_MEASUREMENT="en_US" LC_IDENTIFICATION="en_US" LC_ALL= Repeat-By: executing this script #!/bin/bash let "STEP=86400" let "CALCTIME=`date +%s`" export DAYSTART=`echo "$CALCTIME - ($CALCTIME % $STEP) - $STEP" |bc -i |tail -1` echo "$DAYSTART" export BASHBUG=`date -d "1970-01-01 $DAYSTART sec" +"%Y-%m-%d %T"` Fix: Yes, please ;)
Re: Variable passed to system contains garbage characters
']' +++ unset LANGUAGE +++ '[' -n '' ']' +++ unset LINGUAS +++ '[' -n '' ']' +++ unset _XKB_CHARSET /sbin/consoletype +++ consoletype=pty +++ '[' -n '' ']' +++ '[' -n '' ']' +++ '[' -n en_US ']' +++ case $LANG in +++ '[' xterm = linux ']' +++ unset SYSFONTACM SYSFONT +++ unset sourced +++ unset langfile ++ for i in '/etc/profile.d/*.sh' ++ '[' -r /etc/profile.d/less.sh ']' ++ . /etc/profile.d/less.sh +++ '[' -x /usr/bin/lesspipe.sh ']' +++ export 'LESSOPEN=|/usr/bin/lesspipe.sh %s' +++ LESSOPEN='|/usr/bin/lesspipe.sh %s' ++ for i in '/etc/profile.d/*.sh' ++ '[' -r /etc/profile.d/mc.sh ']' ++ . /etc/profile.d/mc.sh +++ alias 'mc=. /usr/share/mc/bin/mc-wrapper.sh' ++ for i in '/etc/profile.d/*.sh' ++ '[' -r /etc/profile.d/vim.sh ']' ++ . /etc/profile.d/vim.sh +++ '[' -n '3.2.33(1)-release' -o -n '' -o -n '' ']' +++ '[' -x //usr/bin/id ']' //usr/bin/id -u +++ '[' 508 -le 100 ']' +++ alias vi +++ alias vi=vim ++ for i in '/etc/profile.d/*.sh' ++ '[' -r /etc/profile.d/which-2.sh ']' ++ . /etc/profile.d/which-2.sh +++ alias 'which=alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde' ++ unset i ++ unset pathmunge + let STEP=86400 ++ date +%s + let CALCTIME=1197919330 ++ echo '1197919330 - (1197919330 % 86400) - 86400' ++ bc -i ++ tail -1 + export 'DAYSTART=1197763200' + DAYSTART='1197763200' + echo '1197763200' 1197763200 ++ date -d '1970-01-01 1197763200 sec' '+%Y-%m-%d %T' date: invalid date `1970-01-01 \033[?1034h1197763200 sec' + export BASHBUG= + BASHBUG= In FC6 it worked fine for me as well :) In fact it did so for two years, but somewhere between late fedora 7 and early fedora 8 it suddenly stopped. Br, Patrick
Re: Variable passed to system contains garbage characters
At 17.12.2007, Bob Proulx wrote: Patrick Nagelschmidt wrote: > The script below fails on my box with the following output: > > 1197676800 > date: invalid date `1970-01-01 \033[?1034h1197676800 sec' > > So for some reason the value passed to date got a nasty prefix. I cannot recreate the problem. However I think your issue is because you are forcing bc into interactive mode. Ok, thanks guys, you are both right, the interactive mode was the cause for this. how could i ever doubt the purity of the bash :) Br, Patrick
Re: problem with echoing script variable to gpg
Thanks. Amazingly, I got it figured out shortly after, and works like
a charm now.
Patrick
On Thu, Dec 18, 2008 at 1:57 PM, Eric Blake wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> According to pwaugh on 12/18/2008 1:49 AM:
>> digest = $( echo -n ${1} | gpg --print-md sha1 )
>> echo digest
>>
>> I do not get the same value I have no clue why. Help.
>
> Too much whitespace, and a misunderstanding about how shell variables
> work. You want:
>
> digest=$( echo -n "$1" | gpg --print-md sha1 )
> echo "$digest"
>
> - --
> Don't work too hard, make some time for fun as well!
>
> Eric Blake e...@byu.net
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (Cygwin)
> Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAklKq0EACgkQ84KuGfSFAYCFRgCgpsZf3FyDYGAn4nMaPmqIoxAq
> No4Anjl+kNnWGowPn+d8U6vpKSY2OV1Y
> =Capc
> -END PGP SIGNATURE-
>
Re: Feature Request: Custom delimeter for single quotes
I just came across: Example 19-8 Here: http://tldp.org/LDP/abs/html/here-docs.html It looks like this example meets my needs. I'm curious why this method is referred to as devious though? -- Patrick On Thu, Oct 31, 2019 at 11:54 PM Patrick Blesi wrote: > I'm looking for a hybrid between single quotes and a here doc or here > string. > > The main use case is for accepting arbitrary user-specified text. I would > like to wrap this text in single quotes so as to prevent any variable > expansion or interpretation of the text of any kind. Additionally, I would > like to allow the users to include single quotes in their text without > requiring that they escape these quotes. > > Something akin to the following would alleviate the need to communicate > that users must escape single quotes, but also provide the same literal > string behavior of single quotes. > > presuming the arbitrarily substituted text is: > > echo 'this command is specified by the user' > > Then a syntax for this single quote heredoc behavior could be like: > > $ sh -c <<^MAGIC_WORD echo 'this command is specified by the user' > MAGIC_WORD > > Everything within the MAGIC_WORD declarations would not have command > substitution, variable expansion, etc, but would be treated as if it were > wrapped in single quotes with the exception that single quotes between the > MAGIC_WORDs need not be escaped. > > Pardon my naïveté, does any such feature exist or are there good ways to > accomplish this? If not, is this something that could feasibly be > implemented? Would it be desirable? > > Thanks, > > Patrick >
Feature Request: Custom delimeter for single quotes
I'm looking for a hybrid between single quotes and a here doc or here string. The main use case is for accepting arbitrary user-specified text. I would like to wrap this text in single quotes so as to prevent any variable expansion or interpretation of the text of any kind. Additionally, I would like to allow the users to include single quotes in their text without requiring that they escape these quotes. Something akin to the following would alleviate the need to communicate that users must escape single quotes, but also provide the same literal string behavior of single quotes. presuming the arbitrarily substituted text is: echo 'this command is specified by the user' Then a syntax for this single quote heredoc behavior could be like: $ sh -c <<^MAGIC_WORD echo 'this command is specified by the user' MAGIC_WORD Everything within the MAGIC_WORD declarations would not have command substitution, variable expansion, etc, but would be treated as if it were wrapped in single quotes with the exception that single quotes between the MAGIC_WORDs need not be escaped. Pardon my naïveté, does any such feature exist or are there good ways to accomplish this? If not, is this something that could feasibly be implemented? Would it be desirable? Thanks, Patrick
Re: Feature Request: Custom delimeter for single quotes
The actual use case is taking a command from a Ruby script:
https://github.com/braintree/runbook/blob/4a0f0770a8a2a7be135cf13ee435d981b5975a06/lib/runbook/helpers/tmux_helper.rb#L23
`tmux send-keys -t #{target} #{_pager_escape_sequence} '#{command}' C-m`
The user specifies the command they want to run as a Ruby string and it
gets interpolated into the above string and then executed (The backticks in
Ruby invoke the command in a subprocess and return the output as a string,
#{} is string interpolation). As you can see, if the user-specified command
has a single quote, it will break this command unless escaped.
I think doing something like this should serve my needs:
`
command=$(cat <<'MAGIC_WORD'
#{command}
MAGIC_WORD
)
tmux send-keys -t #{target} #{_pager_escape_sequence} "$command" C-m
`
So that no single quote escaping is required. The non-valid input for the
command would be MAGIC_WORD. Do you know if this command is POSIX
compliant/supported by a large number of shells? Is is supported by the
bourne shell?
On Fri, Nov 1, 2019 at 3:37 AM Ilkka Virta wrote:
> On 1.11. 06:54, Patrick Blesi wrote:
> > I'm looking for a hybrid between single quotes and a here doc or here
> > string.
> >
> > The main use case is for accepting arbitrary user-specified text.
>
> Do your users enter the text by directly editing the script?
> Would it make more sense to use e.g. 'read' to read the input directly
> from the user?
>
> input=""
> nl='
> '
> echo "Enter text, end with ^D:"
> while IFS= read -r line; do
> input="$input$line$nl"
> done
>
> printf "You entered:\n---\n%s---\n" "$input"
>
>
> or to just have the text in a separate file (not the script) and read it
> from there?
>
> input=$(< inputfile)
>
>
> That way, the text appears in a variable, and you don't need to care
> about quotes inside it.
>
>
> (You could also read from stdin with just input=$(cat) instead of the
> while read loop but that feels a bit odd to me for some reason.)
>
> > I would
> > like to wrap this text in single quotes so as to prevent any variable
> > expansion or interpretation of the text of any kind. Additionally, I
> would
> > like to allow the users to include single quotes in their text without
> > requiring that they escape these quotes.
> >
> > Something akin to the following would alleviate the need to communicate
> > that users must escape single quotes, but also provide the same literal
> > string behavior of single quotes.
> >
> > presuming the arbitrarily substituted text is:
> >
> > echo 'this command is specified by the user'
> >
> > Then a syntax for this single quote heredoc behavior could be like:
> >
> > $ sh -c <<^MAGIC_WORD echo 'this command is specified by the user'
> > MAGIC_WORD
> >
> > Everything within the MAGIC_WORD declarations would not have command
> > substitution, variable expansion, etc, but would be treated as if it were
> > wrapped in single quotes with the exception that single quotes between
> the
> > MAGIC_WORDs need not be escaped.
> >
> > Pardon my naïveté, does any such feature exist or are there good ways to
> > accomplish this? If not, is this something that could feasibly be
> > implemented? Would it be desirable?
> >
> > Thanks,
> >
> > Patrick
> >
>
>
> --
> Ilkka Virta / itvi...@iki.fi
>
Re: Feature Request: Custom delimeter for single quotes
Upon further inspection, what Andreas pointed out is actually what I need.
Just to close the loop on everything...
It looks like Ruby does support execution with and without a shell:
https://apidock.com/ruby/Kernel/system.
The reasoning for using two programming languages is that sometimes it is
easier to accomplish things in Ruby and sometimes it is easier to
accomplish things in a shell. Providing the user the option to implement
something via shell or via Ruby allows for maximum flexibility and utility.
Regarding, posix compliance, I was specifically asking about whether the
following code could be expected to be supported by any POSIX-compliant
shell:
command=$(cat <<'MAGIC_WORD'
#{command}
MAGIC_WORD
)
tmux send-keys -t %1 q C-u "$command" C-m
Not sure if this is the canonical POSIX shell reference:
https://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_07_04,
but it appears that the quoted here-doc and command substitution are shell
features defined by the POSIX standard.
Thank you very much for your help.
-- Patrick
On Fri, Nov 1, 2019 at 3:44 PM Eli Schwartz wrote:
> On 11/1/19 3:57 PM, Patrick Blesi wrote:
> > The actual use case is taking a command from a Ruby script:
> >
> >
> https://github.com/braintree/runbook/blob/4a0f0770a8a2a7be135cf13ee435d981b5975a06/lib/runbook/helpers/tmux_helper.rb#L23
> >
> > `tmux send-keys -t #{target} #{_pager_escape_sequence} '#{command}' C-m`
> >
> > The user specifies the command they want to run as a Ruby string and it
> > gets interpolated into the above string and then executed (The backticks
> in
> > Ruby invoke the command in a subprocess and return the output as a
> string,
> > #{} is string interpolation). As you can see, if the user-specified
> command
> > has a single quote, it will break this command unless escaped.
>
> I don't know about ruby.
>
> I know that in, say, python, the subprocess module can take an array
> with a command executable and its arguments, and execute it using the
> exec() family of functions.
>
> You can optionally request that the subprocess module do its execution
> via a shell, just like system() does, but it's generally not exactly
> recommended.
>
> Have you considered rewriting your ruby program to not use vulnerable
> methods of executing subprocesses? Given that ruby is, presumably, a
> powerful programming language, I don't understand why you would want to
> write a program that now uses *two* programming languages:
>
> - ruby
> - /bin/sh
>
> when you could do all your work in ruby.
>
> If you absolutely require using shell syntax in your subprocess for
> inexplicable reasons, you can use the shell syntax embedded within this
> pseudocode, which would be executed using the exec() family of functions:
>
> {'sh', '-c', 'do_things "$1"', '_', 'argv_containing_user_input'}
>
> given sh is being passed an argument without introducing a shell, and
> that argument is assigned to the shell variable $1, that argument can be
> defined and passed to exec() containing anything which ruby wants to put
> there.
>
> Safely.
>
> > I think doing something like this should serve my needs:
> >
> > `
> > command=$(cat <<'MAGIC_WORD'
> > #{command}
> > MAGIC_WORD
> > )
> > tmux send-keys -t #{target} #{_pager_escape_sequence} "$command" C-m
> > `
> > So that no single quote escaping is required. The non-valid input for the
> > command would be MAGIC_WORD. Do you know if this command is POSIX
> > compliant/supported by a large number of shells? Is is supported by the
> > bourne shell?
>
> Is what command POSIX compliant?
>
> - The one you're proposing be added, right now, to bash and bash alone?
> - tmux?
> - cat with quoted delimiter tokens?
>
> --
> Eli Schwartz
> Arch Linux Bug Wrangler and Trusted User
>
>
Re: Feature Request: Custom delimeter for single quotes
I was originally thinking I did not want to shell-quote these because I may
have wanted part of the user input to be interpreted by the shell. Thinking
more about this, I want the entirety of user input to be treated as a
verbatim string to be passed to the tmux command, so this is exactly what I
want.
Regarding security issues, that isn't a concern for this use case because
the user is allowed to execute arbitrary commands within the application.
The application is invoked at the command line as opposed to through a web
or other restricted interface. Any security concerns would be implemented
at the OS level (file permissions, etc.).
Thank you for your help.
-- Patrick
On Fri, Nov 1, 2019 at 3:57 PM Andreas Schwab wrote:
> On Nov 01 2019, Patrick Blesi wrote:
>
> > The actual use case is taking a command from a Ruby script:
> >
> >
> https://github.com/braintree/runbook/blob/4a0f0770a8a2a7be135cf13ee435d981b5975a06/lib/runbook/helpers/tmux_helper.rb#L23
> >
> > `tmux send-keys -t #{target} #{_pager_escape_sequence} '#{command}' C-m`
> >
> > The user specifies the command they want to run as a Ruby string and it
> > gets interpolated into the above string and then executed (The backticks
> in
> > Ruby invoke the command in a subprocess and return the output as a
> string,
> > #{} is string interpolation). As you can see, if the user-specified
> command
> > has a single quote, it will break this command unless escaped.
>
> Just shell-quote the characters in the interpolated string, as you need
> to do anyway for the other interpolated strings. Not doing this would
> be a security bug waiting to happen.
>
> Andreas.
>
> --
> Andreas Schwab, sch...@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
> "And now for something completely different."
>
Bug where SIGINT trap handler isn't called
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu'
-DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib
-D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe
-fstack-protector-strong --param=ssp-buffer-size=4
-DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/bin'
-DSTANDARD_UTILS_PATH='/usr/bin' -DSYS_BASHRC='/etc/bash.bashrc'
-DSYS_BASH_LOGOUT='/etc/bash.bash_logout'
uname output: Linux PatrickDesktop 3.19.2-1-ARCH #1 SMP PREEMPT Wed Mar
18 16:21:02 CET 2015 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu
Bash Version: 4.3
Patch Level: 39
Release Status: release
Description:
There's a bug that happens when waiting for a child process to complete
(via wait_for in jobs.c) that isn't part of a job, like command
substitution subshells. If a SIGINT is caught by the
wait_sigint_handler, the handler sets the wait_sigint_received flag
which is checked in set_job_status_and_cleanup. But
set_job_status_and_cleanup is called from waitchld only if the pid
belongs to a process that is part of a job. The result is that a SIGINT
trap, if it exists, is not called if a SIGINT is received while waiting
for a comsub subshell.
Repeat-By:
Running the following script (requires python) and giving it a SIGINT
(via ^C) when it blocks doesn't produce any output from the script.
Because SIGINT is trapped it should print “trapped”, however. Closing
stdout causes wait_for to be called in command_substitute:subst.c.
subst-sleep.sh:
trap 'echo trapped >&2; exit 1' INT
foo="$(python -c 'import time, os; os.close(1); time.sleep(20)')"
Fix:
The attached patch makes a conservative change that moves the code that
calls the SIGINT trap handler from set_job_status_and_cleanup to a new
function handle_sigint_caught_during_wait. This function is then
additionally called from waitchld for dead children that don't belong to
a job.
Related to the code the fix touches: handle_sigint_caught_during_wait
contains some duplicated code. Also, wait_for calls waitchld as long as
the child or job is still running and waitchld and
set_job_status_and_cleanup in turn call handle_sigint_caught_during_wait
if the child or job is dead. This seems to ensure that the SIGINT trap
handler is called at most once for a call to wait_for but isn't wait_for
a better place to check wait_sigint_received after having restored the
sigint handler (restore_sigint_handler()), especially since it already
contains CHECK_WAIT_INTR. Not to mention that a SIGINT could be received
after checking the wait_sigint_received flag and before restoring the
signal handler, as far as I understand it.
--- a/jobs.c
+++ b/jobs.c
@@ -260,6 +260,7 @@ static int print_job __P((JOB *, int, int, int));
static int process_exit_status __P((WAIT));
static int process_exit_signal __P((WAIT));
static int set_job_status_and_cleanup __P((int));
+static void handle_sigint_caught_during_wait __P((PROCESS *, int));
static WAIT job_signal_status __P((int));
static WAIT raw_job_exit_status __P((int));
@@ -3311,7 +3312,11 @@ itrace("waitchld: waitpid returns %d block = %d", pid,
block);
}
if (job == NO_JOB)
- continue;
+{
+ if (PEXITED (child))
+handle_sigint_caught_during_wait (child, NO_JOB);
+ continue;
+}
call_set_current += set_job_status_and_cleanup (job);
@@ -3371,18 +3376,104 @@ itrace("waitchld: waitpid returns %d block = %d", pid,
block);
return (children_exited);
}
-/* Set the status of JOB and perform any necessary cleanup if the job is
- marked as JDEAD.
+/* Handle SIGINTs received while waiting for a foreground JOB, or CHILD if it
+ isn't part of a job, to finish. */
+static void handle_sigint_caught_during_wait (child, job)
+ PROCESS *child;
+ int job;
+{
+ SigHandler *temp_handler;
+ if (job != NO_JOB)
+child = jobs[job]->pipe;
+ int tstatus, job_has_job_control, is_foreground_job;
+
+ job_has_job_control = job == NO_JOB ? job_control != 0 : IS_JOBCONTROL (job);
+ is_foreground_job = job == NO_JOB ? interactive_shell == 0 :
+IS_FOREGROUND (job);
+
+ if (!(wait_sigint_received && is_foreground_job))
+return;
+
+ /* If we're running a shell script and we get a SIGINT with a
+ SIGINT trap handler, but the foreground job handles it and
+ does not exit due to SIGINT, run the trap handler but do not
+ otherwise act as if we got the interrupt. */
+ if (interactive_shell == 0 &&
+ child_caught_sigint && signal_is_trapped (SIGINT))
+ {
+int old_frozen;
+wait_sigint_received = 0;
+last_command_exit_value = process_exit_status (child->status);
+
+old_frozen = jobs_list_frozen;
+jobs_list_frozen = 1;
+tstatus = maybe_call_trap_handler (SI
Re: Bug where SIGINT trap handler isn't called
On 30/06/15 02:27, Chet Ramey wrote: > On 6/29/15 3:41 PM, Patrick Plagwitz wrote: > >> Bash Version: 4.3 >> Patch Level: 39 >> Release Status: release >> >> Description: >> There's a bug that happens when waiting for a child process to complete >> (via wait_for in jobs.c) that isn't part of a job, like command >> substitution subshells. If a SIGINT is caught by the >> wait_sigint_handler, the handler sets the wait_sigint_received flag >> which is checked in set_job_status_and_cleanup. But >> set_job_status_and_cleanup is called from waitchld only if the pid >> belongs to a process that is part of a job. The result is that a SIGINT >> trap, if it exists, is not called if a SIGINT is received while waiting >> for a comsub subshell. > > This isn't exactly correct. The SIGINT trap will be called if the process > in the command substitution exits due to SIGINT. For instance, if you > change the assignment to something like: > > foo=$(sleep 20 >&- ) > or > foo=$(exec 2>&- ; sleep 20) > > both of which I think correctly reproduce the python command, the SIGINT > trap will execute. > > This is another case of the scenario most recently described in > > http://lists.gnu.org/archive/html/bug-bash/2014-03/msg00108.html > > In this case, python appears to catch the SIGINT (it looks like a > KeyboardInterrupt exception), print the message, and exit with status 1. > > Chet > Ok, I see. However, there appears to be some race condition when waiting for a command substitution. I have the attached combination of scripts. When run with $ bash run-until-end-of-loop.sh bash start-subst-loop.sh '2>/dev/null' , the script will try to launch and SIGINT subst-loop.sh repeatedly until the SIGINT trap is once *not* called which will happen in at most 200 repetitions on my machine. The script then ends after printing “end of loop”. The python script execute-with-sigint.py is only there to enable subst-loop.sh to receive a SIGINT at all. subst-loop.sh calls date(1) in a loop; date should have SIGINT set to SIG_DFL other than python initially. I analyzed the execution by inserting some debug output into the bash code. It seems that in the case that the SIGINT trap is not called subst-loop.sh gets the SIGINT while (or shortly before or after) calling waitpid in waitchld:jobs.c which will then return without errno == EINTR. The wait_sigint_handler will be called, though, and so wait_sigint_received will be true. If you try the script often enough, it will sometimes block. I saw that this is because wait_sigint_handler calls itself in an infinite loop. If I understand it correctly, this happens since wait_for:jobs.c calls old_sigint_handler = set_signal_handler (SIGINT, wait_sigint_handler); and if a SIGINT is caught after sigaction in set_signal_handler was called but before old_sigint_handler is set, restore_sigint_handler (); kill (getpid (), SIGINT); in wait_sigint_handler will leave wait_sigint_handler as handler. run-until-end-of-loop.sh Description: application/shellscript start-subst-loop.sh Description: application/shellscript import signal import os import sys signal.signal(signal.SIGINT, signal.SIG_DFL) sys.stderr.write(os.environ["SHELL"] + "\n") os.execv(os.environ["SHELL"], [os.environ["SHELL"]] + sys.argv[1:]) subst-loop.sh Description: application/shellscript
Re: Bug where SIGINT trap handler isn't called
On 22/07/15 04:01, Chet Ramey wrote: > On 7/16/15 12:05 AM, Patrick Plagwitz wrote: > >>> This is another case of the scenario most recently described in >>> >>> http://lists.gnu.org/archive/html/bug-bash/2014-03/msg00108.html >>> >>> In this case, python appears to catch the SIGINT (it looks like a >>> KeyboardInterrupt exception), print the message, and exit with status 1. >>> >>> Chet >>> >> >> Ok, I see. >> However, there appears to be some race condition when waiting for a >> command substitution. >> I have the attached combination of scripts. >> >> When run with >> $ bash run-until-end-of-loop.sh bash start-subst-loop.sh '2>/dev/null' >> , the script will try to launch and SIGINT subst-loop.sh repeatedly >> until the SIGINT trap is once *not* called which will happen in at most >> 200 repetitions on my machine. The script then ends after printing “end >> of loop”. The python script execute-with-sigint.py is only there to >> enable subst-loop.sh to receive a SIGINT at all. >> subst-loop.sh calls date(1) in a loop; date should have SIGINT set to >> SIG_DFL other than python initially. >> >> I analyzed the execution by inserting some debug output into the bash >> code. It seems that in the case that the SIGINT trap is not called >> subst-loop.sh gets the SIGINT while (or shortly before or after) calling >> waitpid in waitchld:jobs.c which will then return without errno == >> EINTR. The wait_sigint_handler will be called, though, and so >> wait_sigint_received will be true. > > This doesn't agree with what I see on RHEL6. I get waitpid() returning -1/ > EINTR, which bash interprets, using a heuristic, to mean that the child > blocked or caught SIGINT, in which case bash should not act as if it > received it. > > There is a small race condition here, which is very hard to close while > maintaining the desired behavior: bash only responds to SIGINT received > while waiting for a child if the child exits due to SIGINT. You appear > to have hit it: the timing of the ^C is such that waitpid returns > -1/EINTR, causing the shell to ignore the ^C. I suspect this is because > the child called exit(0) before the ^C arrived and the shell got the > signal, but the child exited successfully, causing the shell to assume > the child blocked or caught the SIGINT. > > That heuristic was developed as the result of an extensive discussion > between me and several Linux kernel developers back in 2011. You can > read that here: > > http://lists.gnu.org/archive/html/bug-bash/2011-02/msg00050.html > http://lists.gnu.org/archive/html/bug-bash/2011-03/msg0.html > > I will look at the signal handler race condition you identified. > > Chet > Thanks. And thanks for the information. So $ bash -c 'python -c "import time; time.sleep(10)"; echo foo' outputs foo when ^Cd because bash suppresses normal SIGINT handling if the child is assumed to have handled it itself. Meanwhile $ bash -c 'sleep 10; echo foo' doesn't output foo. But custom trap handlers are called either way in set_job_status_and_cleanup. $ bash -c 'trap "echo trap; exit" INT; python -c "import time; time.sleep(10)"; echo foo' prints trap, not foo, as does $ bash -c 'trap "echo trap; exit" INT; sleep 10; echo foo' I think command substitution has two issues. Those are because set_job_and_cleanup isn't called for the single child made for comsub. (1) The bug that was fixed in the discussion you linked is still in for comsub waits: $ bash -c 'while [ "$(exec >&-; sleep 0.001)" = "" ]; do :; done' sometimes requires two ^Cs to stop. The reason seems to be the same race condition described in http://lists.gnu.org/archive/html/bug-bash/2011-02/msg00073.html and in your and in my last mail. Checking last_command_exit_value == (128 + SIGINT) in command_substitute reflects the way the now-called child_caught_sigint was determined before the patch made during the discussion (i.e. it doesn't implement the heuristics). As a side note, http://lists.gnu.org/archive/html/bug-bash/2011-03/msg00039.html explained to me why this outcome of the race condition is at all likely. (2) Running the comsub version of one of the above scripts: $ bash -c 'trap "echo trap; exit" INT; foo="$(exec >&-; python -c "import time; time.sleep(10)")"; echo foo' yields another result (it prints foo, not trap). This is the behavior the original bug report was about and you already replied to it but is the difference between comsub children and normal ones here intended?
bash integer overflow (?) with large HISTFILESIZE
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' -DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/bin' -DSTANDARD_UTILS_PATH='/usr/bin' -DSYS_BASHRC='/etc/bash.bashrc' -DSYS_BASH_LOGOUT='/etc/bash.bash_logout' -Wno-parentheses -Wno-format-security uname output: Linux icewind 4.8.7-1-ARCH #1 SMP PREEMPT Thu Nov 10 17:22:48 CET 2016 x86_64 GNU/Linux Machine Type: x86_64-unknown-linux-gnu Bash Version: 4.4 Patch Level: 0 Release Status: release Description: With the latest version of bash just deployed in the Arch Linux repositories, bash fails with HISTSIZE=2147483647: bash: xmalloc: cannot allocate 18446744056529682440 bytes Repeat-By: $ env HISTSIZE=2147483647 /bin/bash --norc bash: xmalloc: cannot allocate 18446744056529682440 bytes Fix: [Description of how to fix the problem. If you don't know a fix for the problem, don't include this section.] -- Patrick Donnelly
Random loss of history entries
Configuration Information [Automatically generated, do not change]: Machine: i386 OS: linux-gnu Compiler: i386-redhat-linux-gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i386' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i386-redhat-linux-gnu' -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -pipe -m32 -march=i386 -mtune=pentium4 uname output: Linux rat 2.6.9-42.0.10.ELsmp #1 SMP Tue Feb 27 08:38:56 CST 2007 i686 i686 i386 GNU/Linux Machine Type: i386-redhat-linux-gnu Bash Version: 3.0 Patch Level: 15 Release Status: release Description: When using bash command history heavily, with long lines, and command review and reuse, command history becomes useless, possibly being reset way back in time. There is more detail at http://shallowsky.com/blog/linux/, the essence of which is repeated below: "Say I type some longish command. After it runs, I hit return a couple of times, wait a while, do a couple of other things, then decide I want to call that command back from history so I can run something similar, maybe with the filename changed or a different flag. I ctrl-P or up-arrow ... and the command isn't there! If I type history at this point, I'll see most of my command history ... with an empty line in place of the line I was hoping to repeat. The command is gone. My only option is to remember what I typed, and type it all again." Repeat-By: I have not been able to discern a pattern, except that it happens at very inconvenient times.
