Old idea for vi-yank-pop
A few years ago, I had an idea for bash, which I recently remembered. I never cared about it enough to propose it, but I figured I should share in case it is helpful to someone else. In vim, you can get access to multiple yanks/cuts/kills, but you can't in bash's vi-mode. It would be a good feature to give vi-mode access to the the stack of yanks from the default emacs mode. I figured it could be called vi_yank_pop. This is a patch I wrote a long time ago. I don't know if it works, and it probably mostly copies the emacs code. Cheers, Ian Kelling --- ../bash-4.0/lib/readline/kill.c 2008-08-12 14:59:31.0 -0700 +++ kill.c 2009-03-12 20:58:07.0 -0700 @@ -550,6 +550,39 @@ } } +int +rl_vi_yank_pop (count, key) + int count, key; +{ + int l, n; + + if (((rl_last_func != rl_vi_yank_pop) && (rl_last_func != rl_vi_put)) || + !rl_kill_ring) +{ + _rl_abort_internal (); + return -1; +} + + l = strlen (rl_kill_ring[rl_kill_index]); + n = rl_point - l; + if (n >= 0 && STREQN (rl_line_buffer + n + 1, rl_kill_ring[rl_kill_index], l)) +{ + rl_delete_text (n, rl_point); + rl_point = n; + rl_kill_index--; + if (rl_kill_index < 0) + rl_kill_index = rl_kill_ring_length - 1; + rl_vi_put (1, 'p'); + return 0; +} + else +{ + _rl_abort_internal (); + return -1; +} +} + + /* Yank the COUNTh argument from the previous history line, skipping HISTORY_SKIP lines before looking for the `previous line'. */ static int --- ../bash-4.0/lib/readline/funmap.c 2009-01-04 11:32:33.0 -0800 +++ funmap.c2009-03-12 18:37:18.0 -0700 @@ -139,6 +139,7 @@ { "yank-last-arg", rl_yank_last_arg }, { "yank-nth-arg", rl_yank_nth_arg }, { "yank-pop", rl_yank_pop }, + { "vi-yank-pop", rl_vi_yank_pop }, #if defined (VI_MODE) { "vi-append-eol", rl_vi_append_eol }, --- ../bash-4.0/lib/readline/readline.h 2009-01-04 11:32:33.0 -0800 +++ readline.h 2009-03-12 18:37:18.0 -0700 @@ -167,6 +167,7 @@ extern int rl_copy_backward_word PARAMS((int, int)); extern int rl_yank PARAMS((int, int)); extern int rl_yank_pop PARAMS((int, int)); +extern int rl_vi_yank_pop PARAMS((int, int)); extern int rl_yank_nth_arg PARAMS((int, int)); extern int rl_yank_last_arg PARAMS((int, int)); /* Not available unless __CYGWIN__ is defined. */
process substitution and trailing file descriptors
Hi, I'm not so sure this is a bug rather than a feature but it has undesirable behaviour to my eye. I found it originally in 3.0.16 but I've just reproduced it in 4.1. If I have a script where I use process substitution to log the output yet keep stdout and stderr as they stand: command1 > >(tee out) 2> >(tee err >&2) I understand that process substitution will find two free high numbered file descriptors (to avoid clashes with low numbered file descriptors that the script might arbitrarily use) and that seems fine. However, the redirection on the command line means I will have: fds 1 and 63 going to "tee out" and fds 2 and 62 going to "tee err". In fact, because of left-to-right processing, "tee err" will also have fd 63 going to "tee out". So far, it's OK as when command1 exits, everything will close up and sub-processes will exit. However, if command1 were to run command2 in the background then exit itself, command2, having inherited fds 62 and 63, will keep the pipes open to the two tee processes and all will hang about until command2 (or its descendants) exit. Before I realised fds 63 and 62 were in play, I thought I could prevent the tee processes hanging about by having command2 redirect its stdout and stderr but no such luck. I can have command2 invoke exec 63>&- 62>&- but the choice of numbers is somewhat arbitrary depending on what has happened in the history of the processes. The manual suggests I could move and close file descriptors with [n]>&digit- but I would need the equivalent of command1 >&>(...)- Digit might very well mean (just a) digit but here the process substitution, of course, is replaced with /dev/fd/63, say, certainly not a digit. Basically, by mixing IO redirection and process substitution I'm left with a trailing file descriptor which can cause scripts to hang around despite any subsequent redirection of stdout/stderr best practices. There's no mechanism to discover these new file descriptors and they are not closed on exec. Is there any hope for me? Cheers, Ian
Re: process substitution and trailing file descriptors
On 11 Feb, 20:29, Greg Wooledge wrote: > When the shortcuts are too short, you need to fall back to the original > tools. In this case, >() is really just a shortcut for "create a FIFO, > and open it". Doing so by hand should give you the manual control you > need. At the very least, you can tell bash which FD number to use when > you open the FIFO. Thanks, Greg. I'd simply not thought of doing the heavy lifting myself. I'd realised afterwards that any soft of close on exec would be fraught with pain as in the /dev/fd/n case, the files you might be giving to diff, as in diff <(sort a) <(sort b) would be deleting the /dev/fd/n that diff was about to open. I think I was hoping that there might be some magic with the redirection operators that might recognise that the word in >word was a process substitution fifo and therefore the high numbered file descriptor created temporarily for the fifo could be closed. There's probably a perfectly reasonable case where the process substitution fifo on the RHS of an IO redirection operator should not be closed. Cheers, Ian
aliases sometimes not expanded even with expand_aliases set
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer uname output: Linux sibyl.beware.dropbear.id.au 6.10.10-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 18:26:09 UTC 2024 x86_64 GNU/Linux Machine Type: x86_64-redhat-linux-gnu Bash Version: 5.2 Patch Level: 26 Release Status: release Description: In the following script, the first alias fails with "comand not found" but the second works. #!/bin/bash shopt -s expand_aliases if :; then alias foo='echo' foo x y z shopt expand_aliases alias -p fi alias foo='echo' foo a b c Running it produces: $ ./alias-test.sh ./alias-test.sh: line 7: foo: command not found expand_aliases on alias foo='echo' a b c This is at least, undocumented behaviour. Repeat-By: Run the script in "Description".
dotgob+extglob bug
Running this script with your own bash path demonstrates the bug. #!/home/ian/opt/bash/bash --norc shopt -s extglob shopt -s dotglob cd $(mktemp -d) mkdir a touch a/.b touch a/c echo a/!(.b) output: a/. a/.. a/c this happens with all bash versions 4.3+ (latest is patch 18). before that, the output is: a/c Another related bug. man bash states: The file names ``.'' and ``..'' are always ignored when GLOBIGNORE is set and not null. Which clearly implies that . and .. should not be ignored in some other case. Well, that was not true before 4.3, but I'm guess this new behavior is a bug, and the doc is a bug, since the doc hasn't changed, and setting GLOBIGNORE doesn't actually make . and .. be ignored.
Re: dot gob+extglob bug
Chet Ramey writes: > On 6/9/14, 3:42 PM, Ian Kelling wrote: > Yes, it's an interesting question: what exactly does pattern negation > include? You can match all filenames beginning with a `.' by using `.' > as the first character of the pattern, so should a negated pattern > beginning with a `.' match all filenames beginning with a `.' that don't > match that particular pattern? The bash-4.3 implementation says yes. > (FWIW, ksh93 disagrees.) Yes, now I understand. I agree with this bash 4.3 behavior. As you pointed out, some of my comments about past bash were wrong. Thank you. Now I think I have I have explored properly the latest version and found some problems I did not fully see before: The doc says "When matching a pathname, the slash character must always be matched explicitly." Shortly thereafter, in the next paragraph of the same section, GLOBIGNORE is described, which does not treat / as special, but this is not mentioned, and is very unexpected to me. Closer inspection, I see same language "filenames matching a pattern" is used in both paragraphs, so I think some clarification is needed. # example: / matters to GLOBIGNORE ~/opt/bash (master) $ ./bash --norc bash-4.3$ cd $(mktemp -d) bash-4.3$ touch a bash-4.3$ GLOBIGNORE=a bash-4.3$ echo * * bash-4.3$ echo ./* ./a # another example of the same phenomenon bash-4.3$ GLOBIGNORE='*a' bash-4.3$ echo ./* ./* And then, this definitely seems like a bug: * matches / in GLOGIGNORE, and so does [/], but ? does not match / # example: ? does not match "/" bash-4.3$ GLOBIGNORE=.?a bash-4.3$ echo ./* ./a # example: ? does match "x" bash-4.3$ touch .xa bash-4.3$ echo .x* .x* # example: [/] matches "/" bash-4.3$ GLOBIGNORE=.[/]a bash-4.3$ echo ./* ./.xa And then, another bug or doc clarification. The various [:class:] forms don't seem to work at all in GLOBIGNORE. Side note, I've added to my bashrc: GLOBIGNORE=*/.:*/..
Re: dot gob+extglob bug
The pathexp-globignore-delim.patch seems to work as advertised. > If *a matches scratch/a, for > example, that's a bug in the matching code I will have to identify and fix. Yes, this is the case. Based on your reply, the examples I showed are definitely a bug. Thank you so much.
Bug in getcwd implementation in lib/sh/getcwd.c
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: /bind/obj/repos/dist-buildroot.hg/build_x86_64/staging_dir/usr/bin/x86_64-linux-uclibc-gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPA uname output: Linux bogga 2.6.18.8-x86_64-xen0 #39 Wed Oct 3 11:13:22 BST 2007 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 3.2 Patch Level: 17 Release Status: release Description: I am cross compiling bash 3.2 in a uclibc environment using buildroot, the target is x86_64. Because of the cross compilation the configure test for "if getcwd() will dynamically allocate memory" fails and so the version in lib/sh/getcwd.c is used. The code to handle buf == NULL in this version of getcwd() is buggy where it sets len = size at line 256: size_t len = pathbuf + pathsize - pathp; if (buf == NULL) { if (len < (size_t) size) len = size; ... (void) memcpy((PTR_T) buf, (PTR_T) pathp, len); ... If size is greater than "pathbuf + pathsize - pathp" then the memcpy at line 267 will copy past the end of the path[] buffer. Depending on the stack layout / depth etc this can result in a segmentation fault as memcpy copies past the top of the stack. Repeat-By: It's a bit tricky but I see it by repeatedly running: chroot /uclibc-root /bin/bash in a Xen domain0 (kernel with some bits from rhel5x). There is some randomisation of the stack in this configuration (exec-shield I think) which means the segfault does not occur every time. Fix: The fix is really to pre-seed config.cache so that it knows that the uclibc version of getcwd() does the right thing but the fix for the problem in lib/sh/getcwd.c is the patch below from my colleague Christian Limpach. Index: bash-3.2/lib/sh/getcwd.c === --- bash-3.2.orig/lib/sh/getcwd.c 2007-10-09 09:53:38.0 +0100 +++ bash-3.2/lib/sh/getcwd.c2007-10-09 09:53:54.0 +0100 @@ -251,19 +251,21 @@ { size_t len = pathbuf + pathsize - pathp; -if (buf == NULL) +if (buf == NULL && size < 0) { - if (len < (size_t) size) - len = size; - buf = (char *) malloc (len); - if (buf == NULL) - goto lose2; +size = len; } -else if ((size_t) size < len) +if ((size_t) size < len) { errno = ERANGE; goto lose2; } +if (buf == NULL) + { + buf = (char *) malloc (size); + if (buf == NULL) + goto lose2; + } (void) memcpy((PTR_T) buf, (PTR_T) pathp, len); }
Invalid array subscript (single backslash) causes assertion failure
Configuration Information [Automatically generated, do not change]: Machine: i486 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i486' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 -Wall uname output: Linux redacted 2.6.22-14-generic #1 SMP Tue Feb 12 07:42:25 UTC 2008 i686 GNU/Linux Machine Type: i486-pc-linux-gnu Bash Version: 3.2 Patch Level: 25 Release Status: release Description: Bash prints the following lines and dumps core: malloc: ../bash/arrayfunc.c:616: assertion botched free: start and end chunk sizes differ last command: ${foo[\]} Aborting...Aborted (core dumped) Repeat-By: Enter the following line in interactive mode: ${foo[\]} Fix: Obviously not a valid construct, but nevertheless shouldn't crash.
Re: Creating directories with sticky bit set
Angel Tsankov wrote: > Greg Wooledge wrote: > Let's say that removing '-p' is straightforward, but what about setting the > sticky bit to every newly created directory component? > mkdir also has the -m argument, so you could do mkdir -m 1755 dir interestingly -m does not apply to parent directories created with -p If you wanted to deal with the -p, I would remove it and not pass it to mkdir, but create them yourself in the function. I'd use parameter expansion to deal with parts of the path 1 by 1. - Ian Kelling
Re: Bash 4 cursor in my prompt
Mike Frysinger wrote: considering it seems to involve the prompt, you really need to post your exact prompt settings (i.e. PS1/PS2/etc...) -mike Realize that in PS vars, \[ and \] should be used around non-printable characters. This surely won't happen if you start with --norc. You've got to track down everything in your invocation files that affects your prompt and narrow it down to something we can test ourselves. - Ian Kelling
Re: Bash 4 cursor in my prompt
Special Sauce wrote: [an...@nobby-nobbs ~]$ echo $PS1 [\[\e[28;1m\...@\h\[ \e[0m\]\w]$ [an...@nobby-nobbs ~]$ echo $PS2 Work fine with the bash in the ubuntu repos I put this at the end of my .bashrc and it works fine for me. PS1='[\[\e[28;1m\...@\h\[ \e[0m\]\w]$ ' Theres probably something else going on in your invocation files. Try making that the only line in your .bashrc and starting a non-login shell so thats the only file read. - Ian Kelling
Re: possible bash 4.0.10 bug: command substitution in PROMPT_COMMAND
Jared Yanovich wrote: Hi, I upgraded to bash 4.0.10 and my PS1 prompt is no longer displaying anything at all. I think this was addressed in a recent patch. Try upgrading to the most recent version. - Ian Kelling
Re: possible bash 4.0.10 bug: command substitution in PROMPT_COMMAND
Chet Ramey wrote: I cannot reproduce this using bash-4.0.10 or bash-4.0.17 on Mac OS X, Ubuntu, RHEL 4, or FreeBSD. Chet I can't reproduce it either. - Ian Kelling
Re: Bash 4 cursor in my prompt
Special Sauce wrote: From: anton To: bug-bash@gnu.org Subject: Cursor starts inside prompt I just noticed this issue too. It seems it was fine under gnu screen, but not with plain xterm or gnome-terminal. Upgrading to the latest patch version 4.0.17 fixed it. - Ian Kelling
Re: Bash 4 cursor in my prompt
Chet Ramey wrote: > Ian Kelling wrote: >> Special Sauce wrote: >>> From: anton >>> To: bug-bash@gnu.org >>> Subject: Cursor starts inside prompt >> I just noticed this issue too. It seems it was fine under gnu screen, >> but not with plain xterm or gnome-terminal. Upgrading to the latest >> patch version 4.0.17 fixed it. > > That's interesting, since none of the patches so far have dealt with > readline display or prompt expansion. > > Chet > Yes, quite odd. In my initial investigation I did grep -ri "prompt" on the patches and found nothing. Some time later after a bunch of testing, I figured, well, I should upgrade from patch level 10 to 17 before reporting it, and then it was gone. If your curious or doubt this was the real issue, I could double check and narrow it down to a specific patch that fixes it. - Ian Kelling
Re: Bash 4 cursor in my prompt
Chet Ramey wrote: Ian Kelling wrote: Special Sauce wrote: From: anton To: bug-bash@gnu.org Subject: Cursor starts inside prompt I just noticed this issue too. It seems it was fine under gnu screen, but not with plain xterm or gnome-terminal. Upgrading to the latest patch version 4.0.17 fixed it. That's interesting, since none of the patches so far have dealt with readline display or prompt expansion. Odd. The earlier version does not cause it to happen now. So, I don't know what caused it. Perhaps I did something screwy in the original patch/compile sequence. - Ian Kelling
Re: declare a=( {1..10} )
Chet Ramey wrote: I will see whether or not I can defer brace expansion on assignment statement arguments. Chet I always remember that brace expansion happens first so I'd prefer that it expand on both kinds of assignments. I suppose it doesn't matter too much as long as its consistent. - Ian Kelling
unable to redirect readonly variable error
Hello, This doesn't seem right: [EMAIL PROTECTED] bash_completion]$ foo=x [EMAIL PROTECTED] bash_completion]$ foo=bar [EMAIL PROTECTED] bash_completion]$ readonly foo [EMAIL PROTECTED] bash_completion]$ foo=bar >/dev/null bash: foo: readonly variable [EMAIL PROTECTED] bash_completion]$ foo=bar 2>/dev/null bash: foo: readonly variable [EMAIL PROTECTED] bash_completion]$ foo=bar &>/dev/null bash: foo: readonly variable [EMAIL PROTECTED] bash_completion]$ bash --version GNU bash, version 3.00.16(1)-release (i686-redhat-linux-gnu) Copyright (C) 2004 Free Software Foundation, Inc. Ian -- Ian Macdonald | "Microsoft is the epitome of innovation and [EMAIL PROTECTED] | product quality." -- This testimonial http://www.caliban.org/ | paid for by Microsoft. | | ___ Bug-bash mailing list Bug-bash@gnu.org http://lists.gnu.org/mailman/listinfo/bug-bash
Re: unable to redirect readonly variable error
On Thu 07 Jul 2005 at 20:07:33 -0400, you wrote: > On Thu, 7 Jul 2005, Ian Macdonald wrote: > > >Hello, > >This doesn't seem right: > >[EMAIL PROTECTED] bash_completion]$ foo=x > >[EMAIL PROTECTED] bash_completion]$ foo=bar > >[EMAIL PROTECTED] bash_completion]$ readonly foo > >[EMAIL PROTECTED] bash_completion]$ foo=bar >/dev/null > >bash: foo: readonly variable > >[EMAIL PROTECTED] bash_completion]$ foo=bar 2>/dev/null > >bash: foo: readonly variable > >[EMAIL PROTECTED] bash_completion]$ foo=bar &>/dev/null > >bash: foo: readonly variable > > What are you tring to redirect? There is no output from your > command; it is only an assignment to a read-only variable (which is > naturally going to fail). I'm trying to redirect the error, because I have a read-only variable that may have attempts made to redefine it. I want those to silently fail. Ian -- Ian Macdonald | "Reality is that which, when you stop [EMAIL PROTECTED] | believing in it, doesn't go away". -- http://www.caliban.org/ | Philip K. Dick | | ___ Bug-bash mailing list Bug-bash@gnu.org http://lists.gnu.org/mailman/listinfo/bug-bash
difference between complete -o default and -o bashdefault
Hello, I can't quite work out what the difference is between -o default and -o bashdefault. One is readline-oriented and the other is not, but the effect appears to be the same for the cases that I have run through. Anyone? Ian -- Ian Macdonald | The one good thing about repeating your [EMAIL PROTECTED] | mistakes is that you know when to cringe. http://www.caliban.org/ | | | ___ Bug-bash mailing list Bug-bash@gnu.org http://lists.gnu.org/mailman/listinfo/bug-bash
set +o noglob within completion function disables alias expansion
Hi, This is a bit of a corner case, but appears to be a bug. If one defines a completion function that calls 'set +o noglob' and this function is then bound to a command, alias expansion is temporarily disabled when the command is executed. If the command is subsequently executed again without calling the completion function, alias expansion occurs as normal. For example: [EMAIL PROTECTED] i386]$ set +o | grep noglob set +o noglob [EMAIL PROTECTED] i386]$ _foo() { local glob=$(set +o|grep noglob); set -f; eval "$glob"; } [EMAIL PROTECTED] i386]$ complete -F _foo -o filenames ls [EMAIL PROTECTED] i386]$ alias ls="echo ALIAS! $1" [EMAIL PROTECTED] i386]$ ls ~/foo ALIAS! /home/ianmacd/foo [EMAIL PROTECTED] i386]$ ls ~/foo accesses.db day12-5day17-2 divzeroprog4 anagram day13-1day17-3 editor.rb prog5 buzz.db day13-2day17-5 executablesruncom.rb callanalyzer day13-3day17-6 fib_test_2.rb spinner1.rb cards day13-4day18-1 findbox.rb spinner2.rb cards.rb day14-2day18-4 foospinner3.rb cards2day14-3.1 day18-5 index.html spinner4.rb cards3day14-3.2 day18-6 inner.rb spintest.rb cards4day14-3.3 day19-1 messagebox.rb student.rb complex.rbday14-4day19-2.rb outer.rb teacher.rb datagram.rb day15-2day20-1 p302 ty_ruby_21_listings day10-1 day15-4day20-2 prog1 ty_ruby_21_listings.tgz day11-2 day15-5day20-3 prog2 day12-4 day15-6deck.rb prog3 [EMAIL PROTECTED] i386]$ bash --version GNU bash, version 3.00.16(1)-release (i686-redhat-linux-gnu) Copyright (C) 2004 Free Software Foundation, Inc. As you can see, in this second invocation, the alias was not used as it should have been. Ian -- Ian Macdonald | I sometimes think that God, in creating [EMAIL PROTECTED] | man, somewhat overestimated his ability. http://www.caliban.org/ | -- Oscar Wilde | | ___ Bug-bash mailing list Bug-bash@gnu.org http://lists.gnu.org/mailman/listinfo/bug-bash
Re: set +o noglob within completion function disables alias expansion
On Sat 23 Jul 2005 at 12:10:53 -0400, you wrote: > This has already been fixed for bash-3.1. The problem is that shell > function execution during completion didn't save and restore enough > of the parser state. Thanks, Chet. Are you able to say anything about the time frame for releasing 3.1? Cheers, Ian -- Ian Macdonald | Once harm has been done, even a fool [EMAIL PROTECTED] | understands it. -- Homer http://www.caliban.org/ | | | ___ Bug-bash mailing list Bug-bash@gnu.org http://lists.gnu.org/mailman/listinfo/bug-bash
Arithmetic expansion with increments and output redirection
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic uname output: Linux s1028505 3.10.0-957.10.1.el7.x86_64 #1 SMP Thu Feb 7 07:12:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-redhat-linux-gnu Bash Version: 4.2 Patch Level: 46 Release Status: release Description: When using arithmetic expansion with variable pre- and post-increments/decrements in the output redirection file path, specifically on external executables (not builtins or functions), the state of the variable being incremented/decremented is not persisted in the environment. Repeat-By: # n=1; true > $((--n)); echo $n; ls outputs: 0 0 but # n=1; /bin/true > $((--n)); echo $n; ls outputs: 1 0
Re: Arithmetic expansion with increments and output redirection
> Date:Tue, 23 Apr 2019 15:49:18 -0600 > From: Ian Neal > Message-ID: 1_z7uy+7rv...@mail.gmail.com> > > | When using arithmetic expansion with variable pre- and > | post-increments/decrements in the output redirection file path, > | specifically on external executables (not builtins or functions), the > state > | of the variable being incremented/decremented is not persisted in the > | environment. > > That is as it should be, redirects are eveluated in the sub-shell > context. If anything the bug you showed is that redirects when the > shell does not fork are being evaluated in the context of the shell > (but I think that might be an unspecified case). > > In general it is best not to even consider using any evaluation with > side effects in any redirection, and unless you really consider what > you're doing, not in var-assigns either (there, if you're not expecting > the side effects to be visible in the same, or any other, assignemnt > in the same command you should be OK). > > kre > At what point is a subshell being invoked? There's no pipeline, command substitution, coprocess, background process, or explicit () subshell here, which are the only cases a subshell should be created. Otherwise, the entire operation should be evaluated by the main shell. Arithmetic expansion is not in that list. >
Re: Arithmetic expansion with increments and output redirection
On Wed, Apr 24, 2019, 07:12 Chet Ramey wrote: > On 4/24/19 8:47 AM, Ian Neal wrote: > > > At what point is a subshell being invoked? There's no pipeline, command > > substitution, coprocess, background process, or explicit () subshell > here, > > which are the only cases a subshell should be created. Otherwise, the > > entire operation should be evaluated by the main shell. Arithmetic > > expansion is not in that list. > > Why would you think that /bin/true would be "evaluated by the main shell?" > It's not a shell compound command or a builtin, and non-builtin commands > are run in child processes. > Calling what should be a simple fork(); exec(); a "subshell" is a little disingenuous unless it has its own parameter expansion options, which isn't true -- if the $((n++)) was on the left side of the file redirection (which is to say a parameter), it would be expanded by the shell before the fork(), so why is this true with the redirection itself? What makes that case so special? I posit that it shouldn't be. >
Re: Arithmetic expansion with increments and output redirection
> > "Utilities other than the special built-ins (see Special Built-In > Utilities) shall be invoked in a separate environment that consists of the > following...[includes redirections specified to the utility]...The > environment of the shell process shall not be changed by the utility" > > > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_12 > The utility here in question is the external command (/bin/true in my example), which doesn't do the expansion itself. In this case, it's not the utility affecting the environment, it's the expansion performed by the (sub)shell itself before the exec*() that has an effect on the environment. They're different from word expansions. > The bash manual calls it a word, which is one of the sources of my confusion. "The general format for redirecting output is: [n]>word" https://linux.die.net/man/1/bash
Shellshock-vulnerable version still most obvious on ftp.gnu.org
If you go to ftp://ftp.gnu.org/pub/gnu/bash/, the most obvious most recent version of bash is this: ftp://ftp.gnu.org/pub/gnu/bash/bash-4.3.tar.gz ftp://ftp.gnu.org/pub/gnu/bash/bash-4.3.tar.gz.sig The shellshock fix is hidden in a subdirectory: ftp://ftp.gnu.org/pub/gnu/bash/bash-4.3-patches/ This is not mentioned on the main bash webpage here: http://www.gnu.org/software/bash/ Could there please be a new full tarball release of the patched version ? Thanks, Ian.
Re: Shellshock-vulnerable version still most obvious on ftp.gnu.org
Chet Ramey writes ("Re: Shellshock-vulnerable version still most obvious on ftp.gnu.org"): > I will put tarballs with patches in the usual places within a few days. Thanks, that would be very helpful. For the future, it might be worth considering whether it's really sensible, nowadays, to be distributing bash as `.0 tarball with patches'. That made sense when bandwidth was much scarcer, disks (and backup systems) much smaller in relation to source code releases, and when most people would get bash directly from ftp.gnu.org. But in the current environment it's looking rather quaint. We could probably provide a full tarball for each patch release. Ian.
Re: Shellshock-vulnerable version still most obvious on ftp.gnu.org
Chet Ramey writes ("Re: Shellshock-vulnerable version still most obvious on ftp.gnu.org"): > On 11/6/14, 7:47 AM, Ian Jackson wrote: > > But in the current environment it's looking rather quaint. We could > > probably provide a full tarball for each patch release. > > That is supposed to be one of the advantages of using git. You can always > get a tarball of the latest release with all patches applied using > > http://git.savannah.gnu.org/cgit/bash.git/snapshot/bash-master.tar.gz Right. That's great. But that's not the official primary distribution channel for bash, as I understand it. Thanks, Ian.
Want way to run background processes with SIGINT unignored
Hi. I've been wrestling recently[1] with a bash script which invokes a number of subprocesses in parallel and collects the output. The problem is that if you ^C the script, the subprocesses carry on running. This is because of the standards-mandated resetting of SIGINT (and QUIT) to SIG_IGN in children. Working around this in a race-free way with additional code in the script is very hard indeed. I can't see how to do it without having the parent install an INT trap handler which synchronises with all the children, or something equally baroque. The reason for SIGINT being ignored is purely historical: back in the dawn of time, there was no job control. If you interactively spawned a background process with &, you wouldn't want your attempts to ^C your foreground process to kill it. This SIGINT-ignoring also applied to noninteractive shells and of course came to be relied on. So it is too late to change the default :-/. However, it would be very easy for bash to provide an option (via `set -o' perhaps) to disable this behaviour. That is, to allow SIGINT to be delivered normally to child processes. With such an option, scripts which run on modern systems and which attempt to parallelise their work, would be able to arrange that ^C properly cleans up the whole process group, rather than leaving the background tasks running (doing needless work and perhaps causing lossage). I suggest that this option should have two possible modes: 1. Current behaviour (the default): SIGINT and SIGQUIT are set to SIG_IGN in the child shortly after fork. 2. In the child, reset SIGINT and SIGQUIT to the values found at shell startup. That is, uninstall trap handlers. This is what most ordinary scripts will want, because they don't want the trap handler running in both parent and child. It's the same as is done for all other signals, and for all signals in non-background subshells and subprocesses. If this sounds like a good idea, I'm happy to write the patch. Please tell me what these options should be called :-). I suggest: set -o nobgchildsigint # posix behaviour set -o bgchildsigint # reset to shell's inherited dispositions I suggest that these options do the same for both signals. Ie that it wouldn't be possible to specify the inheritance separately for INT and QUIT. This whole discussion, and these proposed options, are relevant only when job control is not in effect. Thanks, Ian. [1] http://lists.xenproject.org/archives/html/xen-devel/2015-10/msg01208.html http://lists.xenproject.org/archives/html/xen-devel/2015-10/msg01211.html
Re: Want way to run background processes with SIGINT unignored
Chet Ramey writes ("Re: Want way to run background processes with SIGINT unignored"): > On 10/9/15 2:42 PM, Ian Jackson wrote: > > However, it would be very easy for bash to provide an option (via `set > > -o' perhaps) to disable this behaviour. That is, to allow SIGINT to > > be delivered normally to child processes. > > I'm restricting non-standard options to `shopt' to avoid any possible > conflict with future posix changes. Right. > > With such an option, scripts which run on modern systems and which > > attempt to parallelise their work, would be able to arrange that ^C > > properly cleans up the whole process group, rather than leaving the > > background tasks running (doing needless work and perhaps causing > > lossage). > > I'd be willing to look at a patch that implemented a new option to > enable this. You only need one option; you only have two behavior > modes and you're introducing one new behavior. OK. > > 2. In the child, reset SIGINT and SIGQUIT to the values found at > > shell startup. That is, uninstall trap handlers. This is what > > most ordinary scripts will want, because they don't want the trap > > handler running in both parent and child. It's the same as is > > done for all other signals, and for all signals in non-background > > subshells and subprocesses. > > This is the behavior that any new option would toggle. Some name like > `async_sig_ignore' or `async_sig_restore' would work. Thanks for your attention and your feedback; I'll get back to you with a patch. FWIW I agree that Linda Walsh seems to be talking about something different; I don't have a clear opinion right now about her observations, but I don't intend to do anything about trying to improve the area she's talking about: Linda Walsh writes ("Re: Want way to run background processes with SIGINT unignored"): > no? they want to run background processes and not have > the sig handlers mangled (and presumably that means getting the > signals in real time -- not waiting for the users next keypress!) With my proposed new option enabled, the affected signals (INT and QUIT) would be set to DFL in the child. This will mean that the child is terminated immediately by the kernel when the signal is sent. In most cases, the programmer doesn't mind whether the resulting SIGCHLD etc. for the parent is dealt with now, or later (when the parent script calls the `wait' builtin). > Requiring timely delivery of said signals, would be required for > their use case of " a bash script which invokes a > number of subprocesses in parallel and collects the output. ". If > the foreground process is also waiting for user input to provide > interactivity, will the SIGCHLD events come in when the children > terminate or when the foreground process leaves readline? I'm talking about noninteractive shells here. (Interactive shells have job control, and their background children do not have INT etc. ignored.) I don't think a noninterative shell is likely to be in readline, although it might be waiting in `read', or for a foreground subprocess to take user input, in which case when a SIGINT comes in the parent's handling of the death of its background children will indeed be delayed until the shell gets round to it. I think this is fine (even, usually, desirable). Regards, Ian.
Re: Want way to run background processes with SIGINT unignored
Ian Jackson writes ("Re: Want way to run background processes with SIGINT unignored"): > Chet Ramey writes ("Re: Want way to run background processes with SIGINT > unignored"): > > This is the behavior that any new option would toggle. Some name like > > `async_sig_ignore' or `async_sig_restore' would work. > > Thanks for your attention and your feedback; I'll get back to you with > a patch. This patch is nearly ready and I will send it shortly. While I was working on the docs aspect I noticed an infelicity in the documentation. It generally talk about the signal disposition that the shell `inherits from its parent'. For example: traps caught by the shell are reset to the values inherited from the shell's parent, and traps ignored by the shell are ignored But this is misleading. The values that the signals are reset to are those which the parent inherited _at shell invocation_ (ie, at exec). If the shell is a subshell (produced with `( )' perhaps), it may have inherited different settings from its _parent_ (ie, the main shell). I think this wording should be changed throughout to read `inherited at shell invocation'. Do you agree ? Would you welcome a patch along those lines ? Thanks, Ian.
[PATCH] Provide shopt -s no_async_sig_ignore
This option disables the standards-mandated resetting of SIGINT (and QUIT) to SIG_IGN in asynchronous children. This resetting makes it very hard to write a bash script which invokes a number of subprocesses in parallel and collects the output. The problem is that if you ^C the script, the subprocesses carry on running (doing needless work and perhaps causing lossage). Working around this in a race-free way with additional code in the script is very hard indeed. However, we can provide an option to do the right thing in modern programs. (The reason for SIGINT being ignored is purely historical: back in the dawn of time, there was no job control. If you interactively spawned a background process with &, you wouldn't want your attempts to ^C your foreground process to kill it. This SIGINT-ignoring also applied to noninteractive shells and of course came to be relied on. So it is too late to change the default.) Signed-off-by: Ian Jackson Signed-off-by: Ian Jackson CC: Ian Campbell --- MANIFEST| 3 +++ builtins/shopt.def | 2 ++ doc/bash.1 | 10 ++ doc/bashref.texi| 13 - execute_cmd.c | 5 - tests/asyncsigint.right | 6 ++ tests/asyncsigint.tests | 27 +++ tests/run-asyncsigint | 2 ++ tests/shopt.right | 3 +++ 9 files changed, 69 insertions(+), 2 deletions(-) create mode 100644 tests/asyncsigint.right create mode 100644 tests/asyncsigint.tests create mode 100644 tests/run-asyncsigint diff --git a/MANIFEST b/MANIFEST index 55c8d1c..1c3106c 100644 --- a/MANIFEST +++ b/MANIFEST @@ -841,6 +841,8 @@ tests/assoc4.subf tests/assoc5.sub f tests/assoc6.sub f tests/assoc7.sub f +tests/asyncsigint.testsf +tests/asyncsigint.rightf tests/braces.tests f tests/braces.right f tests/builtins.tests f @@ -1107,6 +1109,7 @@ tests/run-arith f tests/run-arrayf tests/run-array2 f tests/run-assocf +tests/run-asyncsigint f tests/run-braces f tests/run-builtins f tests/run-case f diff --git a/builtins/shopt.def b/builtins/shopt.def index 6050a14..89a9aae 100644 --- a/builtins/shopt.def +++ b/builtins/shopt.def @@ -90,6 +90,7 @@ extern int autocd; extern int glob_star; extern int glob_asciirange; extern int lastpipe_opt; +extern int no_async_sig_ignore_opt; #if defined (EXTENDED_GLOB) extern int extended_glob; @@ -199,6 +200,7 @@ static struct { #endif { "login_shell", &shopt_login_shell, set_login_shell }, { "mailwarn", &mail_warning, (shopt_set_func_t *)NULL }, + { "no_async_sig_ignore", &no_async_sig_ignore_opt, (shopt_set_func_t *)NULL }, #if defined (READLINE) { "no_empty_cmd_completion", &no_empty_command_completion, (shopt_set_func_t *)NULL }, #endif diff --git a/doc/bash.1 b/doc/bash.1 index 2b1ca15b..f318ab1 100644 --- a/doc/bash.1 +++ b/doc/bash.1 @@ -9535,6 +9535,16 @@ If set, and a file that \fBbash\fP is checking for mail has been accessed since the last time it was checked, the message ``The mail in \fImailfile\fP has been read'' is displayed. .TP 8 +.B no_async_sig_ignore +If set in a noninteractive shell, asynchronous commands will +not have +.B SIGINT +and +.BR SIGQUIT +ignored; +instead, those signals' handlers will be reset to those +inherited by the shell from its parent, as for other signals. +.TP 8 .B no_empty_cmd_completion If set, and .B readline diff --git a/doc/bashref.texi b/doc/bashref.texi index 597274b..2556b60 100644 --- a/doc/bashref.texi +++ b/doc/bashref.texi @@ -3060,7 +3060,10 @@ Non-builtin commands started by Bash have signal handlers set to the values inherited by the shell from its parent. When job control is not in effect, asynchronous commands ignore @code{SIGINT} and @code{SIGQUIT} in addition to these inherited -handlers. +handlers, +unless the shell option @code{no_async_sig_ignore} +(see the description of @code{shopt} in @ref{The Shopt Builtin}) +is enabled. Commands run as a result of command substitution ignore the keyboard-generated job control signals @code{SIGTTIN}, @code{SIGTTOU}, and @code{SIGTSTP}. @@ -5146,6 +5149,14 @@ If set, and a file that Bash is checking for mail has been accessed since the last time it was checked, the message @code{"The mail in @var{mailfile} has been read"} is displayed. +@item no_async_sig_ignore +If set in a noninteractive shell, asynchronous commands will not +have +@code{SIGINT} and @code{SIGQUIT} +ignored; +instead, those signals' handlers will be reset to those +inherited by the shell from its parent, as for other signals. + @item no_empty_cmd_completion If set, and Readline is being used, Bash will not attempt to search the @env{PATH} for possible completions when completion is attempted diff --git a/execute_cmd.c b/execute_cmd.c index 9cebaef..c3
doc improvement for dotglob
The docs currently say: "When a pattern is used for filename expansion, the character '.' at the start of a filename or immediately following a slash must be matched explicitly, unless the shell option 'dotglob' is set. The last bit should be changed to say unless the shell option 'dotglob' is set and the file is not named '.' or '..'. And dotglob's dedicated section has similar wording to change. I looked at making a patch, but it seems the docs are repeated in many formats, so I figured I'd just post this first. -- Ian Kelling https://iankelling.org
Re: doc improvement for dotglob
On Tue, Sep 27, 2016, at 09:41 AM, L. A. Walsh wrote: > > > Ian Kelling wrote: > > The docs currently say: > > > > "When a pattern is used for filename expansion, the character '.' at > > the start of a filename or immediately following a slash must be matched > > explicitly, unless the shell option 'dotglob' is set. > > > > The last bit should be changed to say > > > > unless the shell option 'dotglob' is set and the file is not named '.' > > or '..'. > > > --- > But that's not what is supposed to happen. It doesn't say it > matches the > filename with "." [or ".."], but the *start* of a filename or after a > "/". > I.e., interactive shell usage should not include > ".profile/.foo/.anything" in > such expansions. I'm not sure your reading it quite right. Chet gets it. Another way of explaining it is: dotglob does not affect the files "." and "..".
Re: doc improvement for dotglob
On Tue, Sep 27, 2016, at 11:08 AM, Chet Ramey wrote: > > Thanks for the report. There should be something in there about `.' > and `..' always having to be matched explicitly. Yw. I noticed another improvement that could be made to the docs. How do you prefer patches? And, I assume you manually deal with the duplication in different doc formats, is that right?
Re: aliases sometimes not expanded even with expand_aliases set
Thanks for that. It is certainly confusing that you can do "alias -p" inside the compound statement and have it print out the alias you think you have defined, but not have it work. But given that bit of the manual, I can see why that would be. On Sat, 2024-10-12 at 15:18 -0400, Chet Ramey wrote: > On 10/12/24 9:31 AM, i...@beware.dropbear.id.au wrote: > > > Bash Version: 5.2 > > Patch Level: 26 > > Release Status: release > > > > Description: > > > > In the following script, the first alias fails with "comand not found" > > but the second works. > > This is covered in the manual (bash.1): > > Bash > always reads at least one complete line of input, > and all lines that make up a compound command, > before executing any of the commands on that line or the compound command. > Aliases are expanded when a command is read, not when it is executed. > Therefore, an > alias definition appearing on the same line as another > command does not take effect until the shell reads the next line of input, > and an alias definition in a compound command does not take > effect until the shell parses and executes the entire compound command. > The commands following the alias definition > on that line, > or in the rest of a compound command, > are not affected by the new alias. > This behavior is also an issue when functions are executed. > Aliases are expanded when a function definition is read, > not when the function is executed, because a function definition > is itself a command. > As a consequence, aliases > defined in a function are not available until after that > function is executed. > > > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer >``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
bash completion mangles file names with newlines
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -g -O2 -fdebug-prefix-map=/build/bash-a6qmCk/bash-5.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -Wno-parentheses -Wno-format-security uname output: Linux ubuntu20-04 5.13.0-30-generic #33~20.04.1-Ubuntu SMP Mon Feb 7 14:25:10 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 5.0 Patch Level: 17 Release Status: release Description: BASH completion cannot correctly handle file names containing newline characters. Repeat-By: # First, get a shell with no completion loaded and show it working: $ bash --norc bash-5.0$ ls -b one\ntwo\nthree\nfour\nfive\nsix bash-5.0$ xxx 'one two three four five six' bash-5.0$ touch foo bash-5.0$ xxx foo one^Jtwo^Jthree^Jfour^Jfive^Jsix bash-5.0$ xxx 'one two three four five six' # Now, load the completion scripts and watch it break: bash-5.0$ source /usr/share/bash-completion/bash_completion bash-5.0$ ls -b foo one\ntwo\nthree\nfour\nfive\nsix bash-5.0$ xxx five foofour onesixthree two bash-5.0$ xxx o five four onesixthree two # The last two completions are garbage. # The file name is being split on newlines. -- | Ian! D. Allen, BA-Psych, MMath-CompSci idal...@idallen.ca Ottawa CANADA | Home: www.idallen.com Contact Improvisation Dance: www.contactimprov.ca | Former college professor of Free/Libre GNU+Linux @ teaching.idallen.com | Improve democracy www.fairvote.ca and defend digital freedom www.eff.org
Re: bash completion mangles file names with newlines
On Wed, Feb 23, 2022 at 03:37:12PM -0500, Chet Ramey wrote: > As others have noted, bash-completion is a separate package. > But we might be able to figure out where it's not being appropriately > quoted if you enable `set -x' before attempting the completion. At > least that would give you a backtrace to send with your bug report. Aye, I put the same bug report into the bash-completion GitHub and the backtrace is there: https://github.com/scop/bash-completion/issues/704 -- | Ian! D. Allen, BA-Psych, MMath-CompSci idal...@idallen.ca Ottawa CANADA | Home: www.idallen.com Contact Improvisation Dance: www.contactimprov.ca | Former college professor of Free/Libre GNU+Linux @ teaching.idallen.com | Improve democracy www.fairvote.ca and defend digital freedom www.eff.org
Re: set -o notify spurious CR when jobs redirected to file
On Fri, Nov 14, 2014 at 10:30:12AM -0500, Chet Ramey wrote: > Yes, it's intended. Piotr identified the code that does this. If the > shell is currently interactive, it can't be sure under what circumstances > it's printing a job notification. If the shell is printing the job notification because a user has typed "jobs" at the command line, there is no uncertainty on the part of the user that the resulting output does not need the extra CR. Perhaps the shell code isn't written to recognize that the request comes from a "jobs" command, but there is no uncertainty from the perspective of the user typing "jobs". Output generated directly as a result of typing "jobs" does not need the extra CR. Now, I can imagine implementations of "jobs" that might make it hard to communicate this case to the job notify code, but that would be a shortcoming in the recognition of the circumstances, not a lack of clarity in what the circumstances are. In short, it's still a bug. One might mitigate the bug by refusing to add the CR if output is not a TTY. That would refrain from creating what look like DOS-format files. -- | Ian! D. Allen - idal...@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/