Re: [PATCH] Switch vi-insert TAB binding only for the default binding

2019-02-17 Thread Chet Ramey
On 2/15/19 6:58 PM, Koichi Murase wrote:
>> There is a third option: turning posix mode on saves the old binding
>> for TAB; turning it off restores it. That, combined with a check of
>> whether or not the binding was still TAB:self-insert, should handle
>> both cases.
> 
> Thank you for your reply. I am fine with the third option as far as
> `local POSIXLY_CORRECT' safely works. 

I believe it does.

> Actually I initially thought
> about that option, but I felt it is a little bit inconsistent because
> with that option only the binding for TAB becomes POSIX compliant, and
> user-defined bindings for all the other keys/keymaps are kept
> user-defined in POSIX mode. 

Yes. Bash has to assume some things, and only restores the key bindings
that it itself changes. Users can modify anything at their own discretion;
bash only claims POSIX conformance if users don't modify the bindings.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: Failed exec resets job control state

2019-02-17 Thread Chet Ramey
On 2/15/19 7:32 PM, Grisha Levit wrote:
> An `exec` that can't execute the supplied command seems to cause the
> status of job control to become reset to the default state:
> 
>$ bash -O execfail -mc 'echo $-; exec xxx; echo $-'
>hmBc
>bash: line 0: exec: xxx: not found
>hBc
> 
>$ set +m; echo $-
>hiBHs
>$ exec xxx
>bash: exec: xxx: not found
>$ echo $-
>himBHs

Thanks for the report. It's an unusual set of circumstances where one
turns off job control in an interactive shell.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: info missing in output of declare -a in 4.4.x

2019-02-17 Thread L A Walsh



On 2/16/2019 4:57 AM, Robert Elz wrote:
> Date:Fri, 15 Feb 2019 22:21:25 -0800
> From:L A Walsh 
> Message-ID:  <5c67abe5.1030...@tlinx.org>
>
>   | Thought about thatrestarted a fresh shell.  Same same.
>   | At least I know it's supposed to...
>
> It isn't just BASH_ALIASES - you'll see the same from lots of
> other dynamic variables.
>   
---
Woe!  That's what threw me -- it says empty, but it isn't.
> Try actually accessing an element, as in ...
>
>
> On occasion, the output from declare will show all the
> aliases in BASH_ALIASES but I have no idea what sequence of
> commands, of state of the universe, causes that to happen.
>   
-

AND -- just did it now, it doesn't even show BASH_ALIASES, but
I know I have aliases defined.  Weird!

So it really is a bug of some sort, not that I use BASH ALIASES
for anything.  Was going to, but ...  and you're right, lots of
things aren't showing there.





Re: info missing in output of declare -a in 4.4.x

2019-02-17 Thread Dennis Williamson
On Sun, Feb 17, 2019, 3:10 PM L A Walsh 
>
> On 2/16/2019 4:57 AM, Robert Elz wrote:
> > Date:Fri, 15 Feb 2019 22:21:25 -0800
> > From:L A Walsh 
> > Message-ID:  <5c67abe5.1030...@tlinx.org>
> >
> >   | Thought about thatrestarted a fresh shell.  Same same.
> >   | At least I know it's supposed to...
> >
> > It isn't just BASH_ALIASES - you'll see the same from lots of
> > other dynamic variables.
> >
> ---
> Woe!  That's what threw me -- it says empty, but it isn't.
> > Try actually accessing an element, as in ...
> >
> >
> > On occasion, the output from declare will show all the
> > aliases in BASH_ALIASES but I have no idea what sequence of
> > commands, of state of the universe, causes that to happen.
> >
> -
>
> AND -- just did it now, it doesn't even show BASH_ALIASES, but
> I know I have aliases defined.  Weird!
>
> So it really is a bug of some sort, not that I use BASH ALIASES
> for anything.  Was going to, but ...  and you're right, lots of
> things aren't showing there.
>
>
>

Have you tried starting Bash without any startup files, creating an alias
and then checking the array?

>


Re: info missing in output of declare -a in 4.4.x

2019-02-17 Thread L A Walsh



On 2/17/2019 2:19 PM, Dennis Williamson wrote:
>
> So it really is a bug of some sort, not that I use BASH ALIASES
> for anything.  Was going to, but ...  and you're right, lots of
> things aren't showing there.
>
>
>  
>
> Have you tried starting Bash without any startup files, creating an
> alias and then checking the array?
yes.  Just now.
out of 14 vars starting with BASH
only BASHOPTS is different in the clean startup.
In a 'normal'[?] startup, there are 3 extra vars starting with BASH_:

BASH_COMPLETION_COMPAT_DIR
BASH_ENV (I set this, but unset it for this test)
BASH_REMATCH

then bash (type -p says /bin/bash)
 --noprofile --norc

Version would be same, and the debug related would be blank, but
hmmm
declare -A BASH_CMDS=()  (command hashing?)

most of the BASH vars show no value (exception of
version-related, BASH=/bin/bash & "BASHOPTS", but they
DO have a value.  If I print it in the shell instead
of using declare -p I see stuff.











Re: [PATCH] Fix a bug that `bind -x '"metachar":command'' fails when rl-variable `convert-meta' is `on'

2019-02-17 Thread Chet Ramey
On 2/11/19 1:57 AM, Koichi Murase wrote:

> Bash Version: 5.0
> Patch Level: 0
> Release Status: release
> 
> Description:
> 
>   When readline variable `convert-meta' is set to `on', `bind -x'
> including meta characters in its keyseq fails to bind.
> 
>   For an actual example, see the next `Repeat-By' section. An
> inconsistency comes from the condition whether to contain ESC (from
> meta chars) in the keyseq stored in keymaps. For example, in
> `rl_generic_bind' (lib/readline/bind.c), ESC is only contained when
> the keymap already has a child ISKMAP entry at ESC as quoted below:
> 
> if (META_CHAR (ic) && _rl_convert_meta_chars_to_ascii)
>   {
> ic = UNMETA (ic);
> if (map[ESC].type == ISKMAP) /* <-- this condition */
>   {
> prevmap = map;
> map = FUNCTION_TO_KEYMAP (map, ESC);
>   }
>   }
> 
>   In addition, the condition in `rl_translate_keyseq'
> (lib/readline/bind.c) always refers to the current top-level keymap
> `_rl_keymap' rather than the corresponding keymap entry in the target
> keymap tree, which appears to be unreasonable:
> 
> if (_rl_convert_meta_chars_to_ascii && _rl_keymap[ESC].type == ISKMAP)
> array[l++] = ESC; /* ESC is meta-prefix */
> 
>   `bind -x' stores data in two different keymaps: the selected keymap
> and the keymap `cmd_xmap' defined in `bashline.c'. When one has a
> child ISKMAP entry at ESC and the other doesn't have, the inconsisty
> arises.
> 
> Repeat-By:
> 
>   The readline variable `convert-meta' is enabled by default when Bash
> starts with `LANG=C'.
> 
> $ LANG=C bash-5.0 --norc
> $ bind -v | grep convert-meta
> set convert-meta on
> 
>   `bind -x' whose keyseq contains meta characters stores the command
> string to wrong places in `cmd_xmap'. In the following example, the
> command string for `\M-c' is stored to the placeholder for `c'.
> 
> $ bind -x '"c": "echo hello"'
> $ bind -x $'"\xE3": "echo world"'
> world # <-- somehow the binding for `\M-c' is invoked by typing `c'
> $
> bash-5.0: bash_execute_unix_command: cannot find keymap for
> command # <-- error is produced by typing `\M-c'
> $ bind -X
> "c": "echo world"
> $
> 
>   For reference, usual macro key bindings work as expected.
> 
> $ bind '"c": "hello"'
> $ bind $'"\xE3": "world"'
> $ echo helloworld # <-- `hello' and `world' is inserted by typing
> `c' and `\M-c'
> helloworld
> $ bind -s
> "\ec": "world"
> "c": "hello"
> 
> Fix:
> 
>   Actually I'm not sure if there is a reason why stored keyseqs
> contains ESC only when there is already ESC entry, but a possible
> solution would be to create a new ISKMAP entry when there is no ISKMAP
> entry at ESC. I attach a patch
> `0001-translate-meta-characters-when-convert-meta-is-set.patch'. I
> have changed `rl_bind_key' (lib/readeline/bind.c) so that the ISKMAP
> entry is explicitly created. I have changed `rl_translate_keyseq'
> (lib/readeline/bind.c) so that meta characters `c' are always
> translated to `ESC' + `UNMETA (c)' when `convert-meta' is set. In this
> way, the array `keys' in `rl_generic_bind' (lib/readeline/bind.c)
> would never contain meta characters when `convert-meta' is set, so the
> codes to check meta characters have been removed from
> `rl_generic_bind'.

I agree that this is a problem. I think an easier solution for rl_bind_key
is to simply create a new key sequence in KEYSEQ, with the first character
as ESC, then pass the whole thing off to rl_generic_bind, which will handle
creating the appropriate shadow keymaps if necessary (your second patch
tries to do this inline, it appears).

The other patches are good. Thanks for the detailed report. Please take a
look at the next devel branch push and make sure it fixes your issues.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: Failure to log in as root to serial console in bash 5.0; works in bash 4.4

2019-02-17 Thread Chet Ramey
On 2/16/19 10:55 AM, jpb...@jfroot.com wrote:

> Bash Version: 5.0
> Patch Level: 2
> Release Status: release
> 
> Description:
>   This all works with bash 4.4; just tested it with it, it works fine.
>   In bash 5.0, an attempt to login as root on a serial console on my
>   machine results in this failure:
> 
>   bash: initialize_job_control: no job control in background: Bad 
> file descriptor
> 
>...and ANY keypress results in an 'exit' being printed and the
>connection dropped. Logging in as any other user works fine, and then
>I can 'su' to root, and all works. Again, this works in 4.4. The code
>that prints this (in jobs.c) seems to be new for 5.0.

This seems to be a timing problem, though I don't know why it would happen
only to root.

This code runs when the shell thinks it's started in the background: job
control is enabled, or the shell thinks it should be enabled and is trying
to initialize it, and the shell's process group (obtained using getpgid)
and the terminal's process group are not the same. The assumption is that
the shell's parent will be around to take care of this and allow the shell
to continue in the foreground.

Instead of trying to read a character, which may or may not generate
SIGTTIN, the shell just resets the handler to SIG_DFL and sends SIGTTIN
to itself.

In bash-4.4, the code to handle this just looped forever, leading to
situations like this one:

http://lists.gnu.org/archive/html/bug-bash/2016-11/msg00137.html

The change that added the warning message capped the number of times this
happened at 16 -- after 16 SIGTTINs return without changing the terminal's
process group, there probably isn't an agent there to change it, and we'll
proceed without job control and take our chances with read(2).

The shell probably exits because it either ends up in an orphaned process
group (read returns -1/EIO) or tries to read from the background process
while ignoring or blocking SIGTTIN (read returns EOF).

I suspect it's a timing issue because bash-4.4, instead of waiting forever,
eventually breaks out of that loop with shell_pgrp == terminal_pgrp,
successfully finishes initializing job control, and goes on. I don't know
how to compute the right number of loops to delay that long.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: info missing in output of declare -a in 4.4.x

2019-02-17 Thread Dennis Williamson
On Sun, Feb 17, 2019, 6:01 PM L A Walsh 
>
> On 2/17/2019 2:19 PM, Dennis Williamson wrote:
> >
> > So it really is a bug of some sort, not that I use BASH ALIASES
> > for anything.  Was going to, but ...  and you're right, lots of
> > things aren't showing there.
> >
> >
> >
> >
> > Have you tried starting Bash without any startup files, creating an
> > alias and then checking the array?
> yes.  Just now.
> out of 14 vars starting with BASH
> only BASHOPTS is different in the clean startup.
> In a 'normal'[?] startup, there are 3 extra vars starting with BASH_:
>
> BASH_COMPLETION_COMPAT_DIR
> BASH_ENV (I set this, but unset it for this test)
> BASH_REMATCH
>
> then bash (type -p says /bin/bash)
>  --noprofile --norc
>
> Version would be same, and the debug related would be blank, but
> hmmm
> declare -A BASH_CMDS=()  (command hashing?)
>
> most of the BASH vars show no value (exception of
> version-related, BASH=/bin/bash & "BASHOPTS", but they
> DO have a value.  If I print it in the shell instead
> of using declare -p I see stuff.
>
>
>
>
>
>
>
>


Oh, interesting! In Bash 4 and 5, I just did declare -p with no args and it
showed BASH_ALIASES empty. But with declare -p BASH_ALIASES it shows the
contents (as does the alias command).

Other arrays that show this difference for me:

BASH_CMDS
DIRSTACK
GROUPS

Scalars that exhibit this issue:

BASHPID
BASH_COMMAND
BASH_SUBSHELL
COMP_WORDBREAKS
HISTCMD
LINENO
RANDOM
SECONDS


Re: ${b+s ''}

2019-02-17 Thread Chet Ramey
On 2/16/19 11:38 AM, sunnycemet...@gmail.com wrote:
> I would like to include a null string as part of a parameter expansion word:
> 
>> mapfile -t${bNullDelimited+d ''}
> 
> When bNullDelimited is set, I expect this to expand to:
> 
>> mapfile -td ''
> 
> However, it expands to:
> 
>> mapfile -td

There are mixed behaviors. The idea behind the bash behavior is that a
null string added to a non-empty word is simply discarded, and this happens
while the word after the `+' is being processed, before word splitting.

bash/ksh93/zsh do it one way, ash-derived shells/mksh/yash do it another.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: nested bash 5 interactive sessions without job control

2019-02-17 Thread Chet Ramey
On 2/15/19 9:43 PM, Grisha Levit wrote:
> When in an interactive shell with job control disabled, starting
> another interactive shell, disabling job control, and then exiting,
> causes the parent shell to exit after any key press.
> 
> bash-5.0$ set +m; $BASH --norc -i -c 'set +m'
> bash-5.0$ [any key]
> exit
> 
> This seems to work fine in bash 4.4 and below.

Thanks for the report. I'll fix it in the next devel branch push.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: bug-bash@gnu.org

2019-02-17 Thread Chet Ramey
On 2/14/19 4:20 PM, rugk wrote:
> Hi,
> regarding the paste security issues (pastejacking) [1] there is one last
> thing that shall be done to make it possible for terminal emulators to
> enable a secure shell by default: Enable bracket pasting mode in bash, by
> default.

That's a good reason to turn this on by default for the next version. In
the meantime, it's easy enough to turn on in a startup file for users (or
distributions) who are concerned about it.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: in bash 4.4.12,want to verify a behavior...

2019-02-17 Thread Chet Ramey
On 2/15/19 11:45 PM, L A Walsh wrote:
> 
> I printed the various declares using:
> 
> declare -p |& more
> 
> One of the early entries is:
> 
> declare -A BASH_ALIASES=()
> 
> Yet if I type 'alias |& wc, I see 56 lines starting with alias.
> 
> I _thought_ BASH_ALIASES was suppose to hold the aliases
> and for an "alias whence='type -a'", I should see something like
> declare -A BASH_ALIASES=([whence]='type -a ...) instead of
> the empty assignment above.

Not quite. Dynamic variables are only defined to return the correct value
when they are referenced. The rest of the time, the values are
indeterminate (and possibly stale). The idea is that you can get an element
given the right index, or all the elements with the `*' or `@' subscripts.

Dynamic scalar variables generally hang onto the last value generated
and generate a new value when referenced.

Dynamic array variables fall into two classes: objects bash maintains
internally as arrays (BASH_ARGV, BASH_SOURCE, BASH_LINENO, FUNCNAME) and
arrays that reflect structures maintained by other commands (BASH_ALIASES,
BASH_CMDS, DIRSTACK, GROUPS). The latter group is generated on demand when
a subscript is referenced. For that group, the array doesn't have any
contents until you ask for something. After you do, of course, the values
show up in `declare' output.

`declare -p' just lists variables and any existing values. It doesn't cause
values to be generated. `declare -p BASH_ALIASES' counts as asking for the
value.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



Re: in bash 4.4.12,want to verify a behavior...

2019-02-17 Thread Chet Ramey
On 2/16/19 7:57 AM, Robert Elz wrote:
> Date:Fri, 15 Feb 2019 22:21:25 -0800
> From:L A Walsh 
> Message-ID:  <5c67abe5.1030...@tlinx.org>
> 
>   | Thought about thatrestarted a fresh shell.  Same same.
>   | At least I know it's supposed to...
> 
> It isn't just BASH_ALIASES - you'll see the same from lots of
> other dynamic variables.
> 
> Try actually accessing an element, as in ...
> 
> jinx$ declare -p | grep BASH_A
> declare -A BASH_ALIASES=()
> declare -a BASH_ARGC=([0]="0")
> declare -a BASH_ARGV=()
> declare -- BASH_ARGV0
> jinx$ echo ${BASH_ALIASES["ap"]}
> addpath
> jinx$ declare -p | grep BASH_A
> declare -A BASH_ALIASES=()
> declare -a BASH_ARGC=([0]="0")
> declare -a BASH_ARGV=()
> declare -- BASH_ARGV0

$ declare -p | grep BASH_ALI
declare -A BASH_ALIASES=()
$ echo ${BASH_ALIASES[m]}
mv
$ declare -p | grep BASH_ALI
declare -A BASH_ALIASES=([jl]="jobs -l" [functions]="typeset -f"
[ll]="/bin/ls -lF" [dir]="/bin/ls -balF" [news]="aterm -g 110x45 -title trn
-e trn -e -S1 -N &" [lsc]="ls -C" [r]="fc -e -" [more]="less" [m]="mv"
[nohup]="nohup " [j]="jobs" [integer]="typeset -i" [ss]="ps -aux"
[h]="history 24" [ls]="/bin/ls -F" [errno]="perl -e '\$! = shift;print
\"\$!\\n\";'" [c]="clear" [lt]="ll -t" )


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/