Re: About `M-C-e` expand result `'` failed

[Chet Ramey]

On 2/3/24 7:44 AM, Martin D Kealey wrote:

On Wed, 31 Jan 2024 at 01:04, Andreas Schwab  wrote:


On Jan 30 2024, Zachary Santer wrote:

There's no way this is the intended behavior, right?


The command is doing exactly what it is documented to do, that is do all
of the shell word expansions.



If that's how the documentation is interpreted, then clearly there's a bug
in the documentation, because this is almost never *useful* behaviour.

It actually says:


shell-expand-line (M-C-e) ... This performs alias and history expansion

as well as all of the shell word expansions.

It does *not* say that it does quote removal.


So your position is that quote removal is not one of the shell word
expansions?


It could be argued that quote removal is implicitly required for some of
the steps to proceed, but in that case it can equally be argued that such
removal must therefore be reversed afterwards.


If you want to undo the word expansions, use the readline undo command
(C-_ in emacs mode). You'll get an idea of what the shell will do with
the command, and your command as you entered it will be restored.



By all means, add a "strip-quotes" command to readline, so the user can use
in the exceptional cases where you want to diverge from what the shell
would have done without M-C-e,


The result is exactly what the shell would have done when it expanded the
command.



but by default any expansion or substitution
that's triggered by a readline command should render a result that's immune
to that same expansion being done again when the user hits enter.


By all means, advocate for a quote-words readline command.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Feature request: prompt strings in output from edit-and-execute-command readline function ( was About `M-C-e` expand result `'` failed )

[Chet Ramey]

On 2/3/24 10:43 PM, Oğuz wrote:


I want to make one too. Multiline commands retrieved from history can be
short-circuited like this:

   $ ls
   x  y
   $ echo '
   > rm *
   > ' >z
   $
   $ ls
   x  y  z
   $ echo ''
   rm *
   ' >z

   > ^C
   $ ls
   $

After the second invocation of `ls' I add a single quote and hit enter
accidentally, and lose all my files. I wish it didn't happen


What's the specific request here?


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Feature request: prompt strings in output from edit-and-execute-command readline function ( was About `M-C-e` expand result `'` failed )

[Oğuz]

On Tuesday, February 6, 2024, Chet Ramey  wrote:
>
> What's the specific request here?
>

That bash interpret multiline commands retrieved from command history as a
whole and not run the rest of the lines as separate commands when the first
line is changed. Try it yourself; run a multiline command, recall it and
edit the first line to introduce a syntax error and hit enter. Does it look
right? Wouldn't it be much better if the entire command were discarded?


-- 
Oğuz

Re: About `M-C-e` expand result `'` failed

[Chet Ramey]

On 2/3/24 10:58 PM, Martin D Kealey wrote:

On Sun, 4 Feb 2024, 02:01 Koichi Murase,  wrote:


I now think I should leave a comment because even Martin (who I believed
was one of the careful people about backward compatibility as seen in
[1,2]) seems to suggest a breaking change.



That's a fair point.

I am generally concerned about breaking changes that affect existing
scripts, but I see changes to readline as less problematic, since an
interactive user gets the chance to check the replacement before hitting
enter.

Yes users have to learn new behaviours, which isn't ideal, but if my
suggested "unquote" ("dequote"? "strip-quoting"?) bindable function was
also added, the impact on users would be minimal: press M-C-e and then a
second keypress to remove quotes, bringing the input buffer to the same
state as would occur under the current arrangement.


Or you could leave shell-expand-line alone, which allows users to review
the expanded line and undo it if they wish, and add a second bindable
function -- which would add the same keypress -- to quote the words.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: About `M-C-e` expand result `'` failed

[Chet Ramey]

On 2/4/24 2:35 AM, Martin D Kealey wrote:


PS: Sadly M-C-r seems to be already taken, so I can't just hop one key over.


You can rebind it, you know. Anyway, let's assume the existence of a new
bindable command that (shell) quotes the (shell? readline?) words on a
line. Let's assume that this new command is bound to some sequence like
\C-xq. Then you can use a macro to create something like what you want:

"\M-\C-w":"\M-\C-e\C-xq"

with each command being separately undoable.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: About `M-C-e` expand result `'` failed

[Zachary Santer]

On Tue, Feb 6, 2024 at 2:35 PM Chet Ramey  wrote:

> and add a second bindable function -- which would add the same keypress --
> to quote the words.
>

Would Bash be saving what the line looked like before it performed quote
removal with M-C-e, and replacing what's on the command-line with that when
this other readline function is invoked? All this talk of a quote-line
readline function doesn't make any sense if Bash doesn't know how you want
it to be quoted.

This seems so obvious that I must be missing something.

Re: Feature request: prompt strings in output from edit-and-execute-command readline function ( was About `M-C-e` expand result `'` failed )

[Chet Ramey]

On 2/3/24 10:18 PM, Zachary Santer wrote:


While I'm making feature requests.

I hit C-x C-e and enter the following into my editor:

var='duck'
declare -p var
(
   var='squirrel'
   declare -p var
)
declare -p var

I save that and exit my editor, and this is what I get in the terminal:

zsant@Zack2021HPPavilion MINGW64 ~
$
var='duck'
declare -p var
declare -- var="duck"
(
   var='squirrel'
   declare -p var
)
declare -- var="squirrel"
declare -p var
declare -- var="duck"

Kinda confusing, right?

I could see adding PS1 and PS2 prompts where they would've been, had I 
typed all my commands into the terminal directly. PS2 prompts in front of 
everything that was entered in the editor might be preferable, though, 
given the complex PS1s you see sometimes.


I'd have to think about it. This isn't interactive input read from the
keyboard, where you'd expect to get a prompt. This is more like sourcing
a file with `set -v' temporarily enabled (in fact, it's very close to
that).

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: About `M-C-e` expand result `'` failed

[Chet Ramey]

On 2/6/24 3:04 PM, Zachary Santer wrote:
On Tue, Feb 6, 2024 at 2:35 PM Chet Ramey > wrote:


and add a second bindable function -- which would add the same keypress
-- to quote the words.


Would Bash be saving what the line looked like before it performed quote 
removal with M-C-e,


It already makes the shell expansions an undoable operation, I think that
would suffice. I'm not talking about changing the behavior of M-C-e, I'm
talking about adding a different function that you could use separately as
well.

and replacing what's on the command-line with that when 
this other readline function is invoked? All this talk of a quote-line 
readline function doesn't make any sense if Bash doesn't know how you want 
it to be quoted.


There are other places (e.g., ${var@Q} where bash chooses the most 
appropriate form of quoting. Why not here?


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Feature request: prompt strings in output from edit-and-execute-command readline function ( was About `M-C-e` expand result `'` failed )

[Chet Ramey]

On 2/6/24 1:39 PM, Oğuz wrote:
On Tuesday, February 6, 2024, Chet Ramey > wrote:


What's the specific request here?


That bash interpret multiline commands retrieved from command history as a 
whole and not run the rest of the lines as separate commands when the first 
line is changed.


Ah, I see: you want any "lines" that readline returns that contain embedded
newlines to be parsed as a compound_list instead of a series of newline-
terminated commands (what bash calls a simple_list). Kind of like they had
been enclosed in braces. I'm not sure I could do that much violence to the
parser.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: About `M-C-e` expand result `'` failed

[Mike Jonkmans]

On Tue, Feb 06, 2024 at 02:59:21PM -0500, Chet Ramey wrote:
> On 2/4/24 2:35 AM, Martin D Kealey wrote:
> 
> > PS: Sadly M-C-r seems to be already taken, so I can't just hop one key over.
> 
> You can rebind it, you know. Anyway, let's assume the existence of a new
> bindable command that (shell) quotes the (shell? readline?) words on a
> line. Let's assume that this new command is bound to some sequence like
> \C-xq. Then you can use a macro to create something like what you want:
> 
> "\M-\C-w":"\M-\C-e\C-xq"
> 
> with each command being separately undoable.

Assuming you want M-C-e, C-xq, Return to be a equivalent to a verbose Return.
I don't think it is doable that way, please consider:

$ printf "<%s>\n" x\ y


$ printf "<%s>\n" x\ y ## M-C-e

$ printf <%s>\n x y ## C-xq (let's assume it quotes the redirections)

$ printf "<%s>\n" x y




The original words before quote removal should be requoted.

Also when expansion introduces quotes:
$ x=\"
$ printf "$x\n" ## M-C-e-new (shell-expand-quoted?)
$ printf "\"\n"

$ printf $x\\n ## M-C-e-new
$ printf \"\\n


Oh well, I am happy with the undo :)

-- 
Regards, Mike Jonkmans

Re: About `M-C-e` expand result `'` failed

[Zachary Santer]

On Tue, Feb 6, 2024 at 3:13 PM Chet Ramey  wrote:

> There are other places (e.g., ${var@Q} where bash chooses the most
> appropriate form of quoting. Why not here?


zsant@Zack2021HPPavilion MINGW64 ~
$ animal='dog'

zsant@Zack2021HPPavilion MINGW64 ~
$ action='chases the cat'

zsant@Zack2021HPPavilion MINGW64 ~
$ printf '%s\n' "${animal}" "${action}"
dog
chases the cat

zsant@Zack2021HPPavilion MINGW64 ~
$ printf %s\n dog chases the cat # Same command as above, but with M-C-e
dognchasesnthencatn
zsant@Zack2021HPPavilion MINGW64 ~
$ printf '%s\n' "${animal@Q}" "${action@Q}"
'dog'
'chases the cat'

When Bash is looking at a line like
$ printf %s\n dog chases the cat
where should it choose to add quotes?

Compare this scenario to a hypothetical readline command that transforms
the command line so as to show the results of expansions, but leaving
quoting in place and adding backslashes, such that the end result of
evaluating the transformed line is identical to what would've happened had
the line not been transformed.

I accept the argument that you can just undo the M-C-e operation, though.

Re: Feature request: prompt strings in output from edit-and-execute-command readline function ( was About `M-C-e` expand result `'` failed )

[Zachary Santer]

On Tue, Feb 6, 2024 at 3:07 PM Chet Ramey  wrote:

> This is more like sourcing a file with `set -v' temporarily enabled (in
> fact, it's very close to that).
>

Can't imagine POSIX would be amenable to all the 'set -v' output lines
being prepended with PS2s, but that actually suffers from the same
ambiguity if stdout and stderr are going to the same place. Not that people
use 'set -v' enough to complain.

[PATCH] retry opening startup files on EINTR

[Grisha Levit]

I have some dotfiles symlinked to storage backed by a macOS File
Provider extension (e.g. Dropbox):

$ realpath ~/.bash_profile
/Users/levit/Library/CloudStorage/Dropbox/profile/.bash_profile

This normally works fine, except when my terminal emulator (tested
both Terminal.app and iTerm) restores sessions after being restarted
-- opening the startup file fails in about half of the restored
sessions, which ends up looking like:

[Restored Feb 7, 2024 at 00:11:37]
Last login: Wed Feb  7 00:11:37 on ttys008
Restored session: Wed Feb  7 00:11:27 EST 2024
-bash: /Users/levit/.bash_profile: Interrupted system call
mbp:~ levit$

If I remove ~/.bash_profile, and make ~/.inputrc and ~/.bash_history
similar symlinks, I see the same issue with loading them (without an
error message).

I'm not sure what the underlying cause is here, but maybe it would be
appropriate to retry open(2) calls for these files if they fail with
EINTR?

diff --git a/bashhist.c b/bashhist.c
index 9e762057..aeaa5234 100644
--- a/bashhist.c
+++ b/bashhist.c
@@ -336,7 +336,8 @@ load_history (void)

   if (hf && *hf && file_exists (hf))
 {
-  read_history (hf);
+  while (read_history (hf) == EINTR)
+   QUIT;
   /* We have read all of the lines from the history file, even if we
 read more lines than $HISTSIZE.  Remember the total number of lines
 we read so we don't count the last N lines as new over and over
diff --git a/builtins/evalfile.c b/builtins/evalfile.c
index 6a242bda..60adca34 100644
--- a/builtins/evalfile.c
+++ b/builtins/evalfile.c
@@ -106,7 +106,8 @@ _evalfile (const char *filename, int flags)
 #  endif
 #endif

-  fd = open (filename, O_RDONLY);
+  while ((fd = open (filename, O_RDONLY)) < 0 && errno == EINTR)
+QUIT;

   if (fd < 0 || (fstat (fd, &finfo) == -1))
 {
diff --git a/lib/readline/bind.c b/lib/readline/bind.c
index 47478f08..44787cfe 100644
--- a/lib/readline/bind.c
+++ b/lib/readline/bind.c
@@ -968,8 +968,10 @@ _rl_read_file (char *filename, size_t *sizep)
   char *buffer;
   int i, file;

-  file = -1;
-  if (((file = open (filename, O_RDONLY, 0666)) < 0) || (fstat (file,
&finfo) < 0))
+  while ((file = open (filename, O_RDONLY, 0666)) < 0 && errno == EINTR)
+RL_CHECK_SIGNALS ();
+
+  if ((file < 0) || (fstat (file, &finfo) < 0))
 {
   if (file >= 0)
close (file);

[PATCH] ASS_NOEVAL for BASHOPTS and SHELLOPTS

[Grisha Levit]

(Prompted by the report from Emanuele Torre in [1])

The value of currently_executing_command is garbage when there is an error
during assignment when popping the variable context:

$ bash -c 'declare -i SHELLOPTS; f() { local -; set -f; }; f' |& cat -v
bash: line 1: M-`^WM-R^N^\{: braceexpand:hashall:interactive-comment...

$ bash-asan -c 'declare -i SHELLOPTS; f() { local -; set -f; }; f'
SUMMARY: AddressSanitizer: heap-use-after-free execute_cmd.c:399 in
executing_line_number

[1]: https://lists.gnu.org/archive/html/bug-bash/2024-01/msg00111.html

I'm not sure that allowing the integer attribute to be changed really makes
sense for readonly variables but, in any case, the following should prevent
the issue here:

diff --git a/builtins/set.def b/builtins/set.def
index e0024a97..05f3f942 100644
--- a/builtins/set.def
+++ b/builtins/set.def
@@ -578,7 +578,7 @@ set_shellopts (void)
   exported = v ? exported_p (v) : 0;

   /* ASS_FORCE so we don't have to temporarily turn off readonly */
-  v = bind_variable ("SHELLOPTS", value, ASS_FORCE);
+  v = bind_variable ("SHELLOPTS", value, ASS_FORCE|ASS_NOEVAL);

   /* Turn the read-only attribute back on, and turn off the export attribute
  if it was set implicitly by mark_modified_vars and SHELLOPTS was not
diff --git a/builtins/shopt.def b/builtins/shopt.def
index e6c77cc5..95459bbc 100644
--- a/builtins/shopt.def
+++ b/builtins/shopt.def
@@ -854,7 +854,7 @@ set_bashopts (void)
   exported = v ? exported_p (v) : 0;

   /* ASS_FORCE so we don't have to temporarily turn off readonly */
-  v = bind_variable ("BASHOPTS", value, ASS_FORCE);
+  v = bind_variable ("BASHOPTS", value, ASS_FORCE|ASS_NOEVAL);

   /* Turn the read-only attribute back on, and turn off the export attribute
  if it was set implicitly by mark_modified_vars and SHELLOPTS was not

declare -Aa var

[Grisha Levit]

If given both the -a and -A flags and an existing scalar variable,
declare will assign both attributes to the variable:

$ V=X; declare -Aa V; echo $?
0
$ echo ${V@a}
aA
$ (declare -p V)
Segmentation fault: 11

declare -A +A

[Grisha Levit]

If a single declare command both sets and unsets the array or assoc
attribute for an existing scalar variable, the `value' member of the
SHELL_VAR is assigned an ARRAY* or HASH_TABLE* as appropriate, but
later ends up treated as a char*:

$ bash-asan -c 'X=Y; declare -A +A X; declare -p X'
declare -- X=$'\200_\020\005\001'

$ bash-lsan -c 'X=Y; declare -A +A X; X=(Z)'
ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1024 byte(s) in 1 object(s) allocated from:
#0 0xda7342f8 in malloc
#1 0xda7f24d0 in xmalloc xmalloc.c:107:10
#2 0xda7b81fc in hash_create hashlib.c:72:25
#3 0xda7cc0c8 in convert_var_to_assoc arrayfunc.c:117:10

Maybe it would be appropriate to reject a request to turn off an
attribute that is being turned on?