Some proposals concerning bash manual

[Ingo]

Hello,

during my job related reentry into bash scripting after 23 years of work with 
other platforms (e.g. mainframes) I made some extensions/changes to a printed 
version of bash manual edition 4.4 of 2016 which might be useful also for 
others.

Here are they:

Page 18, section 3.4 Shell Parameters
...All values undergo tilde expansion, parameter and variable expansion, 
command substitution *), arithmetic expansion and quote removal (detailed 
below).
...
Footnote *) E.g. the output of a pipe may be assigned to a parameter using 
command substitution, see page 29.


Page 19, section 3.4.1 Positional Parameters
...The set and shift builtins are used to set and unset them (see Chapter 4 
[Shell Builin Commands], page 46).  (not 42)


Page 39, section 3.7.5 Exit Status
...
All builtins return an exit status of 2 to indicate incorrect usage, generally 
invalid options or missing arguments.
The exit status is accessible via special parameter $?, see page 20.


Page 42, section 4.1 Bourne Shell Builtins
...
. (a period)
...
If filename is not found, or cannot be read, the return status is non-zero. No 
execution right needed to be set for filename. No subshell opened so filename 
shouldn't contain final exit. This builtin is equivalent to source.


Page 46
...
return
...
set (see page 60)
shift
...



Page 171, section D.5 Concept Index
A
...
arrays...91
assigning values.18
assigning pipe output...18 (new footnote)
...
H
here document..35
history builtins..137
...


Hope this improves.

Best wishes and many thanks for your awesome and yet free work :-)
Ingo Kelmes, Cologne


-- 
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

Re: [BUG] Bash not reacting to Ctrl-C

[Ingo Molnar]

* Chet Ramey  wrote:

> On 2/8/11 4:17 PM, Oleg Nesterov wrote:
> 
> > Once again. If bash gets ^C and at the same time the current foreground
> > child exits normally (either because this jctl signal races with exit()
> > or because the child hooks SIGINT and exits after that) SIGINT is lost.
> > 
> > set_job_status_and_cleanup() insists that WTERMSIG(child->status) should
> > be SIGINT, iow the child should be killed by the same signal. Otherwise
> > it is not going to kill itself, and the next wait_for() clears
> > wait_sigint_received.
> > 
> > This all looks intentional, but this means ^C can never work reliably.
> 
> It depends on what you mean by `reliably'.  Consider a script that runs
> emacs, then does other processing when emacs completes.  Emacs uses SIGINT
> internally to interrupt editing commands, but handles it and does not exit
> as a result.  Since emacs is run from a script, and job control is not
> enabled, the shell receives the SIGINT also, because it is in the
> terminal's foreground process group.  Should the shell abort the script
> when emacs exits?

Could you try the reproducer please?

Once you run it, try to stop it via Ctrl-C, and try to do this a couple of 
times.

Do you consider it normal that it often takes 2-3 Ctrl-C attempts to interrupt 
that 
script, that it is not possible to stop the script reliably with a single 
Ctrl-C?

This issue occurs all the time on different hardware and on different 
distributions, 
during make jobs and other simpler scripts, etc. - i just simplified it down to 
a 
simple testcase.

Oleg also found another simple testcase i think - and Thomas (Cc:-ed) reported 
similar Ctrl-C problems with Bash as well.

Thanks,

Ingo

Re: [BUG] Bash not reacting to Ctrl-C

[Ingo Molnar]

* Oleg Nesterov  wrote:

> That is why I provided another test-case, let me repeat it:
> 
>   #!./bash
> 
>   perl -we '$SIG{INT} = sub {exit}; sleep'
> 
>   echo "Hehe, I am going to sleep after ^C"
>   sleep 100

This reliably reproduces the (formerly sporadic) script Ctrl-C bug here,
100% of the time:

 aldebaran:~> ./test-signal-perl.sh 
 ^CHehe, I am going to sleep after ^C

 [ it waits 100 seconds ]

Thanks,

Ingo

Re: [BUG] Bash not reacting to Ctrl-C

[Ingo Molnar]

* Chet Ramey  wrote:

> > So the above trace is one that my patch would have handled correctly
> > (since it has no EINTR). Maybe a combination of the two approaches
> > would work even better?
> 
> I installed what is essentially the union of your changes and mine.  Thanks
> for everyone's help with this.

Thanks! Mind attaching the final patch to this thread, so that people can try 
out 
and test the patch you mean?

Thanks,

Ingo

Command name dequote does not work

[ingo . krabbe]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/lib  
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat 
-Werror=format-security -Wall
uname output: Linux k210app1 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 
GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.2
Patch Level: 37
Release Status: release

Description:
I wrote a simple shell script using sed to convert \n into newlines and 
other \\(.) to \\\1 and called that script dequote.
When I'm trying to call it nothing happens, when I'm calling $HOME/bin/dequote 
it works. There's no alias called dequote, there's no other dequote in path.
When I rename the script and call it it just works as expected.

This looks like a secret command, that just echos its arguments, as calling 
"dequote xyz" echoes "xyz".

But having a secret alias or command is very evil. What's up here?


Repeat-By:
see above

Fix:
DonÃ't use secret commands. Any builtin has to be documented on the 
help page.

Re: Command name dequote does not work

[ingo . krabbe]

I finally found it. I forgot to check the function names. It's a function of 
bash-completion, though I don't think that such a function name should be 
exported.
--- Begin Message ---
Not sure we have enough info here. Have you tried "set -x" to see what's really 
going on? Have you also confirmed that it isn't actually calling your script 
and the script simply isn't functioning as expected?
-- 
Dan Douglas--- End Message ---

potential bash bug, weird script behavior, Linux, SIGCHLD

[Ingo Molnar]

Oleg Nesterov has distilled a very simple (and reproducable) testcase 
below for what appears to be a potential long-existing bash bug. This is 
a problem that triggers on Linux quite frequently. (i can also send the 
configs.tar.bz2 testcase i made - but i think Oleg's is far simpler) I 
used bash-3.2-19.fc8 for my tests, on Linux 2.6.24-0.39.rc3.git1.fc9.

    Ingo

- Forwarded message from Oleg Nesterov <[EMAIL PROTECTED]> -

Date: Mon, 3 Dec 2007 18:42:51 +0300
From: Oleg Nesterov <[EMAIL PROTECTED]>
To: Ingo Molnar <[EMAIL PROTECTED]>
Subject: Re: weird script behavior, signals?
Cc: Jan Kratochvil <[EMAIL PROTECTED]>,
Roland McGrath <[EMAIL PROTECTED]>

On 12/03, Ingo Molnar wrote:
>
> here's a fresh incident that is 100% reproducible. I constructed the
> following simple oneliner script to analyze saved kernel config files:
>
>  for N in `grep 'is not set' config* | cut -d\# -f2- | cut -d' ' -f2 |
>  sort | uniq`; do printf "%10d %s\n" `grep "$N=y" config* | wc -l` $N; done
>
> the script starts printing results like this:
>
> [...]
> 30 CONFIG_B43LEGACY_DEBUG
> 15 CONFIG_B43LEGACY_DMA_AND_PIO_MODE
> 18 CONFIG_B43LEGACY_DMA_MODE
> 19 CONFIG_B43LEGACY_PIO_MODE
> 21 CONFIG_B43_DEBUG
> 15 CONFIG_B43_DMA_AND_PIO_MODE
> 17 CONFIG_B43_DMA_MODE
>  6 CONFIG_B43_PCMCIA
> [...]
>
> now if i Ctrl-C the script, i get:
>
>-bash: printf: CONFIG_AFS_FS: invalid number
>
> if i Ctrl-Z the script, i get hung output, due to:
>
> |-login(2068)---bash(2306)---bash(10838)-+-grep(10839)
> |`-wc(10840)
>
> both grep and wc are in T+ state:
>
>  mingo10839  0.0  0.0   6088   676 tty2 T+   06:14
>  mingo10840  0.0  0.0   3800   428 tty2 T+   06:14   0:00 wc -l
>
> is this signal behavior really expected? I cannot kill the script - i

I assume you still can kill it doing "kill" aon another console, yes?

> have to manually kill the wc and grep tasks and then have to wait until
> its finished. Is this normal?

Looks like a bash bug to me.

$ echo `echo >&2 XXX; sleep 1`
$ ps ax
...
2549 tty1 S  0:00 -bash
2550 tty1 S+ 0:00 sleep 1
...

Small note, the job control rules is a black magic to me, so I assume it
is correct that "sleep" is in "foreground process group", but "bash" is not.
This -bash btw is the child of login shell, it executes `...`.

$ cat /proc/2549/status
...
ShdPnd: 
SigBlk: 0001
...

No pending signals, but SIGCHLD is blocked, I think this is the reason.

$ cat /proc/2549/wchan; echo
do_wait

Now I press Ctrl-Z, SIGTSTP goes to "sleep" and stopes it.

$ cat /proc/2550/status
...
State:  T (stopped)
...

"sleep" notifies the parent,

$ cat /proc/2549/status
...
ShdPnd: 0001
SigBlk: 0001
...

note the pending SIGCHLD. But it is blocked, signal_pending() is not true.
do_notify_parent_cldstop() does __wake_up_parent() anyway, but this doesn't
help because according to strace the "bash" does waitpid(-1, 0xafd37628, 0).

So do_wait() was called with options == WEXITED, it blocks again after wakeup.
This is correct because !signal_pending().

Unless I missed something, perhaps this should be reported to bash developers?

Oleg.

- End forwarded message -

- End forwarded message -

- End forwarded message -

- End forwarded message -

[PATCH 1/1] Exit status of fc -e is the wrong way around

[Ingo Karkat]

Hello Bash developers!

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' 
-DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' 
-DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib   -g -O2
uname output: Linux nashi 3.19.0-61-generic #69~14.04.1-Ubuntu SMP Thu Jun 9 
09:09:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu

Bash Version: 4.3
Patch Level: 46
Release Status: release
Also seen in prior versions, at least down to 4.2.46(1)-release

Description:

I'm trying to edit and recall a previous command with Bash's built-in "fc -e 
vi" command, but somehow the exit status always indicates failure (1) on 
successful edit, even if that editing is a no-op. The recalled command itself 
executes just fine, just $? is wrong.
Originally reported (and more background) here: 
http://unix.stackexchange.com/questions/292847/exit-status-of-fc-e-is-the-wrong-way-around

Repeat-By:

$ uname
Linux
$ fc -e true
uname
Linux
$ echo $?
1
$ # Would have expected 0 here: successful re-invocation

Fix:

This is my first dive into Bash's sources; patch follows and is also attached 
to this mail. Please let me know if you need anything else. As a longtime Bash 
user, I'm very happy to contribute to my favorite shell!

-- regards, ingo

From 1cf392a401c67c2f8437f2da459dfcf0f675dc55 Mon Sep 17 00:00:00 2001
From: Ingo Karkat 
Date: Thu, 30 Jun 2016 12:30:59 +0200
Subject: [PATCH 1/1] Exit status of fc -e is the wrong way around

fc_execute_file() delegates to _evalfile(), which only returns the result of 
the file's execution if FEVAL_BUILTIN is set (exemplified by source_file()). If 
unset, an indication of whether the file exists is returned instead 
(exemplified by maybe_execute_file(), which is used for the .bash_profile, 
.bash_login, .profile optional init chain).

According to the manual (and common sense), fc -e editor should return the 
recalled command's success. For that, the FEVAL_BUILTIN flag needs to be set.
---
 builtins/evalfile.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/builtins/evalfile.c b/builtins/evalfile.c
index 058d99d..e5c118b 100644
--- a/builtins/evalfile.c
+++ b/builtins/evalfile.c
@@ -331,7 +331,7 @@ fc_execute_file (filename)
 
   /* We want these commands to show up in the history list if
  remember_on_history is set. */
-  flags = FEVAL_ENOENTOK|FEVAL_HISTORY|FEVAL_REGFILE;
+  flags = FEVAL_ENOENTOK|FEVAL_BUILTIN|FEVAL_HISTORY|FEVAL_REGFILE;
   return (_evalfile (filename, flags));
 }
 #endif /* HISTORY */
-- 
1.9.1

-- 
  -- Ingo Karkat --  /^-- /^-- /^-- /^-- /^-- http://ingo-karkat.de/ --
From 1cf392a401c67c2f8437f2da459dfcf0f675dc55 Mon Sep 17 00:00:00 2001
From: Ingo Karkat 
Date: Thu, 30 Jun 2016 12:30:59 +0200
Subject: [PATCH 1/1] Exit status of fc -e is the wrong way around

fc_execute_file() delegates to _evalfile(), which only returns the result of the file's execution if FEVAL_BUILTIN is set (exemplified by source_file()). If unset, an indication of whether the file exists is returned instead (exemplified by maybe_execute_file(), which is used for the .bash_profile, .bash_login, .profile optional init chain).

According to the manual (and common sense), fc -e editor should return the recalled command's success. For that, the FEVAL_BUILTIN flag needs to be set.
---
 builtins/evalfile.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/builtins/evalfile.c b/builtins/evalfile.c
index 058d99d..e5c118b 100644
--- a/builtins/evalfile.c
+++ b/builtins/evalfile.c
@@ -331,7 +331,7 @@ fc_execute_file (filename)
 
   /* We want these commands to show up in the history list if
  remember_on_history is set. */
-  flags = FEVAL_ENOENTOK|FEVAL_HISTORY|FEVAL_REGFILE;
+  flags = FEVAL_ENOENTOK|FEVAL_BUILTIN|FEVAL_HISTORY|FEVAL_REGFILE;
   return (_evalfile (filename, flags));
 }
 #endif /* HISTORY */
-- 
1.9.1



signature.asc
Description: OpenPGP digital signature

Re: The Manual about the MAIL environment is misleading

[Ingo Krabbe]

Hi Chet,

thank you for your answer. I will send a summary about this problem to the 
gentoo maintainers, then.

Best Regards,

Ingo


> On 7/13/16 8:37 AM, ikrabbe@gmail.com wrote:
> 
>> Bash Version: 4.3
>> Patch Level: 42
>> Release Status: release
>> 
>> Description:
>>  In a bash session where the MAIL variable is not explicitly set, the 
>> MAIL variable might be auto-set through a compile time definition of 
>> DEFAULT_MAIL_DIRECTORY. There is no method (no method I know of) to find 
>> about about these compile time definitions and / or configuration flags.
>>  To find out about this bevaiour I needed to read the source code and 
>> finally found this line in CHANGES:
>> 
>>  hhh. Improved the mail checking code so it won't check (and possibly 
>> cause an
>>   NFS file system mount) until MAILPATH or MAIL is given a value -- 
>> there
>>   is no default if DEFAULT_MAIL_DIRECTORY is not defined at compile 
>> time.
>>  
>> 
>> Repeat-By:
>>  Install bash and remove all occurences of MAIL= or similar from the 
>> several profile and rc files.
>> 
>> Fix:
>>  Please describe the dependency of the MAIL variables on the manual 
>> page. If you manage to do so, add the value of DEFAULT_MAIL_DIRECTORY and 
>> it's effects in the generated manual pages.
> 
> The precedence of the MAIL and MAILPATH variables is already described in
> the man page.  I will look at changes to the man page to indicate that
> there is no default value for MAILPATH; there is some language in there
> that implies bash sets a default value (which it did in the past).
> 
> The manual page describes bash as I distribute it.  If a vendor or
> distribution wishes to change the default, to supply a default for
> MAILPATH, for example, I expect them to modify the documentation for
> their distribution to reflect that.  Some do, many don't.
> 
> -- 
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/