bash test: lastpipe.tests failed

[dengke...@windriver.com]

Hi all

When I run the bash test lastpipe.tests, there were some errors occurred.

The code in lastpipe.tests:

shopt -s lastpipe
...
exit 142 | false
echo $? -- ${PIPESTATUS[@]}

The error output:

lastpipe.tests: line 41:   708 Exit 142exit 142
703 Hangup  | false
129 -- 142 129

The lastpipe.right is:

...
1 -- 142 1
...

When I run the above line by line, it was correct.

$ shopt -s lastpipe
$ exit 142 | false
$ echo $? -- ${PIPESTATUS[@]}
$ 1 -- 142 1

That is strange.

//dengke

Re: bash: remove the format string "%q" in the unicode3.sub

[Chet Ramey]

On 8/10/16 2:42 AM, dengke...@windriver.com wrote:
> Hi chet
> 
> Thanks for you help.
> 
> 1. This file:
> 
>
> http://git.savannah.gnu.org/cgit/bash.git/tree/tests/unicode3.sub?h=devel&id=74b8cbb41398b4453d8ba04d0cdd1b25f9dcb9e3
> 
> When executed to the 3 line:
> 
> "$payload"
> 
> On my target the output was:
> 
> unicode3.sub: line 3: 5�@3�+�S8: command not found

You don't give any information about your target, but using the devel
version of bash on that file produces output using $'...' quoting on all
the systems I have tested, since the string contains an invalid multibyte
sequence.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: bash test: lastpipe.tests failed

[Chet Ramey]

On 8/10/16 3:32 AM, dengke...@windriver.com wrote:
> Hi all
> 
> When I run the bash test lastpipe.tests, there were some errors occurred.
> 
> The code in lastpipe.tests:
> 
> shopt -s lastpipe
> ...
> exit 142 | false
> echo $? -- ${PIPESTATUS[@]}
> 
> The error output:
> 
> lastpipe.tests: line 41:   708 Exit 142exit 142
> 703 Hangup  | false
> 129 -- 142 129

I can't reproduce this.  It looks system-specific; there shouldn't be
anything sending a SIGHUP to a simple command there.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Bash crashes while handling very long string in parameter expansion

[Siteshwar Vashisht]

- Original Message -
> From: "Chet Ramey" 
> To: "Siteshwar Vashisht" , bug-bash@gnu.org
> Cc: "chet ramey" 
> Sent: Tuesday, August 9, 2016 7:43:54 PM
> Subject: Re: Bash crashes while handling very long string in parameter
> expansion
> 
> You exceed the hard resource limit for your data segment size, and either
> the kernel kills the process or malloc fails and xmalloc() aborts the
> process.  If malloc fails and returns 0, the shell will attempt to print
> an explanatory message.  If that's not happening, the kernel is killing it.

Bash is not crashing due to exhausting system limits or by kernel OOM killer. 
This bug reproduces on systems that have more than 4 GB RAM. 

Bash uses signed int variable to store return value of strlen() function.  
While doing parameter expansion when string length goes beyond signed int 
limits, it causes a crash with below backtrace :

(gdb) bt
#0  __memmove_sse2_unaligned_erms () at 
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:364
#1  0x00455a4a in sub_append_string (
source=0x7ffef75de010 
"\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001."...,
 target=0x74aad0 "\001C\001o\001m\001m\001a\001n\001d\001 
\001O\001u\001t\001p\001u\001t\001 :\001 \001c\001a\001t\001 
\001d\001a\001t\001a\001\061\001g\001 ", 
indx=0x7fffdd30, size=0x7fffdd34) at subst.c:722
#2  0x00467bd0 in expand_word_internal (word=0x74cdc0, quoted=1, 
isexp=0, contains_dollar_at=0x7fffded0, expanded_something=0x0) at 
subst.c:9068
#3  0x00468db6 in expand_word_internal (word=0x74d0a0, quoted=0, 
isexp=0, contains_dollar_at=0x7fffe078, expanded_something=0x7fffe07c) 
at subst.c:9387
#4  0x0046ac21 in shell_expand_word_list (tlist=0x74d080, eflags=31) at 
subst.c:10490
#5  0x0046af10 in expand_word_list_internal (list=0x74a7c0, eflags=31) 
at subst.c:10613
#6  0x0046a137 in expand_words (list=0x74a7c0) at subst.c:10135
#7  0x0043d7ff in execute_simple_command (simple_command=0x74a7a0, 
pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x74cc60) at execute_cmd.c:4153
#8  0x00437a4e in execute_command_internal (command=0x74cd70, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x74cc60) at 
execute_cmd.c:802
#9  0x00437030 in execute_command (command=0x74cd70) at 
execute_cmd.c:405
#10 0x00422045 in reader_loop () at eval.c:180
#11 0x0041fd8a in main (argc=3, argv=0x7fffe458, 
env=0x7fffe478) at shell.c:792

(gdb) frame 1
#1  0x00455a4a in sub_append_string (
source=0x7ffef75de010 
"\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001."...,
 target=0x74aad0 "\001C\001o\001m\001m\001a\001n\001d\001 
\001O\001u\001t\001p\001u\001t\001 :\001 \001c\001a\001t\001 
\001d\001a\001t\001a\001\061\001g\001 ", 
indx=0x7fffdd30, size=0x7fffdd34) at subst.c:722
722   FASTCOPY (source, target + *indx, srclen);


(gdb) l 713,722
713
714   srclen = STRLEN (source);
715   if (srclen >= (int)(*size - *indx))
716 {
717   n = srclen + *indx;
718   n = (n + DEFAULT_ARRAY_SIZE) - (n % DEFAULT_ARRAY_SIZE);
719   target = (char *)xrealloc (target, (*size = n));
720 }
721
722   FASTCOPY (source, target + *indx, srclen);

(gdb) p srclen
$4 = -2147483648

I can see that bash uses signed int variables through out it's code to store 
return value of strlen() function, so such issues may crop up at other places 
too.

> 
> 
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
> 

-- 
--
Siteshwar Vashisht

Re: Bash crashes while handling very long string in parameter expansion

[Chet Ramey]

On 8/10/16 7:20 AM, Siteshwar Vashisht wrote:
>
> (gdb) frame 1
> #1  0x00455a4a in sub_append_string (
> source=0x7ffef75de010 
> "\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001.\001."...,
>  target=0x74aad0 "\001C\001o\001m\001m\001a\001n\001d\001 
> \001O\001u\001t\001p\001u\001t\001 :\001 \001c\001a\001t\001 
> \001d\001a\001t\001a\001\061\001g\001 ", 
> indx=0x7fffdd30, size=0x7fffdd34) at subst.c:722
> 722   FASTCOPY (source, target + *indx, srclen);
> 
> 
> (gdb) l 713,722
> 713
> 714   srclen = STRLEN (source);
> 715   if (srclen >= (int)(*size - *indx))
> 716 {
> 717   n = srclen + *indx;
> 718   n = (n + DEFAULT_ARRAY_SIZE) - (n % DEFAULT_ARRAY_SIZE);
> 719   target = (char *)xrealloc (target, (*size = n));
> 720 }
> 721
> 722   FASTCOPY (source, target + *indx, srclen);
> 
> (gdb) p srclen
> $4 = -2147483648

Thanks, I'll take a look.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

a patch to fix sh_stat on SunOS

[Dmitry Goncharov]

Good morning,

The following piece of shell code does not work on sunos.
mkfifo /tmp/pipe
cat < /tmp/pipe &
exec 6>/tmp/pipe
test -p /dev/fd/6 && echo pipe

The reason is on sunos /dev/fd/6 is a special character file, not a pipe
(unlike bsd), not a symlink (unlike linux, cygwin).
sh_stat ends up calling stat("/dev/fd/6", finfo) and obtains stat of /dev/fd/6
file rather than stat of the file pointed to by fd 6.

The patch in the attachment makes sh_stat always use fstat regardless of
HAVE_DEV_FD.
I've tested this patch on linux, sunos and aix.

regards, Dmitry
diff --git a/lib/sh/eaccess.c b/lib/sh/eaccess.c
index 8132a9a..1575dd2 100644
--- a/lib/sh/eaccess.c
+++ b/lib/sh/eaccess.c
@@ -82,8 +82,6 @@ sh_stat (path, finfo)
  const char *path;
  struct stat *finfo;
 {
-  static char *pbuf = 0;
-
   if (*path == '\0')
 {
   errno = ENOENT;
@@ -91,7 +89,6 @@ sh_stat (path, finfo)
 }
   if (path[0] == '/' && path[1] == 'd' && strncmp (path, "/dev/fd/", 8) == 0)
 {
-#if !defined (HAVE_DEV_FD)
   intmax_t fd;
   int r;
 
@@ -103,16 +100,6 @@ sh_stat (path, finfo)
 }
   errno = ENOENT;
   return (-1);
-#else
-  /* If HAVE_DEV_FD is defined, DEV_FD_PREFIX is defined also, and has a
- trailing slash.  Make sure /dev/fd/xx really uses DEV_FD_PREFIX/xx.
- On most systems, with the notable exception of linux, this is
- effectively a no-op. */
-  pbuf = xrealloc (pbuf, sizeof (DEV_FD_PREFIX) + strlen (path + 8));
-  strcpy (pbuf, DEV_FD_PREFIX);
-  strcat (pbuf, path + 8);
-  return (stat (pbuf, finfo));
-#endif /* !HAVE_DEV_FD */
 }
 #if !defined (HAVE_DEV_STDIN)
   else if (STREQN (path, "/dev/std", 8))
diff --git a/tests/test.tests b/tests/test.tests
index 47eef35..5a790d8 100644
--- a/tests/test.tests
+++ b/tests/test.tests
@@ -436,3 +436,17 @@ t -t /dev/tty444...
 
 # fixed in bash-4.0-beta
 t -t ' '
+
+exec 6>&-
+echo "t -p /dev/fd/6"
+t -p /dev/fd/6
+
+rm /tmp/pipe 2>/dev/null
+mkfifo /tmp/pipe
+cat < /tmp/pipe &
+exec 6>&-
+exec 6>/tmp/pipe
+echo "t -p /dev/fd/6"
+t -p /dev/fd/6
+kill $! 2>/dev/null
+rm /tmp/pipe 2>/dev/null

Re: a patch to fix sh_stat on SunOS

[Chet Ramey]

On 8/9/16 9:12 PM, Dmitry Goncharov wrote:
> Good morning,
> 
> The following piece of shell code does not work on sunos.
> mkfifo /tmp/pipe
> cat < /tmp/pipe &
> exec 6>/tmp/pipe
> test -p /dev/fd/6 && echo pipe
> 
> The reason is on sunos /dev/fd/6 is a special character file, not a pipe
> (unlike bsd), not a symlink (unlike linux, cygwin).
> sh_stat ends up calling stat("/dev/fd/6", finfo) and obtains stat of /dev/fd/6
> file rather than stat of the file pointed to by fd 6.

So what you're saying is that you don't like how SunOS implements /dev/fd
and you'd like bash to override the native implementation semantics.  Bash
is going to preserve the OS behavior of /dev/fd wherever possible, so I'm
not going to apply this patch without a much more compelling reason.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: a patch to fix sh_stat on SunOS

[Dmitry Goncharov]

On Wed, Aug 10, 2016 at 9:56 AM, Chet Ramey  wrote:

> So what you're saying is that you don't like how SunOS implements /dev/fd

> > and you'd like bash to override the native implementation semantics.

i don't care how sunos implements /dev/fd. i care that bash fails on sunos.
bash does not do what the following quote from the bash man page says it
does "If any file argument to one of the primaries is of the form
/dev/fd/n, then file descriptor n is checked.".
Other shells wok on sunos, though.
i am willing to redo it. Which patch would you consider acceptable?

regards, Dmitry

Re: a patch to fix sh_stat on SunOS

[Chet Ramey]

On 8/10/16 10:19 AM, Dmitry Goncharov wrote:
> 
> 
> On Wed, Aug 10, 2016 at 9:56 AM, Chet Ramey  > wrote:
> 
>> So what you're saying is that you don't like how SunOS implements /dev/fd
> 
>> and you'd like bash to override the native implementation semantics.
> 
> i don't care how sunos implements /dev/fd. i care that bash fails on sunos.
> bash does not do what the following quote from the bash man page says it
> does "If any file argument to one of the primaries is of the form
> /dev/fd/n, then file descriptor n is checked.".

It seems like a documentation error.  Bash doesn't second-guess the OS's
/dev/fd implementation if it's there.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Bash bind bug: infinite loop with memory leak in Readline

[Christian Klomp]

Hi,

I found a problem with the binding of key sequences to macros that
results in bash allocating all the memory until the process is killed.
It happens when a key is mapped to itself, e.g., `bind '"3":"3"'`. Now
when '3' is typed the while loop inside the readline_internal_charloop
function will never end.

Besides manually, the flaw can be triggered via programs like expect,
screen and tmux, and works over SSH. When scripted it can result in
significant resource usage which might ultimately result in a denial
of service (although impact can be considered low since users need to
be authenticated and mitigation should be fairly easy, e.g., via
ulimit).

As far as I can tell every distribution and every Bash version is
affected by this (oldest I tried is Damn Small Linux 0.4.10 with Bash
2.05b) and it also happens on the new Windows Ubuntu subsystem.

I've written a preliminary patch that prevents triggering the flaw.
The first part prevents adding problematic macros and the second part
will escape the loop if a problematic key mapping occurs (I'm not sure
this second part is necessary assuming the first part properly
prevents adding any problematic mapping, but I've included it since
I'm not sure whether there are other ways to add mappings). So the
underlying problem is not addressed by the patch.

It also doesn't catch every case, e.g., `bind '"3":"123"'` will still
result in an infinite loop but in that case accumulation of memory is
insignificant.

Since I am not known with the internals of Bash/Readline and I am not
a C programmer I've refrained myself from digging further.

Regards,
Christian Klomp


# Bash 4.3.46:
--- ../bash-4.3/lib/readline/bind.c2013-04-06 23:46:38.0 +0200
+++ lib/readline/bind.c2016-08-09 21:55:56.500772211 +0200
@@ -313,6 +313,14 @@
  const char *keyseq, *macro;
  Keymap map;
 {
+  /* don't add macro if input is the same as output
+ in order to avoid infinite loop. */
+  if (*keyseq == *macro)
+{
+  _rl_init_file_error ("keyseq cannot be the same as macro");
+  return -1;
+}
+
   char *macro_keys;
   int macro_keys_len;

--- ../bash-4.3/lib/readline/readline.c2016-08-09 17:14:28.0 +0200
+++ lib/readline/readline.c2016-08-09 21:55:08.596984135 +0200
@@ -957,6 +957,14 @@
 {
   rl_executing_keyseq[rl_key_sequence_length] = '\0';
   macro = savestring ((char *)map[key].function);
+
+  /* avoid infinite loop if key and macro are equal. */
+  if (key == *macro)
+{
+  _rl_abort_internal ();
+  return -1;
+}
+
   _rl_with_macro_input (macro);
   return 0;
 }


# Bash 4.4-beta-2 (only line numbers differ):
--- ../bash-4.4-beta2/lib/readline/bind.c2016-01-25 16:33:57.0 +0100
+++ lib/readline/bind.c2016-08-09 22:06:42.602407370 +0200
@@ -337,6 +337,14 @@
  const char *keyseq, *macro;
  Keymap map;
 {
+  /* don't add macro if input is the same as output
+ in order to avoid infinite loop. */
+  if (*keyseq == *macro)
+{
+  _rl_init_file_error ("keyseq cannot be the same as macro");
+  return -1;
+}
+
   char *macro_keys;
   int macro_keys_len;

--- ../bash-4.4-beta2/lib/readline/readline.c2016-04-20
21:53:52.0 +0200
+++ lib/readline/readline.c2016-08-09 22:07:58.824249663 +0200
@@ -996,6 +996,14 @@
 {
   rl_executing_keyseq[rl_key_sequence_length] = '\0';
   macro = savestring ((char *)map[key].function);
+
+  /* avoid infinite loop if key and macro are equal. */
+  if (key == *macro)
+{
+  _rl_abort_internal ();
+  return -1;
+}
+
   _rl_with_macro_input (macro);
   return 0;
 }

Unable to dereference function-local nameref to global variable of same name

[Andreas Kusalananda Kähäri]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: openbsd6.0
Compiler: cc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='openbsd6.0' -DCONF_MACHTYPE='x86_64-unknown-openbsd6.0' 
-DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' 
-DSHELL  -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib  -DUSE_MKTEMP 
-DUSE_MKSTEMP -I/usr/local/include -O2 -pipe
uname output: OpenBSD uerfale 6.0 GENERIC.MP#38 amd64
Machine Type: x86_64-unknown-openbsd6.0

Bash Version: 4.3
Patch Level: 46
Release Status: release

Description:
When declaring a variable in a function as a nameref, it can not
be dereferenced if the variable it's a nameref to happen to have
the same name as the nameref itself.  This imposes unnecessary
restrictions on the caller.

Also seen in GNU bash, version 4.3.39(1)-release (x86_64-apple-darwin15).

Repeat-By:
function bug {
typeset -n var="$1"
printf "%s\n" "$var"
}

var="hello"
bug var

Result:
bash: warning: var: circular name reference

Expected result: The string "hello" outputted with a terminating
newline (as with ksh93).

Re: a patch to fix sh_stat on SunOS

[Dmitry Goncharov]

On Wed, Aug 10, 2016 at 10:30 AM, Chet Ramey  wrote:

> > On 8/10/16 10:19 AM, Dmitry Goncharov wrote:

> > > bash does not do what the following quote from the bash man page says
it

> > > does "If any file argument to one of the primaries is of the form

> > > /dev/fd/n, then file descriptor n is checked.".

>
> It seems like a documentation error.

What should the man page say instead?

regards, Dmitry