segmentation fault after interrupting function that uses "time"
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Wall
uname output: Linux nb 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1
(2024-04-11) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu
Bash Version: 5.2
Patch Level: 15
Release Status: release
Description:
Calling a function that itself uses the keyword "time" to call another
function that performs a long task, and then interrupting with Ctrl+C before it
is complete, causes a segfault. Behavior varies depending on whether or not the
output of time is piped into another command, and whether the functions were
sourced from a file or typed in directly.
Seems similar to the bug previously reported here:
https://lists.gnu.org/archive/html/bug-bash/2019-07/msg4.html
Repeat-By:
The simplest case can be reproduced with the following steps:
foo () { sleep 10; echo 'output from long task'; }
bar_1 () { time foo; }
bar_1 # interrupt after a few seconds
bar_1 # Segmentation fault
Additionally, consider these variations:
bar_2 () { time foo | cat; }
bar_3 () { time (foo | cat); }
If defined directly in the shell, calling and interrupting these
functions causes no issues. If you source them from a script, however, bar_2
behaves like bar_1 did above:
. functions.sh # contains definition of foo and bar_[1-3]
bar_2 # interrupt
bar_2 # Segmentation fault
Unless bar_3 is called AND interrupted beforehand:
. functions.sh
bar_3 # interrupt
bar_2 # interrupt
bar_2 # still works!
If bar_3 finishes without interruption, bar_2 will still segfault
afterwards. But bar_3 itself does not seem to have any issues.
Re: sh vs. bash -xc 'a=b c=$a'
On 5/22/24 6:56 PM, Dan Jacobson wrote: It seems these should both make one line "+ a=b c=b" output, for s in sh bash do $s -xc 'a=b c=$a' done I mean they give the same results, but bash splits it into two lines, so the user reading the bash -x output cannot tell if one (correct) or two (incorrect) lines were used. There's nothing incorrect about it. The bash output clearly tells you the order of operations, which is the whole purpose of -x. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: sh vs. bash -xc 'a=b c=$a'
On 5/22/24 10:57 PM, Oğuz wrote: And some Bourne shells expand command substitutions first $ (exit 5) $ x=$? y=`exit 10` $ echo $x 10 It's worse than that. The v7 sh performed assignments right to left. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: sh vs. bash -xc 'a=b c=$a'
Date:Thu, 23 May 2024 09:04:48 -0400 From:Chet Ramey Message-ID: | The bash output clearly tells you the | order of operations, which is the whole purpose of -x. But it can be horribly misleading. Consider what bash does with this similar case (I used 5.3a1 but I suspect any version would do) bash-5.3a1 -xc 'a=a b=b; a=1 b=2 cat /dev/null; echo $a' + a=a + b=b + a=1 + b=2 + cat /dev/null + echo a a versus bash-5.3a1 -xc 'a=a b=b; a=1 b=2; cat /dev/null; echo $a' + a=a + b=b + a=1 + b=2 + cat /dev/null + echo 1 1 The only difference in the two outputs from -x is the arg to the echo command at the end, no indication at all why they're different. On the other hand: sh -xc 'a=a b=b; a=1 b=2 cat /dev/null; echo $a' + a=a b=b + a=1 b=2 cat /dev/null + echo a a sh -xc 'a=a b=b; a=1 b=2 ; cat /dev/null; echo $a' + a=a b=b + a=1 b=2 + cat /dev/null + echo 1 1 and it is quite clear what is happening. Personally I'd think it more likely that your average reader would interpret "a=1 b=2" as being executed left to right than that they'd somehow guess that "a=1" isn't really happening in shell environment. kre
Re: sh vs. bash -xc 'a=b c=$a'
if u var=value command ; the var's only for cmd and disappears afterwards if u var=val ; cmd ; its not for cmd , excepts exported and is set after cmd , usual behav old rule On Thu, May 23, 2024, 3:34 PM Robert Elz wrote: > Date:Thu, 23 May 2024 09:04:48 -0400 > From:Chet Ramey > Message-ID: > > > | The bash output clearly tells you the > | order of operations, which is the whole purpose of -x. > > But it can be horribly misleading. Consider what bash does with > this similar case (I used 5.3a1 but I suspect any version would do) > > bash-5.3a1 -xc 'a=a b=b; a=1 b=2 cat /dev/null; echo $a' > + a=a > + b=b > + a=1 > + b=2 > + cat /dev/null > + echo a > a > > versus > > bash-5.3a1 -xc 'a=a b=b; a=1 b=2; cat /dev/null; echo $a' > + a=a > + b=b > + a=1 > + b=2 > + cat /dev/null > + echo 1 > 1 > > The only difference in the two outputs from -x is the arg to the echo > command at the end, no indication at all why they're different. > > > On the other hand: > > sh -xc 'a=a b=b; a=1 b=2 cat /dev/null; echo $a' > + a=a b=b > + a=1 b=2 cat /dev/null > + echo a > a > > sh -xc 'a=a b=b; a=1 b=2 ; cat /dev/null; echo $a' > + a=a b=b > + a=1 b=2 > + cat /dev/null > + echo 1 > 1 > > and it is quite clear what is happening. > > Personally I'd think it more likely that your average reader would > interpret "a=1 b=2" as being executed left to right than that they'd > somehow guess that "a=1" isn't really happening in shell environment. > > kre > > >
Re: [PATCH] rl_change_case: skip over invalid mbchars
On 5/21/24 2:42 PM, Grisha Levit wrote: Avoid using (size_t)-1 as an offset. I can't reproduce this on macOS. Where is the code that's using -1 as an offset? Also, not sure it makes sense to change the case of an invalid byte cast to (wchar_t). Bash and readline are pretty consistent about treating invalid multibyte characters as a sequence of bytes. Maybe the thing to do here is to add a goto to the single-byte case if m == (size_t)-1 or (size_t)-2, but assigning a byte to a wchar_t variable isn't harmful, since wchar_t is guaranteed to be an integral type. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
yet another $(case ... parse bug
See: $ bash -c 'for (( $(case x in x) esac);; )); do :; done' bash: -c: line 1: syntax error: `;' unexpected bash: -c: line 1: syntax error: `(( $(case x in x) ;; esac);; ))' This is reproducible on 5.3 alpha too. Oğuz
Re: [PATCH] rl_change_case: skip over invalid mbchars
On Thu, May 23, 2024 at 10:25 AM Chet Ramey wrote:
>
> On 5/21/24 2:42 PM, Grisha Levit wrote:
> > Avoid using (size_t)-1 as an offset.
>
> I can't reproduce this on macOS. Where is the code that's using -1 as an
> offset?
The loop in rl_change_case does the following:
rl_change_case(count=-1, op=2) at text.c:1483:9
1481 while (start < end)
1482 {
-> 1483 c = _rl_char_value (rl_line_buffer, start);
_rl_char_value(buf="\xc0", ind=0) at mbutil.c:493:23
491l = strlen (buf);
492if (ind + 1 >= l)
-> 493 return ((WCHAR_T) buf[ind]);
(wchar_t) c = L'À'
This seems questionable since a string consisting of \xC0, and a string
actually representing \u00C0 (\xC3\x80) will both return the same thing.
The next check passes, since C is LATIN CAPITAL LETTER A WITH GRAVE
rl_change_case(count=-1, op=2) at text.c:1487:28
-> 1487 if (_rl_walphabetic (c) == 0)
1488 {
1489 inword = 0;
1490 start = next;
1450 continue;
_rl_walphabetic(wc=L'À') at util.c:89:5
88 if (iswalnum (wc))
-> 89 return (1);
So we call mbrtowc on the same string position and since this is not a
valid multibyte character, (size_t)-1 is stored in M.
rl_change_case(count=-1, op=2) at text.c:1512:22
-> 1512 m = MBRTOWC (&wc, rl_line_buffer + start, end - start,
&mps);
(size_t) m = 18446744073709551615
Then we again interpret \xC0 as if it were \u00C0:
rl_change_case(count=-1, op=2) at text.c:1514:20
1513 if (MB_INVALIDCH (m))
-> 1514 wc = (WCHAR_T)rl_line_buffer[start];
(wchar_t) wc = L'À'
And lowercase that character, storing its length in MLEN.
rl_change_case(count=-1, op=2) at text.c:1517:11
-> 1517 nwc = (nop == UpCase) ? _rl_to_wupper (wc) :
_rl_to_wlower (wc);
rl_change_case(count=-1, op=2) at text.c:1524:28
-> 1524 mlen = WCRTOMB (mb, nwc, &ts);
(wchar_t) nwc = L'à'
(int) mlen = 2
Since WC and NWC are different, and M (being (size_t)-1) is greater than MLEN:
rl_change_case(count=-1, op=2) at text.c:1544:13
1541 else if (m > mlen)
1542 {
1543 memcpy (s, mb, mlen);
-> 1544 memmove (s + mlen, s + m, (e - s) - m);
So the second arg to memmove is a pointer one behind S.
Re: segmentation fault after interrupting function that uses "time"
On 5/23/24 7:43 AM, Michael Maurer wrote:
Bash Version: 5.2
Patch Level: 15
Release Status: release
Description:
Calling a function that itself uses the keyword "time" to call another
function that performs a long task, and then interrupting with Ctrl+C before it is
complete, causes a segfault. Behavior varies depending on whether or not the output of
time is piped into another command, and whether the functions were sourced from a file or
typed in directly.
Seems similar to the bug previously reported here:
https://lists.gnu.org/archive/html/bug-bash/2019-07/msg4.html
I can't reproduce this on macOS, RHEL 9, or Debian 12 with either the
latest devel branch or bash-5.2.26.
Here's a sample transcript from the Debian 12 instance:
admin@ip-172-29-1-176:~/build/bash-5.2.26$ ./bash
admin@ip-172-29-1-176:~/build/bash-5.2.26$ cd scratch/
admin@ip-172-29-1-176:~/build/bash-5.2.26/scratch$ foo () { sleep 10; echo
'output from long task'; }
admin@ip-172-29-1-176:~/build/bash-5.2.26/scratch$ bar_1 () { time foo; }
admin@ip-172-29-1-176:~/build/bash-5.2.26/scratch$ bar_1
^C
real0m5.780s
user0m0.001s
sys 0m0.000s
admin@ip-172-29-1-176:~/build/bash-5.2.26/scratch$ bar_1
output from long task
real0m10.002s
user0m0.002s
sys 0m0.000s
admin@ip-172-29-1-176:~/build/bash-5.2.26/scratch$ echo $BASH_VERSION
5.2.26(1)-release
admin@ip-172-29-1-176:~/build/bash-5.2.26/scratch$ uname -a
Linux ip-172-29-1-176 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian
6.1.90-1 (2024-05-03) x86_64 GNU/Linux
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: [PATCH] rl_change_case: skip over invalid mbchars
On 5/23/24 3:25 PM, Grisha Levit wrote:
On Thu, May 23, 2024 at 10:25 AM Chet Ramey wrote:
On 5/21/24 2:42 PM, Grisha Levit wrote:
Avoid using (size_t)-1 as an offset.
I can't reproduce this on macOS. Where is the code that's using -1 as an
offset?
The loop in rl_change_case does the following:
rl_change_case(count=-1, op=2) at text.c:1483:9
1481 while (start < end)
1482 {
-> 1483 c = _rl_char_value (rl_line_buffer, start);
_rl_char_value(buf="\xc0", ind=0) at mbutil.c:493:23
491l = strlen (buf);
492if (ind + 1 >= l)
-> 493 return ((WCHAR_T) buf[ind]);
(wchar_t) c = L'À'
Nope, this is where you lose me. Using lldb with an input file created
from the string you sent, I get c = (wchar_t) L'\Ufffd', which fails
the rl_walphabetic test. Even running the command as you posted it just
prints `?'. What os are you using?
Maybe I'll try it some other places, but I don't have any other readily
available machines to run address sanitizer, so I'll have to eyeball it.
Maybe later.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: [PATCH] rl_change_case: skip over invalid mbchars
On Thu, May 23, 2024 at 4:11 PM Chet Ramey wrote:
>
> On 5/23/24 3:25 PM, Grisha Levit wrote:
> > On Thu, May 23, 2024 at 10:25 AM Chet Ramey wrote:
> >>
> >> On 5/21/24 2:42 PM, Grisha Levit wrote:
> >>> Avoid using (size_t)-1 as an offset.
> >>
> >> I can't reproduce this on macOS. Where is the code that's using -1 as an
> >> offset?
> >
> > The loop in rl_change_case does the following:
> >
> > rl_change_case(count=-1, op=2) at text.c:1483:9
> > 1481 while (start < end)
> > 1482 {
> > -> 1483 c = _rl_char_value (rl_line_buffer, start);
> >
> > _rl_char_value(buf="\xc0", ind=0) at mbutil.c:493:23
> > 491l = strlen (buf);
> > 492if (ind + 1 >= l)
> > -> 493 return ((WCHAR_T) buf[ind]);
> >
> > (wchar_t) c = L'À'
>
> Nope, this is where you lose me. Using lldb with an input file created
> from the string you sent, I get c = (wchar_t) L'\Ufffd', which fails
> the rl_walphabetic test. Even running the command as you posted it just
> prints `?'. What os are you using?
I think this is lldb being too clever and showing _any_ negative wchar_t
as the unicode replacement character.
(lldb) p (wchar_t)-1
(wchar_t) L'\Ufffd'
(lldb) p (wchar_t)-64
(wchar_t) L'\Ufffd'
The issue here is that on arm64 linux, char is unsigned, so the (wchar_t)
conversion of a plain char in the '\x80'-'\xFF' range yields a valid wide
character.
(lldb) p (wchar_t)( signed char)'\xC0' == L'\u00C0'
(bool) false
(lldb) p (wchar_t)(unsigned char)'\xC0' == L'\u00C0'
(bool) true
Re: segmentation fault after interrupting function that uses "time"
On Thu, May 23, 2024 at 3:55 PM Chet Ramey wrote:
>
> On 5/23/24 7:43 AM, Michael Maurer wrote:
>
> > Bash Version: 5.2
> > Patch Level: 15
> > Release Status: release
> >
> > Description:
> > Calling a function that itself uses the keyword "time" to call
> > another function that performs a long task, and then interrupting with
> > Ctrl+C before it is complete, causes a segfault. Behavior varies depending
> > on whether or not the output of time is piped into another command, and
> > whether the functions were sourced from a file or typed in directly.
> > Seems similar to the bug previously reported here:
> > https://lists.gnu.org/archive/html/bug-bash/2019-07/msg4.html
>
> I can't reproduce this on macOS, RHEL 9, or Debian 12 with either the
> latest devel branch or bash-5.2.26.
This shows up with ASAN:
bash-5.3$ g() { sleep 10; :; }; f() { time g; }; f
^C
=
==12333==ERROR: AddressSanitizer: heap-use-after-free on address
0x503098b4 at pc 0xc26d0bcf8268 bp 0xfbfb85c0 sp
0xfbfb85b0
WRITE of size 4 at 0x503098b4 thread T0
#0 0xc26d0bcf8264 in time_command
/home/vm/src/bash/origin/execute_cmd.c:1456
#1 0xc26d0bcf0148 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:796
#2 0xc26d0bcf30b4 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:1073
#3 0xc26d0bd1eda0 in execute_function
/home/vm/src/bash/origin/execute_cmd.c:5373
#4 0xc26d0bd1f900 in execute_builtin_or_function
/home/vm/src/bash/origin/execute_cmd.c:5611
0x503098b4 is located 4 bytes inside of 32-byte region
[0x503098b0,0x503098d0)
freed by thread T0 here:
#0 0xe4b56d0dfb0c in free
../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
#1 0xc26d0bceab7c in dispose_command
/home/vm/src/bash/origin/dispose_cmd.c:204
#2 0xc26d0bce9b4c in dispose_command
/home/vm/src/bash/origin/dispose_cmd.c:82
#3 0xc26d0bceaba4 in uw_dispose_command
/home/vm/src/bash/origin/dispose_cmd.c:210
#4 0xc26d0be53868 in unwind_frame_run_internal
/home/vm/src/bash/origin/unwind_prot.c:286
#5 0xc26d0be51b64 in run_unwind_protects_internal
/home/vm/src/bash/origin/unwind_prot.c:211
#6 0xc26d0be50ee8 in run_unwind_protects
/home/vm/src/bash/origin/unwind_prot.c:145
#7 0xc26d0be61998 in throw_to_top_level /home/vm/src/bash/origin/sig.c:465
#8 0xc26d0bcee4f4 in execute_command
/home/vm/src/bash/origin/execute_cmd.c:455
#9 0xc26d0bd04a88 in execute_connection
/home/vm/src/bash/origin/execute_cmd.c:2832
#10 0xc26d0bcf30d4 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:1080
#11 0xc26d0bcf30b4 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:1073
#12 0xc26d0bd1eda0 in execute_function
/home/vm/src/bash/origin/execute_cmd.c:5373
#13 0xc26d0bd1f900 in execute_builtin_or_function
/home/vm/src/bash/origin/execute_cmd.c:5611
#14 0xc26d0bd1b9e8 in execute_simple_command
/home/vm/src/bash/origin/execute_cmd.c:4817
#15 0xc26d0bcf1240 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:905
#16 0xc26d0bcf8198 in time_command
/home/vm/src/bash/origin/execute_cmd.c:1453
#17 0xc26d0bcf0148 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:796
#18 0xc26d0bcf30b4 in execute_command_internal
/home/vm/src/bash/origin/execute_cmd.c:1073
#19 0xc26d0bd1eda0 in execute_function
/home/vm/src/bash/origin/execute_cmd.c:5373
#20 0xc26d0bd1f900 in execute_builtin_or_function
/home/vm/src/bash/origin/execute_cmd.c:5611
previously allocated by thread T0 here:
#0 0xe4b56d0e0f28 in malloc
../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0xc26d0befdd40 in xmalloc /home/vm/src/bash/origin/xmalloc.c:104
#2 0xc26d0bd5d000 in copy_command /home/vm/src/bash/origin/copy_cmd.c:350
#3 0xc26d0bd5a09c in copy_group_command
/home/vm/src/bash/origin/copy_cmd.c:206
#4 0xc26d0bd5d8a8 in copy_command /home/vm/src/bash/origin/copy_cmd.c:378
#5 0xc26d0bd1d790 in execute_function
/home/vm/src/bash/origin/execute_cmd.c:5192
#6 0xc26d0bd1f900 in execute_builtin_or_function
/home/vm/src/bash/origin/execute_cmd.c:5611
Incorrect string length check
Hi Chet,
In this commit in the devel branch:
commit 03c8c43b79177fa678714893e9f05b1c517592c0
Author: Chet Ramey
Date: Fri Apr 5 09:03:52 2024 -0400
man page typesetting updates for compatibilityand layout issues
I think there was a typo in execute_cmd.c:
diff --git a/execute_cmd.c b/execute_cmd.c
[...]
void
coproc_setvars (struct coproc *cp)
{
@@ -6072,14 +6073,14 @@ shell_execve (char *command, char **args, char **env)
interp = getinterp (sample, sample_len, (int *)NULL);
ilen = strlen (interp);
errno = i;
- if (interp[ilen - 1] == '\r')
+ if (interp > 0 && interp[ilen - 1] == '\r')
{
interp = xrealloc (interp, ilen + 2);
interp[ilen - 1] = '^';
interp[ilen] = 'M';
interp[ilen + 1] = '\0';
}
Shouldn't that condition be something like this:
if (ilen > 0 && interp[ilen - 1] == '\r')
{
/* Rest of code. */
}
Since you want to protect against an '#!' without an interpreter
following it. I'm thinking it was just a typo but feel free to correct
me if I am missing something.
Thanks,
Collin
Re: yet another $(case ... parse bug
While you're at it take a look at this too:
true; for (( ; $? == 0; ${ ! break;} )); do uname; done
Once you run this command bash gets stuck in a state where it prints
the prompt string and reads commands but doesn't execute anything. If
you press Ctrl+C it returns to normal.
Oğuz
