Re: exit status issue
"Steven W. Orr" writes: > I think we're beating this one to death, but I have point out that > telling perl to run a print command whose output is redirected by bash > is not the same as telling bash to run a builtin echo command that is > redirected as an integral operation of the same interpreter. They are really the same, but even if you change it bash still wins: $ bash -c 'echo "hello"' >/dev/full bash: line 0: echo: write error: No space left on device Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: exit status issue
On Wednesday 23 November 2011, Andreas Schwab wrote:
> "Steven W. Orr" writes:
>
> > I think we're beating this one to death, but I have point out that
> > telling perl to run a print command whose output is redirected by bash
> > is not the same as telling bash to run a builtin echo command that is
> > redirected as an integral operation of the same interpreter.
>
> They are really the same, but even if you change it bash still wins:
>
> $ bash -c 'echo "hello"' >/dev/full
> bash: line 0: echo: write error: No space left on device
>
IMO this is not really perl's fault, but rather the programmer's
fault, since he has forgotten to check for possible errors. If
did, he would get:
$ perl -e 'print "hello"; close(STDOUT) or die "$!\n";' >/dev/full
No space left on device
$ echo $?
28
Something similar happens with C, of course, if you forget to check
error conditions:
$ cat > foo.c <<'END'
#include
int main (void)
{
printf("hello\n");
return 0;
}
END
$ gcc foo.c
$ ./a.out >/dev/null # Oops, no error reported.
$ cat > foo.c <<'END'
#include
int main (void)
{
printf("hello\n");
if (fclose(stdout) != 0)
perror("Write error");
return 0;
}
END
$ gcc foo.c
$ ./a.out >/dev/null # Error will be reported now.
Write error: No space left on device
More "modern" languages, with built-in exception handling, are somewhat
better in this regard:
$ python3 -c 'print("foo")' >/dev/full
Exception IOError: (28, 'No space left on device') in ... ignored
Just my 2 cents.
Regards,
Stefano
set -e works incorrectly in subshells
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 -Wall uname output: Linux mmarkk-work 2.6.38-12-generic #51-Ubuntu SMP Wed Sep 28 14:27:32 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.2 Patch Level: 8 Release Status: release Repeat-By: mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) aaa mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) || true aaa bbb mmarkk@mmarkk-work:~$ Expect: mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) aaa mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) || true aaa mmarkk@mmarkk-work:~$mmarkk@mmarkk-work:~$
Re: set -e works incorrectly in subshells
On 11/23/2011 11:26 AM, Марк Коренберг wrote: Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 -Wall uname output: Linux mmarkk-work 2.6.38-12-generic #51-Ubuntu SMP Wed Sep 28 14:27:32 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.2 Patch Level: 8 Release Status: release Repeat-By: mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) aaa mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) || true aaa bbb mmarkk@mmarkk-work:~$ Expect: mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) aaa mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) || true aaa mmarkk@mmarkk-work:~$mmarkk@mmarkk-work:~$ Not a bug. See man page of set builtin; quoting: The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test following the if or elif reserved words, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted with !.
Re: set -e works incorrectly in subshells
On 11/23/2011 03:26 AM, Марк Коренберг wrote: > Repeat-By: > mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) > aaa > mmarkk@mmarkk-work:~$ ( set -e; echo aaa; false; echo bbb ) || true > aaa > bbb > mmarkk@mmarkk-work:~$ ksh has the same behavior, and POSIX requires it (basically, running the subshell on the left of || has a higher precedence than the explicit 'set -e' within the subshell). http://austingroupbugs.net/view.php?id=52 Expected behavior. And one of the arguments I give why using the crutch of 'set -e' is almost always the wrong thing in a complex script. -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: feature request: printf %(%s)T
On 11/22/11 4:53 PM, Greg Wooledge wrote: > This is a feature request, rather than a bug. Bash 4.2's printf command > has a lovely %(datefmt)T feature that allows it to print out formatted > timestamps using the underlying operating system's strftime(3) routine. > It even allows bash to print the current time, or the time the current > shell was invoked. I wonder if a better way to handle this is to require the %s expansion at configure time and use the strftime replacement in lib/sh if the C library's strftime doesn't implement it. What systems, if you know, do not handle %s? Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: feature request: printf %(%s)T
On Wed, Nov 23, 2011 at 12:00:34PM -0500, Chet Ramey wrote: > I wonder if a better way to handle this is to require the %s expansion > at configure time and use the strftime replacement in lib/sh if the C > library's strftime doesn't implement it. What systems, if you know, do > not handle %s? HP-UX 10.20 and 11.11 are the ones I have. It's a somewhat commonly requested feature among Solaris users too, but I don't know which versions of Solaris they are using. There are probably many others. At a guess, I would assume everything that isn't GNU/Linux or BSD.
Re: feature request: printf %(%s)T
On 11/23/2011 10:11 AM, Greg Wooledge wrote: > On Wed, Nov 23, 2011 at 12:00:34PM -0500, Chet Ramey wrote: >> I wonder if a better way to handle this is to require the %s expansion >> at configure time and use the strftime replacement in lib/sh if the C >> library's strftime doesn't implement it. What systems, if you know, do >> not handle %s? > > HP-UX 10.20 and 11.11 are the ones I have. It's a somewhat commonly > requested feature among Solaris users too, but I don't know which > versions of Solaris they are using. There are probably many others. > At a guess, I would assume everything that isn't GNU/Linux or BSD. Gnulib already provides a strftime replacement that guarantees %s, which is handy since the next version of POSIX is considering requiring %s in strftime(). -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: feature request: printf %(%s)T
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/23/11 12:29 PM, Eric Blake wrote: > On 11/23/2011 10:11 AM, Greg Wooledge wrote: >> On Wed, Nov 23, 2011 at 12:00:34PM -0500, Chet Ramey wrote: >>> I wonder if a better way to handle this is to require the %s expansion >>> at configure time and use the strftime replacement in lib/sh if the C >>> library's strftime doesn't implement it. What systems, if you know, do >>> not handle %s? >> >> HP-UX 10.20 and 11.11 are the ones I have. It's a somewhat commonly >> requested feature among Solaris users too, but I don't know which >> versions of Solaris they are using. There are probably many others. >> At a guess, I would assume everything that isn't GNU/Linux or BSD. > > Gnulib already provides a strftime replacement that guarantees %s, which > is handy since the next version of POSIX is considering requiring %s in > strftime(). Bash already has Arnold's strftime for those systems that don't provide it, and it implements %s. Chet - -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7NRwoACgkQu1hp8GTqdKu94ACeL7yFHDMaHgjv1ytZ9OvmgAm9 DS4Ani3cj1VXrbP9yuJEAf2iQANoY667 =w0fE -END PGP SIGNATURE-
Re: exit status issue
Hi All,
Just wanted to let you know that I have found a solution to my
problem, though I can't entirely explain it yet. As mentioned
earlier, this script was designed to run as a daemon, i.e. run in the
background. This is what I was doing to daemonize the script:
Daemon()
{
while :
do
checkStatus
sleep 1
done
}
trap CleanupExit INT
trap CleanupExit TERM
trap CleanupExit ILL
trap CleanupExit QUIT
trap CleanupExit ABRT
trap CleanupExit KILL
if [ x"$1" = x"start" ]; then
Daemon 0<&- 1>/dev/null 2>&1 &
echo $! > $PIDFILE
elif [ x"$1" = x"stop" ]; then
TMP=`cat $PIDFILE`
kill $TMP
CleanupExit
else
:
fi
When running like this, I was seeing 'touch' command failures inside
function fsaccesstest() shown previously.
I did an experiment last night and ran this script in the foreground
(removed the &) from the call to Daemon. To my surprise, I did not
see any errors with the 'touch' command.
I then went back and changed how I was turning this script into a
daemon, by using the start-stop-daemon utility instead like this:
start-stop-daemon --start --background --quiet --exec $DAEMON
Once I did this, my shell script is now running in the background
without any errors. Life is good.
Perhaps when I was calling Daemon & as shown above, the parent process
was not exiting properly and was somehow interfering in the child
process execution. I'm not really sure.
Anyhow, I'm past this problem now. Thank you all for the good ideas
and generous help.
Re: exit status issue
On Wed, Nov 23, 2011 at 02:09:57PM -0600, Dallas Clement wrote: > trap CleanupExit KILL For the record, you cannot trap SIGKILL.
Re: feature request: printf %(%s)T
2011-11-23, 12:00(-05), Chet Ramey: > On 11/22/11 4:53 PM, Greg Wooledge wrote: >> This is a feature request, rather than a bug. Bash 4.2's printf command >> has a lovely %(datefmt)T feature that allows it to print out formatted >> timestamps using the underlying operating system's strftime(3) routine. >> It even allows bash to print the current time, or the time the current >> shell was invoked. > > I wonder if a better way to handle this is to require the %s expansion > at configure time and use the strftime replacement in lib/sh if the C > library's strftime doesn't implement it. What systems, if you know, do > not handle %s? [...] Or just have a special variable for that like zsh's $EPOCHSECONDS. Note that GNU strftime has more extensions than just %s. See also http://stchaz.free.fr/wide_strftime for a POSIX shell implementation of strftime (limited to GMT timezone and POSIX locale though). -- Stephane
read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
Bash Version: GNU bash, version 4.1.7(1)-release (amd64-portbld-freebsd8.0)
OS: FreeBSD 8.0
Hardware: amd64
Environment: jail
Description: read terminates reading all records at first null-byte ( chr(0) )
in a stream, null-bytes are valid ascii characters and should not cause read to
stop reading
a line this behavior is not reproducible using bourne shell.
Steps To Reproduce:
[bash ~]$ printf 'foo\0bar\n' | while read line; do echo "$line"; done
foo
[bash ~]$ # verify that printf is yielding the expected output
[bash ~]$ printf 'foo\0bar\n' | od -a
000f o o nul b a r nl
010
[bash ~]$ # verify that it is not just echo with awk
[bash ~]$ printf 'foo\0bar\n' | while read line; do awk -v line="$line" 'BEGIN
{ print line; }'; done | od -a
000f o o nl
004
[bash ~]$ # same awk test with a subshell and no read -- note that null-byte is
removed, but feed does not end
[bash ~]$ awk -v line=`printf 'foo\0bar\n'` 'BEGIN { print line; }' | od -a
000f o o b a r nl
007'
[bash ~]$ # behavior with multiple lines
[bash ~]$ printf 'foo\0bar\nbaz\n' | while read line; do echo "$line"; done
foo
baz
[bash ~]$ # behavior with read -r
[bash ~]$ printf 'foo\0bar\nbaz\n' | while read -r line; do echo "$line"; done
foo
baz
Behavior in bourne shell:
$ # note that the null-byte is dropped, but the line is read
$ printf 'foo\0bar\nbaz\n' | while read line; do echo "$line"; done | od -a
000f o o b a r nl b a z nl
013
$ # test with awk instead of echo
$ printf 'foo\0bar\nbaz\n' | while read line; do awk -v line="$line" 'BEGIN {
print line; }'; done | od -a
000f o o b a r nl b a z nl
013
bash process substitution process not awaited before next statement started?
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 -Wall
uname output: Linux torchio 2.6.32-5-amd64 #1 SMP Fri Sep 9 20:23:16 UTC 2011
x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu
Bash Version: 4.1
Patch Level: 5
Release Status: release
Description:
It looks as if bash is not waiting for a process-substition process
which is reading stdin to complete before bash moves on to executing
the next statement.
Repeat-By:
#!/bin/bash
LOCK_DIR=/tmp/$$
spew_and_slurp_with_lock()
{
local I
for ((I=0; I<1000; I++)); do
echo "some junk"
done > >(mkdir $LOCK_DIR; cat > /dev/null; rmdir $LOCK_DIR)
}
main()
{
local J
rm -fr $LOCK_DIR
for ((J=0; J<1000; J++)); do
spew_and_slurp_with_lock
done
}
main
Expected output: nothing
Actual output: rmdir errors ('cos dir already deleted) and mkdir
errors ('cos dir already exists)
A bit more info ...
The actual process in my real script's process substitution list was
sqlite3, which was randomly complaining that the database was locked;
for the purposes of demonstrating the problem mkdir+cat+rmdir is a
reasonable simulation of sqlite3 (both sqlite3 and mkdir+cat+rmdir
slurp stdin and use locking).
(I use a main() function here simply to allow be to below unambiguously
references bits of code above.)
The expected output was nothing. The actual output was:
mkdir: cannot create directory `/tmp/2076': File exists
rmdir: failed to remove `/tmp/2076': No such file or directory
mkdir: cannot create directory `/tmp/2076': File exists
rmdir: failed to remove `/tmp/2076': No such file or directory
...
The number of failing mkdir/rmdir pairs is not consistent:
fiori$ ./demo 2>&1 | wc -l
468
fiori$ ./demo 2>&1 | wc -l
470
fiori$ ./demo 2>&1 | wc -l
458
fiori$
I.e. somewhere between 20-25%. But that's just due to timing.
It seems to me that the process-substituted list has not finished
before bash moves on to executing the next commmand (in this case:
looping back round in main() to call spew_and_slurp_with_lock()
again). I.e. the N+1'th loop's mkdir is running before the N'th
loop's rmdir, and that results in the 'File exists' message.
The bash man page does not mention that the sustitute process runs
asynchronously, and, indeed, an added call to 'wait' immediately after
the 'for' loop in spew_and_slurp_with_lock() reaps nothing.
A second odd behaviour, which might just be another symptom of
an un-waited-for child process is that when the script finishes
the following things happen in the following order: I get a prompt,
an rmdir complains. Like this:
...
mkdir: cannot create directory `/tmp/12961': File exists
rmdir: failed to remove `/tmp/12961': No such file or directory
mkdir: cannot create directory `/tmp/12961': File exists
fiori$ rmdir: failed to remove `/tmp/12961': No such file or directory
Thanks!
Alexis
Re: bash process substitution process not awaited before next statement started?
On Wed, Nov 23, 2011 at 08:41:56AM +0100, ale...@pasta.net wrote:
> Description:
> It looks as if bash is not waiting for a process-substition process
> which is reading stdin to complete before bash moves on to executing
> the next statement.
Process substitutions are background processes. They are exactly the same
as doing:
mkfifo tmp-fifo
your job tmp-fifo
> spew_and_slurp_with_lock()
> {
> local I
>
> for ((I=0; I<1000; I++)); do
> echo "some junk"
> done > >(mkdir $LOCK_DIR; cat > /dev/null; rmdir $LOCK_DIR)
> }
In this case, the entire mkdir;cat;rmdir job is launched in the background,
with a file descriptor pointing to it. The for loop is being redirected to
that. The for loop should block until the cat command reads from the
pipe, at which point the main shell may move on. Meanwhile, the cat command
which is in the background does its processing, and terminates; and then
finally rmdir gets called.
So yes, the main shell and the rmdir could be doing things simultaneously.
That's how process substitutions are defined.
If you want to ensure that the main shell waits for the rmdir, then
you must abandon the process substitution syntax and use your own explicit
FIFOs:
mkfifo tmp-fifo
{ mkdir "$LOCK_DIR"; cat >/dev/null; rmdir "$LOCK_DIR"; } < tmp-fifo & pid=$!
for ... done > tmp-fifo
wait $pid
rm tmp-fifo
Only then will you be assured that things will run the way you need them to.
Bash-4.2 Official patch 11
BASH PATCH REPORT = Bash-Release: 4.2 Patch-ID: bash42-011 Bug-Reported-by:"David Parks" Bug-Reference-ID: <014101cc82c6$46ac1540$d4043fc0$@com> Bug-Reference-URL: http://lists.gnu.org/archive/html/bug-bash/2011-10/msg00031.html Bug-Description: Overwriting a value in an associative array causes the memory allocated to store the key on the second and subsequent assignments to leak. Patch (apply with `patch -p0'): *** ../bash-4.2-patched/assoc.c 2009-08-05 20:19:40.0 -0400 --- assoc.c 2011-10-04 20:23:07.0 -0400 *** *** 78,81 --- 78,86 if (b == 0) return -1; + /* If we are overwriting an existing element's value, we're not going to + use the key. Nothing in the array assignment code path frees the key + string, so we can free it here to avoid a memory leak. */ + if (b->key != key) + free (key); FREE (b->data); b->data = value ? savestring (value) : (char *)0; *** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010 --- patchlevel.hThu Feb 24 21:41:34 2011 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 10 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 11 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 12
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-012
Bug-Reported-by:Rui Santos
Bug-Reference-ID: <4e04c6d0.2020...@grupopie.com>
Bug-Reference-URL:
http://lists.gnu.org/archive/html/bug-bash/2011-06/msg00079.html
Bug-Description:
When calling the parser to recursively parse a command substitution within
an arithmetic expansion, the shell overwrote the saved shell input line and
associated state, resulting in a garbled command.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/parse.y 2011-02-26 19:19:05.0 -0500
--- parse.y 2011-06-24 20:08:22.0 -0400
***
*** 3843,3846
--- 3849,3853
{
sh_parser_state_t ps;
+ sh_input_line_state_t ls;
int orig_ind, nc, sflags;
char *ret, *s, *ep, *ostring;
***
*** 3850,3857
--- 3857,3866
ostring = string;
+ /*itrace("xparse_dolparen: size = %d shell_input_line = `%s'",
shell_input_line_size, shell_input_line);*/
sflags = SEVAL_NONINT|SEVAL_NOHIST|SEVAL_NOFREE;
if (flags & SX_NOLONGJMP)
sflags |= SEVAL_NOLONGJMP;
save_parser_state (&ps);
+ save_input_line_state (&ls);
/*(*/
***
*** 3862,3865
--- 3871,3876
restore_parser_state (&ps);
reset_parser ();
+ /* reset_parser clears shell_input_line and associated variables */
+ restore_input_line_state (&ls);
if (interactive)
token_to_read = 0;
***
*** 5909,5912
--- 5920,5929
ps->echo_input_at_read = echo_input_at_read;
+ ps->token = token;
+ ps->token_buffer_size = token_buffer_size;
+ /* Force reallocation on next call to read_token_word */
+ token = 0;
+ token_buffer_size = 0;
+
return (ps);
}
***
*** 5950,5953
--- 5967,6006
expand_aliases = ps->expand_aliases;
echo_input_at_read = ps->echo_input_at_read;
+
+ FREE (token);
+ token = ps->token;
+ token_buffer_size = ps->token_buffer_size;
+ }
+
+ sh_input_line_state_t *
+ save_input_line_state (ls)
+ sh_input_line_state_t *ls;
+ {
+ if (ls == 0)
+ ls = (sh_input_line_state_t *)xmalloc (sizeof (sh_input_line_state_t));
+ if (ls == 0)
+ return ((sh_input_line_state_t *)NULL);
+
+ ls->input_line = shell_input_line;
+ ls->input_line_size = shell_input_line_size;
+ ls->input_line_len = shell_input_line_len;
+ ls->input_line_index = shell_input_line_index;
+
+ /* force reallocation */
+ shell_input_line = 0;
+ shell_input_line_size = shell_input_line_len = shell_input_line_index = 0;
+ }
+
+ void
+ restore_input_line_state (ls)
+ sh_input_line_state_t *ls;
+ {
+ FREE (shell_input_line);
+ shell_input_line = ls->input_line;
+ shell_input_line_size = ls->input_line_size;
+ shell_input_line_len = ls->input_line_len;
+ shell_input_line_index = ls->input_line_index;
+
+ set_line_mbstate ();
}
*** ../bash-4.2-patched/shell.h 2011-01-06 22:16:55.0 -0500
--- shell.h 2011-06-24 19:12:25.0 -0400
***
*** 137,140
--- 139,145
int *token_state;
+ char *token;
+ int token_buffer_size;
+
/* input line state -- line number saved elsewhere */
int input_line_terminator;
***
*** 167,171
--- 172,186
} sh_parser_state_t;
+ typedef struct _sh_input_line_state_t {
+ char *input_line;
+ int input_line_index;
+ int input_line_size;
+ int input_line_len;
+ } sh_input_line_state_t;
+
/* Let's try declaring these here. */
extern sh_parser_state_t *save_parser_state __P((sh_parser_state_t *));
extern void restore_parser_state __P((sh_parser_state_t *));
+
+ extern sh_input_line_state_t *save_input_line_state
__P((sh_input_line_state_t *));
+ extern void restore_input_line_state __P((sh_input_line_state_t *));
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 11
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 12
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 13
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-013
Bug-Reported-by:Marten Wikstrom
Bug-Reference-ID:
Bug-Reference-URL:
http://lists.gnu.org/archive/html/bug-bash/2011-05/msg00049.html
Bug-Description:
An off-by-one error caused the shell to skip over CTLNUL characters,
which are used internally to mark quoted null strings. The effect
was to have stray 0x7f characters left after expanding words like
aa.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/subst.c 2011-03-06 14:11:11.0 -0500
--- subst.c 2011-05-11 11:23:33.0 -0400
***
*** 3707,3711
}
else if (string[i] == CTLNUL)
! i++;
prev_i = i;
--- 3710,3717
}
else if (string[i] == CTLNUL)
! {
! i++;
! continue;
! }
prev_i = i;
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 12
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 13
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 14
BASH PATCH REPORT = Bash-Release: 4.2 Patch-ID: bash42-014 Bug-Reported-by:Shawn Bohrer Bug-Reference-ID: <20110504152320.6e8f28130...@dev1.rgmadvisors.com> Bug-Reference-URL: http://lists.gnu.org/archive/html/bug-bash/2011-05/msg00018.html Bug-Description: The regular expression matching operator did not correctly match expressions with an embedded ^A. Patch (apply with `patch -p0'): *** ../bash-4.2-patched/pathexp.c 2010-08-13 23:21:57.0 -0400 --- pathexp.c 2011-05-05 16:40:58.0 -0400 *** *** 197,201 if ((qflags & QGLOB_FILENAME) && pathname[i+1] == '/') continue; ! if ((qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0) continue; temp[j++] = '\\'; --- 197,201 if ((qflags & QGLOB_FILENAME) && pathname[i+1] == '/') continue; ! if (pathname[i+1] != CTLESC && (qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0) continue; temp[j++] = '\\'; *** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010 --- patchlevel.hThu Feb 24 21:41:34 2011 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 13 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 14 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 15
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-015
Bug-Reported-by:
Bug-Reference-ID:
<728_1312188080_4e3666b0_728_118711_1_3b5d3e0f95cc5c478d6500cdce8b691f74a...@puexcb2b.nanterre.francetelecom.fr>
Bug-Reference-URL:
http://lists.gnu.org/archive/html/bug-bash/2011-08/msg0.html
Bug-Description:
When in a context where arithmetic evaluation is not taking place, the
evaluator should not check for division by 0.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/expr.c 2010-12-21 11:12:13.0 -0500
--- expr.c 2011-08-02 20:58:28.0 -0400
***
*** 477,480
--- 481,492
if (special)
{
+ if ((op == DIV || op == MOD) && value == 0)
+ {
+ if (noeval == 0)
+ evalerror (_("division by 0"));
+ else
+ value = 1;
+ }
+
switch (op)
{
***
*** 483,493
break;
case DIV:
- if (value == 0)
- evalerror (_("division by 0"));
lvalue /= value;
break;
case MOD:
- if (value == 0)
- evalerror (_("division by 0"));
lvalue %= value;
break;
--- 495,501
***
*** 805,809
if (((op == DIV) || (op == MOD)) && (val2 == 0))
! evalerror (_("division by 0"));
if (op == MUL)
--- 813,822
if (((op == DIV) || (op == MOD)) && (val2 == 0))
! {
! if (noeval == 0)
! evalerror (_("division by 0"));
! else
! val2 = 1;
! }
if (op == MUL)
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 14
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 15
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 16
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-016
Bug-Reported-by:Martin von Gagern
Bug-Reference-ID: <4e43ad9e.8060...@gmx.net>
Bug-Reference-URL:
http://lists.gnu.org/archive/html/bug-bash/2011-08/msg00141.html
Bug-Description:
Bash should not check for mail while executing the `eval' builtin.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/parse.y Fri Feb 25 12:07:41 2011
--- parse.y Thu Aug 11 19:02:26 2011
***
*** 2500,2504
is the mail alarm reset; nothing takes place in check_mail ()
except the checking of mail. Please don't change this. */
! if (prompt_is_ps1 && time_to_check_mail ())
{
check_mail ();
--- 2498,2502
is the mail alarm reset; nothing takes place in check_mail ()
except the checking of mail. Please don't change this. */
! if (prompt_is_ps1 && parse_and_execute_level == 0 && time_to_check_mail
())
{
check_mail ();
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 15
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 16
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 17
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-017
Bug-Reported-by:Curtis Doty
Bug-Reference-ID: <20110621035324.a4f70849...@mx1.iparadigms.net>
Bug-Reference-URL:
http://lists.gnu.org/archive/html/bug-bash/2011-06/msg00053.html
Bug-Description:
Using `read -a foo' where foo was an already-declared associative array
caused the shell to die with a segmentation fault.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/builtins/read.def 2011-01-04 11:43:36.0
-0500
--- builtins/read.def 2011-06-21 10:31:02.0 -0400
***
*** 643,646
--- 642,651
return EXECUTION_FAILURE; /* readonly or noassign */
}
+ if (assoc_p (var))
+ {
+ builtin_error (_("%s: cannot convert associative to indexed
array"), arrayname);
+ xfree (input_string);
+ return EXECUTION_FAILURE; /* existing associative array */
+ }
array_flush (array_cell (var));
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 16
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 17
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 18
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-018
Bug-Reported-by:Thomas Cort
Bug-Reference-ID:
Bug-Reference-URL:
http://lists.gnu.org/archive/html/bug-bash/2011-06/msg00110.html
Bug-Description:
Bash fails to compile unless JOB_CONTROL is defined.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/execute_cmd.c 2011-02-09 17:32:25.0 -0500
--- execute_cmd.c 2011-11-06 15:12:48.0 -0500
***
*** 2197,2200
--- 2315,2319
cmd->flags |= CMD_IGNORE_RETURN;
+ #if defined (JOB_CONTROL)
lastpipe_flag = 0;
begin_unwind_frame ("lastpipe-exec");
***
*** 2216,2228
add_unwind_protect (lastpipe_cleanup, lastpipe_jid);
}
! cmd->flags |= CMD_LASTPIPE;
}
if (prev >= 0)
add_unwind_protect (close, prev);
exec_result = execute_command_internal (cmd, asynchronous, prev, pipe_out,
fds_to_close);
if (lstdin > 0)
restore_stdin (lstdin);
if (prev >= 0)
--- 2335,2351
add_unwind_protect (lastpipe_cleanup, lastpipe_jid);
}
! if (cmd)
! cmd->flags |= CMD_LASTPIPE;
}
if (prev >= 0)
add_unwind_protect (close, prev);
+ #endif
exec_result = execute_command_internal (cmd, asynchronous, prev, pipe_out,
fds_to_close);
+ #if defined (JOB_CONTROL)
if (lstdin > 0)
restore_stdin (lstdin);
+ #endif
if (prev >= 0)
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 17
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 18
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 19
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-019
Bug-Reported-by:Diego Augusto Molina
Bug-Reference-ID:
Bug-Reference-URL:
lists.gnu.org/archive/html/bug-bash/2011-09/msg00047.html
Bug-Description:
Using `declare' with attributes and an invalid array variable name or
assignment reference resulted in a segmentation fault instead of a
declaration error.
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/builtins/declare.def2010-05-30 18:25:21.0
-0400
--- builtins/declare.def2011-09-15 15:20:20.0 -0400
***
*** 514,517
--- 514,522
var = assign_array_element (name, value, 0); /* XXX - not
aflags */
*subscript_start = '\0';
+ if (var == 0) /* some kind of assignment error */
+ {
+ assign_error++;
+ NEXT_VARIABLE ();
+ }
}
else if (simple_array_assign)
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 18
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 19
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bash-4.2 Official patch 20
BASH PATCH REPORT
=
Bash-Release: 4.2
Patch-ID: bash42-020
Bug-Reported-by:Vincent Sheffer
Bug-Reference-ID:
Bug-Reference-URL:
https://lists.gnu.org/archive/html/bug-readline/2011-08/msg0.html
Bug-Description:
The shared object helper script needs to be updated for Mac OS X 10.7
(Lion, darwin11).
Patch (apply with `patch -p0'):
*** ../bash-4.2-patched/support/shobj-conf 2009-10-28 09:20:21.0
-0400
--- support/shobj-conf 2011-08-27 13:25:23.0 -0400
***
*** 158,162
# Darwin/MacOS X
! darwin[89]*|darwin10*)
SHOBJ_STATUS=supported
SHLIB_STATUS=supported
--- 172,176
# Darwin/MacOS X
! darwin[89]*|darwin1[012]*)
SHOBJ_STATUS=supported
SHLIB_STATUS=supported
***
*** 187,191
case "${host_os}" in
! darwin[789]*|darwin10*) SHOBJ_LDFLAGS=''
SHLIB_XLDFLAGS='-dynamiclib -arch_only `/usr/bin/arch`
-install_name $(libdir)/$@ -current_version $(SHLIB_MAJOR)$(SHLIB_MINOR)
-compatibility_version $(SHLIB_MAJOR) -v'
;;
--- 201,205
case "${host_os}" in
! darwin[789]*|darwin1[012]*) SHOBJ_LDFLAGS=''
SHLIB_XLDFLAGS='-dynamiclib -arch_only `/usr/bin/arch`
-install_name $(libdir)/$@ -current_version $(SHLIB_MAJOR)$(SHLIB_MINOR)
-compatibility_version $(SHLIB_MAJOR) -v'
;;
*** ../bash-4.2-patched/patchlevel.hSat Jun 12 20:14:48 2010
--- patchlevel.hThu Feb 24 21:41:34 2011
***
*** 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 19
#endif /* _PATCHLEVEL_H_ */
--- 26,30
looks for to find the patch level (for the sccs version string). */
! #define PATCHLEVEL 20
#endif /* _PATCHLEVEL_H_ */
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
On Wed, Nov 23, 2011 at 09:03:33AM -0500, Matthew Story wrote:
> Description: read terminates reading all records at first null-byte ( chr(0)
> ) in a stream, null-bytes are valid ascii characters and should not cause
> read to stop reading
> a line this behavior is not reproducible using bourne shell.
Actually, read does not stop reading at a NUL byte unless you tell it to
by using -d ''. It reads until the newline.
> Steps To Reproduce:
>
> [bash ~]$ printf 'foo\0bar\n' | while read line; do echo "$line"; done
> foo
What happens here is bash reads until the newline, but only the "foo"
part is visible in the variable, because the NUL effective ends the
string. Whether the "bar" part is copied into memory or not is not
relevant and cannot be determined without poking around inside the bash
process itself.
If bash had actually stopped reading at the NUL, then the "bar" part
would still be sitting there on stdin waiting for the next read. But
we can see that this is not the case:
$ printf 'foo\0bar\n' | { read line; echo "$line"; read line; echo "$line"; }
foo
$
The second read gets nothing. If we want to see bash stop at the NUL, then
using -d '' will accomplish that:
$ printf 'foo\0bar\n' | { read -d '' line; echo "$line"; read line; echo
"$line"; }
foo
bar
$
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
On 11/23/11 9:03 AM, Matthew Story wrote: > Bash Version: GNU bash, version 4.1.7(1)-release (amd64-portbld-freebsd8.0) > OS: FreeBSD 8.0 > Hardware: amd64 > Environment: jail > Description: read terminates reading all records at first null-byte ( chr(0) > ) in a stream, null-bytes are valid ascii characters and should not cause > read to stop reading > a line this behavior is not reproducible using bourne shell. Bash doesn't stop reading at the NUL; it reads it and the rest of the line up to a newline. Since bash treats the line read as a C string, the NUL terminates the value assigned to `foo'. Bash doesn't drop NULs like the FreeBSD (not the Bourne) shell. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: bash tab variable expansion question?
> On 10/17/11 3:07 PM, Michael Kalisz wrote:
> > Hi Chet,
> >
> > The shopt "direxpand" feature works as advertised (Thanks!) except that I
> > noticed it seems to break the name-completion of executables which are not
> > in you path.
>
> Yes, it expands the directory name. In this case, it expands `.' to $PWD,
> and the differing prefixes make the names not match. I'll have to see
> what I can do about that. It might be a nasty fix, since the same function
> has to make sure that `.' does expand to $PWD to honor the shell's logical
> view of the file system. I'm not sure what negative consequences dropping
> that rewrite might bring.
I found the easy way out and took it. Try the attached patch and see if
it fixes the problem for you. It does for my testing.
Chet
*** ../bash-4.2-direxpand/bashline.c2011-11-23 17:11:31.0 -0500
--- bashline.c 2011-11-05 18:46:35.0 -0400
***
*** 1292,1295
--- 1339,1343
rl_filename_quote_characters = default_filename_quote_characters;
+ set_directory_hook ();
/* Determine if this could be a command word. It is if it appears at
***
*** 1605,1608
--- 1672,1681
else
{
+if (dircomplete_expand && dot_or_dotdot (filename_hint))
+ {
+ dircomplete_expand = 0;
+ set_directory_hook ();
+ dircomplete_expand = 1;
+ }
mapping_over = 4;
goto inner;
***
*** 1805,1808
--- 1878,1884
inner:
val = rl_filename_completion_function (filename_hint, istate);
+ if (mapping_over == 4 && dircomplete_expand)
+ set_directory_hook ();
+
istate = 1;
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Bash-4.2 Official patch 12
--- parse.y +++ parse.y @@ -5983,6 +5983,8 @@ save_input_line_state (ls) /* force reallocation */ shell_input_line = 0; shell_input_line_size = shell_input_line_len = shell_input_line_index = 0; + + return ls; } void Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: Bash-4.2 Official patch 12
On 11/23/11 6:51 PM, Andreas Schwab wrote: > --- parse.y > +++ parse.y > @@ -5983,6 +5983,8 @@ save_input_line_state (ls) >/* force reallocation */ >shell_input_line = 0; >shell_input_line_size = shell_input_line_len = shell_input_line_index = 0; > + > + return ls; > } Thanks. The return value isn't used, so it's not fatal. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
On Nov 23, 2011, at 4:47 PM, Chet Ramey wrote: > On 11/23/11 9:03 AM, Matthew Story wrote: >> Bash Version: GNU bash, version 4.1.7(1)-release (amd64-portbld-freebsd8.0) >> OS: FreeBSD 8.0 >>Hardware: amd64 >> Environment: jail >> Description: read terminates reading all records at first null-byte ( chr(0) >> ) in a stream, null-bytes are valid ascii characters and should not cause >> read to stop reading >> a line this behavior is not reproducible using bourne shell. > > Bash doesn't stop reading at the NUL; it reads it and the rest of the line > up to a newline. Since bash treats the line read as a C string, the NUL > terminates the value assigned to `foo'. it seems to terminate all assignment of the current line at the first `\0', not merely the value assigned to `foo': [bash ~]$ printf '%s\0 %s\n%s\n' foo bar baz | while read foo bar; do echo "$foo" "$bar"; done | od -a 000f o o sp nl b a z sp nl 012 it's clear from this example (and from what you've said above) that read does not halt entirely here as it processes the next line correctly, and ` bar' is not left in the buffer after first read. What I find confusing about this comment and Greg's comment (copied in below) ... > [Greg Wooledge ] > What happens here is bash reads until the newline, but only the "foo" > part is visible in the variable, because the NUL effective ends the > string. ... is that if the line were fully assigned along the lines of my understanding of read, foo should be `foo', up to the null-bye which effectively terminates the string, and then bar should be `bar'. > Bash doesn't drop NULs like the > FreeBSD (not the Bourne) shell. FreeBSD sh indeed, apologies for the misstatement. > > Chet > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer >``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ >
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
On 11/23/11 6:54 PM, Matthew Story wrote: > On Nov 23, 2011, at 4:47 PM, Chet Ramey wrote: > >> On 11/23/11 9:03 AM, Matthew Story wrote: >>> Bash Version: GNU bash, version 4.1.7(1)-release (amd64-portbld-freebsd8.0) >>> OS: FreeBSD 8.0 >>>Hardware: amd64 >>> Environment: jail >>> Description: read terminates reading all records at first null-byte ( >>> chr(0) ) in a stream, null-bytes are valid ascii characters and should not >>> cause read to stop reading >>> a line this behavior is not reproducible using bourne shell. >> >> Bash doesn't stop reading at the NUL; it reads it and the rest of the line >> up to a newline. Since bash treats the line read as a C string, the NUL >> terminates the value assigned to `foo'. > > it seems to terminate all assignment of the current line at the first `\0', > not merely the value assigned to `foo': Yes, sorry. That's what the "bash treats the line read as a C string" was intended to imply. Since the line read is a C string, the NUL terminates it and what remains is assigned to the named variables. I should have used `line' in my explanation instead of `foo'. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
>> Bash doesn't drop NULs like the >> FreeBSD (not the Bourne) shell. one last note on bash dropping NULs: [bash ~]$ foo=`printf 'foo\0bar'` [bash ~]$ echo $foo |od -a 000f o o b a r nl 007 > > FreeBSD sh indeed, apologies for the misstatement. > >> >> Chet >> -- >> ``The lyf so short, the craft so long to lerne.'' - Chaucer >> ``Ars longa, vita brevis'' - Hippocrates >> Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ >> >
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
On Nov 23, 2011, at 7:09 PM, Chet Ramey wrote: > On 11/23/11 6:54 PM, Matthew Story wrote: >> On Nov 23, 2011, at 4:47 PM, Chet Ramey wrote: >> >>> On 11/23/11 9:03 AM, Matthew Story wrote: [... snip] > > Yes, sorry. That's what the "bash treats the line read as a C string" > was intended to imply. Since the line read is a C string, the NUL > terminates it and what remains is assigned to the named variables. I > should have used `line' in my explanation instead of `foo'. I understand that the underlying implementation of the bash builtins is `C', and I understand that `C' stings are NUL terminated. It seems unreasonable to me to expect understanding of this implementation detail when using bash to read streams into variables via the `read' builtin. Further-more, neither the man-page nor the gnu website document this behavior of bash: read read [-ers] [-a aname] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [-t timeout] [-u fd] [name ...] One line is read from the standard input, or from the file descriptor fd supplied as an argument to the -u option, and the first word is assigned to the first name, the second word to the secondname, and so on, with leftover words and their intervening separators assigned to the last name. If there are fewer words read from the input stream than names, the remaining names are assigned empty values. The characters in the value of the IFS variable are used to split the line into words. The backslash character ‘\’ may be used to remove any special meaning for the next character read and for line continuation. If no names are supplied, the line read is assigned to the variable REPLY. The return code is zero, unless end-of-file is encountered, read times out (in which case the return code is greater than 128), or an invalid file descriptor is supplied as the argument to -u. I personally do not read "One line" as meaning "One string of characters terminated either by a null byte or a new-line", I read it as "One string of characters terminated by a new-line". But "One string of characters terminated either by a null byte or a new line" is not the actual functionality. The actual functionality is: "One line is read from the standard input, or from the file descriptor fd supplied as an argument to the -u option, then read byte-wise up to the first contained NUL, or end of string, ..." Furthermore, I do not see the use-case for this behavior ... I simply cannot fathom a case of I/O redirection in shell where I would choose to inject a NUL byte to coerce this sort of behavior from the read builtin, and can't imagine that anyone is relying on this `C string' feature of read currently in bash, especially considering that it is not consistent with NUL handling in other assignments in bash: [matt@matt0 ~]$ foo=`printf 'foo\0bar'`; echo "$foo" | od -a 000f o o b a r nl 007 [bash ~]$ foo=$(printf 'foo\0bar'); echo "$foo" | od -a 000f o o b a r nl 007 which strip NUL. I see one of three possible resolutions here: 1. NUL bytes do not terminate variable assignment from `read', behavior of echo/variable assignments persists as is 2. NUL bytes are stripped by read on assignment, and this functionality is documented as expected. 3. the existing functionality of the system is documented in the man-page and on gnu.org as expected I would prefer the first, and would be happy to attempt in providing a patch, if that's useful. cheers, -matt > > Chet > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer >``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ Additional Notes: The only occurrence of the pattern `NUL' in the FreeBSD man-page for bash is: Pattern Matching Any character that appears in a pattern, other than the special pattern characters described below, matches itself. The NUL character may not occur in a pattern. A backslash escapes the following character; the escaping backslash is discarded when matching. The special pattern characters must be quoted if they are to be matched literally. All other references in the man-page are to the null string (empty string) not to an explicit NUL byte (e.g. ascii 0), the same is true of the gnu.org documentation.
Bash git repository on savannah
I spent a little while messing around with git over the past couple of days, and ended up updating the bash git repository on savannah (http://git.savannah.gnu.org/cgit/bash.git to browse the sources). Bash-4.2 patch 20 is the head of the tree, and there's a branch containing the `direxpand' patches that I've posted here. Each bash-4.2 patch is in there as a separate commit. All the released sources back to bash-1.14.7 are there, building on the work Jari Aalto started a couple of years ago. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: read fails on null-byte: v4.1.7 FreeBSD 8.0 (amd64)
On 11/23/11 9:44 PM, Matthew Story wrote: > > On Nov 23, 2011, at 7:09 PM, Chet Ramey wrote: > >> On 11/23/11 6:54 PM, Matthew Story wrote: >>> On Nov 23, 2011, at 4:47 PM, Chet Ramey wrote: >>> On 11/23/11 9:03 AM, Matthew Story wrote: > [... snip] >> >> Yes, sorry. That's what the "bash treats the line read as a C string" >> was intended to imply. Since the line read is a C string, the NUL >> terminates it and what remains is assigned to the named variables. I >> should have used `line' in my explanation instead of `foo'. > > I understand that the underlying implementation of the bash builtins is > `C', and I understand that `C' stings are NUL terminated. It seems > unreasonable to me to expect understanding of this implementation detail > when using bash to read streams into variables via the `read' builtin. I took a look around at Posix and some other shells. Posix passes on the issue completely: the input to read may not contain NUL bytes at all. The Bourne shell, from v7 to SVR4.2, uses NUL as a line terminator. Other shells, including ksh93, ash and pdksh derivatives like dash and mksh, discard NUL bytes in read. zsh doesn't discard NULs and handles them pretty well, putting them into a variable's value. The discard behavior seems fairly standard, and I will look at putting it into the next version of bash. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
