Re: Regression -- can't read input w/stderr redirect
On Mon, Jun 19, 2017 at 5:17 AM, L A Walsh wrote: > > > Chet Ramey wrote: > >> On 6/18/17 6:59 PM, L A Walsh wrote: >> >> >>> Chet Ramey wrote: >>> >>> Bash has always stripped NULL bytes. Now it tells you it's doing it. >>> Why? Did I toggle a flag asking for the warning? Seems like it >>> was sorta sprung on users w/no way to disable it. >>> >>> >> >> Users asked why bash transformed input without warning, even though it >> had been doing that it's entire lifetime. A warning is appropriate. >> >> > Maybe - but links to, at least, 2-3 users who filed bug reports about this > problem in bug-bash would be appropriate as well to justify the inclusion > of the text. > > I don't recall it ever coming up until the warning message was discussed > as being unwelcome. So please, I'd like to see the bug-report filings > where this > happened. > > > >>>But things are changing -- people have asked for zero-terminated >>> read's >>> and readarrays. More unix utils are offering NUL termination as an >>> option >>> because newlines alone don't cut it in some instances. >>> >>> >> >> And bash provides mechanisms to deal with the relatively few use cases >> where it is a problem. >> >> Recall that the only thing that has changed is that bash now provides a >> warning about what it's doing. >> >> > Oh? I want to read in a value from the registry where something may have > zeros > after the data. Please tell me the mechanism to read this in w/no warnings > that won't silence more serious cases of zero's other than at the end of > line. > > I want to see the hyperlinks to archived bug-discussions on bug-bash where > users complained about this and where it was at the end of a string where > they expected to be able to read past a binary-0 in the input. > > I know I would have like the ability to read binary data into to a var that > might "include a NUL", but I don't recall ever complaining about > end-of-string > NUL's being trimmed -- and it was drummed home to me how the null's were > the > end of the string -- not how bash read everything but nulls from input. > > I'm sorry to say that your behavior on this list is just not acceptable. If you were on IRC I would have banned you much earlier, yet after all these years of trolling the list, Chet is going to great length to explain the rationale of his choices while you keep whining for the shell to just do what you want in the random particular case you happen to work at the moment. Your response: you accuse him to lie to you. I don't think this is constructive in any way and I'm sure that, even if Chet has probably experienced this kind of online situation more than most, it's not a pleasant one. Pierre.
Re: RFE: Fix the name collision issues and typing issues with namerefs, improve various issues for function libraries
On Mon, Jun 19, 2017 at 12:07:31AM -0400, George wrote: > ultimately identified in the nameref also comes from the caller) but the > inconsistency there, of "typeset -n" behaving differently depending on where > the name comes from, is very unappealing to me. I'd much rather see this > behavior activated by an explicit argument to declare, like this: > declare --caller-scope-nameref result_var=$whatever > Or this: > declare -n --scope=caller result_var=$whatever Or this: upvar "$1" myvar ~50% joking...
Re: RFE: Fix the name collision issues and typing issues with namerefs, improve various issues for function libraries
As a bonus you'd be able to lure the uninitiated into asking questions like "What's upvar?" - Original Message - From: "Greg Wooledge" To:"bug-bash" Cc: Sent:Mon, 19 Jun 2017 08:18:14 -0400 Subject:Re: RFE: Fix the name collision issues and typing issues with namerefs, improve various issues for function libraries On Mon, Jun 19, 2017 at 12:07:31AM -0400, George wrote: > ultimately identified in the nameref also comes from the caller) but the inconsistency there, of "typeset -n" behaving differently depending on where > the name comes from, is very unappealing to me. I'd much rather see this behavior activated by an explicit argument to declare, like this: > declare --caller-scope-nameref result_var=$whatever > Or this: > declare -n --scope=caller result_var=$whatever Or this: upvar "$1" myvar ~50% joking...
Re: Regression -- can't read input w/stderr redirect
On 6/18/17 10:17 PM, L A Walsh wrote: > > > Chet Ramey wrote: >> On 6/18/17 6:59 PM, L A Walsh wrote: >> >>> Chet Ramey wrote: >>> Bash has always stripped NULL bytes. Now it tells you it's doing it. >>> Why? Did I toggle a flag asking for the warning? Seems like it >>> was sorta sprung on users w/no way to disable it. I was wondering why you chose this hill to die on, but then I remembered you've chosen so many in the past. The worst part is that you could have spent all this time developing a patch for the behavior you want, but you chose to spend it complaining. There is (once again) no path forward for this conversation that isn't a huge waste of time. I don't plan to continue it. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: RFE: Fix the name collision issues and typing issues with namerefs, improve various issues for function libraries
On 6/19/17 9:12 AM, tetsu...@scope-eye.net wrote: > > > As a bonus you'd be able to lure the uninitiated into asking questions > like "What's upvar?" The same old, what's up with you? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Regression -- can't read input w/stderr redirect
On Sun, Jun 18, 2017 at 07:17:46PM -0700, L A Walsh wrote: > Oh? I want to read in a value from the registry where something may > have zeros > after the data. Please tell me the mechanism to read this in w/no warnings > that won't silence more serious cases of zero's other than at the end of > line. IFS= read -rd '' regvalue < <(regcommand) This will read the content from the stream provided by regcommand, up to the first NUL, where it will stop. This is the general method used to read a NUL-delimited value from a command or pseudo-file. A command substitution is NOT preferred, because it converts the stream to a string (by dropping all NUL bytes, including those that are in between two separate values). An analogous common use-case on Linux: wooledg:~$ hd /proc/$$/cmdline 62 61 73 68 00|bash.| 0005 wooledg:~$ IFS= read -rd '' foo < /proc/$$/cmdline wooledg:~$ declare -p foo declare -- foo="bash" If your regcommand produces a stream with multiple items in it, like this: foo\0bar\0quux\0 Then the IFS= read -rd '' regvalue command will read "foo" (the first value) and stop, whereas a command substitution will give you the string "foobarquux" which is nonsense. (And in bash 4.4, you also get a warning, because doing that is bad. You should stop doing that.) Now, most of us on this mailing list do not use Microsoft Windows, and do not know what commands you are using, or what output they produce. So, if you want any more specific advice, you'll have to go back a step and explain what you're doing, what input you're dealing with, and what results you expect to achieve. It also helps if you use standard bash commands, and not your idiosyncratic aliases like "my" and "int". Some of us may remember that you do things like alias my=declare, but others will not. When reporting a bug, you want your report to be easily understandable and reproducible, without external references or guessing.
Re: Read builtin -e (readline) oddities
On 6/18/17 9:39 PM, Eduardo A. Bustamante López wrote: > I'm trying to figure out a way to fuzz >>read -e -d ""<<, without having the > fuzzer break due to the temporary files created by fc. > > While doing this, I noticed the oddities described below. > > > #1 > Hit `C-x C-e' twice. The value of PATH seems to be ignored for the second > line. I can't reproduce this. I suspect one of your startup files is somehow being read and setting PATH. It may also be an interaction with the command-not-found hook. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Read builtin -e (readline) oddities
On Mon, Jun 19, 2017 at 10:08:30AM -0400, Chet Ramey wrote:
[...]
> I can't reproduce this. I suspect one of your startup files is somehow
> being read and setting PATH. It may also be an interaction with the
> command-not-found hook.
Hm, I can still reproduce it under Debian 9, using the `devel' branch, and I'm
sure no startup files are being sourced.
dualbus@debian:~/src/gnu/bash-builds/devel$ ./bash --norc --noprofile
bash-4.4$ declare -p command-not-found
bash: declare: command-not-found: not found
bash-4.4$
GNU bash, version 4.4.12(3)-maint (x86_64-unknown-linux-gnu)
bash-4.4$ PATH= read -e
bash: vim: No such file or directory
echo this is vim
this is vim
I then patched bash like this:
dualbus@debian:~/src/gnu/bash$ git diff -- execute_cmd.c
diff --git a/execute_cmd.c b/execute_cmd.c
index 0183a105..50f4e2f0 100644
--- a/execute_cmd.c
+++ b/execute_cmd.c
@@ -5550,6 +5550,7 @@ shell_execve (command, args, env)
char sample[HASH_BANG_BUFSIZ];
int sample_len;
+ if (strlen(get_string_value ("PATH")) > 0) abort();
SETOSTYPE (0); /* Some systems use for USG/POSIX semantics */
execve (command, args, env);
i = errno; /* error from execve() */
And I got (the first `\C-x \C-e' gets to shell_execve, but PATH is empty):
dualbus@debian:~/src/gnu/bash-builds/devel$ ./bash --norc --noprofile
bash-4.4$ ulimit -c unlimited
bash-4.4$
GNU bash, version 4.4.12(9)-maint (x86_64-unknown-linux-gnu)
bash-4.4$ PATH= read -e
bash: vim: No such file or directory
Aborted (core dumped)
Core was generated by `./bash --norc --noprofile'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x7f30b14c93fa in __GI_abort () at abort.c:89
#2 0x55a0df6527c4 in shell_execve (command=0x55a0e0a21020
"/usr/bin/vim", args=0x55a0e0a64610, env=0x55a0e0a207f0) at
../../bash/execute_cmd.c:5553
#3 0x55a0df6525b1 in execute_disk_command (words=0x55a0e0a63c20,
redirects=0x0, command_line=0x55a0e0a20f60 "vim /tmp/bash-fc.HTeaRz",
pipe_in=-1, pipe_out=-1, async=0,
fds_to_close=0x55a0e0a42d80, cmdflags=0) at ../../bash/execute_cmd.c:5348
#4 0x55a0df650d4f in execute_simple_command
(simple_command=0x55a0e0a62780, pipe_in=-1, pipe_out=-1, async=0,
fds_to_close=0x55a0e0a42d80) at ../../bash/execute_cmd.c:4466
#5 0x55a0df64a465 in execute_command_internal (command=0x55a0e0a20e80,
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55a0e0a42d80) at
../../bash/execute_cmd.c:811
#6 0x55a0df6ad6a8 in parse_and_execute (string=0x55a0e0a61b10 "vim
/tmp/bash-fc.HTeaRz", from_file=0x55a0df71cecd "fc", flags=4) at
../../../bash/builtins/evalstring.c:430
#7 0x55a0df6aee93 in fc_builtin (list=0x0) at
../../../bash/builtins/../../../bash/builtins/fc.def:438
#8 0x55a0df6511cb in execute_builtin (builtin=0x55a0df6ae4ae
, words=0x55a0e0a625d0, flags=0, subshell=0) at
../../bash/execute_cmd.c:4609
#9 0x55a0df6520a7 in execute_builtin_or_function (words=0x55a0e0a625d0,
builtin=0x55a0df6ae4ae , var=0x0, redirects=0x0,
fds_to_close=0x55a0e0a63df0, flags=0)
at ../../bash/execute_cmd.c:5107
#10 0x55a0df650ad1 in execute_simple_command
(simple_command=0x55a0e0a62540, pipe_in=-1, pipe_out=-1, async=0,
fds_to_close=0x55a0e0a63df0) at ../../bash/execute_cmd.c:4395
#11 0x55a0df64a465 in execute_command_internal (command=0x55a0e0a20400,
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55a0e0a63df0) at
../../bash/execute_cmd.c:811
#12 0x55a0df6ad6a8 in parse_and_execute (string=0x55a0e0a613b0 "fc -e
\"${VISUAL:-${EDITOR:-emacs}}\"", from_file=0x55a0df70ccd5 "C-xC-e", flags=4)
at ../../../bash/builtins/evalstring.c:430
#13 0x55a0df696203 in edit_and_execute_command (count=1, c=5,
editing_mode=1, edit_command=0x55a0df70cd10 "fc -e
\"${VISUAL:-${EDITOR:-emacs}}\"") at ../../bash/bashline.c:977
#14 0x55a0df6962f0 in emacs_edit_and_execute_command (count=1, c=5) at
../../bash/bashline.c:1013
#15 0x55a0df6dccd5 in _rl_dispatch_subseq (key=5, map=0x55a0df9461a0
, got_subseq=0) at
../../../../bash/lib/readline/readline.c:851
#16 0x55a0df6dd19a in _rl_dispatch_subseq (key=24, map=0x55a0df944160
, got_subseq=0) at
../../../../bash/lib/readline/readline.c:985
#17 0x55a0df6dca4c in _rl_dispatch (key=-1320386609, map=0x55a0df944160
) at ../../../../bash/lib/readline/readline.c:797
#18 0x55a0df6dc6d4 in readline_internal_char () at
../../../../bash/lib/readline/readline.c:629
#19 0x55a0df6dc72c in readline_internal_charloop () at
../../../../bash/lib/readline/readline.c:656
#20 0x55a0df6dc750 in readline_internal () at
../../../../bash/lib/readlin
Re: Read builtin -e (readline) oddities
On Mon, Jun 19, 2017 at 9:57 AM, Eduardo A. Bustamante López wrote: [...] > Hm, I can still reproduce it under Debian 9, using the `devel' branch, and I'm > sure no startup files are being sourced. > > dualbus@debian:~/src/gnu/bash-builds/devel$ ./bash --norc --noprofile > bash-4.4$ declare -p command-not-found > bash: declare: command-not-found: not found Heh, actually: dualbus@debian:~/src/gnu/bash-builds/devel$ ./bash --norc --noprofile bash-4.4$ declare -fp command_not_found_handle bash: declare: command_not_found_handle: not found The rest is correct though.
Re: Read builtin -e (readline) oddities
On 6/18/17 9:39 PM, Eduardo A. Bustamante López wrote: > #1 > Hit `C-x C-e' twice. The value of PATH seems to be ignored for the second > line. The shell should ensure that the temporary env is preserved across the entire execution of any builtin that can call parse_and_execute (like `fc' and `read' when using `read -e'). > #2 > Spurious `;'s are inserted into the history. The fc builtin (via edit_and_execute_command) shouldn't try to insert empty lines into the history list. The only time this can happen is via `read -e'. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Read builtin -e (readline) oddities
On 6/18/17 9:39 PM, Eduardo A. Bustamante López wrote:
> I'm trying to figure out a way to fuzz >>read -e -d ""<<, without having the
> fuzzer break due to the temporary files created by fc.
>
> While doing this, I noticed the oddities described below.
>
>
> #1
> Hit `C-x C-e' twice. The value of PATH seems to be ignored for the second
> line.
>
> dualbus@debian:~$ PATH= read -e
>
> bash: vim: No such file or directory
>
> echo hi
> hi
>
>
> #2
> Spurious `;'s are inserted into the history.
>
> dualbus@debian:~$ history -c
> dualbus@debian:~$ PATH= read -e
>
> bash: vim: No such file or directory
>
> echo hi
> hi
>
> dualbus@debian:~$ history
> 1 PATH= read -e; ;
> 2 echo hi
> 3 history
>
>
> #3
> Hit `C-x C-e' thrice. Also, the mapping of `\C-m' became `self-insert' instead
> of `accept-line'.
`read -e' changes it because it's no longer the line delimiter.
> dualbus@debian:~$ PATH= EDITOR=: read -e -d ''
>
> PATH= EDITOR=: read -e -d '';
>
> fc -e "${VISUAL:-${EDITOR:-$(command -v editor || echo emacs)}}"
> : /tmp/bash-fc.IZAzmF
> PATH= EDITOR=: read -e -d ''; ;
> bash: syntax error near unexpected token `;'
>
> fc -e "${VISUAL:-${EDITOR:-$(command -v editor || echo emacs)}}"
> vim /tmp/bash-fc.35ePED
> ^M^M^C
I'll have to check why it doesn't get restored on ^C.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
"realpath" loadable (in the examples/loadables dir) infinite loops
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu' -DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -g -O2 -Wno-parentheses -Wno-format-security uname output: Linux shell 4.4.0-79-generic #100-Ubuntu SMP Wed May 17 19:58:14 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-unknown-linux-gnu Bash Version: 4.4 Patch Level: 0 Release Status: release Description: Command "realpath --help" causes infinite loop. Repeat-By: enable -f /usr/local/lib/bash/realpath realpath realpath --help causes infinite loop of: realpath: usage: realpath [-csv] pathname [pathname...] realpath: usage: realpath [-csv] pathname [pathname...] realpath: usage: realpath [-csv] pathname [pathname...] realpath: usage: realpath [-csv] pathname [pathname...] realpath: usage: realpath [-csv] pathname [pathname...] realpath: usage: realpath [-csv] pathname [pathname...] until the shell is killed by signal 9 from another terminal.
[PATCH] Unix domain socket filename redirection
This patch adds support for Unix domain sockets. A filename of the form
/dev/unixstream/path used in a redirection makes bash attempt to open a
connection to the Unix domain stream socket at /path. Similarly a filename
of the form /dev/unixdgram/path makes bash attempt to open a connection to
the Unix domain datagram socket at /path.
Zartaj Majeed
Makefile.in | 2 +-
config.h.in | 7
configure.ac | 9 +++-
externs.h| 3 ++
lib/sh/Makefile.in | 8 +++-
lib/sh/usockopen.c | 104
+++
redir.c | 12 ++
tests/misc/devunix.tests | 18
8 files changed, 158 insertions(+), 5 deletions(-)
diff --git a/Makefile.in b/Makefile.in
index c7b62bc0..542cede8 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -222,7 +222,7 @@ SHLIB_SOURCE = ${SH_LIBSRC}/clktck.c
${SH_LIBSRC}/getcwd.c \
${SH_LIBSRC}/input_avail.c ${SH_LIBSRC}/mbscasecmp.c \
${SH_LIBSRC}/fnxform.c ${SH_LIBSRC}/unicode.c \
${SH_LIBSRC}/wcswidth.c ${SH_LIBSRC}/wcsnwidth.c \
- ${SH_LIBSRC}/shmbchar.c
+ ${SH_LIBSRC}/shmbchar.c ${SH_LIBSRC}/usockopen.c
SHLIB_LIB = -lsh
SHLIB_LIBNAME = libsh.a
diff --git a/config.h.in b/config.h.in
index a5ad9e72..36e8f5dd 100644
--- a/config.h.in
+++ b/config.h.in
@@ -151,6 +151,10 @@
socket connections when used in redirections */
#undef NETWORK_REDIRECTIONS
+/* Define UNIXSOCK_REDIRECTIONS if you want
/dev/(unixstream|unixdgram)/path to open
+ socket connections when used in redirections */
+#undef UNIXSOCK_REDIRECTIONS
+
/* Define PROGRAMMABLE_COMPLETION for the programmable completion features
and the complete builtin. */
#undef PROGRAMMABLE_COMPLETION
@@ -1042,6 +1046,9 @@
/* Define if you have the header file. */
#undef HAVE_SYS_SOCKET_H
+/* Define if you have the header file. */
+#undef HAVE_SYS_UN_H
+
/* Define if you have the header file. */
#undef HAVE_SYS_STAT_H
diff --git a/configure.ac b/configure.ac
index ce4e9b60..a8e9f43c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -168,6 +168,7 @@ opt_cond_regexp=yes
opt_coproc=yes
opt_arith_for_command=yes
opt_net_redirs=yes
+opt_usock_redirs=yes
opt_progcomp=yes
opt_separate_help=no
opt_multibyte=yes
@@ -196,7 +197,7 @@ if test $opt_minimal_config = yes; then
opt_select=no opt_help=no opt_array_variables=no opt_dparen_arith=no
opt_brace_expansion=no opt_disabled_builtins=no
opt_command_timing=no
opt_extended_glob=no opt_cond_command=no opt_arith_for_command=no
- opt_net_redirs=no opt_progcomp=no opt_separate_help=no
+ opt_net_redirs=no opt_usock_redirs=no opt_progcomp=no
opt_separate_help=no
opt_multibyte=yes opt_cond_regexp=no opt_coproc=no
opt_casemod_attrs=no opt_casemod_expansions=no
opt_extglob_default=no
opt_globascii_default=no
@@ -227,6 +228,7 @@ AC_ARG_ENABLE(history,
AC_HELP_STRING([--enable-history], [turn on command histo
AC_ARG_ENABLE(job-control, AC_HELP_STRING([--enable-job-control], [enable
job control features]), opt_job_control=$enableval)
AC_ARG_ENABLE(multibyte, AC_HELP_STRING([--enable-multibyte], [enable
multibyte characters if OS supports them]), opt_multibyte=$enableval)
AC_ARG_ENABLE(net-redirections,
AC_HELP_STRING([--enable-net-redirections], [enable /dev/tcp/host/port
redirection]), opt_net_redirs=$enableval)
+AC_ARG_ENABLE(usock-redirections,
AC_HELP_STRING([--enable-usock-redirections], [enable /dev/unixstream/path
redirection]), opt_usock_redirs=$enableval)
AC_ARG_ENABLE(process-substitution,
AC_HELP_STRING([--enable-process-substitution], [enable process
substitution]), opt_process_subst=$enableval)
AC_ARG_ENABLE(progcomp, AC_HELP_STRING([--enable-progcomp], [enable
programmable completion and the complete builtin]), opt_progcomp=$enableval)
AC_ARG_ENABLE(prompt-string-decoding,
AC_HELP_STRING([--enable-prompt-string-decoding], [turn on escape character
decoding in prompts]), opt_prompt_decoding=$enableval)
@@ -321,6 +323,9 @@ fi
if test $opt_net_redirs = yes; then
AC_DEFINE(NETWORK_REDIRECTIONS)
fi
+if test $opt_usock_redirs = yes; then
+AC_DEFINE(UNIXSOCK_REDIRECTIONS)
+fi
if test $opt_progcomp = yes; then
AC_DEFINE(PROGRAMMABLE_COMPLETION)
fi
@@ -702,7 +707,7 @@ AC_CHECK_HEADERS(unistd.h stdlib.h stdarg.h varargs.h
limits.h string.h \
stdbool.h stddef.h stdint.h netdb.h pwd.h grp.h strings.h \
regex.h syslog.h ulimit.h)
AC_CHECK_HEADERS(sys/pte.h sys/stream.h sys/select.h sys/file.h
sys/ioctl.h \
-sys/param.h sys/socket.h sys/stat.h \
+sys/param.h sys/socket.h sys/un.h sys/stat.h \
sys/time.h sys/times.h sys/types.h sys/wait.h)
AC_CHECK_HEADERS(netinet/in.h arpa/inet.h)
diff --git a/externs.h b/externs.h
index fa0603d3..1315866c 100644
--- a/externs.h
+++ b/externs.h
@@ -270,6 +270,9 @@ extern int isnetconn __P((int));
Unexpected word splitting on $* when IFS is unset
Hi,
When IFS is unset, unquoted $* undergoes word splitting as if IFS=' ',
and not the expected IFS=$' \t\n'. All the other unquoted mass
expansions ($@, array[*], array[@]) are word-split as if IFS=$'
\t\n'.For instance:
nb_args() { echo $#; }
set -- $'a\nb'
unset IFS
# Expected: 2, actual: 2
nb_args $@
# Expected: 2, actual: 1
nb_args $*
ar=("$@")
# Expected: 2, actual: 2
nb_args ${ar[*]}
# Expected: 2, actual: 2
nb_args ${ar[@]}
Note that this only occurs if IFS is *globally* unset. If made local and
then unset (as in f() { local IFS; unset IFS; ... }), $* is word-split
as expected.
This is a regression that appeared in 4.3 and is still present on devel
(bash-snap-20170616). A git bisect on devel shows that commit
1a81420a36fa (bash-20130125 snapshot) introduced this change. It seems
indeed that this commit is related to the handling of $* when IFS is
unset, but my knowledge of Bash's sources is too limited to tell what's
wrong with it :-)
Kevin
[PATCH] Unix domain socket filename redirection
This patch adds support for Unix domain sockets. A filename of the
form /dev/unixstream/path used in a redirection makes bash attempt to
open a connection to the Unix domain stream socket at /path. Similarly
a filename of the form /dev/unixdgram/path makes bash attempt to open
a connection to the Unix domain datagram socket at /path.
Zartaj Majeed
Makefile.in | 2 +-
config.h.in | 7
configure.ac | 9 +++-
externs.h| 3 ++
lib/sh/Makefile.in | 8 +++-
lib/sh/usockopen.c | 104 +++
redir.c | 12 ++
tests/misc/devunix.tests | 18
8 files changed, 158 insertions(+), 5 deletions(-)
diff --git a/Makefile.in b/Makefile.in
index c7b62bc0..542cede8 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -222,7 +222,7 @@ SHLIB_SOURCE = ${SH_LIBSRC}/clktck.c
${SH_LIBSRC}/getcwd.c \
${SH_LIBSRC}/input_avail.c ${SH_LIBSRC}/mbscasecmp.c \
${SH_LIBSRC}/fnxform.c ${SH_LIBSRC}/unicode.c \
${SH_LIBSRC}/wcswidth.c ${SH_LIBSRC}/wcsnwidth.c \
- ${SH_LIBSRC}/shmbchar.c
+ ${SH_LIBSRC}/shmbchar.c ${SH_LIBSRC}/usockopen.c
SHLIB_LIB = -lsh
SHLIB_LIBNAME = libsh.a
diff --git a/config.h.in b/config.h.in
index a5ad9e72..36e8f5dd 100644
--- a/config.h.in
+++ b/config.h.in
@@ -151,6 +151,10 @@
socket connections when used in redirections */
#undef NETWORK_REDIRECTIONS
+/* Define UNIXSOCK_REDIRECTIONS if you want
/dev/(unixstream|unixdgram)/path to open
+ socket connections when used in redirections */
+#undef UNIXSOCK_REDIRECTIONS
+
/* Define PROGRAMMABLE_COMPLETION for the programmable completion features
and the complete builtin. */
#undef PROGRAMMABLE_COMPLETION
@@ -1042,6 +1046,9 @@
/* Define if you have the header file. */
#undef HAVE_SYS_SOCKET_H
+/* Define if you have the header file. */
+#undef HAVE_SYS_UN_H
+
/* Define if you have the header file. */
#undef HAVE_SYS_STAT_H
diff --git a/configure.ac b/configure.ac
index ce4e9b60..a8e9f43c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -168,6 +168,7 @@ opt_cond_regexp=yes
opt_coproc=yes
opt_arith_for_command=yes
opt_net_redirs=yes
+opt_usock_redirs=yes
opt_progcomp=yes
opt_separate_help=no
opt_multibyte=yes
@@ -196,7 +197,7 @@ if test $opt_minimal_config = yes; then
opt_select=no opt_help=no opt_array_variables=no opt_dparen_arith=no
opt_brace_expansion=no opt_disabled_builtins=no opt_command_timing=no
opt_extended_glob=no opt_cond_command=no opt_arith_for_command=no
- opt_net_redirs=no opt_progcomp=no opt_separate_help=no
+ opt_net_redirs=no opt_usock_redirs=no opt_progcomp=no
opt_separate_help=no
opt_multibyte=yes opt_cond_regexp=no opt_coproc=no
opt_casemod_attrs=no opt_casemod_expansions=no opt_extglob_default=no
opt_globascii_default=no
@@ -227,6 +228,7 @@ AC_ARG_ENABLE(history,
AC_HELP_STRING([--enable-history], [turn on command histo
AC_ARG_ENABLE(job-control, AC_HELP_STRING([--enable-job-control],
[enable job control features]), opt_job_control=$enableval)
AC_ARG_ENABLE(multibyte, AC_HELP_STRING([--enable-multibyte], [enable
multibyte characters if OS supports them]), opt_multibyte=$enableval)
AC_ARG_ENABLE(net-redirections,
AC_HELP_STRING([--enable-net-redirections], [enable /dev/tcp/host/port
redirection]), opt_net_redirs=$enableval)
+AC_ARG_ENABLE(usock-redirections,
AC_HELP_STRING([--enable-usock-redirections], [enable
/dev/unixstream/path redirection]), opt_usock_redirs=$enableval)
AC_ARG_ENABLE(process-substitution,
AC_HELP_STRING([--enable-process-substitution], [enable process
substitution]), opt_process_subst=$enableval)
AC_ARG_ENABLE(progcomp, AC_HELP_STRING([--enable-progcomp], [enable
programmable completion and the complete builtin]),
opt_progcomp=$enableval)
AC_ARG_ENABLE(prompt-string-decoding,
AC_HELP_STRING([--enable-prompt-string-decoding], [turn on escape
character decoding in prompts]), opt_prompt_decoding=$enableval)
@@ -321,6 +323,9 @@ fi
if test $opt_net_redirs = yes; then
AC_DEFINE(NETWORK_REDIRECTIONS)
fi
+if test $opt_usock_redirs = yes; then
+AC_DEFINE(UNIXSOCK_REDIRECTIONS)
+fi
if test $opt_progcomp = yes; then
AC_DEFINE(PROGRAMMABLE_COMPLETION)
fi
@@ -702,7 +707,7 @@ AC_CHECK_HEADERS(unistd.h stdlib.h stdarg.h
varargs.h limits.h string.h \
stdbool.h stddef.h stdint.h netdb.h pwd.h grp.h strings.h \
regex.h syslog.h ulimit.h)
AC_CHECK_HEADERS(sys/pte.h sys/stream.h sys/select.h sys/file.h sys/ioctl.h \
-sys/param.h sys/socket.h sys/stat.h \
+sys/param.h sys/socket.h sys/un.h sys/stat.h \
sys/time.h sys/times.h sys/types.h sys/wait.h)
AC_CHECK_HEADERS(netinet/in.h arpa/inet.h)
diff --git a/externs.h b/externs.h
index fa0603d3..1315866c 100644
--- a/externs.h
+++ b/externs.h
@@ -270,6 +270,9 @@ extern int isnetconn __P((int));
Re: "realpath" loadable (in the examples/loadables dir) infinite loops
On Mon, Jun 19, 2017 at 07:18:11PM -0600, gaze...@xmission.com wrote: [...] > Description: > Command "realpath --help" causes infinite loop. I can reproduce this in the `devel' branch. The getopts loop is missing a `return' to break out of it. This should fix it: diff --git a/examples/loadables/realpath.c b/examples/loadables/realpath.c index b19b87fb..7ea9a995 100644 --- a/examples/loadables/realpath.c +++ b/examples/loadables/realpath.c @@ -88,6 +88,7 @@ WORD_LIST *list; break; default: builtin_usage(); + return (EX_USAGE); } } -- Eduardo Bustamante https://dualbus.me/
Re: [PATCH] Unix domain socket filename redirection
On Mon, Jun 19, 2017 at 07:30:42PM -0400, Z wrote: > This patch adds support for Unix domain sockets. A filename of the form > /dev/unixstream/path used in a redirection makes bash attempt to open a > connection to the Unix domain stream socket at /path. Similarly a filename > of the form /dev/unixdgram/path makes bash attempt to open a connection to > the Unix domain datagram socket at /path. Excuse my ignorance: - What would be an expected use case of this feature? /dev/tcp and /dev/udp are commonly used because it allows the shell script writer to interact with popular text based protocols like HTTP or SMTP. It would seem that this UNIX domain socket patch targets a niche instead? - Why should this be integrated into the core of bash, and not provided as a loadable builtin instead? It makes more sense to me to provide this as an optional shell builtin, which also make the usage less awkward (not a fan of the /dev/unixstream/the/path/is/here thing). By the way, the patch does not apply. It would seem like you copy-pasted the diff, so it got wrapped at some points and git is rejecting it. -- Eduardo Bustamante https://dualbus.me/
Re: [PATCH] Unix domain socket filename redirection
Attaching the patch usock.diff.txt
On Mon, Jun 19, 2017 at 11:36 PM, Eduardo A. Bustamante López
wrote:
> On Mon, Jun 19, 2017 at 07:30:42PM -0400, Z wrote:
>> This patch adds support for Unix domain sockets. A filename of the form
>> /dev/unixstream/path used in a redirection makes bash attempt to open a
>> connection to the Unix domain stream socket at /path. Similarly a filename
>> of the form /dev/unixdgram/path makes bash attempt to open a connection to
>> the Unix domain datagram socket at /path.
>
> Excuse my ignorance:
>
> - What would be an expected use case of this feature? /dev/tcp and
> /dev/udp are commonly used because it allows the shell script writer
> to interact with popular text based protocols like HTTP or SMTP. It
> would seem that this UNIX domain socket patch targets a niche instead?
>
> - Why should this be integrated into the core of bash, and not provided
> as a loadable builtin instead? It makes more sense to me to provide
> this as an optional shell builtin, which also make the usage less
> awkward (not a fan of the /dev/unixstream/the/path/is/here thing).
>
>
> By the way, the patch does not apply. It would seem like you copy-pasted
> the diff, so it got wrapped at some points and git is rejecting it.
>
> --
> Eduardo Bustamante
> https://dualbus.me/
diff --git a/Makefile.in b/Makefile.in
index c7b62bc0..542cede8 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -222,7 +222,7 @@ SHLIB_SOURCE = ${SH_LIBSRC}/clktck.c
${SH_LIBSRC}/getcwd.c \
${SH_LIBSRC}/input_avail.c ${SH_LIBSRC}/mbscasecmp.c \
${SH_LIBSRC}/fnxform.c ${SH_LIBSRC}/unicode.c \
${SH_LIBSRC}/wcswidth.c ${SH_LIBSRC}/wcsnwidth.c \
- ${SH_LIBSRC}/shmbchar.c
+ ${SH_LIBSRC}/shmbchar.c ${SH_LIBSRC}/usockopen.c
SHLIB_LIB = -lsh
SHLIB_LIBNAME = libsh.a
diff --git a/config.h.in b/config.h.in
index a5ad9e72..36e8f5dd 100644
--- a/config.h.in
+++ b/config.h.in
@@ -151,6 +151,10 @@
socket connections when used in redirections */
#undef NETWORK_REDIRECTIONS
+/* Define UNIXSOCK_REDIRECTIONS if you want /dev/(unixstream|unixdgram)/path
to open
+ socket connections when used in redirections */
+#undef UNIXSOCK_REDIRECTIONS
+
/* Define PROGRAMMABLE_COMPLETION for the programmable completion features
and the complete builtin. */
#undef PROGRAMMABLE_COMPLETION
@@ -1042,6 +1046,9 @@
/* Define if you have the header file. */
#undef HAVE_SYS_SOCKET_H
+/* Define if you have the header file. */
+#undef HAVE_SYS_UN_H
+
/* Define if you have the header file. */
#undef HAVE_SYS_STAT_H
diff --git a/configure.ac b/configure.ac
index ce4e9b60..a8e9f43c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -168,6 +168,7 @@ opt_cond_regexp=yes
opt_coproc=yes
opt_arith_for_command=yes
opt_net_redirs=yes
+opt_usock_redirs=yes
opt_progcomp=yes
opt_separate_help=no
opt_multibyte=yes
@@ -196,7 +197,7 @@ if test $opt_minimal_config = yes; then
opt_select=no opt_help=no opt_array_variables=no opt_dparen_arith=no
opt_brace_expansion=no opt_disabled_builtins=no opt_command_timing=no
opt_extended_glob=no opt_cond_command=no opt_arith_for_command=no
- opt_net_redirs=no opt_progcomp=no opt_separate_help=no
+ opt_net_redirs=no opt_usock_redirs=no opt_progcomp=no
opt_separate_help=no
opt_multibyte=yes opt_cond_regexp=no opt_coproc=no
opt_casemod_attrs=no opt_casemod_expansions=no opt_extglob_default=no
opt_globascii_default=no
@@ -227,6 +228,7 @@ AC_ARG_ENABLE(history, AC_HELP_STRING([--enable-history],
[turn on command histo
AC_ARG_ENABLE(job-control, AC_HELP_STRING([--enable-job-control], [enable job
control features]), opt_job_control=$enableval)
AC_ARG_ENABLE(multibyte, AC_HELP_STRING([--enable-multibyte], [enable
multibyte characters if OS supports them]), opt_multibyte=$enableval)
AC_ARG_ENABLE(net-redirections, AC_HELP_STRING([--enable-net-redirections],
[enable /dev/tcp/host/port redirection]), opt_net_redirs=$enableval)
+AC_ARG_ENABLE(usock-redirections,
AC_HELP_STRING([--enable-usock-redirections], [enable /dev/unixstream/path
redirection]), opt_usock_redirs=$enableval)
AC_ARG_ENABLE(process-substitution,
AC_HELP_STRING([--enable-process-substitution], [enable process substitution]),
opt_process_subst=$enableval)
AC_ARG_ENABLE(progcomp, AC_HELP_STRING([--enable-progcomp], [enable
programmable completion and the complete builtin]), opt_progcomp=$enableval)
AC_ARG_ENABLE(prompt-string-decoding,
AC_HELP_STRING([--enable-prompt-string-decoding], [turn on escape character
decoding in prompts]), opt_prompt_decoding=$enableval)
@@ -321,6 +323,9 @@ fi
if test $opt_net_redirs = yes; then
AC_DEFINE(NETWORK_REDIRECTIONS)
fi
+if test $opt_usock_redirs = yes; then
+AC_DEFINE(UNIXSOCK_REDIRECTIONS)
+fi
if test $opt_progcomp = yes; then
AC_DEFINE(PROGRAMMABLE_COMPLETION)
fi
@@ -702,7 +707,7 @@ AC_CHECK_HEADERS(unistd.h stdlib.h stdarg.h varargs.h
limits.h str
devel: Builtin wait infinite loop when SIGINT received
There's something weird about the wait builtin + SIGINT handling in the latest
`devel' branch (i.e. 6374eecf232e70e45fe9c49cc8335e8779c07979)
If you interrupt the second `wait' with C-c, the shell will go into an infinite
loop.
dualbus@debian:~/src/gnu/bash-builds/devel$ cat /tmp/s
#!/bin/bash
: &
wait
sleep inf &
wait
dualbus@debian:~/src/gnu/bash-builds/devel$ ./bash -x /tmp/s
+ wait
+ :
+ wait
+ sleep inf
^C^Z
[1]+ Stopped ./bash -x /tmp/s
dualbus@debian:~/src/gnu/bash-builds/devel$ kill -6 %%
[1]+ Stopped ./bash -x /tmp/s
dualbus@debian:~/src/gnu/bash-builds/devel$ fg
./bash -x /tmp/s
Aborted (core dumped)
(I suspended the process and killed it with SIGABRT to get a nice stack trace).
Core was generated by `./bash -x /tmp/s'.
Program terminated with signal SIGABRT, Aborted.
#0 wait_sigint_handler (sig=2) at ../../bash/jobs.c:2527
2527{
(gdb) bt
#0 wait_sigint_handler (sig=2) at ../../bash/jobs.c:2527
#1
#2 0x7fcbc52e4fba in __GI___waitpid (pid=-1, stat_loc=0x7ffc4d9ec21c,
options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#3 0x563ed5620317 in waitchld (wpid=8312, block=1) at
../../bash/jobs.c:3566
#4 0x563ed561e6a7 in wait_for (pid=8312) at ../../bash/jobs.c:2772
#5 0x563ed561db90 in wait_for_single_pid (pid=8312, flags=1) at
../../bash/jobs.c:2393
#6 0x563ed561df8b in wait_for_background_pids () at
../../bash/jobs.c:2461
#7 0x563ed5678220 in wait_builtin (list=0x0) at
../../../bash/builtins/../../../bash/builtins/wait.def:171
#8 0x563ed5609ad2 in execute_builtin (builtin=0x563ed56780c7
, words=0x563ed6ec7b48, flags=0, subshell=0) at
../../bash/execute_cmd.c:4609
#9 0x563ed560aa32 in execute_builtin_or_function (words=0x563ed6ec7b48,
builtin=0x563ed56780c7 , var=0x0, redirects=0x0,
fds_to_close=0x563ed6ec3e08, flags=0)
at ../../bash/execute_cmd.c:5107
#10 0x563ed56093b4 in execute_simple_command
(simple_command=0x563ed6ec7a08, pipe_in=-1, pipe_out=-1, async=0,
fds_to_close=0x563ed6ec3e08) at ../../bash/execute_cmd.c:4395
#11 0x563ed5602ac8 in execute_command_internal (command=0x563ed6ec7ec8,
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x563ed6ec3e08) at
../../bash/execute_cmd.c:811
#12 0x563ed5601fe4 in execute_command (command=0x563ed6ec7ec8) at
../../bash/execute_cmd.c:393
#13 0x563ed55eb51c in reader_loop () at ../../bash/eval.c:172
#14 0x563ed55e8fd1 in main (argc=3, argv=0x7ffc4d9ecb38,
env=0x7ffc4d9ecb58) at ../../bash/shell.c:794
--
Eduardo Bustamante
https://dualbus.me/
Re: Unexpected word splitting on $* when IFS is unset
On Tue, Jun 20, 2017 at 01:13:07AM +0100, Kevin Brodsky wrote: [...] > When IFS is unset, unquoted $* undergoes word splitting as if IFS=' ', > and not the expected IFS=$' \t\n'. All the other unquoted mass > This is a regression that appeared in 4.3 and is still present on devel [...] AFAICT, the following patch fixes this. diff --git a/subst.c b/subst.c index 3093309f..bf73a35f 100644 --- a/subst.c +++ b/subst.c @@ -10158,7 +10158,7 @@ finished_with_string: or we expanded "$@" with IFS null and we need to split the positional parameters into separate words. */ if (split_on_spaces) - list = list_string (istring, " ", 1); /* XXX quoted == 1? */ + list = list_string (istring, " \t\n", 1); /* XXX quoted == 1? */ /* If we have $@ (has_dollar_at != 0) and we are in a context where we don't want to split the result (W_NOSPLIT2), and we are not quoted, we have already separated the arguments with the first character of The above code (`list_string' function call when split_on_spaces != 0) seems to be executed when IFS is either unset or the empty string. The comment above that block of code mentions that in either case, the result should be "split on spaces", although it has a narrow definition of space. -- Eduardo Bustamante https://dualbus.me/
