Re: Reading bytes one-by-one from pipe of process substitution
On Thu, Sep 28, 2006 at 09:12:47PM +0600, [EMAIL PROTECTED] wrote: [...] > Description: > When Process substitution is used, BASH > reads from such descriptor by one bytes!! Of course it does. A shell is a shell, it's a command that runs other commands. read var Reads one line of input (potentially more without -r). You don't want it to read more than one line of input, because you want the next command to be able to read the next line. And the only way to make sure that only one line is read and not more is to read one character at a time until the LF character is found. For regular files, shells like bash or ksh do some optimisations, they read a buffer worth of data, and then, in case another command is to be run, they seek back (move the "read" cursor position if you prefer) to the character after the LF. This kind of thing can't be done with a pipe of course, you can't tell the process at the other end "please restart from that newline character". It can only be done with regular files (not with devices, sockets, pipes...). So, it's a perfectly normal behavior. That's rather the optimisation that is not normal (in the sense that it reads more than what you may expect it to). > Repeat-By: > -- > /tmp/test: > --begin-- > #!/bin/bash > while read line; do > qwe=qwe > done < <( cat /etc/passwd ) > --end-- > chmod +x /tmp/test > strace /tmp/test > -- > You'll see that BASH read bytes from pipe one-by-one. > Another script will not trigger this problem: > #!/bin/bash > while read line; do > qwe=qwe > done < /etc/passwd [...] -- Stephane ___ Bug-bash mailing list Bug-bash@gnu.org http://lists.gnu.org/mailman/listinfo/bug-bash
"." (aka source) works only with regular files
Configuration Information [Automatically generated, do not change]:
Machine: i486
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i486'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2
uname output: Linux sc 2.6.17.11 #1 PREEMPT Sun Sep 24 13:29:05 BST 2006 i686
GNU/Linux
Machine Type: i486-pc-linux-gnu
Bash Version: 3.1
Patch Level: 17
Release Status: release
Description:
$ echo echo echo | bash -c '. /dev/fd/0'
$ echo echo echo | zsh -c '. /dev/fd/0'
echo
$ echo echo echo | ksh -c '. /dev/fd/0'
echo
$ echo echo echo | ash -c '. /dev/fd/0'
echo
$ echo echo echo | posh -c '. /dev/fd/0'
echo
Same problem if you do a ". fifo".
$ echo echo echo | strace bash -c '. /dev/fd/0'
[...]
stat64("/dev/fd/0", {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
open("/dev/fd/0", O_RDONLY|O_LARGEFILE) = 5
fstat64(5, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
read(5, "", 0) = 0
[...]
The problem being that bash does a stat on the file, and reads
st_size bytes from the file only, instead of reading til eof.
Repeat-By:
echo echo echo | bash -c '. /dev/fd/0'
Fix:
Not use stat(2).
I think the file should be read the same way as standard input
is read.
Like
echo 'echo echo echo >> a' > a
. ./a
should output "echo". (It does with ash, pdksh and zsh, not with
AT&T ksh). SUSv3 is not specific on that case.
--
Stephane
___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash
"type ls unexisting" gives 0 exit status
Configuration Information [Automatically generated, do not change]:
Machine: i486
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i486'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2
uname output: Linux sc 2.6.19-rc1 #5 PREEMPT Fri Oct 6 19:52:16 BST 2006 i686
GNU/Linux
Machine Type: i486-pc-linux-gnu
Bash Version: 3.1
Patch Level: 17
Release Status: release
Description:
Hi
$ bash -c 'type ls foo; echo "$?"'
ls is /bin/ls
bash: line 0: type: foo: not found
0
http://www.opengroup.org/onlinepubs/009695399/utilities/type.html
suggests that the exit status should be >0 in that case which is
the case with other shells (ash, zsh, pdksh, ksh93).
Same for command -v ls foo
but command -v is not meant to accept more than 1 arg according
to POSIX. But you may want to change that as well for
consistency.
Repeat-By:
bash -c 'type ls foo; echo "$?"'
last line of output is "0" instead of "1" or "127".
Fix:
--- /home/chazelas/tmp/type.def~2005-08-24 13:38:34.0 +0100
+++ type.def2006-11-10 12:43:34.0 +
@@ -108,14 +108,14 @@
type_builtin (list)
WORD_LIST *list;
{
- int dflags, successful_finds, opt;
+ int dflags, unsuccessful_finds, opt;
WORD_LIST *this;
if (list == 0)
return (EXECUTION_SUCCESS);
dflags = CDESC_SHORTDESC;/* default */
- successful_finds = 0;
+ unsuccessful_finds = 0;
/* Handle the obsolescent `-type', `-path', and `-all' by prescanning
the arguments and converting those options to the form that
@@ -181,13 +181,13 @@
if (!found && (dflags & (CDESC_PATH_ONLY|CDESC_TYPE)) == 0)
sh_notfound (list->word->word);
- successful_finds += found;
+ unsuccessful_finds += !found;
list = list->next;
}
fflush (stdout);
- return ((successful_finds != 0) ? EXECUTION_SUCCESS : EXECUTION_FAILURE);
+ return ((unsuccessful_finds == 0) ? EXECUTION_SUCCESS : EXECUTION_FAILURE);
}
/*
___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash
[POSIX conformance] "." looks for file in the current directory
Configuration Information [Automatically generated, do not change]: Machine: i486 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i486' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 -Wall uname output: Linux sc.homeunix.net 2.6.25-rc8 #1 PREEMPT Fri Apr 4 08:56:07 BST 2008 i686 GNU/Linux Machine Type: i486-pc-linux-gnu Bash Version: 3.2 Patch Level: 39 Release Status: release Description: I don't know if that's a known issue, but it seems to break POSIX and be inconsistent with the documentation. $ POSIXLY_CORRECT=1 sh --norc sh-3.2$ cat > a echo foo sh-3.2$ type a sh: type: a: not found sh-3.2$ . a foo sh-3.2$ help . .: . filename [arguments] Read and execute commands from FILENAME and return. The pathnames in $PATH are used to find the directory containing FILENAME. If any ARGUMENTS are supplied, they become the positional parameters when FILENAME is executed. sh-3.2$ echo "$PATH" /usr/local/bin:/usr/bin:/bin:/usr/games:/usr/sbin:/sbin:/usr/local/sbin According to POSIX, "." shouldn't look for "a" in the current directory because "." and "" (and of course any path to the current directory) are not in my $PATH. $ ash -c '. a' .: 1: a: not found $ posh -c '. a' posh: .: a: not found $ ksh -c '. a' ksh[1]: .: a: cannot open [No such file or directory] $ pdksh -c '. a' pdksh: .: a: not found $ zsh -c '. a' zsh:.:1: no such file or directory: a SUSv3> If file does not contain a slash, the shell shall use the SUSv3> search path specified by PATH to find the directory SUSv3> containing file. Unlike normal command search, however, the SUSv3> file searched for by the dot utility need not be SUSv3> executable. If no readable file is found, a non-interactive SUSv3> shell shall abort; an interactive shell shall write a SUSv3> diagnostic message to standard error, but this condition SUSv3> shall not be considered a syntax error. [...] SUSv3> Some older implementations searched the current directory SUSv3> for the file, even if the value of PATH disallowed it. This SUSv3> behavior was omitted from this volume of IEEE Std SUSv3> 1003.1-2001 due to concerns about introducing the SUSv3> susceptibility to trojan horses that the user might be SUSv3> trying to avoid by leaving dot out of PATH . Cheers, Stephane
