builtin echo command redirection misbehaves in detached scripts when terminal is closed
Configuration Information [Automatically generated, do not change]: Machine: i486 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i486' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../bash -I../bash/include -I../bash/lib -g -O2 uname output: Linux akula 2.6.17.7 #4 Sat Sep 8 23:10:46 CEST 2007 i686 GNU/Linux Machine Type: i486-pc-linux-gnu Bash Version: 3.1 Patch Level: 17 Release Status: release Description: With Unix-98 ptys, the builtin echo command gets executed even when writing to stdout or redirecting to stderr fails, and the output gets written to the wrong file descriptor if any other redirection is used in the script. Repeat-By: For example, with the following script: while [ 1 ];do echo Test1 echo Test2 >> file.txt sleep 1 done As expected, when this script is run in the background (&), the console slowly fills with "Test1" lines, and the file.txt file slowly fills with "Test2" lines. Now exit the shell leaving the script running (don't simply close the xterm, that'd kill the script. Type "exit"). Since the terminal has closed, stdout is closed, so "echo Test1" should fail. It doesn't, instead it writes "Test1" lines into whatever open file descriptor it can find. In this case, file.txt starts filling up with Test2 Test1 Test2 Test1 ... This does not happen with BSD-style ptys, because apparently when the terminal is closed, the tty seen by the detached bash script stays intact, and whatever is written to the now-closed terminal is simply discarded by the kernel, so the script keeps seeing open stdout and stderr file descriptors. In the case of Unix-98 ptys, this bug happens because the tty file descriptors the bash script uses are really closed This also does not happen with an external echo command: with /bin/echo, the redirection fails and the command is not executed, as expected.
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
On Sun, Sep 09, 2007 at 04:58:23PM +0200, Pierre-Philippe Coupard wrote:
[...]
>while [ 1 ];do
> echo Test1
> echo Test2 >> file.txt
> sleep 1
>done
>
>As expected, when this script is run in the background (&), the
> console
>slowly fills with "Test1" lines, and the file.txt file slowly fills
> with
>"Test2" lines.
>
>Now exit the shell leaving the script running (don't simply close
> the
>xterm, that'd kill the script. Type "exit"). Since the terminal has
>closed, stdout is closed, so "echo Test1" should fail. It doesn't,
>instead it writes "Test1" lines into whatever open file descriptor
> it
>can find. In this case, file.txt starts filling up with
>
>Test2
>Test1
>Test2
>Test1
[...]
Bonjour Pierre-Philippe,
can be reproduced with 3.2.25 and with:
bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | :
It seems to be down to the usage of stdio.
According to ltrace, echo seems to be doing printf("Test1\n")
followed by fflush(stdout). When the write(2) underneath
fflush() fails, "Test1\n" remains in the stdio buffer.
Then bash does an dup2(open("file.txt"), fileno(stdout)) instead
of doing an stdio freopen(3), so the next fflush(3) flushes both
"Test1\n" and "Test2\n" to the now working stdout.
Maybe bash can't use freopen(3) as that would mean closing the
original fd. Best is probably not to use stdio at all.
Note that zsh has the same problem, and AT&T ksh seems to have
an even worse problem (in the example above, it outputs "b\n"
twice). ash and pdksh are OK.
Best regards,
Stéphane
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
Stephane Chazelas <[EMAIL PROTECTED]> writes: > Bonjour Pierre-Philippe, > > can be reproduced with 3.2.25 and with: > > bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | : I get this: $ bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | : bash: line 0: echo: write error: Broken pipe and the file contains only one line. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
On Sun, Sep 09, 2007 at 07:36:52PM +0200, Andreas Schwab wrote: > Stephane Chazelas <[EMAIL PROTECTED]> writes: > > > Bonjour Pierre-Philippe, > > > > can be reproduced with 3.2.25 and with: > > > > bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | : > > I get this: > > $ bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | : > bash: line 0: echo: write error: Broken pipe > > and the file contains only one line. [...] Hi Andreas, What OS and version of glibc? I do get the error message but I get both a and b in the file. That was on Linux, glibc 2.6.1. -- Stéphane
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
On Sun, Sep 09, 2007 at 07:10:59PM +0100, Stephane Chazelas wrote:
[...]
> What OS and version of glibc? I do get the error message but I
> get both a and b in the file.
>
> That was on Linux, glibc 2.6.1.
[...]
Actually,
bash -c 'echo a; echo b > a' >&-
is enough for me to reproduce the problem.
And that program below shows the same behavior when run as
./a.out >&-
#include
#include
int main()
{
printf("a\n");
fflush(stdout);
dup2(open("a", O_WRONLY|O_CREAT, 0644), 1);
printf("b\n");
fflush(stdout);
return 0;
}
--
Stéphane
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
Andreas Schwab wrote: I get this: $ bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | : bash: line 0: echo: write error: Broken pipe and the file contains only one line. Andreas. I did more tests, and this is what I came up with: - akula, my bleeding edge box, is a Debian-unstable box upgraded yesterday sept 8, 2007. It runs linux-2.6.17.7, libc6-2.6.1 - kilo, my most up-to-date box where bash still seems to behave properly with regard to that problem, is also a Debian-unstable box, upgraded on may 1st, 2007. It runs linux2.6.18, libc6-2.5 On both boxes, I tried Stephane's test line with bash-3.1.17 and bash-2.05b: On both boxes, with bash-3.1.17, I get the "bash: line 0: echo: write error: Broken pipe" message, and no error message with bash-2.05b. On akula, both versions of bash generate a file with 2 lines. On kilo, both version of bash generate a correct file with 1 line. I hope this helps. I'll keep the kilo box in "working bash state" if anybody wants ssh access to it to test further.
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
Stephane Chazelas <[EMAIL PROTECTED]> writes: > That was on Linux, glibc 2.6.1. Same. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
On Sun, Sep 09, 2007 at 08:44:25PM +0200, Pierre-Philippe Coupard wrote: [...] > - akula, my bleeding edge box, is a Debian-unstable box upgraded yesterday > sept 8, 2007. It runs linux-2.6.17.7, libc6-2.6.1 > > - kilo, my most up-to-date box where bash still seems to behave properly > with regard to that problem, is also a Debian-unstable box, upgraded on may > 1st, 2007. It runs linux2.6.18, libc6-2.5 > > On both boxes, I tried Stephane's test line with bash-3.1.17 and > bash-2.05b: > > On both boxes, with bash-3.1.17, I get the "bash: line 0: echo: write > error: Broken pipe" message, and no error message with bash-2.05b. > > On akula, both versions of bash generate a file with 2 lines. > On kilo, both version of bash generate a correct file with 1 line. [...] Would seem to be down to the version of glibc where the behavior of fflush() would have changed. But I don't explain why Andreas doesn't get the same behavior as me with the same version of glibc. I tried on a glibc 2.3.4 and with the C file I see only "b" in the "a" file. Same thing with Solaris and HPUX with the system's libc. So that would confirm that the behavior changed in the glibc (probably somewhere after 2.5). And the problem may be more of a glibc problem than a bash problem. I've checked SUSv3 and it doesn't say wether the output buffer should be emptied after a non-successful fflush() (as older glibc seemed to be doing but as newer ones seem no longer to be doing). -- Stéphane
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
Stephane Chazelas <[EMAIL PROTECTED]> writes: > On Sun, Sep 09, 2007 at 07:10:59PM +0100, Stephane Chazelas wrote: > [...] >> What OS and version of glibc? I do get the error message but I >> get both a and b in the file. >> >> That was on Linux, glibc 2.6.1. > [...] > > Actually, > > bash -c 'echo a; echo b > a' >&- > > is enough for me to reproduce the problem. Guess you have a buggy libc, then. Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
On Sun, Sep 09, 2007 at 10:08:14PM +0200, Andreas Schwab wrote: > Stephane Chazelas <[EMAIL PROTECTED]> writes: > > > On Sun, Sep 09, 2007 at 07:10:59PM +0100, Stephane Chazelas wrote: > > [...] > >> What OS and version of glibc? I do get the error message but I > >> get both a and b in the file. > >> > >> That was on Linux, glibc 2.6.1. > > [...] > > > > Actually, > > > > bash -c 'echo a; echo b > a' >&- > > > > is enough for me to reproduce the problem. > > Guess you have a buggy libc, then. [...] I wouldn't be surprised if it has to do with the fix to debian bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021 (I'm CCing Dmitry who is the author of that change according to bugs.debian.org) I was testing with debian package 2.6.1-2 that includes Dmitry's fix for that bug. I don't know if that fix will is planned to be included in the GNU tree, it doesn't seem it yet in the glibc CVS repository. Now, I'm not sure if we can say that the new glibc behavior observed is bogus (other than it's different from the behavior observed in all the libcs I tried with). It is not a harmless change, for sure as it seems to have broken at least bash, zsh and possibly ksh93. Dmitry, you may find that whole thread at: http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/e311bdd4f945a21e/621b7189217760f1 Best regards, Stéphane
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
The change is far from trivial or harmless, if it was intended. I had to rebuild a custom server I run in a hurry because it was flooding an IRC channel with log lines a backend bash script sent to stderr. And I can think of plenty of ways to trash files with this bug. Anyway, thanks a lot Stéphane and Andreas for testing this! Stephane Chazelas wrote: Now, I'm not sure if we can say that the new glibc behavior observed is bogus (other than it's different from the behavior observed in all the libcs I tried with). It is not a harmless change, for sure as it seems to have broken at least bash, zsh and possibly ksh93. Dmitry, you may find that whole thread at: http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/e311bdd4f945a21e/621b7189217760f1 Best regards, Stéphane
Re: builtin echo command redirection misbehaves in detached scripts when terminal is closed
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Stephane Chazelas on 9/9/2007 11:17 AM: > can be reproduced with 3.2.25 and with: > > bash -c 'trap "" PIPE; sleep 1; echo a; echo b > a' | : > > It seems to be down to the usage of stdio. Indeed. I raised this very bug several months ago: http://lists.gnu.org/archive/html/bug-bash/2007-04/msg00070.html where the cd builtin has the same issue. I also proposed several approaches for fixing the issue. - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG5Kom84KuGfSFAYARAgOfAKCiUMKGYRG8+xRJRIoxM5PSnkMoNACgqane BFh6hhMjAibDs0PgSf032Xw= =OF9G -END PGP SIGNATURE-
Re: Bug#429021: builtin echo command redirection misbehaves in detached scripts when terminal is closed
Stephane Chazelas a écrit : > On Sun, Sep 09, 2007 at 10:08:14PM +0200, Andreas Schwab wrote: >> Stephane Chazelas <[EMAIL PROTECTED]> writes: >> >>> On Sun, Sep 09, 2007 at 07:10:59PM +0100, Stephane Chazelas wrote: >>> [...] What OS and version of glibc? I do get the error message but I get both a and b in the file. That was on Linux, glibc 2.6.1. >>> [...] >>> >>> Actually, >>> >>> bash -c 'echo a; echo b > a' >&- >>> >>> is enough for me to reproduce the problem. >> Guess you have a buggy libc, then. > [...] > > I wouldn't be surprised if it has to do with the fix to debian > bug #429021. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=429021 > (I'm CCing Dmitry who is the author of that change according to > bugs.debian.org) > I can reproduce the "bug" with glibc from etch, or even from sarge, so I really doubt that it comes from this change. -- .''`. Aurelien Jarno | GPG: 1024D/F1BCDB73 : :' : Debian developer | Electrical Engineer `. `' [EMAIL PROTECTED] | [EMAIL PROTECTED] `-people.debian.org/~aurel32 | www.aurel32.net
