ordering of printed lines changes when redirecting
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.2
-L/home/abuild/rpmbuild/BUILD/bash-4.2/../readline-6.2
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu'
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H -I. -I. -I./include -I./lib
-fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2
-fstack-protector -funwind-tables -fasynchronous-unwind-tables -g
-D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g -Wuninitialized -Wextra
-Wno-unprototyped-calls -Wno-switch-enum -Wno-unused-variable
-Wno-unused-parameter -ftree-loop-linear -pipe -DBNC382214=0 -fprofile-use
uname output: Linux omnfr121 4.5.0-4.g3d86af7-default #1 SMP PREEMPT Fri Mar 18
13:03:45 UTC 2016 (3d86af7) x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-suse-linux-gnu
Bash Version: 4.2
Patch Level: 47
Release Status: release
Description: ordering of printed lines changes when merging stdin/stdout and
redirecting
Repeat-By:
#include
int main()
{
printf("hallo %d\n",__LINE__);
fprintf(stderr,"hallo %d\n",__LINE__);
printf("hallo %d\n",__LINE__);
fprintf(stderr,"hallo %d\n",__LINE__);
printf("hallo %d\n",__LINE__);
return 0;
}
compile: gcc prg.c
then:
( ./a.out 2>&1 )
hallo 5
hallo 6
hallo 7
hallo 8
hallo 9
( ./a.out 2>&1 ) >xx
cat xx
hallo 6
hallo 8
hallo 5
hallo 7
hallo 9
note: the same happens with
./a.out >xx 2>&1
cat xx
hallo 6
hallo 8
hallo 5
hallo 7
hallo 9
I am not sure if that is an intended behavior. at least it is annoying
when trying to analyze bugs.
If this is intended behavior it would be nice if that would be included
into the manual (i have found no hint, maybe wrong keyword ?) and a way
to fix that.
re,
wh
Re: ordering of printed lines changes when redirecting
On Mon, Jul 18, 2016 at 11:22 AM, walter harms wrote: > > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc -I/home/abuild/rpmbuild/BUILD/bash-4.2 > -L/home/abuild/rpmbuild/BUILD/bash-4.2/../readline-6.2 > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-suse-linux-gnu' > -DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' > -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib > -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 > -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g > -D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g -Wuninitialized -Wextra > -Wno-unprototyped-calls -Wno-switch-enum -Wno-unused-variable > -Wno-unused-parameter -ftree-loop-linear -pipe -DBNC382214=0 -fprofile-use > uname output: Linux omnfr121 4.5.0-4.g3d86af7-default #1 SMP PREEMPT Fri > Mar 18 13:03:45 UTC 2016 (3d86af7) x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-suse-linux-gnu > > Bash Version: 4.2 > Patch Level: 47 > Release Status: release > > Description: ordering of printed lines changes when merging stdin/stdout > and redirecting > > The problem is that buffering of stdout changes depending on where it goes (line buffered in a terminal, fully buffered in a file), see for instance: http://www.pixelbeat.org/programming/stdio_buffering/ This should be documented in your libc library. You could flush stdout in your program, for other workarounds see http://mywiki.wooledge.org/BashFAQ/009
Re: ordering of printed lines changes when redirecting
On Mon, Jul 18, 2016 at 10:22:46AM +0200, walter harms wrote: > ( ./a.out 2>&1 ) > hallo 5 > hallo 6 > hallo 7 > hallo 8 > hallo 9 (snip) > ./a.out >xx 2>&1 > cat xx > hallo 6 > hallo 8 > hallo 5 > hallo 7 > hallo 9 Looks like an artifact of stdio (libc) buffering. When stdout and stderr are going to a terminal (first example), you get line buffering (flushed after each newline), and thus the order you expect. When stdout and stderr are going to a file, you get much larger buffers. Looks like the flush happens implicitly when the program exits. In your second example here, you happen to get all of the stderr lines first, followed by all of the stdout lines. See if the behavior changes when you add these lines to the start of the C program: setvbuf(stdout, NULL, _IOLBF, 0); setvbuf(stderr, NULL, _IOLBF, 0);
Re: ordering of printed lines changes when redirecting
Am 18.07.2016 15:59, schrieb Greg Wooledge: > On Mon, Jul 18, 2016 at 10:22:46AM +0200, walter harms wrote: >> ( ./a.out 2>&1 ) >> hallo 5 >> hallo 6 >> hallo 7 >> hallo 8 >> hallo 9 > > (snip) > >> ./a.out >xx 2>&1 >> cat xx >> hallo 6 >> hallo 8 >> hallo 5 >> hallo 7 >> hallo 9 > > Looks like an artifact of stdio (libc) buffering. When stdout and > stderr are going to a terminal (first example), you get line buffering > (flushed after each newline), and thus the order you expect. When stdout > and stderr are going to a file, you get much larger buffers. Looks like > the flush happens implicitly when the program exits. In your second > example here, you happen to get all of the stderr lines first, followed > by all of the stdout lines. > > See if the behavior changes when you add these lines to the start of > the C program: > > setvbuf(stdout, NULL, _IOLBF, 0); > setvbuf(stderr, NULL, _IOLBF, 0); thx for the fast reply, my testprogramm works as expected after the changes. My basic mistake was that my understand was that \n should flush the buffer, that was obviously wrong. NTL i would recommend to add one or two line about buffering into the ducomentation :) thx for help, re, wh
Re: ordering of printed lines changes when redirecting
On 7/18/16 10:11 AM, walter harms wrote: > NTL i would recommend to add one or two line about buffering into > the ducomentation :) It's already documented in the right place: https://www.gnu.org/software/libc/manual/html_node/Stream-Buffering.html#Stream-Buffering -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: ordering of printed lines changes when redirecting
Am 18.07.2016 16:36, schrieb Chet Ramey: > On 7/18/16 10:11 AM, walter harms wrote: > >> NTL i would recommend to add one or two line about buffering into >> the ducomentation :) > > It's already documented in the right place: > > https://www.gnu.org/software/libc/manual/html_node/Stream-Buffering.html#Stream-Buffering > > to be serious, who will look into libc documentation when ./a.out >xx 2>&1 and ./a.out 2>&1 behave differently ? They will, as i did, check if there is something special with redirections. what about: Note: be aware that when merging output (e.g. 2&>1) the sequence can differ depending on the buffering used by your programm. use setvbuf() to control buffering. see also: https://www.gnu.org/software/libc/manual/html_node/Stream-Buffering.html#Stream-Buffering or see man setvbuf adjust as you like re, wh
Re: ordering of printed lines changes when redirecting
On 18/07/16 14:59, Greg Wooledge wrote: > On Mon, Jul 18, 2016 at 10:22:46AM +0200, walter harms wrote: >> ( ./a.out 2>&1 ) >> hallo 5 >> hallo 6 >> hallo 7 >> hallo 8 >> hallo 9 > > (snip) > >> ./a.out >xx 2>&1 >> cat xx >> hallo 6 >> hallo 8 >> hallo 5 >> hallo 7 >> hallo 9 > > Looks like an artifact of stdio (libc) buffering. When stdout and > stderr are going to a terminal (first example), you get line buffering > (flushed after each newline), and thus the order you expect. When stdout > and stderr are going to a file, you get much larger buffers. Looks like > the flush happens implicitly when the program exits. In your second > example here, you happen to get all of the stderr lines first, followed > by all of the stdout lines. > > See if the behavior changes when you add these lines to the start of > the C program: > > setvbuf(stdout, NULL, _IOLBF, 0); > setvbuf(stderr, NULL, _IOLBF, 0); One might be able to control from outside the program as well like: stdbuf -oL your_prog
