I'm trying to set up a pipe and it's not doing what I expect. By way of background, I'm using mencoder to create a video file. mencoder prints a status line every 0.1 seconds in the following format:
Movie-Aspect is 1.33:1 - prescaling to correct movie aspect. videocodec: libavcodec (704x464 fourcc=34504d46 [FMP4]) [mpeg2video @ 0x257e870]ac-tex damaged at 19 6 [mpeg2video @ 0x257e870]Warning MVs not available [mpeg2video @ 0x257e870]concealing 1080 DC, 1080 AC, 1080 MV errors Pos: 0.0s 1f ( 0%) 0.00fps Trem: 1min 0mb A-V:0.000 [0:0]^MWriting header... ODML: vprp aspect is 4:3. Writing header... ODML: vprp aspect is 4:3. Pos: 0.0s 2f ( 0%) 0.00fps Trem: 1min 11mb A-V:0.003 [0:0]^MPos: 0.1s 3f ( 0%) 0.00fps Trem: 1min 11mb A-V: 0.007 [0:0]^MPos: 0.1s 4f ( 0%) 0.00fps Trem: 1min 12mb A-V: 0.010 [0:0]^MPos: 0.1s 5f ( 0%) 0.00fps Trem: 1min 12mb A-V: 0.013 [0:0]^MPos: 0.2s 6f ( 0%) 0.00fps Trem: 2min 12mb A-V: 0.017 [0:0]^MPos: 0.2s 7f ( 0%) 0.00fps Trem: 1min 12mb A-V: 0.020 [0:0]^MPos: 0.2s 8f ( 0%) 0.00fps Trem: 2min 12mb A-V: 0.023 [0:0]^MPos: 0.3s 9f ( 0%) 0.00fps Trem: 2min 12mb A-V: 0.027 [0:0]^MPos: 0.3s 10f ( 0%) 0.00fps Trem: 2min 13mb A-V: 0.030 [0:0]^MPos: 0.3s 11f ( 0%) 0.00fps Trem: 2min 12mb A-V: 0.033 [0:0]^MPos: 0.4s 12f ( 0%) 0.00fps Trem: 2min 13mb A-V: 0.037 [0:0]^MPos: 0.4s 13f ( 0%) 0.00fps Trem: 2min 12mb A-V: 0.040 [0:0]^MPos: 0.4s 14f ( 0%) 0.00fps Trem: 2min 13mb A-V: 0.043 [0:0]^MPos: 0.5s 15f ( 0%) 0.00fps Trem: 2min 13mb A-V: 0.047 [0:0]^MPos: 0.5s 16f ( 0%) 0.00fps Trem: 2min 12mb A-V: 0.050 [0:0]^MPos: 0.5s 17f ( 0%) 0.00fps Trem: 2min 13mb A-V: 0.053 [0:181]^MPos: 0.6s 18f ( 0%) 0.00fps Trem: 2min 13mb A-V:0.057 [0:180]^MPos: 0.6s 19f ( 0%) 0.00fps Trem: 2min 13mb A-V:0.060 [0:180]^MPos: 0.6s 20f ( 0%) 0.00fps Trem: 2min 14mb A-V:0.063 [0:180]^MPos: 0.7s 21f ( 0%) 0.00fps Trem: 2min 15mb A-V:0.067 [0:180]^MPos: 0.7s 22f ( 0%) 0.00fps Trem: 2min 15mb A-V:0.070 [0:180]^M Skipping frame! Pos: 0.7s 23f ( 0%) 0.00fps Trem: 2min 15mb A-V:0.040 [0:180]^MPos: 0.7s 24f ( 0%) 0.00fps Trem: 2min 16mb A-V:0.043 [0:180]^MPos: 0.8s 25f ( 0%) 0.00fps Trem: 2min 17mb A-V:0.047 [0:181]^MPos: 0.8s 26f ( 0%) 0.00fps Trem: 3min 19mb A-V:0.050 [0:181]^MPos: 0.8s 27f ( 0%) 0.00fps Trem: 3min 20mb A-V:0.053 [0:181]^MPos: 0.9s 28f ( 0%) 0.00fps Trem: 3min 22mb A-V:0.057 [0:181]^MPos: 0.9s 29f ( 0%) 0.00fps Trem: 3min 23mb A-V:0.060 [0:181]^MPos: 0.9s 30f ( 0%) 0.00fps Trem: 3min 26mb A-V:0.063 [0:182]^MPos: 1.0s 31f ( 0%) 0.00fps Trem: 3min 30mb A-V:0.067 [0:182]^MPos: 1.0s 32f ( 0%) 0.00fps Trem: 3min 32mb A-V:0.070 [677:181]^M Note that it's using \r instead of \n for line breaks; that's to overwrite the status line without scrolling the screen. this works fine when displaying on a terminal. I'm trying to reduce this to where it only prints the status line if it gets a Skipped frame warning, so the above would look like this: Pos: 0.7s 22f ( 0%) 0.00fps Trem: 2min 15mb A-V:0.070 [0:180] Skipping frame! ..... To do this, I have the following pipeline: mencoder .... 2>&1 | tr '\r' '\n' | grep -v -B 1 '^Pos' this eventually creates the correct output, but it's buffered until mencoder completes. I have no idea why; the tr command streams without buffering and I've used grep for years and never run into this behavior. If I output to a file, tr is correctly substituting a newline for the return, and if I cat from the file grep behaves as one would assume - the output is created as a stream and it doesn't wait until the file closes. Something with the pipe must be causing the buffering, but i have no idea what it is. I've tried various combinations of sed -ue s/\r\n/g sed -ue s/\r/\n\r/g and assorted changes to the grep command, but nothing seems to work; the output from grep is buffered until mencoder completes. This is a problem in that excessive "Skipping frame" errors indicate a problem with the mencoder command. It can take 2 - 4 hours for mencoder to get done, and I don't want to wait that long to find out I need to change the command.
