Package: xterm Version: 278-4 (Summary, if I'm guessing right: the terminal emulation code has a bug related to the ESC [ K control sequence if it occurs when the line is just about to wrap. And grep uses ESC [ K when changing colors, so the bug is quite easy to see.)
The command: perl -e 'print "a"x79 . "bc" . "a"x5 . "\n";' | grep --color=always c when run on a terminal exactly 80 characters wide should print a line containing a's and one b, and on the next line a colored (red) c plus five a's, like this: rjs@anar:~/t$ perl -e 'print "a"x79 . "bc" . "a"x5 . "\n";' | grep --color=always c aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab caaaaa rjs@anar:~/t$ (the above lines were cut-and-pasted from a gnome-terminal window, with a line break added manually after "b", as displayed on-screen; the "c" is red) However, on xterm, the b and c go missing; the following is cut-and-pasted from an xterm window with a line-break added (that's how it looks on screen, with all characters the same color): rjs@anar:~/t$ perl -e 'print "a"x79 . "bc" . "a"x5 . "\n";' | grep --color=always c aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa rjs@anar:~/t$ Rxvt breaks in the same way as xterm. But the command works properly in gnome-terminal and konsole, as well as in tmux, even when tmux is run inside an xterm. But both rxvt-unicode and screen break in another way: the b at the end of the line goes missing, but the red c remains (in screen, apparently regardless of where screen is run). I just found out that (apparently) the same bug has been reported five years ago for rxvt as bug #477463 <http://bugs.debian.org/477463>. Which escape sequences does grep actually produce? In an xterm (TERM=xterm): rjs@anar:~/t$ perl -e 'print "a"x79 . "bc" . "a"x5 . "\n";' | grep --color=always c | cat -A aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab^[[01;31m^[[Kc^[[m^[[Kaaaaa$ rjs@anar:~/t$ (the bug remains when adding | cat without -A to the end of the command) So grep prints extra ESC [ K control sequences before and after the match. Why? The grep source code has a long comment about it: ,---- Debian wheezy grep-2.12/src/main.c lines 143-201 | /* Select Graphic Rendition (SGR, "\33[...m") strings. */ | /* Also Erase in Line (EL) to Right ("\33[K") by default. */ | /* Why have EL to Right after SGR? | -- The behavior of line-wrapping when at the bottom of the | terminal screen and at the end of the current line is often | such that a new line is introduced, entirely cleared with | the current background color which may be different from the | default one (see the boolean back_color_erase terminfo(5) | capability), thus scrolling the display by one line. | The end of this new line will stay in this background color | even after reverting to the default background color with | "\33[m', unless it is explicitly cleared again with "\33[K" | (which is the behavior the user would instinctively expect | from the whole thing). There may be some unavoidable | background-color flicker at the end of this new line because | of this (when timing with the monitor's redraw is just right). | -- The behavior of HT (tab, "\t") is usually the same as that of | Cursor Forward Tabulation (CHT) with a default parameter | of 1 ("\33[I"), i.e., it performs pure movement to the next | tab stop, without any clearing of either content or screen | attributes (including background color); try | printf 'asdfqwerzxcv\rASDF\tZXCV\n' | in a bash(1) shell to demonstrate this. This is not what the | user would instinctively expect of HT (but is ok for CHT). | The instinctive behavior would include clearing the terminal | cells that are skipped over by HT with blank cells in the | current screen attributes, including background color; | the boolean dest_tabs_magic_smso terminfo(5) capability | indicates this saner behavior for HT, but only some rare | terminals have it (although it also indicates a special | glitch with standout mode in the Teleray terminal for which | it was initially introduced). The remedy is to add "\33K" | after each SGR sequence, be it START (to fix the behavior | of any HT after that before another SGR) or END (to fix the | behavior of an HT in default background color that would | follow a line-wrapping at the bottom of the screen in another | background color, and to complement doing it after START). | Piping grep's output through a pager such as less(1) avoids | any HT problems since the pager performs tab expansion. | | Generic disadvantages of this remedy are: | -- Some very rare terminals might support SGR but not EL (nobody | will use "grep --color" on a terminal that does not support | SGR in the first place). | -- Having these extra control sequences might somewhat complicate | the task of any program trying to parse "grep --color" | output in order to extract structuring information from it. | A specific disadvantage to doing it after SGR START is: | -- Even more possible background color flicker (when timing | with the monitor's redraw is just right), even when not at the | bottom of the screen. | There are no additional disadvantages specific to doing it after | SGR END. | | It would be impractical for GNU grep to become a full-fledged | terminal program linked against ncurses or the like, so it will | not detect terminfo(5) capabilities. */ | static const char *sgr_start = "\33[%sm\33[K"; | static const char *sgr_end = "\33[m\33[K"; `---- This bug can of course also be reproduced without using grep: perl -e 'print "a"x79 . "b\e[01;31m\e[Kc\e[m\e[K" . "a"x5 . "\n";' However, removing the ESC [ K control sequence fixes the bug: rjs@anar:~/t$ perl -e 'print "a"x79 . "b\e[01;31mc\e[m" . "a"x5 . "\n";' aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab caaaaa rjs@anar:~/t$ (cut-and-pasted from xterm with the line break added; the c is red) But the comment quoted above suggests that ESC [ K is necessary for non-default background colors. (And of course ESC [ K should probably not erase extra characters if it occurs close to a line break...) It appears that the bug is not related to color changes; removing them still breaks xterm: rjs@anar:~/t$ perl -e 'print "a"x79 . "b\e[Kc\e[K" . "a"x5 . "\n";' aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa rjs@anar:~/t$ (cut-and-pasted from xterm with the line break added; note that the b and c are missing) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
