bash / readline should support EXTPROC / LINEMODE

[Howard Chu]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/lib   -g -O2 -Wall
uname output: Linux violino 2.6.34-5-generic #14~lucid1-Ubuntu SMP Thu Jun 3 
14:51:20 UTC 2010 x86_64 GNU/Linux

Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.1
Patch Level: 5
Release Status: release

Description:
bash / readline currently forces terminals into character-at-a-time 
operation. This is unfriendly for users on slow networks or networks with high 
per-byte fees. Since 1989 there has been an approach called LINEMODE for 
clients to interact with remote servers while still handling all line editing 
and character processing locally. (See RFC1116 and RFC1184.) But this feature 
is rendered useless when such ubiquitous software as bash and readline force 
character-at-a-time processing.


Repeat-By:
Try to connect to your favorite server using a cellphone network 
connection. Aside from the terrible input lag, if you do this while roaming 
you'll run up a nasty bill at the same time. If you happen to have an up to 
date Telnet client and a BSD server, you'll notice that input responsiveness 
is a lot better, as long as you use a "dumb" shell. You can't repeat this 
experiment on current Linux systems because the Linux kernel never adopted the 
tty driver feature needed to support it.


Fix:
The attached patch will make readline check for the EXTPROC tty mode and 
bypass most of its I/O processing features if EXTPROC is detected. (Input 
prompt handling must still be done.)


Patches for the Linux kernel to support the EXTPROC features are 
available here

http://lkml.org/lkml/2010/6/11/403

Patches to get Telnet working with the Linux support are available here
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=585527

Preliminary patches to get OpenSSH working with linemode are available here
http://marc.info/?l=openssh-unix-dev&m=127655706101347&w=2

Note that these OpenSSH patches were a very rough cut. I'll be posting an 
updated patch shortly that also supports command history. In the meantime I'm 
working on further readline patches to support command completion between the 
client and the server. (And no one will be more surprised than me if that 
actually works...)


This combination of patches is giving me pretty reasonable terminal behavior 
now, but obviously a lot of it is rough. Feedback appreciated.

--
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/
diff -wur bash-4.1/bash/lib/readline/display.c 
bash-4.1.N/bash/lib/readline/display.c
--- bash-4.1/bash/lib/readline/display.c2009-09-26 11:37:33.0 
-0700
+++ bash-4.1.N/bash/lib/readline/display.c  2010-06-14 22:54:30.0 
-0700
@@ -65,6 +65,8 @@
 static void insert_some_chars PARAMS((char *, int, int));
 static void cr PARAMS((void));
 
+int _rl_extproc;
+
 /* State of visible and invisible lines. */
 struct line_state
   {
@@ -728,6 +730,8 @@
  It maintains an array of line breaks for display (inv_lbreaks).
  This handles expanding tabs for display and displaying meta characters. */
   lb_linenum = 0;
+  if (_rl_extproc) goto do_ext;
+
 #if defined (HANDLE_MULTIBYTE)
   in = 0;
   if (MB_CUR_MAX > 1 && rl_byte_oriented == 0)
@@ -902,6 +906,7 @@
 #endif
 
 }
+do_ext:
   line[out] = '\0';
   if (cpos_buffer_position < 0)
 {
diff -wur bash-4.1/bash/lib/readline/rlprivate.h 
bash-4.1.N/bash/lib/readline/rlprivate.h
--- bash-4.1/bash/lib/readline/rlprivate.h  2009-08-31 05:46:04.0 
-0700
+++ bash-4.1.N/bash/lib/readline/rlprivate.h2010-06-14 22:20:03.0 
-0700
@@ -396,6 +396,7 @@
 extern int _rl_last_c_pos;
 extern int _rl_suppress_redisplay;
 extern int _rl_want_redisplay;
+extern int _rl_extproc;
 
 /* isearch.c */
 extern char *_rl_isearch_terminators;
diff -wur bash-4.1/bash/lib/readline/rltty.c 
bash-4.1.N/bash/lib/readline/rltty.c
--- bash-4.1/bash/lib/readline/rltty.c  2009-11-19 06:42:02.0 -0800
+++ bash-4.1.N/bash/lib/readline/rltty.c2010-06-14 22:18:32.0 
-0700
@@ -521,6 +521,12 @@
   _rl_echoctl = (oldtio.c_lflag & ECHOCTL);
 #endif
 
+  if (oldtio.c_lflag & EXTPROC) {
+tiop->c_cc[VEOL] = 9;  /* TAB */
+   _rl_extproc = 1;
+   return;
+  }
+
   tiop->c_lflag &= ~(ICANON | ECHO);
 
   if ((unsigned char) oldtio.c_cc[VEOF] != (unsigned char) _POSIX_VDISABLE)
diff -wur bash-4.1/bash/lib/readline/text.c bash-4.1.N/bash/lib/readline/text.c
--- bash-4.1/bash/lib/readline/text.c   2009-09-07 10:40:57.0 -0700
+++ bash-4.1.N/bash/lib/readline/text.c

Question if bug: declaration of an associative array within a function

[Rainer Rehak]

Hey there,
is it a bug, that the declaration of an associative array within a
function does not survive till after the function, with primitive
variables on the other hand that concept works well.

Greetings
Rainer



bug_bash_declare-A.sh
Description: Bourne shell script


signature.asc
Description: OpenPGP digital signature

Re: Question if bug: declaration of an associative array within a function

[Chris F.A. Johnson]

On Tue, 15 Jun 2010, Rainer Rehak wrote:

> Hey there,
> is it a bug, that the declaration of an associative array within a
> function does not survive till after the function, with primitive
> variables on the other hand that concept works well.

   It's not a bug. It's an unfortunate side effect from 'declare'
   being a duplicate of the (non-standard) typeset builtin.

   If would be much more useful (and semantically accurate), if it
   just did what its name implies, and left the 'local' builtin for
   declaring variable local to a function.

-- 
   Chris F.A. Johnson, 
   Author:
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

Re: Question if bug: declaration of an associative array within a function

[Greg Wooledge]

On Tue, Jun 15, 2010 at 01:21:51PM +0200, Rainer Rehak wrote:
> Hey there,
> is it a bug, that the declaration of an associative array within a
> function does not survive till after the function, with primitive
> variables on the other hand that concept works well.

`declare' has the side effect of making things local to a function, if
it is used within a function.  There is currently no way to create a
global associative array while inside a function; you'd have to do it
outside the function.

Chet has said, I think, that he might consider a `global' (or similar)
keyword at some point in the future.

Re: Question if bug: declaration of an associative array within a function

[Chet Ramey]

On 6/15/10 5:22 PM, Greg Wooledge wrote:

> `declare' has the side effect of making things local to a function, if
> it is used within a function.  There is currently no way to create a
> global associative array while inside a function; you'd have to do it
> outside the function.
> 
> Chet has said, I think, that he might consider a `global' (or similar)
> keyword at some point in the future.

Actually, it would be a `-g' option to declare, and that will probably
get in there.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/