Re: echo $'\0' >a does not write the nul byte

[Eduardo Bustamante]

On Sun, Jan 17, 2021 at 12:05 PM  wrote:
> (...)
> The nul byte is not echoed by $'\0'.

This is expected. Bash uses NUL-byte terminated character sequences to
store strings, so it can't actually store NUL bytes themselves.

$'\0'  is the same as '' (i.e. an empty string).

If you want to output an actual NUL byte, try using:

printf '\0'

Instead

Re: Use of `mktemp' is dangerous, better use `mkstemp'...

[Chet Ramey]

On 1/15/21 4:00 PM, Ángel wrote:

On 2021-01-15 at 10:00 -0500, Chet Ramey wrote:

On 1/14/21 11:45 PM, Jeffrey Walton wrote:

Hi Everyone,

I'm building Bash 5.1 from sources. This may be of interest:

/usr/bin/ld: ./lib/sh/libsh.a(tmpfile.o): in function
`sh_mktmpname':
/home/jwalton/Build-Scripts/bash-5.1/lib/sh/tmpfile.c:160: warning:
the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp'


It's a spurious warning.


mktemp() would only end up being used (with all its issues) if the OS
doesn't provide a DEV FD or there's no mkdtemp(). In other (safe)
Operating Systems, it would be possible to avoid the warning by
building a slightly different libsh.a in such case.


The warning can be ignored.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

echo $'\0' >a does not write the nul byte

[hans]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/lib  
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat 
-Werror=format-security -Wall
uname output: Linux artax 3.2.0-4-amd64 #1 SMP Debian 3.2.96-2 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.2
Patch Level: 37
Release Status: release

Description:
Command
echo $'\0' |od -c
writes
000  \n
001
in contrast to
echo $'\1' |od -c
000 001  \n
002
The nul byte is not echoed by $'\0'.

Repeat-By:
echo $'\0' |od -c
echo $'\1' |od -c

Re: echo $'\0' >a does not write the nul byte

[Martin Schulte]

Hello Hans, hello Eduardo!

> On Sun, Jan 17, 2021 at 12:05 PM  wrote:
> > (...)
> > The nul byte is not echoed by $'\0'.
> 
> This is expected. Bash uses NUL-byte terminated character sequences to
> store strings, so it can't actually store NUL bytes themselves.

To be exact, this is already caused by a Unix Kernel - strings passed by the 
exex* system calls are null terminated, too. As it is the default in the C 
programming language. Thus you can't pass a null byte in an argument when 
invoking a program.

Best regards

Martin

Re: echo $'\0' >a does not write the nul byte

[Chet Ramey]

On 1/17/21 3:05 PM, h...@artax.karlin.mff.cuni.cz wrote:


Description:
Command
echo $'\0' |od -c
writes
000  \n
001
in contrast to
echo $'\1' |od -c
000 001  \n
002
The nul byte is not echoed by $'\0'.

Repeat-By:
echo $'\0' |od -c
echo $'\1' |od -c


Shell builtin commands obey the same argv conventions as any other Unix
program: arguments are null-terminated strings. That means that

echo $'\0'
echo ''
echo ""

are all equivalent, and none of them will output a null byte.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Crash in malloc.c : internal_malloc under Asan

[Chet Ramey]

On 1/15/21 7:11 PM, Ángel wrote:



lib/malloc/malloc.c, if you want to view it online:
https://git.savannah.gnu.org/cgit/bash.git/tree/lib/malloc/malloc.c

Maybe libasan is providing its own malloc and there is a conflict of
calls between theirs and bash mallocs. I would recommend to rebuild
bash with the parameter --without-bash-malloc and see if that makes the
error go away (which is likely).


It's not a conflict so much as the bash malloc using different function
names to implement the conventional malloc functionality, and address
sanitizer (and valgrind, for that matter) not interposing those functions.

For instance, the bash debugging malloc uses sh_malloc and sh_free, and
both those functions and the conventional malloc/free call internal_malloc
and internal_free, respectively. Bash also uses xfree through a function
pointer in a callback fashion to implement unwind-protect cleanups, which
confuses valgrind, at least, because it doesn't think the allocations and
frees match. Thestructure is described here:

https://lists.gnu.org/archive/html/bug-bash/2017-04/msg00053.html

Asan and valgrind are easy to confuse; their assumptions are consistent
with `normal' usage of the libc malloc implementation(s). If you want to
use asan all the time, configure bash --without-bash-malloc as Ángel
suggested.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: echo $'\0' >a does not write the nul byte

[Ilkka Virta]

On Mon, Jan 18, 2021 at 12:02 AM Martin Schulte 
wrote:

> To be exact, this is already caused by a Unix Kernel - strings passed by
> the exex* system calls are null terminated, too. As it is the default in
> the C programming language. Thus you can't pass a null byte in an argument
> when invoking a program.
>

Bash's echo is a builtin, so using it doesn't involve an execve(). Most
shells still don't allow passing NULs
to builtins. Zsh is the exception, it does allow NUL bytes in internal
variables and in arguments to builtins.
But, that's Zsh, and of course even it can't pass the NUL as an argument to
an external program, exactly
because of the execve() interface.

Re: echo $'\0' >a does not write the nul byte

[Léa Gris]

Le 17/01/2021 à 22:02, Chet Ramey écrivait :

On 1/17/21 3:05 PM, h...@artax.karlin.mff.cuni.cz wrote:


Description:
Command
    echo $'\0' |od -c
writes
    000  \n
    001
in contrast to
    echo $'\1' |od -c
    000 001  \n
    002
The nul byte is not echoed by $'\0'.

Repeat-By:
echo $'\0' |od -c
echo $'\1' |od -c


Shell builtin commands obey the same argv conventions as any other Unix
program: arguments are null-terminated strings. That means that

echo $'\0'
echo ''
echo ""

are all equivalent, and none of them will output a null byte.



The only way to output a null byte with shell built-in is:

printf '\0'
or non portable: echo -ne '\0'

This is because `\0' is not a null byte or a nulll string but 
interpreted internally to the command to print a null byte.


--
Léa Gris

Re: Use of `mktemp' is dangerous, better use `mkstemp'...

[Ángel]

On 2021-01-17 at 15:33 -0500, Chet Ramey wrote:
> On 1/15/21 4:00 PM, Ángel wrote:
> > On 2021-01-15 at 10:00 -0500, Chet Ramey wrote:
> > > On 1/14/21 11:45 PM, Jeffrey Walton wrote:
> > > > Hi Everyone,
> > > > 
> > > > I'm building Bash 5.1 from sources. This may be of interest:
> > > > 
> > > > /usr/bin/ld: ./lib/sh/libsh.a(tmpfile.o): in function
> > > > `sh_mktmpname':
> > > > /home/jwalton/Build-Scripts/bash-5.1/lib/sh/tmpfile.c:160: warning:
> > > > the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp'
> > > 
> > > It's a spurious warning.
> > 
> > [Explanation of when is mktemp() used and patch to remove the
> > warning]
> 
> The warning can be ignored.

Sure. However, actually removing the spurious warning would not only
make (some) build systems happier, but also produce less confused
users, so it seemed (potentially) valuable.


Best regards

Feature: where terminal reports mouse click to bash, support positioning the cursor accordingly

[mcarans--- via Bug reports for the GNU Bourne Again SHell]

Hi,
 It would be great to be able to use the mouse to click to position the cursor 
in bash. I raised this with the Gnome terminal emulator here and they said "The 
cursor position is under the control of the application, not the terminal 
emulator. vte supports reporting the mouse click to the application, and the 
application can reposition the cursor accordingly."
My request is that bash support the mouse information that terminal emulators 
pass to it so that it is possible to click the mouse button to position the 
cursor.

Thanks,Mike

Re: Feature: where terminal reports mouse click to bash, support positioning the cursor accordingly

[IFo Hancroft]

Hi,

This has nothing to do with BASH or any shell for that matter. What you
want (the ability to reposition the cursor in the terminal with a mouse
click) can be done with just the terminal itself.

It varies which part needs to do this depending on whether your terminal
emulator application is separate from the terminal library or not, but
generally an application has access to the mouse click events and the
terminal library (if separate) is the one drawing and positioning the
cursor on screen.

Best Regards,
IFo Hancroft

On 1/18/21 1:57 AM, mcarans--- via Bug reports for the GNU Bourne Again
SHell wrote:
> Hi,
>  It would be great to be able to use the mouse to click to position the 
> cursor in bash. I raised this with the Gnome terminal emulator here and they 
> said "The cursor position is under the control of the application, not the 
> terminal emulator. vte supports reporting the mouse click to the application, 
> and the application can reposition the cursor accordingly."
> My request is that bash support the mouse information that terminal emulators 
> pass to it so that it is possible to click the mouse button to position the 
> cursor.
> 
> Thanks,Mike
> 

Re: Feature: where terminal reports mouse click to bash, support positioning the cursor accordingly

[Koichi Murase]

2021年1月18日(月) 12:32 IFo Hancroft :
> What you want (the ability to reposition the cursor in the terminal
> with a mouse click) can be done with just the terminal itself.

I think Mike wanted to move the current text insertion point in
Readline (or, in other words, the command line of Bash) but not just
move the current cursor position in the terminal.  If the terminal
just moves its cursor position without sharing the information with
the application (i.e., Bash in this case), the result would be just
broken rendering.

So, this does have something to do with Bash or any text editors
working in terminals.  In fact, recent versions of Emacs and Vim
supports mouse using the mouse escape sequences of terminals.  A
terminal application (such as Bash or text editors) could receive from
the terminal the mouse events encoded in terminal sequences, move its
internal cursor position, render the updated text content and move
the terminal cursor position by sending escape sequences to the
terminal.

There has been some attempt to support the mouse in zsh:

https://unix.stackexchange.com/questions/444601/any-terminal-shell-with-mouse-support

but I think this implementation is not robust.  For example, I think
this doesn't work when there are wide characters in the command line
or the text content doesn't fit into a single terminal line.

Actually, there is some practical problem of supporting mouse by
shells: the shell is not a full-screen terminal application so that
it cannot handle the mouse events that occurred outside the command
line.  So, one needs to compromise with some features or integrate
something like terminal multiplexers in the shell to manage all the
terminal contents in the shell.  There were some discussions in Oil
Shell:

https://oilshell.zulipchat.com/#narrow/stream/121539-oil-dev/topic/Playing.20With.20Debuggers
https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/non-linear.20shell.20UI

--
Koichi

Re: Feature: where terminal reports mouse click to bash, support positioning the cursor accordingly

[Chris F.A. Johnson]

On Sun, 17 Jan 2021, mcarans--- via Bug reports for the GNU Bourne Again SHell 
wrote:


Hi,
It would be great to be able to use the mouse to click to position the cursor in bash. I 
raised this with the Gnome terminal emulator here and they said "The cursor position 
is under the control of the application, not the terminal emulator. vte supports 
reporting the mouse click to the application, and the application can reposition the 
cursor accordingly."
My request is that bash support the mouse information that terminal emulators 
pass to it so that it is possible to click the mouse button to position the 
cursor.


https://cfajohnson.com/shell/?2005-07-15_mousetraps

--
   Chris F.A. Johnson 
   === Author: ===
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
   Pro Bash Programming: Scripting the GNU/Linux shell (2009, Apress)

Fwd: Feature: where terminal reports mouse click to bash, support positioning the cursor accordingly

[IFo Hancroft]

Good point! Thank you!

I forgot to take full screen applications like Vim and such into account
and the fact that they draw to the screen themselves instead of making
the terminal to do it.

 Forwarded Message 
Subject: Re: Feature: where terminal reports mouse click to bash,
support positioning the cursor accordingly
Date: Mon, 18 Jan 2021 14:01:42 +0800
From: Koichi Murase 
To: IFo Hancroft 
CC: bash.bug list 

2021年1月18日(月) 12:32 IFo Hancroft :
> What you want (the ability to reposition the cursor in the terminal
> with a mouse click) can be done with just the terminal itself.

I think Mike wanted to move the current text insertion point in
Readline (or, in other words, the command line of Bash) but not just
move the current cursor position in the terminal.  If the terminal
just moves its cursor position without sharing the information with
the application (i.e., Bash in this case), the result would be just
broken rendering.

So, this does have something to do with Bash or any text editors
working in terminals.  In fact, recent versions of Emacs and Vim
supports mouse using the mouse escape sequences of terminals.  A
terminal application (such as Bash or text editors) could receive from
the terminal the mouse events encoded in terminal sequences, move its
internal cursor position, render the updated text content and move
the terminal cursor position by sending escape sequences to the
terminal.

There has been some attempt to support the mouse in zsh:

https://unix.stackexchange.com/questions/444601/any-terminal-shell-with-mouse-support

but I think this implementation is not robust.  For example, I think
this doesn't work when there are wide characters in the command line
or the text content doesn't fit into a single terminal line.

Actually, there is some practical problem of supporting mouse by
shells: the shell is not a full-screen terminal application so that
it cannot handle the mouse events that occurred outside the command
line.  So, one needs to compromise with some features or integrate
something like terminal multiplexers in the shell to manage all the
terminal contents in the shell.  There were some discussions in Oil
Shell:

https://oilshell.zulipchat.com/#narrow/stream/121539-oil-dev/topic/Playing.20With.20Debuggers
https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/non-linear.20shell.20UI

--
Koichi

Re: echo $'\0' >a does not write the nul byte

[Martin Schulte]

Hello Ilkka, hello *!

> Bash's echo is a builtin, so using it doesn't involve an execve().

Sure – but while passing a string (however it will be encoded) containing a 
null byte to builtins would be possible in principle (as zsh shows) this would 
lead to a large bunch of problems, e.g. another incompatibility between the 
builtin and the external echo.

Best regards

Martin