Re: `history -r` can not read from /dev/stdin ?

[Techlive Zheng]

2012/8/17 郑文辉(Techlive Zheng) :
> 2012/8/17 Chet Ramey :
>> On 8/16/12 10:11 PM, 郑文辉(Techlive Zheng) wrote:
>>> 2012/8/17 Chet Ramey :
 On 8/16/12 9:17 AM, 郑文辉(Techlive Zheng) wrote:
> I was trying to reload the bash history file which changed by another
> bash session with the following commands, but it wouldn't work, please
> help me, why?
>
> ```
> new_history=$(history -a /dev/stdout)
> history -c
> history -r
> echo "$new_history" | history -r /dev/stdin
> ```

 One possible cause that springs to mind is the fact that the `history -r'
 at the end of the pipeline is run in a subshell and cannot affect its
 parent's history list.
>>>
>>> So, How could I accomplish this kind of thing?
>>
>> Why not just use a regular file?
>>
>> --
>> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>>  ``Ars longa, vita brevis'' - Hippocrates
>> Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
>
> Actually, I was tring to erase duplicate entries and share history
> across bash sessions.'erasedups' in `HISTCONTROL` only have effect for
> history list in the memory, so my solution is to load entire history
> file into memory and save it after every command finished.
>
> Here is what I am currently have in .bashrc, and it works as expected.
>
> reload_history() {
> local HISTHASH_NEW=`md5sum $HOME/.bash_history | cut -d' ' -f1`
> if [ "$HISTHASH" = "$HISTHASH_NEW" ]; then
> history -w
> # This is necessay because we need
> # to clear the last append signture
> history -c
> history -r
> else
> HISTTEMP=`mktemp`
> history -a $HISTTEMP
> history -c
> history -r
> history -r $HISTTEMP
> history -w
> rm $HISTTEMP
> fi
> HISTHASH=`md5sum $HOME/.bash_history | cut -d' ' -f1`
> }
>
> export PROMPT_COMMAND="reload_history;$PROMPT_COMMAND"
>
> Considering `mkemp` then remove the temp file on every prompt command
> is a little bit expensive, I want to directly pipe the output of the
> `history -a` to `hisotory -r` like below, unfortunately, this wouldn't
> work, because `history -r` could not handle /dev/stdin.
>
> reload_history() {
> local HISTHASH_NEW=`md5sum $HOME/.bash_history | cut -d' ' -f1`
> if [ "$HISTHASH" = "$HISTHASH_NEW" ]; then
> history -w
> # This is necessay because we need
> # to clear the last append signture
> history -c
> history -r
> else
> new_history=$(history -a /dev/stdout)
> history -c
> history -r
> echo "$new_history" | history -r /dev/stdin
> history -w
> fi
> HISTHASH=`md5sum $HOME/.bash_history | cut -d' ' -f1`
> }
>
> export PROMPT_COMMAND="reload_history;$PROMPT_COMMAND"

I have just checked out the source of bash and found that `read()` or
`mmap` is used to read the history file as below. The following code
could not handle /dev/stdin properly. As I am not a C expert, I could
not come up a patch for this, maybe someone here could give me a
little help?

>From lib/readline/histfile.c:200

#ifdef HISTORY_USE_MMAP
  /* We map read/write and private so we can change newlines to NULs without
 affecting the underlying object. */
  buffer = (char *)mmap (0, file_size, PROT_READ|PROT_WRITE,
MAP_RFLAGS, file, 0);
  if ((void *)buffer == MAP_FAILED)
{
  errno = overflow_errno;
  goto error_and_exit;
}
  chars_read = file_size;
#else
  buffer = (char *)malloc (file_size + 1);
  if (buffer == 0)
{
  errno = overflow_errno;
  goto error_and_exit;
}

  chars_read = read (file, buffer, file_size);
#endif

Re: `history -r` can not read from /dev/stdin ?

[Techlive Zheng]

2012/8/17 郑文辉(Techlive Zheng) :
> 2012/8/17 郑文辉(Techlive Zheng) :
>> 2012/8/17 Chet Ramey :
>>> On 8/16/12 10:11 PM, 郑文辉(Techlive Zheng) wrote:
 2012/8/17 Chet Ramey :
> On 8/16/12 9:17 AM, 郑文辉(Techlive Zheng) wrote:
>> I was trying to reload the bash history file which changed by another
>> bash session with the following commands, but it wouldn't work, please
>> help me, why?
>>
>> ```
>> new_history=$(history -a /dev/stdout)
>> history -c
>> history -r
>> echo "$new_history" | history -r /dev/stdin
>> ```
>
> One possible cause that springs to mind is the fact that the `history -r'
> at the end of the pipeline is run in a subshell and cannot affect its
> parent's history list.

 So, How could I accomplish this kind of thing?
>>>
>>> Why not just use a regular file?
>>>
>>> --
>>> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>>>  ``Ars longa, vita brevis'' - Hippocrates
>>> Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
>>
>> Actually, I was tring to erase duplicate entries and share history
>> across bash sessions.'erasedups' in `HISTCONTROL` only have effect for
>> history list in the memory, so my solution is to load entire history
>> file into memory and save it after every command finished.
>>
>> Here is what I am currently have in .bashrc, and it works as expected.
>>
>> reload_history() {
>> local HISTHASH_NEW=`md5sum $HOME/.bash_history | cut -d' ' -f1`
>> if [ "$HISTHASH" = "$HISTHASH_NEW" ]; then
>> history -w
>> # This is necessay because we need
>> # to clear the last append signture
>> history -c
>> history -r
>> else
>> HISTTEMP=`mktemp`
>> history -a $HISTTEMP
>> history -c
>> history -r
>> history -r $HISTTEMP
>> history -w
>> rm $HISTTEMP
>> fi
>> HISTHASH=`md5sum $HOME/.bash_history | cut -d' ' -f1`
>> }
>>
>> export PROMPT_COMMAND="reload_history;$PROMPT_COMMAND"
>>
>> Considering `mkemp` then remove the temp file on every prompt command
>> is a little bit expensive, I want to directly pipe the output of the
>> `history -a` to `hisotory -r` like below, unfortunately, this wouldn't
>> work, because `history -r` could not handle /dev/stdin.
>>
>> reload_history() {
>> local HISTHASH_NEW=`md5sum $HOME/.bash_history | cut -d' ' -f1`
>> if [ "$HISTHASH" = "$HISTHASH_NEW" ]; then
>> history -w
>> # This is necessay because we need
>> # to clear the last append signture
>> history -c
>> history -r
>> else
>> new_history=$(history -a /dev/stdout)
>> history -c
>> history -r
>> echo "$new_history" | history -r /dev/stdin
>> history -w
>> fi
>> HISTHASH=`md5sum $HOME/.bash_history | cut -d' ' -f1`
>> }
>>
>> export PROMPT_COMMAND="reload_history;$PROMPT_COMMAND"
>
> I have just checked out the source of bash and found that `read()` or
> `mmap` is used to read the history file as below. The following code
> could not handle /dev/stdin properly. As I am not a C expert, I could
> not come up a patch for this, maybe someone here could give me a
> little help?
>
> From lib/readline/histfile.c:200
>
> #ifdef HISTORY_USE_MMAP
>   /* We map read/write and private so we can change newlines to NULs without
>  affecting the underlying object. */
>   buffer = (char *)mmap (0, file_size, PROT_READ|PROT_WRITE,
> MAP_RFLAGS, file, 0);
>   if ((void *)buffer == MAP_FAILED)
> {
>   errno = overflow_errno;
>   goto error_and_exit;
> }
>   chars_read = file_size;
> #else
>   buffer = (char *)malloc (file_size + 1);
>   if (buffer == 0)
> {
>   errno = overflow_errno;
>   goto error_and_exit;
> }
>
>   chars_read = read (file, buffer, file_size);
> #endif

I have tried using a named pipe or process substition, neither worked.

Bash 4.1 doesn't behave as I think it should: arrays and the environment

[John Summerfield]

GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 



This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I am running on 64-bit CentOS 6.

I have been writing a script that reads a text file and munges it to 
create a shell command to run a script.


In two cases I wish to pass an array in the environment, like so:
14:28 john@Boomer$ STUFF[1]=one STUFFX=stuffx env | grep ^ST
STUFFX=stuffx
STUFF[1]=one
14:28 john@Boomer$

Of course, my script is not called "env," but this shows the sort of 
thing that is clearly visible within my script.


The symptom I observed is that the elements of STUFF are visible in the 
environment, but expand to NULL in my script. However, things like 
STUFFX are visible in the environment and their values are accessible in 
the script.


The man page for bash contains a para entitled ENVIRONMENT which doesn't 
mention arrays, leaving the reader to assume they are not different from 
other shell variables.


It was some time (hours) before I tried this:
14:28 john@Boomer$ export STUFF[1]=one
bash: export: `STUFF[1]': not a valid identifier
14:33 john@Boomer$

I have found no documentation (and I have searched 
http://www.gnu.org/software/bash/manual/bashref.html ) that clarifies 
what is going on.


Now, passing arrays via the environment is useful, I found a use for it.

It is possible (more or less) to get elements of the array into the 
environment. It's unclear to me, but I suppose the above example may 
have created a variable "STUFF[1]."


I note that IEEE Std 1003.1, 2004 Edition 6 allows implementations to 
expand the base rules regarding variable names so that "STUFF[1]" is 
permissible in the environment.


I suggest bash be enhanced to allow the export statement, or something 
analogous applying to arrays, so that array elements can be passed in 
the same way as ordinary variables.


If there exists  a way to do what I have been trying, then it needs to 
be documented somewhere and mentioned in the ENVIRONMENT para and in 
documentation of the set command and arrays, and maybe other places too.


--
John Summerfield

Re: Bash 4.1 doesn't behave as I think it should: arrays and the environment

[Greg Wooledge]

On Fri, Aug 17, 2012 at 03:19:56PM +0800, John Summerfield wrote:
> In two cases I wish to pass an array in the environment, like so:
> 14:28 john@Boomer$ STUFF[1]=one STUFFX=stuffx env | grep ^ST
> STUFFX=stuffx
> STUFF[1]=one

You are creating an environment variable called "STUFF[1]".  This is
an invalid environment variable name, but env(1) receives it and
dutifully prints it out.  However, Bash recognizes that it is an
invalid name, and does not turn it into a usable shell variable.

Some older versions of Bash would also strip such invalid variables
from the environment before invoking child processes.  Bash 4.2
leaves them in the environment because of complaints about build
systems that were using them for some purpose.  I don't know whether
Bash 4.1 is one that stripped them or preserved them.  In either
case, you should not be writing programs that rely on invalid variable
names.

`bash --debugger` in interactive mode

[Techlive Zheng]

I happened to have `shopt -s extdebug` in my .bashrc recently.

After a new bash session started, I got the following output.

bash debugger, bashdb, release 4.2-0.8

Copyright 2002, 2003, 2004, 2006, 2007, 2008, 2009, 2010, 2011
Rocky Bernstein
This is free software, covered by the GNU General Public License,
and you are
welcome to change it and/or distribute copies of it under certain
conditions.

** Internal debug error _Dbg_is_file(): file argument null
bash: _Dbg_filenames[$fullname]: bad array subscript

I guess `--debugger` or `shopt -s extdebug` is not supposed to be run
under interactive mode?

If so, maybe a detection should be done for this situation?