Re: Default time for unmarked history lines

[Reuben Thomas]

On 11 January 2016 at 14:22, Chet Ramey  wrote:

> For a history file without any timestamps, using
> the current default and setting the history entry timestamp to the current
> time is more appropriate.
>

​Why is that? The only similar thing I can think of is file systems, where
if you zero the metadata you get timestamps of the epoch, not the current
time.​
​


> You probably also have histappend set or didn't use something like
> `history -w' to force the entire history file to be written when the shell
> exits.
>

​That's right.

--
​ ​

http://rrt.sc3d.org

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Greg Wooledge]

On Sat, Jan 16, 2016 at 02:28:27PM -0500, Adam Danischewski wrote:
> If it is expected behavior I didn't see it in the documents.

I suspect the interpretation of -d '' to mean a NUL byte delimiter
may have been an accident originally (but that's a guess).  But it's
an incredibly useful feature, and has been used in countless real
life scripts.  At this point, while it is still undocumented, it is
nevertheless a feature whose omission would be considered a regression.

Others have correctly described the difference between -d'' and -d ''
so I won't repeat that, except to say that they are right.

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Chet Ramey]

On 1/18/16 8:14 AM, Greg Wooledge wrote:
> On Sat, Jan 16, 2016 at 02:28:27PM -0500, Adam Danischewski wrote:
>> If it is expected behavior I didn't see it in the documents.
> 
> I suspect the interpretation of -d '' to mean a NUL byte delimiter
> may have been an accident originally (but that's a guess).

I guess that depends on what you mean by `accident'.  There's nothing
special in the code that checks for NUL; it's just another delimiter
and another value returned by read(2).  There's no reason it should
not just work.

> But it's
> an incredibly useful feature, and has been used in countless real
> life scripts.  At this point, while it is still undocumented, it is
> nevertheless a feature whose omission would be considered a regression.

It's not a special case that needs to be documented as such.  It's a
straightforward application of the rules for assigning the delimiter
and reading it.  You do have to understand how C strings work and how
they are used when passing arguments to commands for it to make sense.

Chet


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Fwd: Re: Default time for unmarked history lines

[Chet Ramey]

--- Begin Message ---
On 1/11/16 11:54 AM, Reuben Thomas wrote:
> On 11 January 2016 at 14:22, Chet Ramey  > wrote:
> 
> For a history file without any timestamps, using
> the current default and setting the history entry timestamp to the current
> time is more appropriate.
> 
> 
> ​Why is that? The only similar thing I can think of is file systems, where
> if you zero the metadata you get timestamps of the epoch, not the current
> time.​

OK, consider the situation when a user has no timestamps in his history
file.  There seem to be three choices for the timestamp: the epoch time,
which to me is clearly incorrect ("what? what is this 1970s stuff?"), the
last modified time on the history file itself, and the current time.  I
left the default as the current time, so commands from the hsitory fi


>  
> 
> You probably also have histappend set or didn't use something like
> `history -w' to force the entire history file to be written when the shell
> exits.
> 
> 
> ​That's right.​

Then you have presented the history library with an ambiguous situation.

> ​​
> 
> -- 
> http://rrt.sc3d.org


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
--- End Message ---

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Greg Wooledge]

On Mon, Jan 18, 2016 at 11:16:06AM -0500, Chet Ramey wrote:
> On 1/18/16 8:14 AM, Greg Wooledge wrote:
> > I suspect the interpretation of -d '' to mean a NUL byte delimiter
> > may have been an accident originally (but that's a guess).
> 
> I guess that depends on what you mean by `accident'.

Happy coincidence, then.

> It's not a special case that needs to be documented as such.  It's a
> straightforward application of the rules for assigning the delimiter
> and reading it.  You do have to understand how C strings work and how
> they are used when passing arguments to commands for it to make sense.

Other shells must go out of their way to suppress it, then.

wooledg@wooledg:~$ while IFS= read -r -d '' foo; do echo "<$foo>"; done < 
<(printf 'one\0two\0')


wooledg@wooledg:~$ ksh
$ while IFS= read -r -d '' foo; do echo "<$foo>"; done < <(printf 'one\0two\0')
$ while IFS= read -r -d x foo; do echo "<$foo>"; done < <(printf 'onextwox')   

Fwd: Default time for unmarked history lines

[Reuben Thomas]

[
​Forwarding reply erroneously not sent to the list.]​


On 15 January 2016 at 15:26, Chet Ramey  wrote:

> On 1/11/16 11:54 AM, Reuben Thomas wrote:
> > On 11 January 2016 at 14:22, Chet Ramey  > > wrote:
> >
> > For a history file without any timestamps, using
> > the current default and setting the history entry timestamp to the
> current
> > time is more appropriate.
> >
> >
> > ​Why is that? The only similar thing I can think of is file systems,
> where
> > if you zero the metadata you get timestamps of the epoch, not the current
> > time.​
>
> OK, consider the situation when a user has no timestamps in his history
> file.  There seem to be three choices for the timestamp: the epoch time,
> which to me is clearly incorrect ("what? what is this 1970s stuff?"), the
> last modified time on the history file itself, and the current time.  I
> left the default as the current time, so commands from the history file
> are the same as commands entered at the terminal.
>

​I agree with the user story thought in principle.

The trouble is that this is an experience users have anyway with
filesystems and with programs that deal with metadata in files, such as
dates in JPEGs: zero (and sometimes absent) metadata result in epoch
times/dates.

​By fixing it just in bash you're actually confusing users (like me) who've
worked out what tends to happen with 0 dates in general.

So, how about instead interpreting a missing/0 date as a NaD (Not A Date),
rather as readline does anyway with time 0, and providing a slightly more
meaningful message than the current "??". Then a) I would be able to remove
all my bogus "1" timestamps, and b) both "0" and missing timestamps would
give the user a clue that data was missing and/or zero?


> > You probably also have histappend set or didn't use something like
> > `history -w' to force the entire history file to be written when the
> shell
> > exits.
> >
> >
> > ​That's right.​
>
> Then you have presented the history library with an ambiguous situation.
>

Correct, and it's entirely natural: for years I didn't use timestamps, now
I do.

Again, with the solution I propose above, I would get a monotonic history
for the part where time is defined, which is good enough.​

-- 
http://rrt.sc3d.org

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Chet Ramey]

On 1/18/16 11:25 AM, Greg Wooledge wrote:
> On Mon, Jan 18, 2016 at 11:16:06AM -0500, Chet Ramey wrote:
>> On 1/18/16 8:14 AM, Greg Wooledge wrote:
>>> I suspect the interpretation of -d '' to mean a NUL byte delimiter
>>> may have been an accident originally (but that's a guess).
>>
>> I guess that depends on what you mean by `accident'.
> 
> Happy coincidence, then.

Sure.

> 
>> It's not a special case that needs to be documented as such.  It's a
>> straightforward application of the rules for assigning the delimiter
>> and reading it.  You do have to understand how C strings work and how
>> they are used when passing arguments to commands for it to make sense.
> 
> Other shells must go out of their way to suppress it, then.

Most of the other shells remove NUL bytes from `read's input.  They
probably do this before checking the delimiter.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Stephane Chazelas]

2016-01-18 11:16:06 -0500, Chet Ramey:
[...]
> > But it's
> > an incredibly useful feature, and has been used in countless real
> > life scripts.  At this point, while it is still undocumented, it is
> > nevertheless a feature whose omission would be considered a regression.
> 
> It's not a special case that needs to be documented as such.  It's a
> straightforward application of the rules for assigning the delimiter
> and reading it.  You do have to understand how C strings work and how
> they are used when passing arguments to commands for it to make sense.
[...]

You should not have to understand C strings to be able to use a
shell.

C strings are *encoded* with a NUL delimiter. Stricktly
speaking, that NUL is not part of the line's content. "read -d
something" sets the delimited to the first character of
"something", with read -d "", there's no character in that
string.

I think there's no way anyone that has no notion of C could
guess that

read -d '' var

reads until the first NUL character.

A more intuitive reading of that would be that it disables
delimiters altogether (reads until end of file).

I'd agree it should be documented and that it's a useful feature.
An I see the same feature is coming to readarray/mapfile in 4.4
which is welcome as well.

Also, we see people doing:

read -d $'\0' var

That is actually doing what it seems to be saying on the can
(read until a $'\0'), except that what it says on the can is
wrong, since

read -d $'\0' var

actually calls "read" with ("read", "-d", "", "var") arguments.

It may be worth mentioning that command line arguments, here
documents  and variables in bash don't support the NUL character
(and how it behaves in various contexts).

-- 
Stephane

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Stephane Chazelas]

2016-01-18 11:25:49 -0500, Greg Wooledge:
[...]
> Other shells must go out of their way to suppress it, then.
> 
> wooledg@wooledg:~$ while IFS= read -r -d '' foo; do echo "<$foo>"; done < 
> <(printf 'one\0two\0')
> 
> 
> wooledg@wooledg:~$ ksh
> $ while IFS= read -r -d '' foo; do echo "<$foo>"; done < <(printf 
> 'one\0two\0')
> $ while IFS= read -r -d x foo; do echo "<$foo>"; done < <(printf 'onextwox')  
>  
> 
> 
[...]

zsh behaves like bash here (wrt to '', not in corner cases like:
$ echo 'aéb' | bash -c "read -d $'\xc3' a; echo \$a"
a
$ echo 'aéb' | zsh -c "read -d $'\xc3' a; echo \$a"
aéb
)

Note that zsh supports passing NUL characters to its builtins, so

read -d $'\0' var

does do what it says on the tin there.

ksh93's one seems to be broken with multi-byte characters:

~$ echo 'aéb' | ksh -c 'read -d é a; echo $a'
aéb
~$ echo 'aéb' | ksh -c "read -d $'\xc3' a; echo \$a"
zsh: doneecho 'aéb' |
zsh: segmentation fault  ksh -c "read -d $'\xc3' a; echo $a"

-- 
Stephane

Re: Fwd: Default time for unmarked history lines

[Chet Ramey]

On 1/18/16 11:53 AM, Reuben Thomas wrote:

> So, how about instead interpreting a missing/0 date as a NaD (Not A Date),
> rather as readline does anyway with time 0, and providing a slightly more
> meaningful message than the current "??". Then a) I would be able to remove
> all my bogus "1" timestamps, and b) both "0" and missing timestamps would
> give the user a clue that data was missing and/or zero?

I think that a more meaningful error string would be useful.  We'll try
reporting on invalid dates if the timestamp string in the history entry is
non-empty.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/