read Built-in Parameter Behavior -- Null Byte Delimiter

[Adam Danischewski]

It seems the parameter for the delimiter for the read built-in behaves
differently for the NULL case, and it is a very useful case. I found this
after a difficult to track down bug appeared in some of  my code, so I
thought I would pass it on to you.

If it is expected behavior I didn't see it in the documents. Most other
options seem to follow the getopts model and allow for no space for
parameter arguments.

It seems to work for the read built-in yet not for NULL read -d''.

E.g.
$> touch file_{1..40..3}.txt
$> while IFS= read -r -d'.' a; do echo "got $a"; done < <(find . -type f
-print0)
 *  has the same results as *
$> while IFS= read -r -d '.' a; do echo "got $a"; done < <(find . -type f
-print0)
got
got /file_40
got txt
got /file_37
...
got txt
got /file_1

Yet if we look for the null byte:
$> while IFS= read -r -d'' a; do echo "got $a"; done < <(find . -type f
-print0)
 *  returns nothing *
$> while IFS= read -r -d '' a; do echo "got $a"; done < <(find . -type f
-print0)
   *returns the expected results*
got ./file_40.txt
got ./file_37.txt
got ./file_34.txt
...
got ./file_16.txt

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Piotr Grzybowski]

hi,

On Sat, Jan 16, 2016 at 8:28 PM, Adam Danischewski
 wrote:
> ..
> Yet if we look for the null byte:
> $> while IFS= read -r -d'' a; do echo "got $a"; done < <(find . -type f
> -print0)
>returns nothing

 this is because it is the same as:

read -r -d a;

 I think there is no way to distinguish the two.
 The documentation clearly says:

-d delim

 so without going into details of how getopt works, it seems to me
that this is not a bug.

> $> while IFS= read -r -d '' a; do echo "got $a"; done < <(find . -type f
> -print0)
>returns the expected results

pg

Re: read Built-in Parameter Behavior -- Null Byte Delimiter

[Martijn Dekker]

Adam Danischewski schreef op 16-01-16 om 20:28:
> Yet if we look for the null byte:

bash, like most UNIX shells, cannot store or handle null bytes. (zsh is
the only exception I know of.)

> $> while IFS= read -r -d'' a; do echo "got $a"; done < <(find . -type f
> -print0)
>  * returns nothing *

This is because of how shell quoting works. The "read" command doesn't
even enter into it. It's all happening before "read" is even executed.

'' does not represent a null byte; it's an empty, as in zero bytes,
surrounded by single quotes.

Before passing the arguments to the command (such as 'read'), the shell
performs quote removal. This removes the quotes from the empty.

That leaves the 'read' command with no way to distinguish between

read -r -d'' a

and

read -r -d a

Meaning, you were actually executing "read" with "a" as the delimiter,
and no variable, so your "echo" command naturally returned nothing.
Meanwhile "read" would have stored whatever it got in the variable REPLY
by default.

Separating the quoted empty from the -d makes it possible to distinguish
it as an empty argument (which is still different from a null byte).

Hope this helps.

- M.