Bug in [ -d ... ] ?

[Michael F Gordon]

Can anyone explain the following?

tla19> set|grep FOOFOOFOO
tla19> if [ -d ${FOOFOOFOO:=""} ]; then echo YES;fi
YES
tla19> echo $BASH_VERSION 
4.4.12(1)-release

I don't get YES printed with 4.1.2(1)-release on Scientific Linux 6. I do
with 4.2.46(1)-release on Scientific Linux 7 and it happens with the latest
release on both.

This is a simplified version of some code from a Cadence setup script; the
above behaviour causes it to fail on SL7.


Michael Gordon
-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: Bug in [ -d ... ] ?

[Greg Wooledge]

On Thu, Nov 02, 2017 at 11:03:27AM +, Michael F Gordon wrote:
> Can anyone explain the following?
> 
> tla19> set|grep FOOFOOFOO
> tla19> if [ -d ${FOOFOOFOO:=""} ]; then echo YES;fi
> YES

Quoting failure.

wooledg:~$ unset FOO
wooledg:~$ if [ -d "${FOO:=""}" ]; then echo yes; else echo no; fi
no

With quotes the command is transformed to

[ -d "" ]

Without quotes it becomes

[ -d ]

A single-argument test command simply checks whether the string (-d in
this case) is nonzero length.  It is equivalent to

[ -n -d ]

Re: Bug in [ -d ... ] ?

[Eric Blake]

On 11/02/2017 06:03 AM, Michael F Gordon wrote:
> Can anyone explain the following?

Yes.  You used incorrect quoting.

> 
> tla19> set|grep FOOFOOFOO
> tla19> if [ -d ${FOOFOOFOO:=""} ]; then echo YES;fi
> YES

Because you forgot to quote the variable expansion, you ended up executing:

if [ -d ]; then echo YES; fi

and since the one-argument form of [] succeeds if the single argument is
not the empty string, and -d is not the empty string, you got an answer
of YES.

If you want the two-argument form of [], which treats -d as a unary
operator testing whether the second argument names a directory, then you
should have written:

if [ -d "${FOOFOOFOO:=""}" ]; ...

> tla19> echo $BASH_VERSION 
> 4.4.12(1)-release
> 
> I don't get YES printed with 4.1.2(1)-release on Scientific Linux 6. I do
> with 4.2.46(1)-release on Scientific Linux 7 and it happens with the latest
> release on both.

Are you sure you tested correctly?  test has always behaved differently
for one argument than for two.

At any rate, this is not a bug in bash, but more likely in your script.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Bug in [ -d ... ] ?

[Chet Ramey]

On 11/2/17 7:03 AM, Michael F Gordon wrote:
> Can anyone explain the following?
> 
> tla19> set|grep FOOFOOFOO
> tla19> if [ -d ${FOOFOOFOO:=""} ]; then echo YES;fi
> YES
> tla19> echo $BASH_VERSION 
> 4.4.12(1)-release

Greg is correct. This expands to [ -d ], which is true.  There was an
expansion bug in bash-4.1 that caused ${FOOFOOFOO:=""} to expand to the
empty string, but that was fixed almost eight years ago as the result of
http://www.austingroupbugs.net/view.php?id=221

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Bug in [ -d ... ] ?

[Michael F Gordon]

On Thu, Nov 02, 2017 at 09:47:24AM -0400, Greg Wooledge wrote:
> Quoting failure.
> 
> wooledg:~$ unset FOO
> wooledg:~$ if [ -d "${FOO:=""}" ]; then echo yes; else echo no; fi
> no
> 
> With quotes the command is transformed to
> 
> [ -d "" ]
> 
> Without quotes it becomes
> 
> [ -d ]
> 
> A single-argument test command simply checks whether the string (-d in
> this case) is nonzero length.  It is equivalent to
> 
> [ -n -d ]

Interesting.  This is the way "[" has always been documented to work, but 
something has changed in newer versions of the shell - the unquoted version

  if [ -d ${FOO:=""} ]; then echo yes; else echo no; fi

gives "yes" on 4.2 onwards but "no" on 4.1.  Also, replacing := with :-
gives "no" on both versions when unquoted or unquoted.  I think I'll just
rewrite this bit of the script to separate the assignment and test.


Michael
-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

Re: New MILLISECONDS “special” variable?

[Eduardo Bustamante]

El mié., nov. 1, 2017 8:48 PM, Eduardo A. Bustamante López <
dual...@gmail.com> escribió:

> [...]
> /* FIXME: do better validation */
> static int parse_timeval (const char *s, struct timeval *tv) {
>   char *ds;
>   timerclear(tv);
>   tv->tv_sec = strtol(s, &ds, 10);
>   if (ds && *ds == '.') {
> tv->tv_usec = strtol(ds + 1, NULL, 10);
>   }
>   return 0;
> }
>
> int timer_builtin (WORD_LIST *list) {

  [...]

parse_timeval (s, &tv2);
>   timersub (&tv1, &tv2, &tv3);
>   snprintf (b2, 64, "%ld.%ld", tv3.tv_sec, tv3.tv_usec);
>   bind_variable (delta, b2, 0);
>
>   update_value:
>   snprintf (b1, 64, "%ld.%ld", tv1.tv_sec, tv1.tv_usec);
>   bind_variable (current, b1, 0);
>

Heh, this is embarassing. Evidently, that is not the right way of parsing a
timeval, or of storing it in a string (i.e. the tv_usec member).

But at least it still works as a demo of how to approach the problem with
loadable builtins.

>

Re: Bug in [ -d ... ] ?

[Chet Ramey]

On 11/2/17 12:54 PM, Michael F Gordon wrote:

> 
> Interesting.  This is the way "[" has always been documented to work, but 
> something has changed in newer versions of the shell 

I answered this.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: New MILLISECONDS “special” variable?

[Chet Ramey]

On 11/1/17 4:19 PM, Alan Dipert wrote:
> Hello all,
> 
> I understand there is a “special” variable, SECONDS, which returns the
> number of seconds since bash invocation when referenced.
> 
> Unfortunately for tasks like profiling one’s .profile, SECONDS is too
> coarse a unit. Milliseconds are more useful, but getting the time in
> milliseconds is only possibly by calling out to date. (At least, I’m not
> aware of a built-in way to get either Unix epoch or elapsed time in any
> sub-second unit) Shelling out to date isn’t desirable as it adds
> significant overhead and so complicates the task of profiling.

There is a special builtin in the devel branch version (EPOCHREALTIME) that
gives you the epoch time with microsecond resolution. It will be in the
next release.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: New MILLISECONDS “special” variable?

[DJ Mills]

On Thu, Nov 2, 2017 at 3:35 PM, Chet Ramey  wrote:
>
> There is a special builtin in the devel branch version (EPOCHREALTIME) that
> gives you the epoch time with microsecond resolution. It will be in the
> next release.
>
>
Out of curiosity, is that functionality going to extend to printf '%(fmt)T'
? A %N or
so would be lovely

Re: New MILLISECONDS “special” variable?

[Chet Ramey]

On 11/2/17 4:09 PM, DJ Mills wrote:
> 
> 
> On Thu, Nov 2, 2017 at 3:35 PM, Chet Ramey  > wrote:
> 
> There is a special builtin in the devel branch version (EPOCHREALTIME) 
> that
> gives you the epoch time with microsecond resolution. It will be in the
> next release.
> 
> 
> Out of curiosity, is that functionality going to extend to printf '%(fmt)T'
> ? A %N or
> so would be lovely

It depends on what strftime(3) provides; the T format just passes what's
between the parens to strftime().

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Bug in [ -d ... ] ?

[Robert Elz]

Date:Thu, 2 Nov 2017 16:54:48 +
From:Michael F Gordon 
Message-ID:  <20171102165448.ga18...@ee.ed.ac.uk>

  | Also, replacing := with :- gives "no" on both versions when unquoted
  | or unquoted.

That's because they're doing different things, the := form assigns "" to
FOOFOOFOO, then expands ${FOOFOOFOO}.   Expanding a var set to null produces
nothing, not a null string.  In the :- case, if FOO is not set (or is null),
a null string results.   [Incidentally, I believe that was the old bug
Chet referred to, the = operator in ${} expansions, used to return the word
assigned, similar to what the - operator does, rather than assigning the
value, and then expanding the variable, which is what it is supposed to do,
and now does - this example illustrates the difference.]

  | I think I'll just rewrite this bit of the script to separate the
  | assignment and test.

That would be a good idea, as the suggested workaround:

if [ -d "${FOOFOOFOO:=""}" ]; ...

is not really portable.   In bash, and several other shells, it does what
it looks like it should do, but in posix (as it stands today) and in some
other shells, including the original Bourne shell, that is parsed as

if [ -d  "${FOOFOOFOO:=""}" ]; ...
 ^=v^=v

where the ^ indicates quoting starts, the v indicates quoting ends, and
the ='s are under quoted characters.

While this would work for this particular usage, as it is identical to

if [ -d "${FOOFOOFOO:=}" ]; ...

which assigns a null string to FOOFOOFOO and then expands "${FOOFOOFOO}"
which gives the expected null string (and would be entirely suitable as
an alternative formulation), but quoting in the word following the - + =
and ? operators in ${} expansions is a bad habit to get into as other similar
uses don't always work out so well.

For ref, this is being made unspecified in some later version of the
standard, which makes it even worse to use, as then shells will be able
to do almost anything they like when something like this is seen.

kre

ps: I'm not really sure what you're attempting to achieve here anyway,
if FOOFOOFOO is a null string already, assigning a different null string
to it doesn't really change anything.  If the issue is just to make sure
it is set, then why?   Unset vars produce the same null string value when
referenced as set ones do, just leave it unset...