builtin test command file existence fails with negation

[Lynn Kerby]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' - 
DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' - 
DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' - 
DSHELL -DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/ 
lib   -g -O2 -Wall
uname output: Linux virtsrv2 2.6.28-13-server #44-Ubuntu SMP Tue Jun 2  
08:40:28 UTC 2009 x86_64 GNU/Linux

Machine Type: x86_64-pc-linux-gnu

Bash Version: 3.2
Patch Level: 48
Release Status: release

Description:
Use of the '-a' option to the builtin test command fails to
produce the correct result when used with negation. The specific
error the case where the file exists and a "test ! -a file" is
executed.  If the script is changed to use '-e' for file
existence the result is correct as expected.  If the script is
modified to place parenthesis around the '-a file' expression,
the result is also correct.

Repeat-By:
Run the following bash script to see all behaviors including the
behavior of the /usr/bin/test command:

---cut
#!/bin/bash
#
fn=._you_better_not_have_this_file

trap "rm -f $fn" 0

for (( tc=0; tc < 4; tc++ ))
do
echo "Starting case $tc"
case $tc in
"0")  # bash test builtin; bracket operator
ts=\[
te=\]
to=-a
;;
"1")  # bash test builtin; test
ts=test
te=
to=-a
;;
"2")  # system standalone test
ts=/usr/bin/test
te=
to=-a
;;
"3")  # bash test builtin; test; -e unary op
ts=test
te=
to=-e
;;
esac

#
if eval $ts $to $fn $te
then
echo "surprised (not good)"
fi
if eval $ts ! $to $fn $te
then
echo "file $fn not present (good)"
fi
touch $fn
if eval $ts $to $fn $te
then
echo "file present (good)"
fi
if eval $ts ! $to $fn $te
then
echo "file $fn not present (bug)"
fi

echo "Completed case $tc"
echo
rm -f $fn
done
---cut

Fix:
Changing the order of clauses in the three_arguments function to
look for negation before the possibility of an and/or expression
solved the problem for me.  It also passed the test cases for the
test operator.  This is consistent with the way things are handled
in the coreutils standalone test program.

--- bash-3.2-dist/test.c2006-01-29 11:44:02.0 -0800
+++ bash-3.2/test.c 2009-07-20 19:20:08.0 -0700
@@ -709,14 +709,6 @@ three_arguments ()
   value = binary_operator ();
   pos = argc;
 }
-  else if (ANDOR (argv[pos+1]))
-{
-  if (argv[pos+1][1] == 'a')
-   value = ONE_ARG_TEST(argv[pos]) && ONE_ARG_TEST(argv[pos+2]);
-  else
-   value = ONE_ARG_TEST(argv[pos]) || ONE_ARG_TEST(argv[pos+2]);
-  pos = argc;
-}
   else if (argv[pos][0] == '!' && argv[pos][1] == '\0')
 {
   advance (1);
@@ -727,6 +719,14 @@ three_arguments ()
   value = ONE_ARG_TEST(argv[pos+1]);
   pos = argc;
 }
+  else if (ANDOR (argv[pos+1]))
+{
+  if (argv[pos+1][1] == 'a')
+   value = ONE_ARG_TEST(argv[pos]) && ONE_ARG_TEST(argv[pos+2]);
+  else
+   value = ONE_ARG_TEST(argv[pos]) || ONE_ARG_TEST(argv[pos+2]);
+  pos = argc;
+}
   else
 test_syntax_error (_("%s: binary operator expected"), argv[pos 
+1]);


---EOF---

Re: builtin test command file existence fails with negation

[Lynn Kerby]

On Jul 21, 2009, at 8:53 AM, Andreas Schwab wrote:


Lynn Kerby  writes:


Use of the '-a' option to the builtin test command fails to
produce the correct result when used with negation. The specific
error the case where the file exists and a "test ! -a file" is
executed.  If the script is changed to use '-e' for file
existence the result is correct as expected.  If the script is
modified to place parenthesis around the '-a file' expression,
the result is also correct.


This is not a bug, but consistent with the POSIX rules.  Better avoid
the (nonstandard) -a unary operator and use -e instead.  The problem  
is
that -a is also a binary operator, and POSIX says that if test is  
called

with three arguments and the second argument is a binary operator it
should be parsed as such.

See
<http://opengroup.org/onlinepubs/9699919799/utilities/test.html#tag_20_128_05 
>

for details.

Andreas.

--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276  
4ED5

"And now for something completely different."


Thanks, I hadn't read the Open Group test command doc.  However, I  
find the doc inconsistent on this matter when read in its entirety.


For starters, POSIX doesn't allow for the use of '-a' as a unary  
operator (according to the RATIONALE section this is because mere  
humans will be confused and should use '-e' instead). In that context,  
the parsing of 3 arguments to test could proceed on the assumption  
that any '-a' option in the middle would be an attempt to perform a  
binary operation.  The GNU bash and test command do support the  
historical '-a' unary operator usage as a unary operator, so its hard  
to justify this behavior as POSIX conformant IMO.  It might be, but it  
is pretty clearly wrong.


Secondly, down in the RATIONALE section the document states:
In evaluating these more complex combined expressions, the following  
precedence rules are used:


	•The unary primaries have higher precedence than the algebraic  
binary primaries.
	•The unary primaries have lower precedence than the string binary  
primaries.
	•The unary and binary primaries have higher precedence than the  
unary string primary.
	•The ! operator has higher precedence than the -a operator, and the  
-a operator has higher precedence than the -o operator.

•The -a and -o operators are left associative.
	•The parentheses can be used to alter the normal precedence and  
associativity.




So if I'm reading that corrrectly, the '!' operator is supposed to  
have a higher precedence than the '-a' operator in complex  
expressions.  Simply adding a "-a 1" to the expression (resulting in  
" ! -a file -a 1") causes a different parsing behavior.  It seems that  
the 3 argument processing rules for the test command violate this by  
having the negation test ordered after binary primaries.  The rules  
for 2 and 4 argument parsing clearly handle negation first.


I still consider this a bug, since this is caused by an ambiguity (at  
some level) in a non-POSIX defined operator.  Blaming it on a POSIX  
defined argument processing rule doesn't really apply here.  Whatever  
the disposition, I'm going to be modifying this bunch of scripts  
(inherited) to use the '-e' unary operator for existence tests.


Lynn Kerby