[PR middle-end/79123] cast false positive in -Walloca-larger-than=

[Aldy Hernandez]

In the attached testcase, we have a clearly bounded case of alloca which 
is being incorrectly reported:

void g (int *p, int *q)
{
   size_t n = (size_t)(p - q);
   if (n < 10)
     f (__builtin_alloca (n));
}

The problem is that VRP gives us an anti-range for `n' which may be out 
of range:

  # RANGE ~[2305843009213693952, 16140901064495857663]
   n_9 = (long unsigned int) _4;

We do a less than stellar job with casts and VR_ANTI_RANGE's, mostly 
because we're trying various heuristics to make up for the fact that we 
have crappy range info from VRP.  More specifically, we're basically 
punting on an VR_ANTI_RANGE and ignoring that the casted result (n_9) 
has a bound later on.

Luckily, we already have code to check simple ranges coming into the 
alloca by looking into all the basic blocks feeding it.  The attached 
patch delays the final decision on anti ranges until we have examined 
the basic blocks and determined for that we are definitely out of range.

I expect all this to disappear with Andrew's upcoming range info overhaul.

OK for trunk?

commit 07677ba03a01cbd1f1c4747b4df333b35d0d3afd
Author: Aldy Hernandez <al...@redhat.com>
Date:   Thu Jan 19 05:44:58 2017 -0500

            PR middle-end/79123
            * gimple-ssa-warn-alloca.c (alloca_call_type): Make sure
            casts from signed to unsigned really don't have a range.

diff --git a/gcc/gimple-ssa-warn-alloca.c b/gcc/gimple-ssa-warn-alloca.c
index a27eea1..d553a34 100644
--- a/gcc/gimple-ssa-warn-alloca.c
+++ b/gcc/gimple-ssa-warn-alloca.c
@@ -272,6 +272,7 @@ static struct alloca_type_and_limit
 alloca_call_type (gimple *stmt, bool is_vla, tree *invalid_casted_type)
 {
   gcc_assert (gimple_alloca_call_p (stmt));
+  bool tentative_cast_from_signed = false;
   tree len = gimple_call_arg (stmt, 0);
   tree len_casted = NULL;
   wide_int min, max;
@@ -352,8 +353,26 @@ alloca_call_type (gimple *stmt, bool is_vla, tree 
*invalid_casted_type)
          // with this heuristic.  Hopefully, this VR_ANTI_RANGE
          // nonsense will go away, and we won't have to catch the
          // sign conversion problems with this crap.
+         //
+         // This is here to catch things like:
+         // void foo(signed int n) {
+         //   if (n < 100)
+         //     alloca(n);
+         //   ...
+         // }
          if (cast_from_signed_p (len, invalid_casted_type))
-           return alloca_type_and_limit (ALLOCA_CAST_FROM_SIGNED);
+           {
+             // Unfortunately this also triggers:
+             //
+             // __SIZE_TYPE__ n = (__SIZE_TYPE__)blah;
+             // if (n < 100)
+             //   alloca(n);
+             //
+             // ...which is clearly bounded.  So, double check that
+             // the paths leading up to the size definitely don't
+             // have a bound.
+             tentative_cast_from_signed = true;
+           }
        }
       // No easily determined range and try other things.
     }
@@ -371,10 +390,12 @@ alloca_call_type (gimple *stmt, bool is_vla, tree 
*invalid_casted_type)
          ret = alloca_call_type_by_arg (len, len_casted,
                                         EDGE_PRED (bb, ix), max_size);
          if (ret.type != ALLOCA_OK)
-           return ret;
+           break;
        }
     }
 
+  if (tentative_cast_from_signed && ret.type != ALLOCA_OK)
+    return alloca_type_and_limit (ALLOCA_CAST_FROM_SIGNED);
   return ret;
 }
 
diff --git a/gcc/testsuite/gcc.dg/Walloca-13.c 
b/gcc/testsuite/gcc.dg/Walloca-13.c
new file mode 100644
index 0000000..f9bdcef
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/Walloca-13.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* { dg-options "-Walloca-larger-than=100 -O2" } */
+
+void f (void*);
+
+void g (int *p, int *q)
+{
+  __SIZE_TYPE__ n = (__SIZE_TYPE__)(p - q);
+  if (n < 100)
+    f (__builtin_alloca (n));
+}