[PATCH] Improve VRP range intersection for partly symbolic ranges

[Richard Biener]

The following makes intersecting [-INF, +10] and [a + -1, +INF]
to [10, a + -1] possible with the chance that for a <= 10 the
resulting range will be empty (but not trivially visible as so).

Bootstrap / regtest running on x86_64-unknown-linux-gnu.

I'll add a testcase later.

Richard.

2017-04-27  Richard Biener  <rguent...@suse.de>

        * tree-vrp.c (intersect_ranges): Better handle partly
        symbolic ranges.

Index: gcc/tree-vrp.c
===================================================================
--- gcc/tree-vrp.c      (revision 247334)
+++ gcc/tree-vrp.c      (working copy)
@@ -8989,6 +8989,28 @@ intersect_ranges (enum value_range_type
       else
        gcc_unreachable ();
     }
+  else if (operand_less_p (*vr0min, vr1max) == 1
+          && operand_less_p (*vr0min, vr1min) == 1
+          && operand_less_p (*vr0max, vr1max) == 1
+          && operand_less_p (vr1min, *vr0max) == -2)
+    {
+      /* [ (] )  with ] and ( being unordered as (partly) symbolic.
+         This can result in ranges that are effectively empty.  */
+      if (*vr0type == VR_RANGE
+         && vr1type == VR_RANGE)
+       *vr0min = vr1min;
+    }
+  else if (operand_less_p (vr1min, *vr0max) == 1
+          && operand_less_p (vr1min, *vr0min) == 1
+          && operand_less_p (vr1max, *vr0max) == 1
+          && operand_less_p (*vr0min, vr1max) == -2)
+    {
+      /* ( [) ]  with ] and ( being unordered as (partly) symbolic.
+         This can result in ranges that are effectively empty.  */
+      if (*vr0type == VR_RANGE
+         && vr1type == VR_RANGE)
+       *vr0max = vr1max;
+    }
 
   /* As a fallback simply use { *VRTYPE, *VR0MIN, *VR0MAX } as
      result for the intersection.  That's always a conservative