[PATCH] Fix PR88217

[Richard Biener]

The following fixes infinite looping in VRP because of tricks we pull
off in PHI handling which are not handling our inconsistent honoring
of TYPE_{MIN,MAX}_VALUE.

Bootstrapped and tested on x86_64-unknown-linux-gnu, applied to trunk.

Richard.

>From 9d76e3649ff28f6abdd4960647de03f10573f546 Mon Sep 17 00:00:00 2001
From: Richard Guenther <rguent...@suse.de>
Date: Wed, 28 Nov 2018 11:22:48 +0100
Subject: [PATCH] fix-pr88217 2018-11-28  Richard Biener  <rguent...@suse.de>

        PR tree-optimization/88217
        * vr-values.c (vr_values::extract_range_from_phi_node): Make
        sure to handle results > +INF and < -INF correctly when
        trying to drop down to +INF - 1 or -INF + 1.

        * g++.dg/pr88217.C: New testcase.

diff --git a/gcc/testsuite/g++.dg/pr88217.C b/gcc/testsuite/g++.dg/pr88217.C
new file mode 100644
index 00000000000..b0506acabd7
--- /dev/null
+++ b/gcc/testsuite/g++.dg/pr88217.C
@@ -0,0 +1,18 @@
+// { dg-do compile { target c++11 } }
+// { dg-options "-O2 -fstrict-enums -fno-tree-forwprop -fno-tree-fre" }
+
+extern "C" int printf (const char *, ...);
+
+enum E { e1, e2, e3, X };
+E operator*(E e) { return e; }
+E begin(E e) { return e; }
+E end(E e) { return X; }
+E operator++(E& e) { return e = E(e+1); }
+
+int main()
+{
+  for (auto e: e1)
+    {
+      printf ("%d ", e);
+    }
+}
diff --git a/gcc/vr-values.c b/gcc/vr-values.c
index 41862b97601..a0027c0b353 100644
--- a/gcc/vr-values.c
+++ b/gcc/vr-values.c
@@ -2857,7 +2857,8 @@ vr_values::extract_range_from_phi_node (gphi *phi, 
value_range *vr_result)
       if (cmp_min < 0)
        new_min = lhs_vr->min ();
       else if (cmp_min > 0
-              && !vrp_val_is_min (vr_result->min ()))
+              && tree_int_cst_lt (vrp_val_min (vr_result->type ()),
+                                  vr_result->min ()))
        new_min = int_const_binop (PLUS_EXPR,
                                   vrp_val_min (vr_result->type ()),
                                   build_int_cst (vr_result->type (), 1));
@@ -2866,7 +2867,8 @@ vr_values::extract_range_from_phi_node (gphi *phi, 
value_range *vr_result)
       if (cmp_max > 0)
        new_max = lhs_vr->max ();
       else if (cmp_max < 0
-              && !vrp_val_is_max (vr_result->max ()))
+              && tree_int_cst_lt (vr_result->max (),
+                                  vrp_val_max (vr_result->type ())))
        new_max = int_const_binop (MINUS_EXPR,
                                   vrp_val_max (vr_result->type ()),
                                   build_int_cst (vr_result->type (), 1));