When encountering a memcmp with something like ULONG_MAX for the size, a
calculation can overflow in by_pieces_ninsns because it uses int rather
than HOST_WIDE_INT.
Bootstrapped & tested on x86_64-linux, committed as obvious.
Bernd
Index: gcc/ChangeLog
===================================================================
--- gcc/ChangeLog (revision 240861)
+++ gcc/ChangeLog (working copy)
@@ -1,3 +1,9 @@
+2016-10-07 Bernd Schmidt <bschm...@redhat.com>
+
+ PR tree-optimization/77880
+ * expr.c (by_pieces_ninsns): Use unsigned HOST_WIDE_INT where
+ necessary.
+
2016-10-07 Marek Polacek <pola...@redhat.com>
PR c++/77803
Index: gcc/expr.c
===================================================================
--- gcc/expr.c (revision 240861)
+++ gcc/expr.c (working copy)
@@ -785,7 +785,7 @@ by_pieces_ninsns (unsigned HOST_WIDE_INT
case COMPARE_BY_PIECES:
int batch = targetm.compare_by_pieces_branch_ratio (mode);
int batch_ops = 4 * batch - 1;
- int full = n_pieces / batch;
+ unsigned HOST_WIDE_INT full = n_pieces / batch;
n_insns += full * batch_ops;
if (n_pieces % batch != 0)
n_insns++;
