When ggc_free()ing TYPE_MIN/MAX_VALUE in type_hash_canon we have to make sure to not leave stale entries in int_hash_table. The following fixes that and also makes two related changes - freeing a built INTEGER_CST with a existing entry and properly hashing and finding redundancies of integer types with max value that doesn't fit into 64bits (any __int128 type).
Bootstrapped on x86_64-unknown-linux-gnu, testing in progress. Richard. 2017-11-27 Richard Biener <rguent...@suse.de> * tree.c (wide_int_to_tree): Free discarded INTEGER_CST. (type_hash_canon): Also clear int_cst_hash_table entry for TYPE_MIN/MAX_VALUE. (build_nonstandard_integer_type): Hash all TYPE_MAX_VALUEs. Index: gcc/tree.c =================================================================== --- gcc/tree.c (revision 255138) +++ gcc/tree.c (working copy) @@ -1576,6 +1576,8 @@ wide_int_to_tree (tree type, const wide_ t = nt; *slot = t; } + else + ggc_free (nt); } return t; @@ -6496,10 +6502,18 @@ type_hash_canon (unsigned int hashcode, { if (TYPE_MIN_VALUE (type) && TREE_TYPE (TYPE_MIN_VALUE (type)) == type) - ggc_free (TYPE_MIN_VALUE (type)); + { + /* Zero is always in TYPE_CACHED_VALUES. */ + if (! TYPE_UNSIGNED (type)) + int_cst_hash_table->remove_elt (TYPE_MIN_VALUE (type)); + ggc_free (TYPE_MIN_VALUE (type)); + } if (TYPE_MAX_VALUE (type) && TREE_TYPE (TYPE_MAX_VALUE (type)) == type) - ggc_free (TYPE_MAX_VALUE (type)); + { + int_cst_hash_table->remove_elt (TYPE_MAX_VALUE (type)); + ggc_free (TYPE_MAX_VALUE (type)); + } if (TYPE_CACHED_VALUES_P (type)) ggc_free (TYPE_CACHED_VALUES (type)); } @@ -7486,8 +7500,10 @@ build_nonstandard_integer_type (unsigned fixup_signed_type (itype); ret = itype; - if (tree_fits_uhwi_p (TYPE_MAX_VALUE (itype))) - ret = type_hash_canon (tree_to_uhwi (TYPE_MAX_VALUE (itype)), itype); + + inchash::hash hstate; + inchash::add_expr (TYPE_MAX_VALUE (itype), hstate); + ret = type_hash_canon (hstate.end (), itype); if (precision <= MAX_INT_CACHED_PREC) nonstandard_integer_type_cache[precision + unsignedp] = ret;