Hi! On the huge testcase in the PR we ICE, because GC needs too deep recursion. I've noticed that part of the problem is deep recursion through TYPE_NEXT_VARIANT pointers. As TREE_CHAIN on types is TYPE_STUB_DECL, therefore should point to a different kind of tree and thus isn't really a pointer through which long chain of trees are linked together, I think we should use TYPE_NEXT_VARIANT as such chain instead. In fact, the C FE already uses it, just for INTEGER_TYPEs only. This patch changes both C and C++ FEs to use it for all types.
The patch fixes the testcase (while without the patch it needs roughly 35MB of stack, with the patch 5.25MB is enough) and has been bootstrapped/regtested on x86_64-linux and i686-linux. Ok for trunk? 2011-06-24 Jakub Jelinek <ja...@redhat.com> PR c++/46400 * cp-tree.h (union lang_tree_node): Use TYPE_NEXT_VARIANT instead of TYPE_CHAIN for chain_next for types. * c-decl.c (union lang_tree_node): Use TYPE_NEXT_VARIANT instead of TYPE_CHAIN for chain_next for types. --- gcc/cp/cp-tree.h.jj 2011-06-21 16:45:52.000000000 +0200 +++ gcc/cp/cp-tree.h 2011-06-24 12:07:42.000000000 +0200 @@ -729,7 +729,7 @@ enum cp_tree_node_structure_enum { /* The resulting tree type. */ union GTY((desc ("cp_tree_node_structure (&%h)"), - chain_next ("CODE_CONTAINS_STRUCT (TREE_CODE (&%h.generic), TS_COMMON) ? ((union lang_tree_node *) TREE_CHAIN (&%h.generic)) : NULL"))) lang_tree_node { + chain_next ("CODE_CONTAINS_STRUCT (TREE_CODE (&%h.generic), TS_TYPE_COMMON) ? ((union lang_tree_node *) TYPE_NEXT_VARIANT (&%h.generic)) : CODE_CONTAINS_STRUCT (TREE_CODE (&%h.generic), TS_COMMON) ? ((union lang_tree_node *) TREE_CHAIN (&%h.generic)) : NULL"))) lang_tree_node { union tree_node GTY ((tag ("TS_CP_GENERIC"), desc ("tree_node_structure (&%h)"))) generic; struct template_parm_index_s GTY ((tag ("TS_CP_TPI"))) tpi; --- gcc/c-decl.c.jj 2011-06-17 11:02:19.000000000 +0200 +++ gcc/c-decl.c 2011-06-24 12:13:35.000000000 +0200 @@ -238,7 +238,7 @@ extern char C_SIZEOF_STRUCT_LANG_IDENTIF /* The resulting tree type. */ union GTY((desc ("TREE_CODE (&%h.generic) == IDENTIFIER_NODE"), - chain_next ("TREE_CODE (&%h.generic) == INTEGER_TYPE ? (union lang_tree_node *) TYPE_NEXT_VARIANT (&%h.generic) : CODE_CONTAINS_STRUCT (TREE_CODE (&%h.generic), TS_COMMON) ? ((union lang_tree_node *) TREE_CHAIN (&%h.generic)) : NULL"))) lang_tree_node + chain_next ("CODE_CONTAINS_STRUCT (TREE_CODE (&%h.generic), TS_TYPE_COMMON) ? (union lang_tree_node *) TYPE_NEXT_VARIANT (&%h.generic) : CODE_CONTAINS_STRUCT (TREE_CODE (&%h.generic), TS_COMMON) ? ((union lang_tree_node *) TREE_CHAIN (&%h.generic)) : NULL"))) lang_tree_node { union tree_node GTY ((tag ("0"), desc ("tree_node_structure (&%h)"))) Jakub