https://gcc.gnu.org/bugzilla/show_bug.cgi?id=117164
--- Comment #6 from GCC Commits <cvs-commit at gcc dot gnu.org> --- The master branch has been updated by Joseph Myers <js...@gcc.gnu.org>: https://gcc.gnu.org/g:3320319ede5ce1cb34ba016f4a0d1e6588059980 commit r15-5321-g3320319ede5ce1cb34ba016f4a0d1e6588059980 Author: Joseph Myers <josmy...@redhat.com> Date: Fri Nov 15 14:08:42 2024 +0000 tree-nested: Do not inline or clone functions with nested functions with VM return type [PR117164] Bug 117164 is an ICE on an existing test with -std=gnu23 involving a nested function returning a variable-size structure (and I think the last bug needing to be resolved before switching to -std=gnu23 as the default, as without fixing this would be a clear regression from a change in default). The problem is a GIMPLE verification failure where (after type remapping from inlining / cloning) the return type of the function no longer exactly matches the type to which it is assigned (these types use structural equality, which means GIMPLE verification can't use TYPE_CANONICAL and expects an exact match). Specifically, the nested function itself is *not* inlined (the -fno-inline-small-functions in the original test nested-func-12.c, I think, or the noinline attribute in some of my variant tests), but the function containing it is either cloned (the --param ipa-cp-eval-threshold=0 in the original test) or inlined. (I'm not sure what role -fno-guess-branch-probability plays in getting the right situation for the ICE; maybe affecting when inlining or cloning is considered profitable?) There is in fact existing code in tree-nested.cc to prevent inlining of a function containing a nested function with variably modified *argument* types. I think the same issue of ensuring consistency of types means such prevention should also apply for a variably modified return type. Furthermore, exactly the same problem applies for cloning for other reasons as it does for inlining. Thus, change the logic to include variably modified return types for nested functions alongside those for arguments of those functions as a reason not to inline, and also add the noclone attribute in these cases. Bootstrapped with no regressions for x86-64-pc-linux-gnu. PR c/117164 gcc/ * tree-nested.cc: Include "attribs.h". (check_for_nested_with_variably_modified): Also return true for variably modified return type. (create_nesting_tree): If check_for_nested_with_variably_modified returns true, also add noclone attribute. gcc/testsuite/ * gcc.dg/nested-func-13.c, gcc.dg/nested-func-14.c: gcc.dg/nested-func-15.c, gcc.dg/nested-func-16.c, gcc.dg/nested-func-17.c: New tests.
