https://gcc.gnu.org/bugzilla/show_bug.cgi?id=122443
Bug ID: 122443
Summary: Huge memory usage in walk_tree_1 with deeply nested
expressions
Product: gcc
Version: 16.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: jit
Assignee: dmalcolm at gcc dot gnu.org
Reporter: antoyo at gcc dot gnu.org
CC: antoyo at gcc dot gnu.org
Target Milestone: ---
Created attachment 62643
--> https://gcc.gnu.org/bugzilla/attachment.cgi?id=62643&action=edit
GIMPLE from a similar function
Hi.
Sam found a case where compiling a Rust project with rustc_codegen_gcc would
lead to an OOM (+64 GB of RAM used for
https://github.com/rust-lang/rustc_codegen_gcc/issues/777).
Some discussion happened regarding deep unsharing
(https://github.com/rust-lang/rustc_codegen_gcc/pull/783#pullrequestreview-3380376351).
I continued investigating and can confirm that this happens in unshare_body.
I tried setting LANG_HOOKS_DEEP_UNSHARING to false and stopped setting
TREE_VISITED to 1, but that doesn't even help.
It seems like walk_tree_1 doesn't like deeply nested expressions.
I've been unable to create a reproducer yet, but one function that
rustc_codegen_gcc gets this issue is this one:
https://github.com/mit-plv/fiat-crypto/blob/5d98fcc6da685ee06302ec43bdcd7c861539b41a/fiat-rust/src/p384_64.rs#L219.
A temporary workaround for this problem is to split the expressions in
temporary variables.
I attached a GIMPLE from a similar function in case that could help pinpoint
the issue.
Unfortunately, I forgot how I got this GIMPLE.
The GIMPLE I now see is much less dense:
_384 = (int128_t) _383;
_385 = _311 - _384;
_386 = _385 + -0xffffffffffffffff;
_387 = _386 >> 64;
_388 = (unsigned char) _387;
_389 = -_388;
_390 = (int128_t) _389;
Any ideas what could cause this issue?
Is it expected that walk_tree_1 consumes a huge amount of RAM for deeply nested
expressions?
Is there anything the other frontends do to split these expressions before they
reach walk_tree_1?
