Ping.
Thanks,
Kyrill
> On 6 Jul 2026, at 20:32, Kyrylo Tkachov <[email protected]> wrote:
>
> Ping.
> https://gcc.gnu.org/pipermail/gcc-patches/2026-June/719927.html
> Thanks,
> Kyrill
>
>> On 10 Jun 2026, at 13:37, Kyrylo Tkachov <[email protected]> wrote:
>>
>> From: Kyrylo Tkachov <[email protected]>
>>
>> eliminated_by_inlining_prob only credited a write to the return value as
>> free-after-inlining when the RHS was a register or invariant (a scalar
>> "return x;"). A function that returns an aggregate by value
>> ("V t = a; ...; return t;") therefore had the copy of the result into the
>> return slot charged in full to the inliner's growth estimate, even though
>> return-slot optimization / SRA eliminates that copy once the call is inlined
>> (the inlined callee writes the caller's destination in place).
>>
>> For a small value-type wrapper -- a SIMD/array lane type, a std::array-like
>> POD, std::complex, a short Tensor/Matrix -- whose overloaded operators are
>> written "V op(const V &a, const V &b) { V t = a; t op= b; return t; }", the
>> mispriced return-slot copy pushes the wrapper over --param
>> early-inlining-insns, so the early inliner leaves it out of line. Such
>> operators are called pervasively, and keeping them out of line also forces
>> their operands and result through memory in the caller.
>>
>> Credit aggregate copies whose destination is the return value (a RESULT_DECL,
>> or a store through the invisible-reference return pointer) as eliminated by
>> inlining, mirroring the existing treatment of reads of by-reference
>> parameters. Statements with volatile operands are excluded: a volatile
>> access is never elided, so such a copy is not free after inlining.
>>
>> This is a size-model refinement only; no IR is changed and no semantics are
>> affected.
>>
>> This gives about 8.5% on an aarch64 machine on the 766.femflow_r
>> benchmark from SPEC2026 where these routines are critical to performance
>> and their vectorised form needs to be inlined for good performance.
>>
>> Bootstrapped and tested on aarch64-none-linux-gnu.
>>
>> Signed-off-by: Kyrylo Tkachov <[email protected]>
>>
>> gcc/ChangeLog:
>>
>> * ipa-fnsummary.cc (eliminated_by_inlining_prob): Credit aggregate
>> copies into the return slot (a RESULT_DECL or invisible-reference
>> return) as eliminated by inlining.
>>
>> gcc/testsuite/ChangeLog:
>>
>> * g++.dg/ipa/inline-aggregate-retval.C: New test.
>> * gcc.dg/ipa/inline-aggregate-retval.c: New test.
>> ---
>> gcc/ipa-fnsummary.cc | 27 ++++++++++++++++
>> .../g++.dg/ipa/inline-aggregate-retval.C | 21 +++++++++++++
>> .../gcc.dg/ipa/inline-aggregate-retval.c | 31 +++++++++++++++++++
>> 3 files changed, 79 insertions(+)
>> create mode 100644 gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C
>> create mode 100644 gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c
>>
>> diff --git a/gcc/ipa-fnsummary.cc b/gcc/ipa-fnsummary.cc
>> index a6ccdb1852a..c05ffc313ed 100644
>> --- a/gcc/ipa-fnsummary.cc
>> +++ b/gcc/ipa-fnsummary.cc
>> @@ -1476,6 +1476,33 @@ eliminated_by_inlining_prob (ipa_func_body_info *fbi,
>> gimple *stmt)
>> if (lhs_free
>> && (is_gimple_reg (rhs) || is_gimple_min_invariant (rhs)))
>> rhs_free = true;
>> + /* A by-value aggregate return ("T t = ...; return t;") stores the
>> + result into the return slot. Under the return-slot optimization the
>> + inlined callee writes the caller's destination in place and SRA then
>> + scalarises the small aggregate, so the store is eliminated by inlining
>> + -- the return-value analogue of the by-reference parameter reads
>> + credited above. The is_gimple_reg/is_gimple_min_invariant test only
>> + credits scalar returns; also credit an aggregate return whose source is
>> + an ordinary memory value. This is the value-type wrapper idiom
>> + (a SIMD/array lane type, std::array, std::complex, a short Tensor):
>> + "T operator OP (const T &a, const T &b) { T t = a; t OP= b; return t;
>> }".
>> + This is the whole-aggregate-copy case. */
>> + if (!rhs_free
>> + && !gimple_has_volatile_ops (stmt)
>> + && AGGREGATE_TYPE_P (TREE_TYPE (lhs))
>> + && gimple_assign_rhs_class (stmt) == GIMPLE_SINGLE_RHS
>> + && (TREE_CODE (inner_lhs) == RESULT_DECL
>> + || (TREE_CODE (inner_lhs) == MEM_REF
>> + && TREE_CODE (TREE_OPERAND (inner_lhs, 0)) == SSA_NAME
>> + && SSA_NAME_VAR (TREE_OPERAND (inner_lhs, 0))
>> + && TREE_CODE (SSA_NAME_VAR (TREE_OPERAND (inner_lhs, 0)))
>> + == RESULT_DECL))
>> + && (TREE_CODE (inner_rhs) == MEM_REF
>> + || TREE_CODE (inner_rhs) == COMPONENT_REF
>> + || VAR_P (inner_rhs)
>> + || TREE_CODE (inner_rhs) == PARM_DECL
>> + || TREE_CODE (inner_rhs) == RESULT_DECL))
>> + return 2;
>> if (lhs_free && rhs_free)
>> return 1;
>> }
>> diff --git a/gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C
>> b/gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C
>> new file mode 100644
>> index 00000000000..a60bad00b66
>> --- /dev/null
>> +++ b/gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C
>> @@ -0,0 +1,21 @@
>> +// The value-type operator-overload idiom: a by-value 'operator*' returns
>> its
>> +// result aggregate, and the copy of that result into the return slot is
>> +// eliminated by inlining (return-slot optimization / SRA). It used to be
>> +// charged to the early-inliner growth, pushing the operator over
>> +// --param early-inlining-insns. -fno-inline-functions isolates the early
>> +// inliner so the operator is inlined only when the copy is credited.
>> +// { dg-do compile }
>> +// { dg-options "-O3 --param early-inlining-insns=14 -fno-inline-functions
>> -fno-partial-inlining -fdump-tree-einline-optimized" }
>> +
>> +struct V {
>> + double d[8];
>> + V &operator*= (const V &o) { for (int i = 0; i < 8; i++) d[i] *= o.d[i];
>> return *this; }
>> +};
>> +
>> +static V operator* (const V &a, const V &b) { V t = a; return t *= b; }
>> +
>> +V ga, gb, gc, r;
>> +
>> +void use () { r = ga * gb * gc; }
>> +
>> +// { dg-final { scan-tree-dump "Inlining.*operator.*into.*use" "einline" } }
>> diff --git a/gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c
>> b/gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c
>> new file mode 100644
>> index 00000000000..e21cff3438f
>> --- /dev/null
>> +++ b/gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c
>> @@ -0,0 +1,31 @@
>> +/* A small by-value wrapper that returns its aggregate result by value. The
>> + copy of the result into the return slot is eliminated by inlining
>> + (return-slot optimization / SRA combines it with the caller's
>> destination),
>> + but it used to be charged in full to the early-inliner growth estimate,
>> + pushing the wrapper over --param early-inlining-insns and leaving it out
>> of
>> + line. -fno-inline-functions isolates the early inliner, so 'mul' can
>> only
>> + be inlined when the return-slot copy is correctly credited as
>> eliminable. */
>> +/* { dg-do compile } */
>> +/* { dg-options "-O3 --param early-inlining-insns=14 -fno-inline-functions
>> -fno-partial-inlining -fdump-tree-einline-optimized" } */
>> +
>> +struct V { double d[8]; };
>> +
>> +static struct V
>> +mul (const struct V *a, const struct V *b)
>> +{
>> + struct V t = *a;
>> + for (int i = 0; i < 8; i++)
>> + t.d[i] *= b->d[i];
>> + return t;
>> +}
>> +
>> +struct V ga, gb, gc, r;
>> +
>> +void
>> +use (void)
>> +{
>> + struct V x = mul (&ga, &gb);
>> + r = mul (&x, &gc);
>> +}
>> +
>> +/* { dg-final { scan-tree-dump "Inlining mul.* into use" "einline" } } */
>> --
>> 2.50.1 (Apple Git-155)
>>
>
> <favicon.ico>
