https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101718
Bug ID: 101718
Summary: Objective-C frontend emits wrong code to call methods
returning _Complex types returned in memory
Product: gcc
Version: 10.2.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: objc
Assignee: unassigned at gcc dot gnu.org
Reporter: mhjacobson at me dot com
Target Milestone: ---
When a method returns a type that the platform ABI says is returned in memory,
the Objective-C frontend (for the NeXT runtime ABIs) emits a call to
`objc_msgSend_stret`; the `_stret` variant of the message dispatcher knows to
look at the *second* argument for `self`, and so on, since the first argument
is the pointer to where the return value will go.
On some platforms, `_Complex double` is too large to be returned through
registers, so it's returned through memory.
But the Objective-C frontend still insists on using `objc_msgSend`, not
`objc_msgSend_stret`.
`objc_msgSend` is thoroughly confused when the first argument, which it expects
to be `self`, is actually a pointer to garbage-filled stack.
I believe this happens because this check:
```
/* If we are returning a struct in memory, and the address
of that memory location is passed as a hidden first
argument, then change which messenger entry point this
expr will call. NB: Note that sender_cast remains
unchanged (it already has a struct return type). */
if (!targetm.calls.struct_value_rtx (0, 0)
&& (TREE_CODE (ret_type) == RECORD_TYPE
|| TREE_CODE (ret_type) == UNION_TYPE)
&& targetm.calls.return_in_memory (ret_type, 0))
{
```
limits the use of `stret` to record/union types. Primitive types that are
returned in memory should also be made to use `stret` (the name's meaning
notwithstanding).
