https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107606
--- Comment #2 from Florian Weimer <fw at gcc dot gnu.org> ---
(In reply to Segher Boessenkool from comment #1)
> Or what else is the intention? Do you have an example of something that was
> hard to debug, maybe?
The prctl(2) manual page documents the prctl function as:
int prctl(int option, unsigned long arg2, unsigned long arg3,
unsigned long arg4, unsigned long arg5);
<https://man7.org/linux/man-pages/man2/prctl.2.html>
But if you use this prototype to call prctl (potentially through a function
pointer), then the glibc implementation will corrupt the stack in the caller
because it is defined as:
int
prctl (int option, ...)
{
…
}
The prototype in <sys/prctl.h> matches this definition. We use to have an
assembler implementation that did not clobber the caller stack even if called
as a non-variadic function. But that got replaced in glibc 2.32, breaking some
programs. We had similar portability issues with open/openat.
Technically these things are all undefined, and the glibc headers and
implementations are consistent. Users can even declare their own matching
(variadic) prototypes, as required by POSIX for such functions. On the other
hand, the prctl prototype is misdocumented, so it's easy to get that wrong, and
the variadic nature of open/openat is not that widely realized, either.
Basically this is an avoidable portability trap for powerpc64le-linux-gnu, and
maybe we can make life for programmers a bit easier.
