[Bug c/27702] New: the trampoline code of nested functions depends on executable stacks

[okuji at enbug dot org]

As discussed in this thread:

<URL:http://gcc.gnu.org/ml/gcc-help/2005-12/msg00173.html>

Many GNU/Linux distributions (such as Debian, Ubuntu and RedHat) are planning
to prohibit executable stacks completely, regardless of the presence of the
executable stack flag. At the moment, GCC produces trampoline code for nested
functions on the stack, so the use of nested functions will disable programs,
thus nested functions won't be usable in reality.

Here is an example of producing trampoline code on the stack:

gcc -O1 -save-temps -c nested_test.c

# 1 "nested_test.c"
# 1 "<built-in>"
# 1 "<command line>"
# 1 "nested_test.c"
void f0(void (*f)());

long f1 (void)
{
        long i = 0;
        void f2(void)
        {
                i++;
        }
        f0(f2);
        return i;
}

void f0(void (*f)())
{
        (*f)();
}

int main()
{
        return f1();
}

As Ian suggested in <URL:http://gcc.gnu.org/ml/gcc-help/2005-12/msg00177>,
it would work on POSIX systems to put trampoline code on read-write pages
allocated by mmap then switch to read-exec by mprotect before jumping
to the trampoline code.


-- 
           Summary: the trampoline code of nested functions depends on
                    executable stacks
           Product: gcc
           Version: 4.0.1
            Status: UNCONFIRMED
          Severity: major
          Priority: P3
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: okuji at enbug dot org
 GCC build triplet: i586-mandriva-linux-gnu
  GCC host triplet: i586-mandriva-linux-gnu
GCC target triplet: i586-mandriva-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27702