http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58378
Bug ID: 58378
Summary: Protect libgomp against child process hanging after a
Unix fork()
Product: gcc
Version: 4.8.1
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: libgomp
Assignee: unassigned at gcc dot gnu.org
Reporter: olivier.grisel at ensta dot org
CC: jakub at gcc dot gnu.org
Created attachment 30784
--> http://gcc.gnu.org/bugzilla/attachment.cgi?id=30784&action=edit
Patch to protect libgomp thread pool against fork()
The problem is discussed in [1]. To summarize if a process uses OpenMP
features and then calls fork, the threads from the OpenMP thread pool
of the parent process are not copied to the child process (which is
expected). Then later if the child process uses some OpenMP feature
again it will hang, waiting for threads that don't exist in its own
process.
In practice this can often happen in Python programs that import
modules that use OpenMP internally while also using the
`multiprocessing` module. This module is in the Python standard
library and uses Unix fork for handling multi-core concurrency
efficiently at the Python level.
I attach the patch to this report and a test that checks that the fix actually
works. The patch can also be visualized on this github branch [2].
When running the example snippet from [1] saved in a file called
`openmp_fork.c` I get the expected output:
$ gcc-head -fopenmp -o openmp_fork openmp_fork.c && ./openmp_fork
para_a
para_a
a ended
para_b
para_b
b ended
instead of a hanging process.
[1] http://bisqwit.iki.fi/story/howto/openmp/#OpenmpAndFork
[2] https://github.com/ogrisel/gcc/compare/forksafe-omp-pthread_atfork
Note that the OpenMP implementation of ICC does not hang either when using
fork.
Note 2: this problem is related to (a duplicate of)
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52303 which was deemed
invalid as the POSIX standard states that it's unsafe to use threading after a
fork and prior to calling exec. However as the `pthread_atfork` protection from
this patch is rather non-invasive and should not impact any process not
initializing the libgomp runtime prior to a fork. Interpreted language
implementations such as CPython that (ab)use Unix fork for efficient
concurrency would really benefit from such a protection against libraries using
OpenMP internally with the caller not necessarily being aware of it.
