From: Stefan Behnel <stefan...@behnel.de>
To: python-dev@python.org
Subject: Re: [Python-Dev] Forking and Multithreading - enemy brothers
Message-ID: <hk0k0q$7i...@ger.gmane.org>
Content-Type: text/plain; charset=ISO-8859-15
Pascal Chambon, 29.01.2010 22:58:
I've just recently realized the huge problems surrounding the mix of
multithreading and fork() - i.e that only the main thread actually
survived the fork(), and that process data (in particular,
synchronization primitives) could be left in a dangerously broken state
because of such forks, if multithreaded programs.
I would *never* have even tried that,
Why not? Actually there are some real-world use-cases.
For example, in 2005 I developed a multi-threaded report generator
application which supports different engines. Some of these engines
allow to generate a report by submitting a http request, while others
require you to start an executable.
The application worked fine, but the multi-threading/forking problem
led to some very-hard-to-detect bugs.
See http://mail.python.org/pipermail/python-dev/2007-June/073745.html
or just search the web for "unwanted handle inheritance".
Even seemingly fool-proof things like using the logging module (when
internally deleting old log files) in the parent process would cause
problems.
I remember Martin von Löwis mentioned that the special case of file
handles on Windows should be solved by the new I/O implementation of
Python 3, but I didn't check this (that particular application is still
working with Python 2.4 (I developed a workaround that required
replacing all calls to open and patching socket.py or whatever).
I would *never* have even tried that, but it doesn't surprise me that it
works basically as expected. I found this as a quick intro:
http://unix.derkeiler.com/Newsgroups/comp.unix.programmer/2003-09/0672.html
... and another interesting link that also describes exec() usage in this
context.
http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them
Stefan
That is indeed very interesting.
Unfortunately Python has to support older Linux kernels as well.
Henning
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
