Calvin Spealman wrote:
Has anyone made the argument for keeping the GIL to discourage
threading? I'm only throwing this out there and I'm sure we'd want to
improve things no matter what, but I would like to voice the concern
anyway. We all know there are people who think threading is the answer
to all things, and who don't understand the difficulties and problems
with threading. Often, when trying to persuade them of alternative
solutions, one selling point is "your python threads can't take
advantage of multiple cores, anyway, because of the GIL."
There are many concurrency problems for which multiprocessing is *not* a
good answer and for which careful use of threads is a relatively
straightforward answer. For people with those sorts of problems CPython
(and therefore the perception is Python) does not offer a usable
concurrency story unless you are prepared to also write in C.
Unfortunately many in the Python community seem pathologically opposed
to even good use of threading and refuse to believe that such situations
even exist. In the meantime, people with these problems (or who believe
they are likely to have such problems) simply turn to other languages.
All the best,
Michael Foord
I worry, if it is ever actually removed, that I won't be able to
convince these people that you really don't need to spawn twenty
threads to write an IRC bot.
On Fri, Oct 31, 2008 at 2:24 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
Neil Schemenauer wrote:
Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote:
For a student project in a course on virtual machines, we are
evaluating the possibility to experiment with removing the GIL
from CPython
Hi,
It's great to hear of this kind of project. I think what you want
to do is difficult but possible. The major compilcation would be
that extension modules would have to re-written since they all
assume a reference counting GC. A foreign function interface like
CMU Lisp's "alien" or GHC's FFI is not necessarily any worse but it
does place different demands on extension module authors.
Michael Foord's comment about the way Ironclad does it suggests a
possible interim step that would allow existing extensions to be used at
the cost of some additional overhead: use free threading in the core,
but have an "extension module lock" that cuts things back to a single
thread whenever non-core code is invoked.
It wouldn't be as fast as a completely free-threaded setup (since the
EML would need to operate as a variant of a 'read/write' lock, with the
core code mostly leaving it in the multiple-threads-allowed 'read' mode,
promoting it to single-thread-only 'write' mode when potentially
dispatching to non-core code, and dropping back to the 'read' mode when
the dispatch landed back in core code again).
Over time, extension modules which were updated to use their own
internal locks could also drop back to free-threading mode. In essence,
the GIL would still exist, but instead of being a simple mutex the way
it is now, it would have an intermediate stage which allowed multiple
threads to continue running. Once most extension modules had been
converted, the function to promote the EML from multi-threaded mode to
single-thread only mode could be removed.
A mechanism like that would avoid one of the more subtle problems in
removing the GIL: currently, CPython-specific Python code can assume
that invocation of a C function from Python code is an atomic operation,
since the body of the function executes as a single bytecode (we're
talking about the case here where the C function is known not to release
the GIL).
I've written code which relied on the GIL in exactly that way, and I'm
sure I'm far from the only Python programmer to do so. If the GIL was
removed without adding an extension module lock, such applications would
require additional locking either in the C extension module or in the
Python code.
A possible starting point for such a project: move the locking
mechanisms that are implemented in Python in the threading module to
C-based implementations in _thread so that a richer low overhead locking
arrangement is available to extensions modules (and the rest of the core).
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---------------------------------------------------------------
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
--
http://www.ironpythoninaction.com/
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
