Francesc Alted, 08.03.2011 20:16:
A Tuesday 08 March 2011 18:50:15 Stefan Behnel escrigué:
mark florisson, 08.03.2011 18:00:
What I meant was that the
wrapper returned by the decorator would have to call the closure
for every iteration, which introduces function call overhead.
[...]
I guess we just have to establish what we want to do: do we
want to support code with Python objects (and exceptions etc), or
just C code written in Cython?
I like the approach that Sturla mentioned: using closures to
implement worker threads. I think that's very pythonic. You could do
something like this, for example:
def worker():
for item in queue:
with nogil:
do_stuff(item)
queue.extend(work_items)
start_threads(worker, count)
Note that the queue is only needed to tell the thread what to work
on. A lot of things can be shared over the closure. So the queue may
not even be required in many cases.
I like this approach too. I suppose that you will need to annotate the
items so that they are not Python objects, no? Something like:
def worker():
cdef int item # tell that item is not a Python object!
for item in queue:
with nogil:
do_stuff(item)
queue.extend(work_items)
start_threads(worker, count)
You *can* use Python references inside of nogil blocks, even if there's a
lot of stuff that you can't do with them, such as looking up Python
attributes, calling them, or even assigning their reference to another
Python variable. But you can access cdef attributes on them, for example,
and some simple conversions to C types may also work (AFAIR). But you're
generally right that passing it into a function (as in the example above)
would likely not work. In any case, Cython will tell you if you try to do
anything that's not allowed.
Stefan
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
