[Python-Dev] Replacing threads with swapcontext()

2010-03-31 Thread Alexander Sabourenkov

Hello.

I have went and replaced Python threads with explicitly-scheduled 
coroutines using swapcontext() family of functions (basically out of 
frustration with the greenlet).


I have two questions.

1. Is it as dangerous to switch C stacks and not switch PyThreadState as 
I suspect? In particular, how tightly is PyThreadState::frame coupled to 
current C stack?


2. How do you all look at merging my work as another threading-related 
configure switch? It's only another thread_something.h file and a small 
patch to threadmodule.c.



Project is at http://code.google.com/p/coev/

--

./lxnt

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Replacing threads with swapcontext()

2010-04-01 Thread Alexander Sabourenkov

Nick Coghlan wrote:

Off the top of my head, recursion limit checks will definitely go
haywire, and I would expect threading.local() and exception handling to
explode at some point (although the latter depends on what you mean by
"switching C stacks" - if you really are just fiddling the stack pointer
without affecting the thread local heap storage mechanisms, the latter
two should be fine. Still doesn't sound like a good idea though).


In my implementation, each context looks like a thread to Python, with 
its own PyThreadState, thread-local storage and locks are taken care of,

so I'm reasonably sure there's nothing scary lurking unseen.

I'm trying to confirm my understanding as to how unreliable the greenlet 
is. The greenlet module basically just switches C stacks and does not 
touch anything else. There evidently are problems with this when there's 
an exception raised, as it's stored in PyThreadState::curexc_*/exc_*.


PyThreadState's profiling/tracing -related members, the tick_counter, 
and per-thread state dict don't appear as a problem at the first glance.


recursion_depth will be messed up unpredictably, as you kindly pointed out.

This leaves PyThreadState::frame a PyFrameObject to be examined.

Do they form a python-side stack of a thread in question?

--

./lxnt




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com