On Thu, Feb 15, 2007 at 04:51:30PM +0100, Joachim K?nig-Baltes wrote:
> The style used in asyncore, inheriting from a class and calling return
> in a method
> and being called later at a different location (different method) just
> interrupts the
> sequential flow of operations and makes it harder to understand. The same is
> true for all other strategies using callbacks or similar mechanisms.
>
> All this can be achieved with a multilevel yield() that is hidden in a
> function call.
> So the task does a small step down (wait) in order to jump up (yield) to
> the scheduler
> without disturbing the eye of the beholder.
I agree -- I find that writing continuations or using asyncore's
structure makes spaghetti out of functionality that requires multiple
blocking operations inside looping or conditional statements. The best
example, for me, was writing a complex site-specific web spider that had
to fetch 5-10 pages in a certain sequence, where each step in that
sequence depended on the results of the previous fetches. I wrote it in
Twisted, but the proliferation of nested callback functions and chained
deferreds made my head explode while trying to debug it. With a decent
microthreading library, that could look like:
def fetchSequence(...):
fetcher = Fetcher()
yield fetcher.fetchHomepage()
firstData = yield fetcher.fetchPage('http://...')
if someCondition(firstData):
while True:
secondData = yield fetcher.fetchPage('http://...')
# ...
if someOtherCondition(secondData): break
else:
# ...
which is *much* easier to read and debug. (FWIW, after I put my head
back together, I rewrote the app with threads, and it now looks like the
above, without the yields. Problem is, throttlling on fetches means 99%
of my threads are blocked on sleep() at any given time, which is just
silly).
All that said, I continue to contend that the microthreading and async
IO operations are separate. The above could be implemented relatively
easily in Twisted with a variant of the microthreading module Phillip
posted earlier. It could also be implemented atop a bare-bones
microthreading module with Fetcher using asyncore on the backend, or
even scheduler urllib.urlopen() calls into OS threads. Presumably, it
could run in NanoThreads and Kamaelia too, among others.
What I want is a consistent syntax for microthreaded code, so that I
could write my function once and run it in *all* of those circumstances.
Dustin
P.S. For the record -- I've written lots of other apps in Twisted with
great success; this one just wasn't a good fit.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
