[Python-Dev] functional continuations
i'm working on some minimalistic asynchronous framework for python, somewhat like twisted or stackless, but for different purposes. i came to the conclusion i want to be able to "freeze" functions, and resume them later, when some condition is matched. the idea i came up with is, using exceptions for functional continuations: after all, the exception's traceback holds the entire context... so with some help from black magic (i.e., ctypes), i can resurrect dead stackframes. i will have a reactor in the background which manages the execution flow, and blocking operations (I/O, etc.) will be wrapped with a thin wrapper, such as class socket2(socket): def recv(self, count = None): raise WaitFor(self, "r") return socket.recv(self, count) now, when read()ing from file2 objects, first an exception will propagate up to the reactor, which will catch it, create a continuation object (which holds the traceback) and register it with the desired event. then, when the event is triggered, the reactor will resurrect the continuation object to the point where it stopped (f_lasti + 2), and execution would continue normally from there. the idea itself is similar to generator objects, and requires some messing-around with f_back and f_lasti which is quite ugly. on the other hand, the beauty of it is, it doesn't require any special design patterns (unlike twisted) or replacing the interpreter (unlike stackless), so it can be transparently used with any third-party libs. all it takes is wrapping I/O objects to "raise WaitFor(...)" prior to doing blocking I/O. moreover, unlike generators, i gain the ability to "propagate" up to the reactor (like normal exceptions), so the code in between the I/O and the reactor needn't be aware of anything (only avoiding unconstrained try-except blocks). i wanted to get some feedback on the issue (i tried c.l.p, but they didn't understand me well enough): * has it been tried before? * do you suppose it will work? are there any drawbacks i didn't anticipate? * is it useful enough to be added to the core interpreter (like generators)? of course if it is added, it could be implemented with a dedicated mechanism, rather than abusing exceptions for that. -tomer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] functional continuations
At 01:04 AM 12/15/2007 -0800, tomer filiba wrote: >* do you suppose it will work? are there any drawbacks i didn't >anticipate? Yes. :) Specifically, think about what happens when a C function is in the call stack, e.g.: def f1(): return map(f2, [1,2,3]) def f2(ob): raise WaitFor(something) return ob print f1() If I understand you correctly, when this program is run, it will print "1", rather than "[1, 2, 3]", because there is no way for you to keep track of the internal state of the 'map()' call. And this isn't just a problem for map() -- even something as simple as a property access passes through C code whose state can get lost. I don't think this approach is practical; you'd be better off using a co-routine stack and trampoline, since the nature of generators naturally forces all the C code out of the picture. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] functional continuations
yeah, i did think native functions wouldn't fit well with it, but then
again, i don't plan to have any c-side functions invoking python
callbacks. i can live with that. but, alas, ceval::EvalFrameEx will
clear the execution stack upon returning [1], so this couldn't work
anyway [2].
[1]
while (STACK_LEVEL() > b->b_level) {
v = POP();
Py_XDECREF(v);
}
[2] hrrrmpff
-tomer
On Dec 15, 2007 6:57 PM, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> At 01:04 AM 12/15/2007 -0800, tomer filiba wrote:
> >* do you suppose it will work? are there any drawbacks i didn't
> >anticipate?
>
> Yes. :)
>
> Specifically, think about what happens when a C function is in the
> call stack, e.g.:
>
> def f1():
> return map(f2, [1,2,3])
>
> def f2(ob):
> raise WaitFor(something)
> return ob
>
> print f1()
>
> If I understand you correctly, when this program is run, it will
> print "1", rather than "[1, 2, 3]", because there is no way for you
> to keep track of the internal state of the 'map()' call.
>
> And this isn't just a problem for map() -- even something as simple
> as a property access passes through C code whose state can get lost.
>
> I don't think this approach is practical; you'd be better off using a
> co-routine stack and trampoline, since the nature of generators
> naturally forces all the C code out of the picture.
>
>
--
An NCO and a Gentleman
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] functional continuations
On Sat, Dec 15, 2007, tomer filiba wrote: > > i wanted to get some feedback on the issue (i tried c.l.p, but they > didn't understand me well enough): python-ideas is the best place for topics like this. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Typing is cheap. Thinking is expensive." --Roy Smith ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] functional continuations
tomer filiba wrote: > the idea i came up with is, using exceptions for functional > continuations: after all, the exception's traceback holds the entire > context... The problem with this is that, if the call chain has passed through a C-implemented function at some point, the traceback *doesn't* contain the entire context -- some of it is in the C stack frames that got unwound during the propagation of the exception. So you may be able to make this work where all the code involved is pure Python, but it can't work in general, unless you redesign the whole interpreter the way the original Stackless did. Having said that, it might still be a useful thing to have, as the pure-Python case is probably fairly common. But I'd want to know what the failure mode is like when the pure-Python case doesn't hold. Do you get a clear indication of what is wrong, or does it misbehave in some obscure way? A test case for this would be to call map() with a function that tries to suspend itself using this mechanism. What happens when you try to resume it? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] functional continuations
By the way, I wouldn't call this a "continuation", as that word implies a bit more (reusability). It's more like a coroutine or lightweight thread. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] need Windows compiles for SSL 1.13
The latest version of the PyPI SSL module is 1.13, and it seems pretty stable. I'd appreciate it if one of you who've compiled it in the past would do so again, and send me Windows binary dists to post to the PyPI site. http://pypi.python.org/pypi/ssl/ Thanks! Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
