Re: [Python-Dev] async/await in Python; v2

[Rajiv Kumar]

I'd like to suggest another way around some of the issues here, with
apologies if this has already been discussed sometime in the past.

>From the viewpoint of a Python programmer, there are two distinct reasons
for wanting to suspend execution in a block of code:

1. To yield a value from an iterator, as Python generators do today.

2. To cede control to the event loop while waiting for an asynchronous task
to make progress in a coroutine.

As of today both of these reasons to suspend are supported by the same
underlying mechanism, i.e. a "yield" at the end of the chain of "yield
from"s. PEPs 492 and 3152 introduce "await" and "cocall", but at the bottom
of it all there's effectively still a yield as I understand it.

I think that the fact that these two concepts use the same mechanism is
what leads to the issues with coroutine-generators that Greg and Yury have
raised.

With that in mind, would it be possible to introduce a second form of
suspension to Python to specifically handle the case of ceding to the event
loop? I don't know what the implementation complexity of this would be, or
if it's even feasible. But roughly speaking, the syntax for this could use
"await", and code would look just like it does in the PEP. The semantics of
"await " would be analogous to "yield from " today, with the
difference that the Task would go up the chain of "await"s to the outermost
caller, which would typically be asyncio, with some modifications from its
form today. Progress would be made via __anext__ instead of __next__.

Again, this might be impossible to do, but the mental model for the Python
programmer becomes cleaner, I think. Most of the issues around combining
generators and coroutines would go away - you could freely use "await"
inside a generator since it cedes control to the event loop, not the caller
of the generator. All of the "async def"/"await" examples in PEP 492 would
work as is. It might also make it easier in the future to add support for
async calls insider __getattr__ etc.

Thanks for reading!
Rajiv
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492: async/await in Python; version 5

[Rajiv Kumar]

I wrote a little example[1] that has a bare-bones implementation of Go
style channels via a custom event loop. I used it to translate the prime
sieve example from Go[2] almost directly to Python. The code uses "message
= await channel.receive()" to mimic Go's "message <- channel". Instead of
using "go func()" to fire off a goroutine, I add the PEP492 coroutine to my
simple event loop.

It's not an efficient implementation - really just a proof of concept that
you can use async/await in your own code without any reference to asyncio.
I ended up writing it as I was thinking about how PEP 342 style coroutines
might look like in an async/await world.

In the course of writing this, I did find that it would be useful to have
the PEP document how event loops should advance the coroutines (via
.send(None) for example). It would also be helpful to have the semantics of
how await interacts with different kinds of awaitables documented. I had to
play with Yury's implementation to see what it does if the __await__ just
returns iter([1,2,3]) for example.

- Rajiv

[1] https://gist.github.com/vrajivk/c505310fb79d412afcd5#file-sieve-py
 https://gist.github.com/vrajivk/c505310fb79d412afcd5#file-channel-py

[2] https://golang.org/doc/play/sieve.go


On Tue, May 5, 2015 at 2:54 PM, Paul Moore  wrote:

> On 5 May 2015 at 22:38, Yury Selivanov  wrote:
> > n 2015-05-05 5:01 PM, Paul Moore wrote:
> >>
> >> On 5 May 2015 at 21:00, Yury Selivanov  wrote:
> >>>
> >>> On 2015-05-05 3:40 PM, Jim J. Jewett wrote:
> 
>  On Tue May 5 18:29:44 CEST 2015, Yury Selivanov posted an updated
>  PEP492.
> 
>  Where are the following over-simplifications wrong?
> 
>  (1)  The PEP is intended for use (almost exclusively) with
>  asychronous IO and a scheduler such as the asynchio event loop.
> >>>
> >>> Yes. You can also use it for UI loops.  Basically, anything
> >>> that can call your code asynchronously.
> >>
> >> Given that the stdlib doesn't provide an example of such a UI loop,
> >> what would a 3rd party module need to implement to provide such a
> >> thing? Can any of the non-IO related parts of asyncio be reused for
> >> the purpose, or must the 3rd party module implement everything from
> >> scratch?
> >
> > The idea is that you integrate processing of UI events to
> > your event loop of choice.  For instance, Twisted has
> > integration for QT and other libraries [1].  This way you
> > can easily combine async network (or OS) calls with your
> > UI logic to avoid "callback hell".
>
> We seem to be talking at cross purposes. You say the PEP is *not*
> exclusively intended for use with asyncio. You mention UI loops, but
> when asked how to implement such a loop, you say that I integrate UI
> events into my event loop of choice. But what options do I have for
> "my event loop of choice"? Please provide a concrete example that
> isn't asyncio. Can I use PEP 492 with Twisted (I doubt it, as Twisted
> doesn't use yield from, which is Python 3.x only)? I contend that
> there *is* no concrete example that currently exists, so I'm asking
> what I'd need to do to write one. You pointed at qamash, but that
> seems to be subclassing asyncio, so isn't "something that isn't
> asyncio".
>
> Note that I don't have a problem with there being no existing
> implementation other than asyncio. I'd just like it if we could be
> clear over exactly what we mean when we say "the PEP is not tied to
> asyncio". It feels like the truth currently is "you can write your own
> async framework that uses the new features introduced by the PEP". I
> fully expect that *if* there's a need for async frameworks that aren't
> fundamentally IO multiplexors, then it'll get easier to write them
> over time (the main problem right now is a lack of good tutorial
> examples of how to do so). But at the moment, asyncio seems to be the
> only game in town (and I can imagine that it'll always be the main IO
> multiplexor, unless existing frameworks like Twisted choose to compete
> rather than integrate).
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rajiv.kumar%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 525

[Rajiv Kumar]

Hi Yury,

I was playing with your implementation to gain a better understanding of
the operation of asend() and friends. Since I was explicitly trying to
"manually" advance the generators, I wasn't using asyncio or other event
loop. This meant that the first thing I ran into with my toy code was the
RuntimeError ("cannot iterate async generator without finalizer set").

As you have argued elsewhere, in practice the finalizer is likely to be set
by the event loop. Since the authors of event loops are likely to know that
they should set the finalizer, would it perhaps be acceptable to merely
issue a warning instead of an error if the finalizer is not set? That way
there isn't an extra hoop to jump through for simple examples.

In my case, I just called
sys.set_asyncgen_finalizer(lambda g: 1)
to get around the error and continue playing :) (I realize that's a bad
thing to do but it didn't matter for the toy cases)

- Rajiv
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com