Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Nick Coghlan
On 7 September 2017 at 07:06, Ethan Furman wrote: > The concern is *how* PEP 550 provides it: > > - explicitly, like threading.local(): has to be set up manually, > preferably with a context manager > > - implicitly: it just happens under certain conditions A recurring point of confusion with t

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
Elvis Pranskevichus wrote: By default, generators reference an empty LogicalContext object that is allocated once (like the None object). We can do that because LCs are immutable. Ah, I see. That wasn't clear from the implementation, where gen.__logical_context__ = contextvars.LogicalCon

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Stefan Krah
On Thu, Sep 07, 2017 at 09:41:10AM -0400, Elvis Pranskevichus wrote: > threading.local(), the isolation mechanism, is *implicit*. > decimal.localcontext() is an *explicit* resource manager that relies on > threading.local() magic. PEP 550 simply provides a threading.local() > alternative that

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Guido van Rossum
I write it in a new thread, but I also want to write it here -- I need a time out in this discussion so I can think about it more. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailma

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Elvis Pranskevichus
On Thursday, September 7, 2017 10:06:14 AM EDT Ethan Furman wrote: > I might be, and I wouldn't be surprised. :) On the other hand, one > can look at isolation as being a resource. > > threading.local(), the isolation mechanism, is *implicit*. > > I don't think so. You don't get threading.local

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Elvis Pranskevichus
On Thursday, September 7, 2017 6:37:58 AM EDT Greg Ewing wrote: > 2) You ignore it and always use a context manager, in > which case it's not strictly necessary for the implicit > context push to occur, since the relevant context managers > can take care of it. > > So there doesn't seem to be any

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Elvis Pranskevichus
On Thursday, September 7, 2017 9:05:58 AM EDT Ethan Furman wrote: > The disagreement seems to be whether a LogicalContext should be > created implicitly vs explicitly (or opt-out vs opt-in). As a user > trying to track down a decimal context change not propagating, I > would not suspect the above c

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Ethan Furman
On 09/07/2017 06:41 AM, Elvis Pranskevichus wrote: On Thursday, September 7, 2017 9:05:58 AM EDT Ethan Furman wrote: The disagreement seems to be whether a LogicalContext should be created implicitly vs explicitly (or opt-out vs opt-in). As a user trying to track down a decimal context change

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Elvis Pranskevichus
On Thursday, September 7, 2017 3:54:15 AM EDT Greg Ewing wrote: > This problem would also not arise if context vars > simply had names instead of being magic key objects: > > def foo(): >contextvars.set("mymodule.myvar", 1) > > That's another thing I think would be an improvement, > b

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Ethan Furman
On 09/06/2017 11:57 PM, Yury Selivanov wrote: On Wed, Sep 6, 2017 at 11:39 PM, Greg Ewing wrote: Here's one way that refactoring could trip you up. Start with this: async def foo(): calculate_something() #in a coroutine, so we can be lazy and not use a cm Where exactly doe

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Ethan Furman
On 09/07/2017 03:37 AM, Greg Ewing wrote: If I understand correctly, instead of using a context manager, your fractions example could be written like this: def fractions(precision, x, y): ctx = decimal.getcontext().copy() decimal.setcontext(ctx) ctx.prec = precision yield My

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Ethan Furman
On 09/07/2017 04:39 AM, Greg Ewing wrote: 1) Under "Generators" it says: once set in the generator, the context variable is guaranteed not to change between iterations; This suggests that you're not allowed to set() a given context variable more than once in a given generator, but some

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
There are a couple of things in the PEP I'm confused about: 1) Under "Generators" it says: once set in the generator, the context variable is guaranteed not to change between iterations; This suggests that you're not allowed to set() a given context variable more than once in a given gene

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
There is one thing I misunderstood. Since generators and coroutines are almost exactly the same underneath, I had thought that the automatic logical_context creation for generators was also going to apply to coroutines, but from reading the PEP again it seems that's not the case. Somehow I missed

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Koos Zevenhoven
On Thu, Sep 7, 2017 at 10:54 AM, Greg Ewing wrote: > Yury Selivanov wrote: > >> def foo(): >> var = ContextVar() >> var.set(1) >> >> for _ in range(10**6): foo() >> >> If 'var' is strongly referenced, we would have a bunch of them. >> > > Erk. This is not how I envisaged

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
Yury Selivanov wrote: I understand what Koos is talking about, but I really don't follow you. Using the "with-statements to be skipped" language is very confusing and doesn't help to understand you. If I understand correctly, instead of using a context manager, your fractions example could be

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
Yury Selivanov wrote: def foo(): var = ContextVar() var.set(1) for _ in range(10**6): foo() If 'var' is strongly referenced, we would have a bunch of them. Erk. This is not how I envisaged context vars would be used. What I thought you would do is this: my_contex

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
Yury Selivanov wrote: The PEP gives you a Task Local Storage, where Task is: 1. your single-threaded code 2. a generator 3. an async task If you correctly use context managers, PEP 550 works intuitively and similar to how one would think that threading.local() should work. My version works *m

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
Yury Selivanov wrote: 1. So essentially this means that we will have one "local context" per context manager storing one value. I can't see that being a major problem. Context vars will (I hope!) be very rare things, and needing to change a bunch of them in one function ought to be rarer still.

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 11:55 PM, Greg Ewing wrote: > Guido van Rossum wrote: >> >> Yeah, so my claim this is simply a non-problem, and you've pretty much >> just proved that by failing to come up with pointers to actual code that >> would suffer from this. Clearly you're not aware of any such code

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Greg Ewing
Yury Selivanov wrote: I still think that giving Python programmers one strong rule: "context mutation is always isolated in generators" makes it easier to reason about the EC and write maintainable code. Whereas I think it makes code *harder* to reason about, because to take advantage of it you

Re: [Python-Dev] PEP 550 v4

2017-09-07 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 11:39 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> It would be great if you or Greg could show a couple of real-world >> examples showing the "issue" (with the current PEP 550 >> APIs/semantics). > > > Here's one way that refactoring could trip you up. > Start with th

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Guido van Rossum wrote: Yeah, so my claim this is simply a non-problem, and you've pretty much just proved that by failing to come up with pointers to actual code that would suffer from this. Clearly you're not aware of any such code. In response I'd ask Yuri to come up with examples of real c

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 11:39 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> It would be great if you or Greg could show a couple of real-world >> examples showing the "issue" (with the current PEP 550 >> APIs/semantics). > > > Here's one way that refactoring could trip you up. > Start with th

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 11:26 PM, Greg Ewing wrote: > Guido van Rossum wrote: >> >> This feels like a very abstract argument. I have a feeling that context >> state propagating out of a call is used relatively rarely -- it must work >> for cases where you refactor something that changes context in

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Yury Selivanov wrote: It would be great if you or Greg could show a couple of real-world examples showing the "issue" (with the current PEP 550 APIs/semantics). Here's one way that refactoring could trip you up. Start with this: async def foo(): calculate_something() #in a corou

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Guido van Rossum wrote: This feels like a very abstract argument. I have a feeling that context state propagating out of a call is used relatively rarely -- it must work for cases where you refactor something that changes context inline into a utility function (e.g. decimal.setcontext()), but

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Elvis Pranskevichus
On Wednesday, September 6, 2017 8:06:36 PM EDT Greg Ewing wrote: > Nathaniel Smith wrote: > > The implementation strategy changed radically between v1 > > and v2 because of considerations around generator (not coroutine) > > semantics. I'm not sure what more it can do to dispel these > > feelings>

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 5:06 PM, Greg Ewing wrote: > Nathaniel Smith wrote: >> >> The implementation strategy changed radically between v1 >> and v2 because of considerations around generator (not coroutine) >> semantics. I'm not sure what more it can do to dispel these feelings >> :-). > > > I can

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 5:00 PM, Greg Ewing wrote: > Nathaniel Smith wrote: >> >> Literally the first motivating example at the beginning of the PEP >> ('def fractions ...') involves only generators, not coroutines, and >> only works correctly if generators get special handling. (In fact, I'd >> be

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 4:27 PM, Greg Ewing wrote: > Ivan Levkivskyi wrote: >> >> Normal generators fall out from this "scheme", and it looks like their >> behavior is determined by the fact that coroutines are implemented as >> generators. > > > This is what I disagree with. Generators don't imple

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Nathaniel Smith wrote: The implementation strategy changed radically between v1 and v2 because of considerations around generator (not coroutine) semantics. I'm not sure what more it can do to dispel these feelings :-). I can't say the changes have dispelled any feelings on my part. The implem

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Nathaniel Smith wrote: Literally the first motivating example at the beginning of the PEP ('def fractions ...') involves only generators, not coroutines, and only works correctly if generators get special handling. (In fact, I'd be curious to see how Greg's {push,pop}_local_storage could handle t

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 1:39 PM, Koos Zevenhoven wrote: [..] > Now this was of course a completely fictional example, and hopefully I > didn't introduce any bugs or syntax errors other than the ones I described. > I haven't seen code like this anywhere, but somehow we caught the problems > anyway.

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Ivan Levkivskyi wrote: Normal generators fall out from this "scheme", and it looks like their behavior is determined by the fact that coroutines are implemented as generators. This is what I disagree with. Generators don't implement coroutines, they implement *parts* of coroutines. We want "t

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Guido van Rossum
On Wed, Sep 6, 2017 at 1:39 PM, Koos Zevenhoven wrote: > On Wed, Sep 6, 2017 at 8:16 PM, Guido van Rossum wrote: > >> On Wed, Sep 6, 2017 at 8:07 AM, Koos Zevenhoven >> wrote: >> >>> I think yield from should have the same semantics as iterating over the >>> generator with next/send, and PEP 55

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Koos Zevenhoven
On Wed, Sep 6, 2017 at 8:22 PM, Yury Selivanov wrote: ​[...] ​ > PEP 550 treats coroutines and generators as objects that support out > of order execution. ​Out of order? More like interleaved.​ ​​ > PEP 555 still doesn't clearly explain how exactly it is different from > PEP 550. Because 55

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Koos Zevenhoven
On Wed, Sep 6, 2017 at 8:16 PM, Guido van Rossum wrote: > On Wed, Sep 6, 2017 at 8:07 AM, Koos Zevenhoven wrote: > >> I think yield from should have the same semantics as iterating over the >> generator with next/send, and PEP 555 has no issues with this. >> > > I think the onus is on you and Gr

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 8:07 AM, Koos Zevenhoven wrote: > On Wed, Sep 6, 2017 at 10:07 AM, Greg Ewing > wrote: >> >> Yury Selivanov wrote: >>> >>> Greg, have you seen this new section: >>> >>> https://www.python.org/dev/peps/pep-0550/#should-yield-from-leak-context-changes >> >> >> That section se

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Guido van Rossum
On Wed, Sep 6, 2017 at 8:07 AM, Koos Zevenhoven wrote: > I think yield from should have the same semantics as iterating over the > generator with next/send, and PEP 555 has no issues with this. > I think the onus is on you and Greg to show a realistic example that shows why this is necessary. S

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Koos Zevenhoven
On Wed, Sep 6, 2017 at 10:07 AM, Greg Ewing wrote: > Yury Selivanov wrote: > >> Greg, have you seen this new section: >> https://www.python.org/dev/peps/pep-0550/#should-yield-from- >> leak-context-changes >> > > That section seems to be addressing the idea of a generator > behaving differently d

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 12:07 AM, Greg Ewing wrote: > Yury Selivanov wrote: [..] > I don't see a lot of value in trying to automagically > isolate changes to global state *only* in generators. > > Under PEP 550, if you want to e.g. change the decimal > context temporarily in a non-generator functio

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Yury Selivanov
On Wed, Sep 6, 2017 at 5:58 AM, Ivan Levkivskyi wrote: > On 6 September 2017 at 11:13, Nathaniel Smith wrote: >> >> On Wed, Sep 6, 2017 at 1:49 AM, Ivan Levkivskyi >> wrote: >> > Normal generators fall out from this "scheme", and it looks like their >> > behavior is determined by the fact that c

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Ivan Levkivskyi
On 6 September 2017 at 11:13, Nathaniel Smith wrote: > On Wed, Sep 6, 2017 at 1:49 AM, Ivan Levkivskyi > wrote: > > Normal generators fall out from this "scheme", and it looks like their > > behavior is determined by the fact that coroutines are implemented as > > generators. What I think miht h

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Koos Zevenhoven
On Wed, Sep 6, 2017 at 12:13 PM, Nathaniel Smith wrote: > On Wed, Sep 6, 2017 at 1:49 AM, Ivan Levkivskyi > wrote: > > Another comment from bystander point of view: it looks like the > discussions > > of API design and implementation are a bit entangled here. > > This is much better in the curre

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Nathaniel Smith
On Wed, Sep 6, 2017 at 1:49 AM, Ivan Levkivskyi wrote: > Another comment from bystander point of view: it looks like the discussions > of API design and implementation are a bit entangled here. > This is much better in the current version of the PEP, but still there is a > _feelling_ that some des

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Ivan Levkivskyi
Another comment from bystander point of view: it looks like the discussions of API design and implementation are a bit entangled here. This is much better in the current version of the PEP, but still there is a _feelling_ that some design decisions are influenced by the implementation strategy. As

Re: [Python-Dev] PEP 550 v4

2017-09-06 Thread Greg Ewing
Yury Selivanov wrote: Greg, have you seen this new section: https://www.python.org/dev/peps/pep-0550/#should-yield-from-leak-context-changes That section seems to be addressing the idea of a generator behaving differently depending on whether you use yield-from on it. I never suggested that, a

Re: [Python-Dev] PEP 550 v4

2017-09-05 Thread Yury Selivanov
On Tue, Sep 5, 2017 at 4:59 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> Question: how to write a context manager with contextvar.new? >> >> var = new_context_var() >> >>class CM: >> >> def __enter__(self): >> var.new(42) >> >>with CM(): >> print(var.

Re: [Python-Dev] PEP 550 v4

2017-09-05 Thread Greg Ewing
Yury Selivanov wrote: Question: how to write a context manager with contextvar.new? var = new_context_var() class CM: def __enter__(self): var.new(42) with CM(): print(var.get() or 'None') My understanding that the above code will print "None", because "

Re: [Python-Dev] PEP 550 v4

2017-08-31 Thread Koos Zevenhoven
On Wed, Aug 30, 2017 at 5:36 PM, Yury Selivanov wrote: > On Wed, Aug 30, 2017 at 9:44 AM, Yury Selivanov > wrote: > [..] > >> FYI, I've been sketching an alternative solution that addresses these > kinds > >> of things. I've been hesitant to post about it, partly because of the > >> PEP550-based

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Kevin Conway
> Can Execution Context be implemented outside of CPython I know I'm well late to the game and a bit dense, but where in the pep is the justification for this assertion? I ask because we buy something to solve the same problem in Twisted some time ago: https://bitbucket.org/hipchat/txlocal . We we

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Yury Selivanov
On Wed, Aug 30, 2017 at 1:39 PM, Kevin Conway wrote: >> Can Execution Context be implemented outside of CPython > > I know I'm well late to the game and a bit dense, but where in the pep is > the justification for this assertion? I ask because we buy something to > solve the same problem in Twiste

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Yury Selivanov
On Wed, Aug 30, 2017 at 8:19 AM, Koos Zevenhoven wrote: > On Wed, Aug 30, 2017 at 2:36 AM, Greg Ewing > wrote: >> >> Yury Selivanov wrote: >>> >>> While we want "yield from" to have semantics close to a function call, >> >> >> That's not what I said! I said that "yield from foo()" should >> have

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Yury Selivanov
On Wed, Aug 30, 2017 at 9:44 AM, Yury Selivanov wrote: [..] >> FYI, I've been sketching an alternative solution that addresses these kinds >> of things. I've been hesitant to post about it, partly because of the >> PEP550-based workarounds that Nick, Nathaniel, Yury etc. have been >> describing, a

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Yury Selivanov
On Wed, Aug 30, 2017 at 8:55 AM, Greg Ewing wrote: > Yury Selivanov wrote: > >> BTW we already have mechanisms to always propagate context to the >> caller -- just use threading.local() or a global variable. > > > But then you don't have a way to *not* propagate the > context change when you don't

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Greg Ewing
Yury Selivanov wrote: BTW we already have mechanisms to always propagate context to the caller -- just use threading.local() or a global variable. But then you don't have a way to *not* propagate the context change when you don't want to. Here's my suggestion: Make an explicit distinction bet

Re: [Python-Dev] PEP 550 v4

2017-08-30 Thread Koos Zevenhoven
On Wed, Aug 30, 2017 at 2:36 AM, Greg Ewing wrote: > Yury Selivanov wrote: > >> While we want "yield from" to have semantics close to a function call, >> > > That's not what I said! I said that "yield from foo()" should > have semantics close to a function call. If you separate the > "yield from"

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Nick Coghlan
On 30 August 2017 at 16:40, Nick Coghlan wrote: > Writing an "update_parent_context" decorator is also trivial (and will > work for both sync and async generators): > > def update_parent_context(gf): > @functools.wraps(gf): > def wrapper(*args, **kwds): > gen = gf(*

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Nick Coghlan
On 30 August 2017 at 10:18, Yury Selivanov wrote: > On Tue, Aug 29, 2017 at 7:36 PM, Greg Ewing > wrote: > [..] >>> For (1) we want the context change to be isolated. For (2) you say >>> that the context change should propagate to the caller. >> >> >> No, I'm saying that the context change shou

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 7:36 PM, Greg Ewing wrote: [..] >> For (1) we want the context change to be isolated. For (2) you say >> that the context change should propagate to the caller. > > > No, I'm saying that the context change should *always* > propagate to the caller, unless you do something

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 7:36 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> While we want "yield from" to have semantics close to a function call, > > > That's not what I said! I said that "yield from foo()" should > have semantics close to a function call. If you separate the > "yield from"

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Greg Ewing
Yury Selivanov wrote: While we want "yield from" to have semantics close to a function call, That's not what I said! I said that "yield from foo()" should have semantics close to a function call. If you separate the "yield from" from the "foo()", then of course you can get different behaviours.

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 7:06 PM, Stefan Krah wrote: > On Tue, Aug 29, 2017 at 06:01:40PM -0400, Yury Selivanov wrote: >> PEP 550 positions itself as a replacement for TLS, and clearly defines >> its semantics for regular functions in a single thread, regular >> functions in multithreaded code, gen

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Stefan Krah
On Tue, Aug 29, 2017 at 06:01:40PM -0400, Yury Selivanov wrote: > PEP 550 positions itself as a replacement for TLS, and clearly defines > its semantics for regular functions in a single thread, regular > functions in multithreaded code, generators, and asynchronous code > (async/await). Everythin

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 5:45 PM, Greg Ewing wrote: [..] > What you seem to be suggesting is that generators shouldn't > leak context changes even when you *don't* use a with-statement. Yes, generators shouldn't leak context changes regardless of what and how changes the context inside them: va

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Greg Ewing
Yury Selivanov wrote: Consider the following generator: def gen(): with decimal.context(...): yield We don't want gen's context to leak to the outer scope That's understandable, but fixing that problem shouldn't come at the expense of breaking the ability to refacto

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 4:33 PM, Antoine Pitrou wrote: > > Le 29/08/2017 à 22:20, Yury Selivanov a écrit : >> >> 2) >> >>gvar = new_context_var() >>var = new_context_var() >> >>async def bar(): >># EC = [current_thread_LC_copy, Task_foo_LC_copy, Task_wait_for_LC] > > Ah, thanks

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
Le 29/08/2017 à 22:20, Yury Selivanov a écrit : > > 2) > >gvar = new_context_var() >var = new_context_var() > >async def bar(): ># EC = [current_thread_LC_copy, Task_foo_LC_copy, Task_wait_for_LC] Ah, thanks!... That explains things, though I don't expect most users to spo

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 4:10 PM, Antoine Pitrou wrote: [..] >> "await bar()" and "await wait_for(bar())" are actually quite >> different. Let me illustrate with an example: >> >> b1 = bar() >> # bar() is not running yet >> await b1 >> >> b2 = wait_for(bar()) >> # bar() was wra

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Nathaniel Smith
On Tue, Aug 29, 2017 at 12:59 PM, Yury Selivanov wrote: > b2 = wait_for(bar()) > # bar() was wrapped into a Task and is being running right now > await b2 Ah not quite. wait_for is itself implemented as a coroutine, so it doesn't spawn off bar() into its own task until you await b

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
Le 29/08/2017 à 21:59, Yury Selivanov a écrit : > > This absolutely needs to be fixed, and the only way (that I know) it > can be fixed is to revert the "every coroutine has its own LC" > statement (going back to the semantics coroutines had in PEP 550 v2 > and v3). I completely agree with this.

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Nathaniel Smith
On Tue, Aug 29, 2017 at 12:32 PM, Antoine Pitrou wrote: > > > Le 29/08/2017 à 21:18, Yury Selivanov a écrit : >> On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou wrote: >>> On Mon, 28 Aug 2017 17:24:29 -0400 >>> Yury Selivanov wrote: Long story short, I think we need to rollback our last dec

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 3:32 PM, Antoine Pitrou wrote: > > > Le 29/08/2017 à 21:18, Yury Selivanov a écrit : >> On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou wrote: >>> On Mon, 28 Aug 2017 17:24:29 -0400 >>> Yury Selivanov wrote: Long story short, I think we need to rollback our last deci

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
Le 29/08/2017 à 21:18, Yury Selivanov a écrit : > On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou wrote: >> On Mon, 28 Aug 2017 17:24:29 -0400 >> Yury Selivanov wrote: >>> Long story short, I think we need to rollback our last decision to >>> prohibit context propagation up the call stack in co

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou wrote: > On Mon, 28 Aug 2017 17:24:29 -0400 > Yury Selivanov wrote: >> Long story short, I think we need to rollback our last decision to >> prohibit context propagation up the call stack in coroutines. In PEP >> 550 v3 and earlier, the following s

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 7:16 PM, Yury Selivanov wrote: > On Mon, Aug 28, 2017 at 6:56 PM, Greg Ewing > wrote: >> Yury Selivanov wrote: >>> >>> I saying that the following should not work: >>> >>> def nested_gen(): >>> set_some_context() >>> yield >>> >>> def gen(): >>>

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
On Mon, 28 Aug 2017 17:24:29 -0400 Yury Selivanov wrote: > Long story short, I think we need to rollback our last decision to > prohibit context propagation up the call stack in coroutines. In PEP > 550 v3 and earlier, the following snippet would work just fine: > >var = new_context_var() >

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Nick Coghlan
On 29 August 2017 at 23:18, Yury Selivanov wrote: > On Tue, Aug 29, 2017 at 5:01 AM, Nick Coghlan wrote: > [..] >> P.S. And I say that as a reader who correctly guessed why you had >> changed the method name in the current iteration of the proposal. I'm >> sympathetic to those reasons, but I thin

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Yury Selivanov
On Tue, Aug 29, 2017 at 5:01 AM, Nick Coghlan wrote: [..] > P.S. And I say that as a reader who correctly guessed why you had > changed the method name in the current iteration of the proposal. I'm > sympathetic to those reasons, but I think sticking with the > conventional API will make this one

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Nick Coghlan
On 29 August 2017 at 07:24, Yury Selivanov wrote: > This means that PEP 550 will have a caveat for async code: don't rely > on context propagation up the call stack, unless you are writing > __aenter__ and __aexit__ that are guaranteed to be called without > being wrapped into a Task. I'm not sur

Re: [Python-Dev] PEP 550 v4

2017-08-29 Thread Nick Coghlan
On 27 August 2017 at 03:23, Yury Selivanov wrote: > On Sat, Aug 26, 2017 at 1:23 PM, Ethan Furman wrote: >> On 08/26/2017 09:25 AM, Yury Selivanov wrote: >>> ContextVar.lookup() method *traverses the stack* until it finds the LC >>> that has a value. "get()" does not reflect this subtle semantic

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 9:50 PM, Guido van Rossum wrote: > On Mon, Aug 28, 2017 at 6:07 PM, Nathaniel Smith wrote: >> >> The important difference between generators/coroutines and normal >> function calls is that with normal function calls, the link between >> the caller and callee is fixed for t

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Glenn Linderman
On 8/28/2017 6:50 PM, Guido van Rossum wrote: FWIW we *could* have a policy that OS threads also inherit the lookup chain from their creator, but I doubt that's going to fly with backwards compatibility. Since LC is new, how could such a policy affect backwards compatibility? The obvious answ

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Guido van Rossum
On Mon, Aug 28, 2017 at 6:07 PM, Nathaniel Smith wrote: > The important difference between generators/coroutines and normal > function calls is that with normal function calls, the link between > the caller and callee is fixed for the entire lifetime of the inner > frame, so there's no way for th

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Nathaniel Smith
On Mon, Aug 28, 2017 at 3:14 PM, Eric Snow wrote: > On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith wrote: >> You might be interested in these notes I wrote to motivate why we need >> a chain of namespaces, and why simple "async task locals" aren't >> sufficient: >> >> https://github.com/njs

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 6:56 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> I saying that the following should not work: >> >> def nested_gen(): >> set_some_context() >> yield >> >> def gen(): >># some_context is not set >>yield from nested_gen() >>

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Greg Ewing
Yury Selivanov wrote: I saying that the following should not work: def nested_gen(): set_some_context() yield def gen(): # some_context is not set yield from nested_gen() # use some_context ??? And I'm saying it *should* work, otherwise it breaks o

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 6:19 PM, Eric Snow wrote: > On Sat, Aug 26, 2017 at 10:31 AM, Yury Selivanov > wrote: >> On Sat, Aug 26, 2017 at 9:33 AM, Sven R. Kunze wrote: >> [..] >>> Why not the same interface as thread-local storage? This has been the >>> question which bothered me from the beginni

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 6:22 PM, Greg Ewing wrote: [..] >> But almost nobody converts the code by simply slapping async/await on >> top of it > > > Maybe not, but it will also affect refactoring of code > that is *already* using async/await, e.g. taking > > >async def foobar(): > # set d

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Greg Ewing
Yury Selivanov wrote: On Mon, Aug 28, 2017 at 1:33 PM, Stefan Krah wrote: [..] * Context managers like decimal contexts, numpy.errstate, and warnings.catch_warnings. The decimal context works like this: 1) There is a default context template (user settable). 2) Whenever the first operati

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Eric Snow
On Sat, Aug 26, 2017 at 10:31 AM, Yury Selivanov wrote: > On Sat, Aug 26, 2017 at 9:33 AM, Sven R. Kunze wrote: > [..] >> Why not the same interface as thread-local storage? This has been the >> question which bothered me from the beginning of PEP550. I don't understand >> what inventing a new wa

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Eric Snow
On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith wrote: > You might be interested in these notes I wrote to motivate why we need > a chain of namespaces, and why simple "async task locals" aren't > sufficient: > > https://github.com/njsmith/pep-550-notes/blob/master/dynamic-scope.ipynb Thanks

[Python-Dev] PEP 550 v4: coroutine policy

2017-08-28 Thread Yury Selivanov
Long story short, I think we need to rollback our last decision to prohibit context propagation up the call stack in coroutines. In PEP 550 v3 and earlier, the following snippet would work just fine: var = new_context_var() async def bar(): var.set(42) async def foo(): aw

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 1:33 PM, Stefan Krah wrote: [..] >> * Context managers like decimal contexts, numpy.errstate, and >> warnings.catch_warnings. > > The decimal context works like this: > > 1) There is a default context template (user settable). > > 2) Whenever the first operation *in a n

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Stefan Krah
On Mon, Aug 28, 2017 at 12:12:00PM -0400, Yury Selivanov wrote: > On Mon, Aug 28, 2017 at 11:52 AM, Stefan Krah wrote: > [..] > > But the state "leaks in" as per your previous example: > > > > async def bar(): > > # use decimal with context=ctx > > > > async def foo(): > >

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 12:43 PM, Ethan Furman wrote: > On 08/28/2017 09:12 AM, Yury Selivanov wrote: > >> If we forget about dynamic scoping (I don't know why it's being brought up >> all the >> time, TBH; nobody uses it, almost no language implements it) > > > Probably because it's not lexical s

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Ethan Furman
On 08/28/2017 09:12 AM, Yury Selivanov wrote: If we forget about dynamic scoping (I don't know why it's being brought up all the time, TBH; nobody uses it, almost no language implements it) Probably because it's not lexical scoping, and possibly because it's possible for a function to be runn

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 11:53 AM, Ivan Levkivskyi wrote: > A question appeared here about a simple mental model for PEP 550. > It looks much clearer now, than in the first version, but I still would like > to clarify: can one say that PEP 550 just provides more fine-grained version > of threading.

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 11:52 AM, Stefan Krah wrote: [..] > But the state "leaks in" as per your previous example: > > async def bar(): > # use decimal with context=ctx > > async def foo(): > decimal.setcontext(ctx) > await bar() > > > IMHO it shouldn't with corou

  1   2   >