Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 9:50 PM, Guido van Rossum wrote: > On Mon, Aug 28, 2017 at 6:07 PM, Nathaniel Smith wrote: >> >> The important difference between generators/coroutines and normal >> function calls is that with normal function calls, the link between >> the caller and callee is fixed for t

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Glenn Linderman
On 8/28/2017 6:50 PM, Guido van Rossum wrote: FWIW we *could* have a policy that OS threads also inherit the lookup chain from their creator, but I doubt that's going to fly with backwards compatibility. Since LC is new, how could such a policy affect backwards compatibility? The obvious answ

Re: [Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Steve Dower
On 28Aug2017 1926, Chris Angelico wrote: On Tue, Aug 29, 2017 at 12:23 PM, Steve Dower wrote: Check your line lengths, I think they may be too long? (Or maybe my mail client is set too short?) Yeah, not sure what's happened here. Are PEPs supposed to be 80? Or 72? According to the emacs st

Re: [Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Steve Dower
On 28Aug2017 1834, Gregory P. Smith wrote: My gut feeling says that there are N interpreters available on just about every bloated system image out there. Multiple pythons are often among them, other we do not control will also continue to exist. I expect a small initial payload can be created th

Re: [Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Chris Angelico
On Tue, Aug 29, 2017 at 12:23 PM, Steve Dower wrote: >> Check your line lengths, I think they may be too long? (Or maybe my mail >> client is set too short?) > > > Yeah, not sure what's happened here. Are PEPs supposed to be 80? Or 72? According to the emacs stanza at the end, 70. I don't know of

Re: [Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Steve Dower
On 28Aug2017 1815, Steven D'Aprano wrote: Very nicely written. A few comments below. On Mon, Aug 28, 2017 at 04:55:19PM -0700, Steve Dower wrote: [...] This PEP describes additions to the Python API and specific behaviors for the CPython implementation that make actions taken by the Python run

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Guido van Rossum
On Mon, Aug 28, 2017 at 6:07 PM, Nathaniel Smith wrote: > The important difference between generators/coroutines and normal > function calls is that with normal function calls, the link between > the caller and callee is fixed for the entire lifetime of the inner > frame, so there's no way for th

Re: [Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Gregory P. Smith
My gut feeling says that there are N interpreters available on just about every bloated system image out there. Multiple pythons are often among them, other we do not control will also continue to exist. I expect a small initial payload can be created that when executed will binary patch the interp

Re: [Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Steven D'Aprano
Very nicely written. A few comments below. On Mon, Aug 28, 2017 at 04:55:19PM -0700, Steve Dower wrote: [...] > This PEP describes additions to the Python API and specific behaviors > for the > CPython implementation that make actions taken by the Python runtime > visible to > security and audi

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Nathaniel Smith
On Mon, Aug 28, 2017 at 3:14 PM, Eric Snow wrote: > On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith wrote: >> You might be interested in these notes I wrote to motivate why we need >> a chain of namespaces, and why simple "async task locals" aren't >> sufficient: >> >> https://github.com/njs

[Python-Dev] PEP 551: Security transparency in the Python runtime

2017-08-28 Thread Steve Dower
Hi python-dev, Those of you who were at the PyCon US language summit this year (or who saw the coverage at https://lwn.net/Articles/723823/) may recall that I talked briefly about the ways Python is used by attackers to gain and/or retain access to systems on local networks. I present here P

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 6:56 PM, Greg Ewing wrote: > Yury Selivanov wrote: >> >> I saying that the following should not work: >> >> def nested_gen(): >> set_some_context() >> yield >> >> def gen(): >># some_context is not set >>yield from nested_gen() >>

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Greg Ewing
Yury Selivanov wrote: I saying that the following should not work: def nested_gen(): set_some_context() yield def gen(): # some_context is not set yield from nested_gen() # use some_context ??? And I'm saying it *should* work, otherwise it breaks o

Re: [Python-Dev] [bpo-30421]: Pull request review

2017-08-28 Thread Terry Reedy
On 8/28/2017 4:43 PM, Mark Lawrence via Python-Dev wrote: The bulk of the work on argparse in recent years has been done by paul.j3.  I have no idea whether or not he is classed as a core developer. 'He' is a CLA-signed contributor, but not a committer, with no GitHub name registered with bpo

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 6:19 PM, Eric Snow wrote: > On Sat, Aug 26, 2017 at 10:31 AM, Yury Selivanov > wrote: >> On Sat, Aug 26, 2017 at 9:33 AM, Sven R. Kunze wrote: >> [..] >>> Why not the same interface as thread-local storage? This has been the >>> question which bothered me from the beginni

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 6:22 PM, Greg Ewing wrote: [..] >> But almost nobody converts the code by simply slapping async/await on >> top of it > > > Maybe not, but it will also affect refactoring of code > that is *already* using async/await, e.g. taking > > >async def foobar(): > # set d

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Greg Ewing
Yury Selivanov wrote: On Mon, Aug 28, 2017 at 1:33 PM, Stefan Krah wrote: [..] * Context managers like decimal contexts, numpy.errstate, and warnings.catch_warnings. The decimal context works like this: 1) There is a default context template (user settable). 2) Whenever the first operati

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Eric Snow
On Sat, Aug 26, 2017 at 10:31 AM, Yury Selivanov wrote: > On Sat, Aug 26, 2017 at 9:33 AM, Sven R. Kunze wrote: > [..] >> Why not the same interface as thread-local storage? This has been the >> question which bothered me from the beginning of PEP550. I don't understand >> what inventing a new wa

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Eric Snow
On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith wrote: > You might be interested in these notes I wrote to motivate why we need > a chain of namespaces, and why simple "async task locals" aren't > sufficient: > > https://github.com/njsmith/pep-550-notes/blob/master/dynamic-scope.ipynb Thanks

[Python-Dev] PEP 550 v4: coroutine policy

2017-08-28 Thread Yury Selivanov
Long story short, I think we need to rollback our last decision to prohibit context propagation up the call stack in coroutines. In PEP 550 v3 and earlier, the following snippet would work just fine: var = new_context_var() async def bar(): var.set(42) async def foo(): aw

Re: [Python-Dev] [bpo-30421]: Pull request review

2017-08-28 Thread Mark Lawrence via Python-Dev
On 28/08/2017 17:46, Terry Reedy wrote: On 8/28/2017 3:42 AM, Robert Schindler wrote: Hello, In May, I submitted a pull request that extends the functionality of argparse.ArgumentParser. The argparse maintainer, bethard (Peter Bethard), was not added to the nosy list.  And he does not seem t

Re: [Python-Dev] Pep 550 and None/masking

2017-08-28 Thread Guido van Rossum
On Mon, Aug 28, 2017 at 9:26 AM, Barry Warsaw wrote: > On Aug 28, 2017, at 11:50, Yury Selivanov wrote: > > > For checking if a context variable has a value in the topmost LC, we > > can add two new keyword arguments to the "ContextVar.lookup()" method: > > > > ContextVar.lookup(*, default=Non

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 1:33 PM, Stefan Krah wrote: [..] >> * Context managers like decimal contexts, numpy.errstate, and >> warnings.catch_warnings. > > The decimal context works like this: > > 1) There is a default context template (user settable). > > 2) Whenever the first operation *in a n

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Stefan Krah
On Mon, Aug 28, 2017 at 12:12:00PM -0400, Yury Selivanov wrote: > On Mon, Aug 28, 2017 at 11:52 AM, Stefan Krah wrote: > [..] > > But the state "leaks in" as per your previous example: > > > > async def bar(): > > # use decimal with context=ctx > > > > async def foo(): > >

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 12:43 PM, Ethan Furman wrote: > On 08/28/2017 09:12 AM, Yury Selivanov wrote: > >> If we forget about dynamic scoping (I don't know why it's being brought up >> all the >> time, TBH; nobody uses it, almost no language implements it) > > > Probably because it's not lexical s

Re: [Python-Dev] [bpo-30421]: Pull request review

2017-08-28 Thread Terry Reedy
On 8/28/2017 3:42 AM, Robert Schindler wrote: Hello, In May, I submitted a pull request that extends the functionality of argparse.ArgumentParser. The argparse maintainer, bethard (Peter Bethard), was not added to the nosy list. And he does not seem to have been active lately -- his bpo pro

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Ethan Furman
On 08/28/2017 09:12 AM, Yury Selivanov wrote: If we forget about dynamic scoping (I don't know why it's being brought up all the time, TBH; nobody uses it, almost no language implements it) Probably because it's not lexical scoping, and possibly because it's possible for a function to be runn

Re: [Python-Dev] Pep 550 and None/masking

2017-08-28 Thread Barry Warsaw
On Aug 28, 2017, at 11:50, Yury Selivanov wrote: > For checking if a context variable has a value in the topmost LC, we > can add two new keyword arguments to the "ContextVar.lookup()" method: > > ContextVar.lookup(*, default=None, topmost=False) > > If `topmost` is set to `True`, `lookup` wi

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 11:53 AM, Ivan Levkivskyi wrote: > A question appeared here about a simple mental model for PEP 550. > It looks much clearer now, than in the first version, but I still would like > to clarify: can one say that PEP 550 just provides more fine-grained version > of threading.

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 11:52 AM, Stefan Krah wrote: [..] > But the state "leaks in" as per your previous example: > > async def bar(): > # use decimal with context=ctx > > async def foo(): > decimal.setcontext(ctx) > await bar() > > > IMHO it shouldn't with corou

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Ivan Levkivskyi
A question appeared here about a simple mental model for PEP 550. It looks much clearer now, than in the first version, but I still would like to clarify: can one say that PEP 550 just provides more fine-grained version of threading.local(), that works not only per thread, but even per coroutine wi

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Stefan Krah
On Mon, Aug 28, 2017 at 11:23:12AM -0400, Yury Selivanov wrote: > On Mon, Aug 28, 2017 at 7:19 AM, Stefan Krah wrote: > > Okay, so if I understand this correctly we actually will not have dynamic > > scoping for regular functions: bar() has returned, so the new context > > would not be found on t

Re: [Python-Dev] Pep 550 and None/masking

2017-08-28 Thread Yury Selivanov
On Sun, Aug 27, 2017 at 4:01 PM, Nathaniel Smith wrote: > I believe that the current status is: > > - assigning None isn't treated specially – it does mask any underlying > values (which I think is what we want) Correct. > > - there is currently no way to "unmask" > > - but it's generally agreed

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 11:26 AM, Ethan Furman wrote: > On 08/28/2017 04:19 AM, Stefan Krah wrote: > >> What about this? >> >> async def bar(): >> setcontext(Context(prec=1)) >> for i in range(10): >> await asyncio.sleep(1) >> yield i >> >> async def foo(): >> asyn

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Ethan Furman
On 08/28/2017 04:19 AM, Stefan Krah wrote: What about this? async def bar(): setcontext(Context(prec=1)) for i in range(10): await asyncio.sleep(1) yield i async def foo(): async for i in bar(): # ctx.prec=1? print(Decimal(100) / 3) If I und

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Sat, Aug 26, 2017 at 4:45 PM, francismb wrote: [..] > it's by design that the execution context for new threads to be empty or > should it be possible to set it to some initial value? Like e.g: > > var = new_context_var('init') > > def sub(): > assert var.lookup() == 'init'

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Yury Selivanov
On Mon, Aug 28, 2017 at 7:19 AM, Stefan Krah wrote: > On Sun, Aug 27, 2017 at 11:19:20AM -0400, Yury Selivanov wrote: >> On Sun, Aug 27, 2017 at 6:08 AM, Stefan Krah wrote: >> > On Sat, Aug 26, 2017 at 04:13:24PM -0700, Nathaniel Smith wrote: >> >> It's perfectly reasonable to have a script where

[Python-Dev] [bpo-30421]: Pull request review

2017-08-28 Thread Robert Schindler
Hello, In May, I submitted a pull request that extends the functionality of argparse.ArgumentParser. To do so, I followed the steps described in the developers guide. According to [1], I already pinged at GitHub but got no response. The next step seems to be writing to this list. I know that no

Re: [Python-Dev] Pep 550 module

2017-08-28 Thread Joao S. O. Bueno
Well, this talk may be a bit of bike-shedding, but +1 for a separate module/sub module And full -1 for something named "dynscopevars" That word is unreadable, barely mnemonic, but plain "ugly" - (I know that this is subjective, but it is just that :-) ) Why not just "execution_context" or "s

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Stefan Krah
On Sun, Aug 27, 2017 at 11:19:20AM -0400, Yury Selivanov wrote: > On Sun, Aug 27, 2017 at 6:08 AM, Stefan Krah wrote: > > On Sat, Aug 26, 2017 at 04:13:24PM -0700, Nathaniel Smith wrote: > >> It's perfectly reasonable to have a script where you call > >> decimal.setcontext or np.seterr somewhere a