[Python-Dev] Whatever happened to 'nonlocal x = y'?

[Nathaniel Smith]

PEP 3104 says:

"""
A shorthand form is also permitted, in which nonlocal is prepended to
an assignment or augmented assignment:

nonlocal x = 3

The above has exactly the same meaning as nonlocal x; x = 3. (Guido
supports a similar form of the global statement [24].)
"""

The PEP metadata says it was approved and implemented in 3.0, yet this
part never seems to have been implemented:

Python 3.7.0a3+ (heads/master:53f9135667, Dec 29 2017, 19:08:19)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
... x = 1
... def g():
... nonlocal x = 2
  File "", line 4
nonlocal x = 2
   ^
SyntaxError: invalid syntax

Was this just an oversight, or did it get rejected at some point and
no-one remembered to update that PEP?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Victor Stinner]

Currently, Context.get(var) returns None when "var in context" is false.
That's surprising and different than var.get(), especially when var has a
default value.

Code:
---
import contextvars

name = contextvars.ContextVar('name', default='victor')
context = contextvars.copy_context()
print(name in context)
print(context.get(name))
print(name.get())
---

Output:
---
False
None
victor
---

Context.get() must raise a lookup error by default if var is not in
context. It should return the default argument if it's set, it's just that
the default parameter must not have a default value (None).

I'l fine that Context.get(default=None) and var.get() behaves differently
(return None vs victor in my example) when var isn't set and var has a
default value.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Paul Moore]

On 4 January 2018 at 23:58, Guido van Rossum  wrote:
> On Thu, Jan 4, 2018 at 9:27 AM, Paul Moore  wrote:
>>
>> On 4 January 2018 at 15:56, Guido van Rossum  wrote:
>> > It was get_context() in an earlier version of PEP 567. We changed it to
>> > copy_context() believing that that would clarify that you get a clone
>> > that
>> > is unaffected by subsequent ContextVar.set() operations (which affect
>> > the
>> > *current* context rather than the copy you just got).
>>
>> Ah thanks. In which case, simply changing the emphasis to avoid the
>> implication that Context objects are immutable (while that may be true
>> in a technical/implementation sense, it's not really true in a design
>> sense if ContextVar.set modifies the value of a variable in a context)
>> is probably sufficient.
>
>
> Do you have a specific proposal for a wording change? PEP 567 describes
> Context as "a read-only mapping, implemented using an immutable dictionary."
> This sounds all right to me -- "read-only" is weaker than "immutable". Maybe
> the implementation should not be mentioned here? (The crux here is that a
> given Context acts as a variable referencing an immutable dict -- but it may
> reference different immutable dicts at different times.)

I've been struggling to think of good alternative wordings (it's a
case of "I'm not sure what you're trying to say, so I can't work out
how you should say it", unfortunately). The best I can come up with is

"""
A Context is a mapping from ContextVar objects to their values. The
Context itself exposes the Mapping interface, so cannot be modified
directly - to modify the value associated with a variable you need to
use the ContextVar.set() method.
"""

Does that explain things correctly? One thing I am sure of is that we
should remove "implemented using an immutable dictionary" - it's an
implementation detail, and adds nothing but confusion to mention it
here.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Victor Stinner]

You can only modify a context when it's the current context, so using
ContextVar.set() in Context.run().

Victor

Le 5 janv. 2018 11:42 AM, "Paul Moore"  a écrit :

On 4 January 2018 at 23:58, Guido van Rossum  wrote:
> On Thu, Jan 4, 2018 at 9:27 AM, Paul Moore  wrote:
>>
>> On 4 January 2018 at 15:56, Guido van Rossum  wrote:
>> > It was get_context() in an earlier version of PEP 567. We changed it to
>> > copy_context() believing that that would clarify that you get a clone
>> > that
>> > is unaffected by subsequent ContextVar.set() operations (which affect
>> > the
>> > *current* context rather than the copy you just got).
>>
>> Ah thanks. In which case, simply changing the emphasis to avoid the
>> implication that Context objects are immutable (while that may be true
>> in a technical/implementation sense, it's not really true in a design
>> sense if ContextVar.set modifies the value of a variable in a context)
>> is probably sufficient.
>
>
> Do you have a specific proposal for a wording change? PEP 567 describes
> Context as "a read-only mapping, implemented using an immutable
dictionary."
> This sounds all right to me -- "read-only" is weaker than "immutable".
Maybe
> the implementation should not be mentioned here? (The crux here is that a
> given Context acts as a variable referencing an immutable dict -- but it
may
> reference different immutable dicts at different times.)

I've been struggling to think of good alternative wordings (it's a
case of "I'm not sure what you're trying to say, so I can't work out
how you should say it", unfortunately). The best I can come up with is

"""
A Context is a mapping from ContextVar objects to their values. The
Context itself exposes the Mapping interface, so cannot be modified
directly - to modify the value associated with a variable you need to
use the ContextVar.set() method.
"""

Does that explain things correctly? One thing I am sure of is that we
should remove "implemented using an immutable dictionary" - it's an
implementation detail, and adds nothing but confusion to mention it
here.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/
victor.stinner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Paul Moore]

On 5 January 2018 at 11:06, Victor Stinner  wrote:
>> Le 5 janv. 2018 11:42 AM, "Paul Moore"  a écrit :
>> """
>> A Context is a mapping from ContextVar objects to their values. The
>> Context itself exposes the Mapping interface, so cannot be modified
>> directly - to modify the value associated with a variable you need to
>> use the ContextVar.set() method.
>> """
>>
>> Does that explain things correctly? One thing I am sure of is that we
>> should remove "implemented using an immutable dictionary" - it's an
>> implementation detail, and adds nothing but confusion to mention it
>> here.

> You can only modify a context when it's the current context, so using
> ContextVar.set() in Context.run().

Thanks. That's a useful qualification. But it may be too detailed for
the summary - it's certainly something that should be covered in the
specification section, though. Maybe "...  you need to use the
ContextVar.set() method from Context.run()" would be OK, although I
don't want the summary to get too long.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/2/2018 12:01 AM, Guido van Rossum wrote:
On Mon, Jan 1, 2018 at 8:50 PM, Ethan Smith > wrote:




On Mon, Jan 1, 2018 at 5:03 PM, Chris Barker mailto:chris.bar...@noaa.gov>> wrote:

On Sat, Dec 30, 2017 at 7:27 AM, Stephen J. Turnbull
mailto:turnbull.stephen...@u.tsukuba.ac.jp>> wrote:

   Just use the simple rule that a new
__repr__ is generated unless provided in the dataclass.


are we only talking about __repr__ here ???

I interpreted Guido's proposal as being about all methods -- we
_may_ want something special for __repr__, but I hope not.

[...]


I interpreted this to be for all methods as well, which makes sense.
Special casing just __repr__ doesn't make sense to me, but I will
wait for Guido to clarify.


Indeed, I just wrote __repr__ for simplicity. This should apply to all 
special methods. (Though there may be some complications for 
__eq__/__ne__ and for the ordering operators.)


On Mon, Jan 1, 2018 at 9:44 PM, Chris Barker > wrote:


On Mon, Jan 1, 2018 at 7:50 PM, Ethan Smith mailto:et...@ethanhs.me>> wrote:


Will you get the "right" __repr__ now if you derive a
dataclass from a dataclass? That would be a nice feature.


The __repr__ will be generated by the child dataclass unless the
user overrides it. So I believe this is the "right" __repr__.


what I was wondering is if the child will know about all the fields
in the parent -- so it could make a full __repr__.


Yes, there's a class variable (__dataclass_fields__) that identifies the 
parent fields. The PEP doesn't mention this or the fact that special 
methods (like __repr__ and __init__) can tell whether a base class is a 
dataclass. It probably should though. (@Eric)


I think that's covered in this section: 
https://www.python.org/dev/peps/pep-0557/#inheritance


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Guido van Rossum]

On Fri, Jan 5, 2018 at 1:57 AM, Nathaniel Smith  wrote:

> PEP 3104 says:
>
> """
> A shorthand form is also permitted, in which nonlocal is prepended to
> an assignment or augmented assignment:
>
> nonlocal x = 3
>
> The above has exactly the same meaning as nonlocal x; x = 3. (Guido
> supports a similar form of the global statement [24].)
> """
>
> The PEP metadata says it was approved and implemented in 3.0, yet this
> part never seems to have been implemented:
>
> Python 3.7.0a3+ (heads/master:53f9135667, Dec 29 2017, 19:08:19)
> [GCC 7.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> def f():
> ... x = 1
> ... def g():
> ... nonlocal x = 2
>   File "", line 4
> nonlocal x = 2
>^
> SyntaxError: invalid syntax
>
> Was this just an oversight, or did it get rejected at some point and
> no-one remembered to update that PEP?
>

I don't recall (though someone with more time might find the discussion in
the archives or on the tracker). It was never implemented and I think it
shouldn't be. So we might as well update the PEP. It wouldn't be
particularly useful, since (by definition) the function that declares the
nonlocal variable is not its owner, and hence it's unlikely to make sense
to initialize it here. The same reasoning applies to global BTW.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Guido van Rossum]

On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith  wrote:

> On 1/2/2018 12:01 AM, Guido van Rossum wrote:
>
>> Yes, there's a class variable (__dataclass_fields__) that identifies the
>> parent fields. The PEP doesn't mention this or the fact that special
>> methods (like __repr__ and __init__) can tell whether a base class is a
>> dataclass. It probably should though. (@Eric)
>>
>
> I think that's covered in this section: https://www.python.org/dev/pep
> s/pep-0557/#inheritance
>

I was specifically talking about the name and contents of __dataclass_fields__,
which are not documented by the PEP. I expect it's inevitable that people
will be looking at this (since they can see it in the source code). Or do
you recommend that people use dataclasses.fields() and catch ValueError? I
notice that _isdataclass() exists but is private and I don't recall why.
(Also now I'm curious what the "pseudo-fields" are that fields() ignores,
but that's OT.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Guido van Rossum]

On Fri, Jan 5, 2018 at 2:05 AM, Victor Stinner 
wrote:

> Currently, Context.get(var) returns None when "var in context" is false.
> That's surprising and different than var.get(), especially when var has a
> default value.
>

I don't see the problem. Context.get() is inherited from Mapping.get(); if
you want it to raise use Context.__getitem__() (i.e. ctx[var]). Lots of
classes define get() methods with various behaviors. Context.get() and
ContextVar.get() are just different -- ContextVar is not a Mapping.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Guido van Rossum]

On Fri, Jan 5, 2018 at 2:41 AM, Paul Moore  wrote:

> On 4 January 2018 at 23:58, Guido van Rossum  wrote:
> > Do you have a specific proposal for a wording change? PEP 567 describes
> > Context as "a read-only mapping, implemented using an immutable
> dictionary."
> > This sounds all right to me -- "read-only" is weaker than "immutable".
> Maybe
> > the implementation should not be mentioned here? (The crux here is that a
> > given Context acts as a variable referencing an immutable dict -- but it
> may
> > reference different immutable dicts at different times.)
>
> I've been struggling to think of good alternative wordings (it's a
> case of "I'm not sure what you're trying to say, so I can't work out
> how you should say it", unfortunately). The best I can come up with is
>
> """
> A Context is a mapping from ContextVar objects to their values. The
> Context itself exposes the Mapping interface, so cannot be modified
> directly - to modify the value associated with a variable you need to
> use the ContextVar.set() method.
> """
>

This is clear, but IMO there's one important detail missing: using
ContextVar.set() you can only modify the current Context.

This part of the PEP (in particular the definition of ContextVar.set() on
line 90) also lies about what ContextVar.set() does -- it implies that
Context is immutable. If it was, the recommended use of Context.run() in
asyncio would make no sense, since run() clearly modifies the Context
object in place. This is not an implementation detail -- it is an API
detail that affects how frameworks should use Context objects.

Re-reading, there's a lot of language in this part of the PEP that needs
updating... :-(


> Does that explain things correctly? One thing I am sure of is that we
> should remove "implemented using an immutable dictionary" - it's an
> implementation detail, and adds nothing but confusion to mention it
> here.


Agreed.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 11:24 AM, Guido van Rossum wrote:
On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith > wrote:


On 1/2/2018 12:01 AM, Guido van Rossum wrote:

Yes, there's a class variable (__dataclass_fields__) that
identifies the parent fields. The PEP doesn't mention this or
the fact that special methods (like __repr__ and __init__) can
tell whether a base class is a dataclass. It probably should
though. (@Eric)


I think that's covered in this section:
https://www.python.org/dev/peps/pep-0557/#inheritance



I was specifically talking about the name and contents of 
__dataclass_fields__, which are not documented by the PEP. I expect it's 
inevitable that people will be looking at this (since they can see it in 
the source code). Or do you recommend that people use 
dataclasses.fields() and catch ValueError?


The expectation is to use dataclasses.fields(). Both it and 
__dataclass_fields__ contain the fields for this class and the parents. 
The only difference is the pseudo-fields.


I can add some words describing .fields() returning which fields are 
present.


I notice that _isdataclass() 
exists but is private and I don't recall why. 


I think the argument was that it's an anti-pattern, and if you really 
want to know, just call dataclasses.fields() and catch the TypeError. I 
have this in a helper file:


def isdataclass(obj):
"""Returns True for dataclass classes and instances."""
try:
dataclasses.fields(obj)
return True
except TypeError:
return False


(Also now I'm curious what

the "pseudo-fields" are that fields() ignores, but that's OT.)


ClassVar and InitVar "fields". dataclasses.fields() doesn't return them.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Guido van Rossum]

On Fri, Jan 5, 2018 at 3:10 AM, Paul Moore  wrote:

> On 5 January 2018 at 11:06, Victor Stinner 
> wrote:
> >> Le 5 janv. 2018 11:42 AM, "Paul Moore"  a écrit :
> >> """
> >> A Context is a mapping from ContextVar objects to their values. The
> >> Context itself exposes the Mapping interface, so cannot be modified
> >> directly - to modify the value associated with a variable you need to
> >> use the ContextVar.set() method.
> >> """
> >>
> >> Does that explain things correctly? One thing I am sure of is that we
> >> should remove "implemented using an immutable dictionary" - it's an
> >> implementation detail, and adds nothing but confusion to mention it
> >> here.
>
> > You can only modify a context when it's the current context, so using
> > ContextVar.set() in Context.run().
>
> Thanks. That's a useful qualification. But it may be too detailed for
> the summary - it's certainly something that should be covered in the
> specification section, though. Maybe "...  you need to use the
> ContextVar.set() method from Context.run()" would be OK, although I
> don't want the summary to get too long.
>

(Sorry, I wrote my previous response before seeing this part of the
exchange.)

Maybe "... you need to use the ContextVar.set() method, which modifies the
current context."

Then we also need to update the summary for ContextVar.set(), which
currently IMO is plain wrong.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2017-12-29 - 2018-01-05)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6377 (+22)
  closed 37871 (+28)
  total  44248 (+50)

Open issues with patches: 2479 


Issues opened (37)
==

#15982: asyncore.dispatcher does not handle windows socket error code 
https://bugs.python.org/issue15982  reopened by vstinner

#32448: subscriptable
https://bugs.python.org/issue32448  opened by thehesiod

#32449: MappingView must inherit from Collection instead of Sized
https://bugs.python.org/issue32449  opened by yahya-abou-imran

#32450: non-descriptive variable name
https://bugs.python.org/issue32450  opened by Yuri Kanivetsky

#32451: python -m venv activation issue when using cygwin on windows
https://bugs.python.org/issue32451  opened by Kevin

#32453: shutil.rmtree can have O(n^2) performance on large dirs
https://bugs.python.org/issue32453  opened by nh2

#32454: Add socket.close(fd) function
https://bugs.python.org/issue32454  opened by christian.heimes

#32455: PyCompile_OpcodeStackEffect() and dis.stack_effect() are not p
https://bugs.python.org/issue32455  opened by serhiy.storchaka

#32456: PYTHONIOENCODING=undefined doesn't work in Python 3
https://bugs.python.org/issue32456  opened by serhiy.storchaka

#32457: Windows Python cannot handle an early PATH entry containing ".
https://bugs.python.org/issue32457  opened by Ray Donnelly

#32458: test_asyncio failures on Windows
https://bugs.python.org/issue32458  opened by pitrou

#32459: Capsule API usage docs are incompatible with module reloading 
https://bugs.python.org/issue32459  opened by ncoghlan

#32461: the first build after a change to Makefile.pre.in uses the old
https://bugs.python.org/issue32461  opened by xdegaye

#32462: mimetypes.guess_type() returns incorrectly formatted type
https://bugs.python.org/issue32462  opened by csabella

#32463: problems with shutil.py and os.get_terminal_size
https://bugs.python.org/issue32463  opened by Dhruve

#32464: raise NotImplemented vs return NotImplemented
https://bugs.python.org/issue32464  opened by thatiparthy

#32465: [urllib] proxy_bypass_registry - extra error handling required
https://bugs.python.org/issue32465  opened by chansol kim

#32466: Fix missing test coverage for fractions.Fraction.__new__
https://bugs.python.org/issue32466  opened by gphemsley

#32467: dict_values isn't considered a Collection nor a Container
https://bugs.python.org/issue32467  opened by yahya-abou-imran

#32469: Generator and coroutine repr could be more helpful
https://bugs.python.org/issue32469  opened by pitrou

#32471: Add an UML class diagram to the collections.abc module documen
https://bugs.python.org/issue32471  opened by yahya-abou-imran

#32473: Readibility of ABCMeta._dump_registry()
https://bugs.python.org/issue32473  opened by yahya-abou-imran

#32475: Add ability to query number of buffered bytes available on buf
https://bugs.python.org/issue32475  opened by kata198

#32476: Add concat functionality to ElementTree xpath find
https://bugs.python.org/issue32476  opened by jjolly

#32477: Move jumps optimization from the peepholer to the compiler
https://bugs.python.org/issue32477  opened by serhiy.storchaka

#32479: inconsistent ImportError message executing same import stateme
https://bugs.python.org/issue32479  opened by xiang.zhang

#32485: Multiprocessing dict sharing between forked processes
https://bugs.python.org/issue32485  opened by André Neto

#32486: tail optimization for 'yield from'
https://bugs.python.org/issue32486  opened by Robert Smart

#32489: Allow 'continue' in 'finally' clause
https://bugs.python.org/issue32489  opened by serhiy.storchaka

#32490: subprocess: duplicate filename in exception message
https://bugs.python.org/issue32490  opened by jwilk

#32491: base64.decode: linebreaks are not ignored
https://bugs.python.org/issue32491  opened by gregory.p.smith

#32492: C Fast path for namedtuple's property/itemgetter pair
https://bugs.python.org/issue32492  opened by rhettinger

#32493: UUID Module - FreeBSD build failure
https://bugs.python.org/issue32493  opened by David Carlier

#32494: interface to gdbm_count
https://bugs.python.org/issue32494  opened by sam-s

#32495: Adding Timer to multiprocessing
https://bugs.python.org/issue32495  opened by jcrotts

#32496: lib2to3 fails to parse a ** of a conditional expression
https://bugs.python.org/issue32496  opened by gregory.p.smith

#32497: datetime.strptime creates tz naive object from value containin
https://bugs.python.org/issue32497  opened by akeeman



Most recent 15 issues with no replies (15)
==

#32496: lib2to3 fails to parse a ** of a conditional expression
https://bugs.python.org/issue32496

#32494: interface to gdbm_count
https://bugs.python.org/issue32494

#32490: subprocess: duplicate filename in exception message
htt

Re: [Python-Dev] PEP 567 v2

[Guido van Rossum]

Some inline responses to Paul (I snipped everything I agree with).

On Wed, Jan 3, 2018 at 3:34 AM, Paul Moore  wrote:

> On 28 December 2017 at 06:08, Yury Selivanov 
> wrote:
> > This is a second version of PEP 567.
> [...]
>
> > The notion of "current value" deserves special consideration:
> > different asynchronous tasks that exist and execute concurrently
> > may have different values for the same key.  This idea is well-known
> > from thread-local storage but in this case the locality of the value is
> > not necessarily bound to a thread.  Instead, there is the notion of the
> > "current ``Context``" which is stored in thread-local storage, and
> > is accessed via ``contextvars.copy_context()`` function.
>
> Accessed by copying it? That seems weird to me. I'd expect either that
> you'd be
> able to access the current Context directly, *or* that you'd say that the
> current Context is not directly accessible by the user, but that a copy
> can be
> obtained using copy_context. But given that the Context is immutable, why
> the
> need to copy it?
>

Because it's not immutable. (I think by now people following this thread
understand that.) The claims or implications in the PEP that Context is
immutable are wrong (and contradict the recommended use of run() for
asyncio, for example).


> Also, the references to threads in the above are confusing. It says that
> this
> is a well-known concept in terms of thread-local storage, but this case is
> different. It then goes on to say that the current Context is stored in
> thread
> local storage, which gives me the impression that the new idea *is*
> related to
> thread local storage...
>

The PEP's language does seem confused. This is because it doesn't come out
and define the concept of "task". The correspondence is roughly that
thread-locals are to threads what Contexts and ContextVars are to tasks.
(This still requires quite a bit of squinting due to the API differences
but it's better than what the PEP says.)


> I think that the fact that a Context is held in thread-local storage is an
> implementation detail. Assuming I'm right, don't bother mentioning it -
> simply
> say that there's a notion of a current Context and leave it at that.
>

No, actually it's important that each thread has its own current context.
It is even possible to pass Contexts between threads, e.g. if you have a
ThreadExecutor e, you can call e.submit(ctx.run, some_function, args...).

However, run() is not thread-safe! @Yury: There's an example that does
almost exactly this in the PEP, but I think it could result in a race
condition if you called run() concurrently on that same context in a
different thread -- whichever run() finishes last will overwrite the
other's state. I think we should just document this and recommend always
using a fresh copy in such scenarios. Hm, does this mean it would be good
to have an explicit Context.copy() method? Or should we show how to create
a copy of an arbitrary Context using ctx.run(copy_context)?


> > Manipulation of the current ``Context`` is the responsibility of the
> > task framework, e.g. asyncio.
> >
> > A ``Context`` is conceptually a read-only mapping, implemented using
> > an immutable dictionary.  The ``ContextVar.get()`` method does a
> > lookup in the current ``Context`` with ``self`` as a key, raising a
> > ``LookupError``  or returning a default value specified in
> > the constructor.
> >
> > The ``ContextVar.set(value)`` method clones the current ``Context``,
> > assigns the ``value`` to it with ``self`` as a key, and sets the
> > new ``Context`` as the new current ``Context``.
> >
>
> On first reading, this confused me because I didn't spot that you're
> saying a
> *Context* is read-only, but a *ContextVar* has get and set methods.
>
> Maybe reword this to say that a Context is a read-only mapping from
> ContextVars
> to values. A ContextVar has a get method that looks up its value in the
> current
> Context, and a set method that replaces the current Context with a new one
> that
> associates the specified value with this ContextVar.
>
> (The current version feels confusing to me because it goes into too much
> detail
> on how the implementation does this, rather than sticking to the high-level
> specification)
>

We went over this passage in another subthread. IMO what it says about
ContextVar.set() is incorrect.


> > Specification
> > =
> >
> > A new standard library module ``contextvars`` is added with the
> > following APIs:
> >
> > 1. ``copy_context() -> Context`` function is used to get a copy of
> >the current ``Context`` object for the current OS thread.
> >
> > 2. ``ContextVar`` class to declare and access context variables.
> >
> > 3. ``Context`` class encapsulates context state.  Every OS thread
> >stores a reference to its current ``Context`` instance.
> >It is not possible to control that reference manually.
> >Instead, the ``Context.run(callable, *args, **kwargs)`` method is
> >use

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Guido van Rossum]

Hm. I don't know that people will conclude that checking for a dataclass is
an anti-pattern. They'll probably just invent a myriad of different hacks
like the one you showed. I recommend making it public.

I still worry a bit about ClassVar and InitVar being potentially useful but
I concede I have no use case so I'll drop it.

On Fri, Jan 5, 2018 at 8:43 AM, Eric V. Smith  wrote:

> On 1/5/2018 11:24 AM, Guido van Rossum wrote:
>
>> On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith > > wrote:
>>
>> On 1/2/2018 12:01 AM, Guido van Rossum wrote:
>>
>> Yes, there's a class variable (__dataclass_fields__) that
>> identifies the parent fields. The PEP doesn't mention this or
>> the fact that special methods (like __repr__ and __init__) can
>> tell whether a base class is a dataclass. It probably should
>> though. (@Eric)
>>
>>
>> I think that's covered in this section:
>> https://www.python.org/dev/peps/pep-0557/#inheritance
>> 
>>
>>
>> I was specifically talking about the name and contents of
>> __dataclass_fields__, which are not documented by the PEP. I expect it's
>> inevitable that people will be looking at this (since they can see it in
>> the source code). Or do you recommend that people use dataclasses.fields()
>> and catch ValueError?
>>
>
> The expectation is to use dataclasses.fields(). Both it and
> __dataclass_fields__ contain the fields for this class and the parents. The
> only difference is the pseudo-fields.
>
> I can add some words describing .fields() returning which fields are
> present.
>
> I notice that _isdataclass() exists but is private and I don't recall why.
>>
>
> I think the argument was that it's an anti-pattern, and if you really want
> to know, just call dataclasses.fields() and catch the TypeError. I have
> this in a helper file:
>
> def isdataclass(obj):
> """Returns True for dataclass classes and instances."""
> try:
> dataclasses.fields(obj)
> return True
> except TypeError:
> return False
>
>
> (Also now I'm curious what
>
>> the "pseudo-fields" are that fields() ignores, but that's OT.)
>>
>
> ClassVar and InitVar "fields". dataclasses.fields() doesn't return them.
>
> Eric.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Random832]

On Fri, Jan 5, 2018, at 10:47, Guido van Rossum wrote:
> I don't recall (though someone with more time might find the discussion in
> the archives or on the tracker). It was never implemented and I think it
> shouldn't be. So we might as well update the PEP. It wouldn't be
> particularly useful, since (by definition) the function that declares the
> nonlocal variable is not its owner, and hence it's unlikely to make sense
> to initialize it here. The same reasoning applies to global BTW.

I'm not so sure...

The only situation in which you're *required* to declare a nonlocal/global 
variable, after all, is if you intend to assign it - a name that you never 
assign is presumed to be non-local. The description in the PEP also applies to 
augmented assignments, and "global some_counter; some_counter += 1" is 
certainly a pattern I've needed in the past.

The PEP also quotes you as endorsing this for global. 
https://mail.python.org/pipermail/python-3000/2006-November/004166.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 12:58 PM, Guido van Rossum wrote:
Hm. I don't know that people will conclude that checking for a dataclass 
is an anti-pattern. They'll probably just invent a myriad of different 
hacks like the one you showed. I recommend making it public.


I'm trying to track down the original discussion. We got bogged down on 
whether it worked for classes or instances or both, then we got tied up 
in naming it (surprise!), then it looks like we decided to just not 
include it since you could make those decisions for yourself.


I think the discussion is buried in this thread:
https://mail.python.org/pipermail/python-dev/2017-November/150966.html

Which references:
https://github.com/ericvsmith/dataclasses/issues/99

So, ignoring the naming issue, I think if we want to revive it, the 
question is: should isdataclass() return True on just instances, just 
classes, or both? And should it ever raise an exception, or just return 
False?


I still worry a bit about ClassVar and InitVar being potentially useful 
but I concede I have no use case so I'll drop it.


IIRC, we decided that we could add a parameter to dataclasses.fields() 
if we ever wanted to return pseudo-fields. But no one came up with a use 
case.


Eric.



On Fri, Jan 5, 2018 at 8:43 AM, Eric V. Smith > wrote:


On 1/5/2018 11:24 AM, Guido van Rossum wrote:

On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith
mailto:e...@trueblade.com>
>> wrote:

     On 1/2/2018 12:01 AM, Guido van Rossum wrote:

         Yes, there's a class variable (__dataclass_fields__) that
         identifies the parent fields. The PEP doesn't mention
this or
         the fact that special methods (like __repr__ and
__init__) can
         tell whether a base class is a dataclass. It probably
should
         though. (@Eric)


     I think that's covered in this section:
https://www.python.org/dev/peps/pep-0557/#inheritance

     >


I was specifically talking about the name and contents of
__dataclass_fields__, which are not documented by the PEP. I
expect it's inevitable that people will be looking at this
(since they can see it in the source code). Or do you recommend
that people use dataclasses.fields() and catch ValueError?


The expectation is to use dataclasses.fields(). Both it and
__dataclass_fields__ contain the fields for this class and the
parents. The only difference is the pseudo-fields.

I can add some words describing .fields() returning which fields are
present.

I notice that _isdataclass() exists but is private and I don't
recall why.


I think the argument was that it's an anti-pattern, and if you
really want to know, just call dataclasses.fields() and catch the
TypeError. I have this in a helper file:

def isdataclass(obj):
     """Returns True for dataclass classes and instances."""
     try:
         dataclasses.fields(obj)
         return True
     except TypeError:
         return False


(Also now I'm curious what

the "pseudo-fields" are that fields() ignores, but that's OT.)


ClassVar and InitVar "fields". dataclasses.fields() doesn't return them.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev

Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org 





--
--Guido van Rossum (python.org/~guido )


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Guido van Rossum]

Yeah, but I've changed my mind on this -- I think it's needless added
complexity that helps save one line of code in very few use cases. And you
don't really think the PEP endorses `nonlocal foo += 1` do you?

On Fri, Jan 5, 2018 at 10:48 AM, Random832  wrote:

> On Fri, Jan 5, 2018, at 10:47, Guido van Rossum wrote:
> > I don't recall (though someone with more time might find the discussion
> in
> > the archives or on the tracker). It was never implemented and I think it
> > shouldn't be. So we might as well update the PEP. It wouldn't be
> > particularly useful, since (by definition) the function that declares the
> > nonlocal variable is not its owner, and hence it's unlikely to make sense
> > to initialize it here. The same reasoning applies to global BTW.
>
> I'm not so sure...
>
> The only situation in which you're *required* to declare a nonlocal/global
> variable, after all, is if you intend to assign it - a name that you never
> assign is presumed to be non-local. The description in the PEP also applies
> to augmented assignments, and "global some_counter; some_counter += 1" is
> certainly a pattern I've needed in the past.
>
> The PEP also quotes you as endorsing this for global.
> https://mail.python.org/pipermail/python-3000/2006-November/004166.html
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Guido van Rossum]

I'm normally no big fan of things that take either a class or an instance,
but since fields() does this, I think is_dataclass() should to. And that's
the name I'd choose. OK on the pseudo-fields.

On Fri, Jan 5, 2018 at 11:06 AM, Eric V. Smith  wrote:

> On 1/5/2018 12:58 PM, Guido van Rossum wrote:
>
>> Hm. I don't know that people will conclude that checking for a dataclass
>> is an anti-pattern. They'll probably just invent a myriad of different
>> hacks like the one you showed. I recommend making it public.
>>
>
> I'm trying to track down the original discussion. We got bogged down on
> whether it worked for classes or instances or both, then we got tied up in
> naming it (surprise!), then it looks like we decided to just not include it
> since you could make those decisions for yourself.
>
> I think the discussion is buried in this thread:
> https://mail.python.org/pipermail/python-dev/2017-November/150966.html
>
> Which references:
> https://github.com/ericvsmith/dataclasses/issues/99
>
> So, ignoring the naming issue, I think if we want to revive it, the
> question is: should isdataclass() return True on just instances, just
> classes, or both? And should it ever raise an exception, or just return
> False?
>
> I still worry a bit about ClassVar and InitVar being potentially useful
>> but I concede I have no use case so I'll drop it.
>>
>
> IIRC, we decided that we could add a parameter to dataclasses.fields() if
> we ever wanted to return pseudo-fields. But no one came up with a use case.
>
> Eric.
>
>
>> On Fri, Jan 5, 2018 at 8:43 AM, Eric V. Smith > > wrote:
>>
>> On 1/5/2018 11:24 AM, Guido van Rossum wrote:
>>
>> On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith
>> mailto:e...@trueblade.com>
>> >> wrote:
>>
>>  On 1/2/2018 12:01 AM, Guido van Rossum wrote:
>>
>>  Yes, there's a class variable (__dataclass_fields__) that
>>  identifies the parent fields. The PEP doesn't mention
>> this or
>>  the fact that special methods (like __repr__ and
>> __init__) can
>>  tell whether a base class is a dataclass. It probably
>> should
>>  though. (@Eric)
>>
>>
>>  I think that's covered in this section:
>> https://www.python.org/dev/peps/pep-0557/#inheritance
>> 
>>  > >
>>
>>
>> I was specifically talking about the name and contents of
>> __dataclass_fields__, which are not documented by the PEP. I
>> expect it's inevitable that people will be looking at this
>> (since they can see it in the source code). Or do you recommend
>> that people use dataclasses.fields() and catch ValueError?
>>
>>
>> The expectation is to use dataclasses.fields(). Both it and
>> __dataclass_fields__ contain the fields for this class and the
>> parents. The only difference is the pseudo-fields.
>>
>> I can add some words describing .fields() returning which fields are
>> present.
>>
>> I notice that _isdataclass() exists but is private and I don't
>> recall why.
>>
>>
>> I think the argument was that it's an anti-pattern, and if you
>> really want to know, just call dataclasses.fields() and catch the
>> TypeError. I have this in a helper file:
>>
>> def isdataclass(obj):
>>  """Returns True for dataclass classes and instances."""
>>  try:
>>  dataclasses.fields(obj)
>>  return True
>>  except TypeError:
>>  return False
>>
>>
>> (Also now I'm curious what
>>
>> the "pseudo-fields" are that fields() ignores, but that's OT.)
>>
>>
>> ClassVar and InitVar "fields". dataclasses.fields() doesn't return
>> them.
>>
>> Eric.
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org 
>> https://mail.python.org/mailman/listinfo/python-dev
>> 
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> 
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido )
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/eric%2Ba-
>> python-dev%40trueblade.com
>>
>>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 2:09 PM, Guido van Rossum wrote:
I'm normally no big fan of things that take either a class or an 
instance, but since fields() does this, I think is_dataclass() should 
to. And that's the name I'd choose. OK on the pseudo-fields.


Sounds good. I'll open a bpo issue.

Eric.

On Fri, Jan 5, 2018 at 11:06 AM, Eric V. Smith > wrote:


On 1/5/2018 12:58 PM, Guido van Rossum wrote:

Hm. I don't know that people will conclude that checking for a
dataclass is an anti-pattern. They'll probably just invent a
myriad of different hacks like the one you showed. I recommend
making it public.


I'm trying to track down the original discussion. We got bogged down
on whether it worked for classes or instances or both, then we got
tied up in naming it (surprise!), then it looks like we decided to
just not include it since you could make those decisions for yourself.

I think the discussion is buried in this thread:
https://mail.python.org/pipermail/python-dev/2017-November/150966.html


Which references:
https://github.com/ericvsmith/dataclasses/issues/99


So, ignoring the naming issue, I think if we want to revive it, the
question is: should isdataclass() return True on just instances,
just classes, or both? And should it ever raise an exception, or
just return False?

I still worry a bit about ClassVar and InitVar being potentially
useful but I concede I have no use case so I'll drop it.


IIRC, we decided that we could add a parameter to
dataclasses.fields() if we ever wanted to return pseudo-fields. But
no one came up with a use case.

Eric.


On Fri, Jan 5, 2018 at 8:43 AM, Eric V. Smith
mailto:e...@trueblade.com>
>> wrote:

     On 1/5/2018 11:24 AM, Guido van Rossum wrote:

         On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith
         mailto:e...@trueblade.com>
>
         

         >
 

         >>


         I was specifically talking about the name and contents of
         __dataclass_fields__, which are not documented by the
PEP. I
         expect it's inevitable that people will be looking at this
         (since they can see it in the source code). Or do you
recommend
         that people use dataclasses.fields() and catch ValueError?


     The expectation is to use dataclasses.fields(). Both it and
     __dataclass_fields__ contain the fields for this class and the
     parents. The only difference is the pseudo-fields.

     I can add some words describing .fields() returning which
fields are
     present.

         I notice that _isdataclass() exists but is private and
I don't
         recall why.


     I think the argument was that it's an anti-pattern, and if you
     really want to know, just call dataclasses.fields() and
catch the
     TypeError. I have this in a helper file:

     def isdataclass(obj):
          """Returns True for dataclass classes and instances."""
          try:
              dataclasses.fields(obj)
              return True
          except TypeError:
              return False


     (Also now I'm curious what

         the "pseudo-fields" are that fields() ignores, but

Re: [Python-Dev] Unique loader per module

[Brett Cannon]

Barry and I had a meeting at work today and we decided to go with Nick's
idea of using a get_resource_reader(fullname) method on loaders. We aren't
going to go with an ABC and simply depend on the method existing as
implementing the API (and then returning None if the loader can't handle
the specified module).

I'll have a PR to update the docs out hopefully today for those that care.

On Thu, 4 Jan 2018 at 23:14 Nick Coghlan  wrote:

> On 3 January 2018 at 06:35, Barry Warsaw  wrote:
> > Brett doesn’t like this, for several reasons (quoting):
> >
> > 1. redundant API in all cases where the loader is unique to the module
> > 2. the memory savings of sharing a loader is small
> > 3. it's implementation complexity/overhead for an optimization case.
> >
> > The second solution, and the one Brett prefers, is to reimplement zip
> importer to not use a shared loader.  This may not be that difficult, if
> for example we were to use a delegate loader wrapping a shared loader.
> >
> > The bigger problem IMHO is two-fold:
> >
> > 1. It would be backward incompatible.  If there’s any code out there
> expecting a shared loader in zipimport, it would break
> > 2. More problematic is that we’d have to impose an additional
> requirement on loaders - that they always be unique per module,
> contradicting the advice in PEP 302
>
> We added module.__spec__.loader_state as part of PEP 451 precisely so
> shared loaders had a place to store per-module state without have to
> switch to a unique-loader-per-module model.
>
> I think the main reason you're seeing a problem here is because
> ResourceReader has currently been designed to be implemented directly
> by loaders, rather than being a subcomponent that you can request
> *from* a loader.
>
> If you instead had an indirection API (that could optionally return
> self in the case of non-shared loaders), you'd keep the current
> resource reader method signatures, but the way you'd access the itself
> would be:
>
> resources = module.__spec__.loader.get_resource_reader(module)
> # resources implements the ResourceReader ABC
>
> For actual use, the loader protocol could be hidden behind a helper
> function:
>
> resources = importlib_resources.get_resource_reader(module)
>
> For a shared loader, get_resource_reader(module) would return a new
> *non*-shared resource reader (perhaps caching it in
> __spec__.loader_state).
>
> For a non-shared loader, get_resource_reader(module) would just return
> self.
>
> In both cases, we'd recommend that loaders ensure "self is
> module.__spec__.loader" as part of their get_resource_reader()
> implementation.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 2:11 PM, Eric V. Smith wrote:

On 1/5/2018 2:09 PM, Guido van Rossum wrote:
I'm normally no big fan of things that take either a class or an 
instance, but since fields() does this, I think is_dataclass() should 
to. And that's the name I'd choose. OK on the pseudo-fields.


Sounds good. I'll open a bpo issue.


https://bugs.python.org/issue32499

I'm slowly reading through the rest of this thread and will respond this 
weekend.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Chris Jerdonek]

On Fri, Jan 5, 2018 at 8:29 AM, Guido van Rossum  wrote:
> On Fri, Jan 5, 2018 at 2:05 AM, Victor Stinner 
> wrote:
>>
>> Currently, Context.get(var) returns None when "var in context" is false.
>> That's surprising and different than var.get(), especially when var has a
>> default value.
>
> I don't see the problem. Context.get() is inherited from Mapping.get(); if
> you want it to raise use Context.__getitem__() (i.e. ctx[var]). Lots of
> classes define get() methods with various behaviors. Context.get() and
> ContextVar.get() are just different -- ContextVar is not a Mapping.

One thing that I think could be contributing to confusion around the
proposed API is that there is a circular relationship between Context
and ContextVar, e.g. ContextVar.get() does a lookup in the current
Context with "self" (the ContextVar object) as a key

Also, it's the "keys" (the ContextVar objects) that have the get()
method that should be used rather than the container object (the
Context). This gives the confusing *feeling* of a mapping of mappings.
This is different from how the containers people are most familiar
with work -- like dict.

Is there a reason ContextVar needs to be exposed publicly at all?  For
example, the API could use string keys like contextvars.get(name) or
Context.get(name) (class method). There could be separate functions to
initialize keys with desired default values, etc (internally creating
ContextVars as needed).

If the issue is key collisions, it seems like this could be handled by
namespacing or using objects (namespaced by modules) instead of
strings.

Maybe this approach was ruled out early on in discussions, but I don't
see it mentioned in the PEP.

--Chris



>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Nathaniel Smith]

On Fri, Jan 5, 2018 at 7:47 AM, Guido van Rossum  wrote:
> I don't recall (though someone with more time might find the discussion in
> the archives or on the tracker). It was never implemented and I think it
> shouldn't be. So we might as well update the PEP. It wouldn't be
> particularly useful, since (by definition) the function that declares the
> nonlocal variable is not its owner, and hence it's unlikely to make sense to
> initialize it here. The same reasoning applies to global BTW.

The reason I got curious and looked into it is that recently I've been
finding myself using it a lot for passing values back out of
concurrent functions (examples below). So it does have use cases, but
I agree that it's not clear how much value is really added by saving a
line here. Maybe in a year or two if this style catches on as
idiomatic then it'd be worth revisiting.

###

Example: run several functions, return the value of the one that
finishes first (or non-deterministic if several finish at ~the same
time):

async def race(*async_fns):
async with trio.open_nursery() as nursery:
winning_value = None

async def driver(async_fn):
nonlocal winning_value
winning_value = await async_fn()
# we're done, so cancel competitors
nursery.cancel_scope.cancel()

for async_fn in async_fns:
nursery.start_soon(driver, async_fn)

return winner

###

Example: an async iterator version of zip, with concurrent evaluation
of the different iterators (based on an idea from github user @matham:
https://github.com/python-trio/trio/issues/393):

async def async_zip(*aiterables):
aiterators = [aiterable.__aiter__() for aiterable in aiterables]
done = False
while True:
items = [None] * len(aiterators)

async def fill_in(i):
try:
items[i] = await aiterators[i].__anext__()
except StopAsyncIteration:
nonlocal done
done = True

async with trio.open_nursery() as nursery:
for i in range(len(aiterators)):
nursery.start_soon(fill_in, i)

if done:
break

yield tuple(items)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Guido van Rossum]

I don't like those examples -- "nonlocal foo = bar" sounds like bar is used
as the *initializer*, but it actually is just an assignment that overwrites
the actual initial value. IMO those shouldn't be combined.

On Fri, Jan 5, 2018 at 2:59 PM, Nathaniel Smith  wrote:

> On Fri, Jan 5, 2018 at 7:47 AM, Guido van Rossum  wrote:
> > I don't recall (though someone with more time might find the discussion
> in
> > the archives or on the tracker). It was never implemented and I think it
> > shouldn't be. So we might as well update the PEP. It wouldn't be
> > particularly useful, since (by definition) the function that declares the
> > nonlocal variable is not its owner, and hence it's unlikely to make
> sense to
> > initialize it here. The same reasoning applies to global BTW.
>
> The reason I got curious and looked into it is that recently I've been
> finding myself using it a lot for passing values back out of
> concurrent functions (examples below). So it does have use cases, but
> I agree that it's not clear how much value is really added by saving a
> line here. Maybe in a year or two if this style catches on as
> idiomatic then it'd be worth revisiting.
>
> ###
>
> Example: run several functions, return the value of the one that
> finishes first (or non-deterministic if several finish at ~the same
> time):
>
> async def race(*async_fns):
> async with trio.open_nursery() as nursery:
> winning_value = None
>
> async def driver(async_fn):
> nonlocal winning_value
> winning_value = await async_fn()
> # we're done, so cancel competitors
> nursery.cancel_scope.cancel()
>
> for async_fn in async_fns:
> nursery.start_soon(driver, async_fn)
>
> return winner
>
> ###
>
> Example: an async iterator version of zip, with concurrent evaluation
> of the different iterators (based on an idea from github user @matham:
> https://github.com/python-trio/trio/issues/393):
>
> async def async_zip(*aiterables):
> aiterators = [aiterable.__aiter__() for aiterable in aiterables]
> done = False
> while True:
> items = [None] * len(aiterators)
>
> async def fill_in(i):
> try:
> items[i] = await aiterators[i].__anext__()
> except StopAsyncIteration:
> nonlocal done
> done = True
>
> async with trio.open_nursery() as nursery:
> for i in range(len(aiterators)):
> nursery.start_soon(fill_in, i)
>
> if done:
> break
>
> yield tuple(items)
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Guido van Rossum]

On Fri, Jan 5, 2018 at 2:43 PM, Chris Jerdonek 
wrote:

> On Fri, Jan 5, 2018 at 8:29 AM, Guido van Rossum  wrote:
> > On Fri, Jan 5, 2018 at 2:05 AM, Victor Stinner  >
> > wrote:
> >>
> >> Currently, Context.get(var) returns None when "var in context" is false.
> >> That's surprising and different than var.get(), especially when var has
> a
> >> default value.
> >
> > I don't see the problem. Context.get() is inherited from Mapping.get();
> if
> > you want it to raise use Context.__getitem__() (i.e. ctx[var]). Lots of
> > classes define get() methods with various behaviors. Context.get() and
> > ContextVar.get() are just different -- ContextVar is not a Mapping.
>
> One thing that I think could be contributing to confusion around the
> proposed API is that there is a circular relationship between Context
> and ContextVar, e.g. ContextVar.get() does a lookup in the current
> Context with "self" (the ContextVar object) as a key
>
> Also, it's the "keys" (the ContextVar objects) that have the get()
> method that should be used rather than the container object (the
> Context). This gives the confusing *feeling* of a mapping of mappings.
> This is different from how the containers people are most familiar
> with work -- like dict.
>

Only if you think of ContextVar as a mapping, which is not at all how it
works. (If anything, its get() method is more like that on weak references.)


> Is there a reason ContextVar needs to be exposed publicly at all?  For
> example, the API could use string keys like contextvars.get(name) or
> Context.get(name) (class method). There could be separate functions to
> initialize keys with desired default values, etc (internally creating
> ContextVars as needed).
>
> If the issue is key collisions, it seems like this could be handled by
> namespacing or using objects (namespaced by modules) instead of
> strings.
>
> Maybe this approach was ruled out early on in discussions, but I don't
> see it mentioned in the PEP.
>

Yes this was litigated repeatedly. Maybe the PEP 550 discussion or Rejected
Ideas section have more.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Paul Moore]

On 5 January 2018 at 23:02, Guido van Rossum  wrote:
> I don't like those examples -- "nonlocal foo = bar" sounds like bar is used
> as the *initializer*, but it actually is just an assignment that overwrites
> the actual initial value. IMO those shouldn't be combined.

That was my immediate reaction too. I would find the "nonlocal x =
value" version more confusing for this reason.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Nathaniel Smith]

On Thu, Jan 4, 2018 at 3:18 PM, Nathaniel Smith  wrote:
> I think the fix is a little bit cumbersome, but straightforward, and
> actually *simplifies* caching.
[...]
> And then the caching in get() becomes:
>
> def get(self):
> if tstate->current_context != self->last_context:
> # Update cache
> self->last_value = tstate->current_context_data->hamt_lookup(self)
> self->last_context = tstate->current_context
> return self->last_value

Actually, this trick of using the Context* as the cache validation key
doesn't work :-/. The problem is that the cache doesn't hold a strong
reference to the Context, so there's an ABA problem [1] if the Context
is deallocated, and then later the same memory location gets used for
a new Context object. It could be fixed by using weakref callbacks,
but that's not really simpler than the current counter-based cache
validation logic.

-n

[1] https://en.wikipedia.org/wiki/ABA_problem

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Elvis Pranskevichus]

On Friday, January 5, 2018 5:43:24 PM EST Chris Jerdonek wrote:
> Is there a reason ContextVar needs to be exposed publicly at all?  For
> example, the API could use string keys like contextvars.get(name) or
> Context.get(name) (class method). There could be separate functions
> to initialize keys with desired default values, etc (internally
> creating ContextVars as needed).

Mainly because the design of contextvars follows the spirit of 
threading.local() (but see [1]).  A ContextVar is not unlike a global 
variable, and we don't normally write globals()['foo'].

  Elvis


[1] 
https://www.python.org/dev/peps/pep-0550/#replication-of-threading-local-interface

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Nathaniel Smith]

On Thu, Jan 4, 2018 at 4:48 PM, Guido van Rossum  wrote:
> Your suggestions sound reasonable, but we are now running into a logistical
> problem -- I don't want to decide this unilaterally but Yury is on vacation
> until Jan 15. That gives us at most 2 weeks for approval of the PEP and
> review + commit of the implementation
> (https://github.com/python/cpython/pull/5027) before the 3.7.0 feature
> freeze / beta (Jan 29).

I don't think there are any fundamental disagreements here, so I guess
we can sort something out pretty quickly once Yury gets back.

> [Even later] Re: your other suggestion, why couldn't the threadstate contain
> just the Context? It seems one would just write
> tstate->current_context->_data everywhere instead of
> tstate->current_context_data.

Yeah, that's probably better :-).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 v2

[Nathaniel Smith]

On Fri, Jan 5, 2018 at 2:43 PM, Chris Jerdonek  wrote:
> One thing that I think could be contributing to confusion around the
> proposed API is that there is a circular relationship between Context
> and ContextVar, e.g. ContextVar.get() does a lookup in the current
> Context with "self" (the ContextVar object) as a key

It's maybe helpful to remember that while Context is an important part
of the overall API, it's not intended to be "user facing" at all. For
folks writing libraries that need context-local state, they just use
ContextVar and ignore the rest of the API.

> Also, it's the "keys" (the ContextVar objects) that have the get()
> method that should be used rather than the container object (the
> Context). This gives the confusing *feeling* of a mapping of mappings.
> This is different from how the containers people are most familiar
> with work -- like dict.
>
> Is there a reason ContextVar needs to be exposed publicly at all?  For
> example, the API could use string keys like contextvars.get(name) or
> Context.get(name) (class method). There could be separate functions to
> initialize keys with desired default values, etc (internally creating
> ContextVars as needed).
>
> If the issue is key collisions, it seems like this could be handled by
> namespacing or using objects (namespaced by modules) instead of
> strings.

With strings you have the problem of collisions, you lose the ability
to cache values to make ContextVar.get() really fast (which is a
critical requirement for decimal and numpy), and you lose the ability
to detect when a ContextVar has been garbage collected and clean up
its values. (We're not bothering to implement that kind of GC right
now, but it's nice to know that we have the option later if it becomes
a problem.) Allowing arbitrary objects as keys somewhat alleviates the
problem with collisions, but not the others.

What we could do is make the API look like:

import contextvars
key = contextvars.ContextKey(...)
contextvars.set(key, value)
value = contextvars.get(key)

That's isomorphic to the current proposal -- it's just renaming
ContextVar and making the existing get/set methods be globals instead
of methods. So then it's just a question of aesthetics. To me, the
current version looks nicer:

from contextvars import ContextVar
var = ContextVar(...)
var.set(value)
value = var.get()

In particular, you can't reasonably do 'from contextvars import get,
set'; that'd be gross :-). But 'from contextvars import ContextVar' is
fine and gives you the whole user-oriented API in one piece.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Benjamin Peterson]

On Fri, Jan 5, 2018, at 01:57, Nathaniel Smith wrote: 
> Was this just an oversight, or did it get rejected at some point and
> no-one remembered to update that PEP?

There was an implementation https://bugs.python.org/issue4199. But several 
years ago, we again reached the conclusion that the feature shouldn't be added.

Normally, I think final PEPs shouldn't be updated. But maybe in this case, it's 
worth deleting those lines to avoid future confusion.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com