Re: [Python-Dev] PEP 567 v2
Hi,
I like the new version of the PEP using "read only mapping" and
copy_context(). It's easier to understand.
I'm ok with seeing a context as a mapping, but I am confused about a
context variable considered as a mapping item. I still see a context
variable as a variable, so something which has a value or not. I just
propose to rename the default parameter of the ContextVar constructor.
Le 28 déc. 2017 7:10 AM, "Yury Selivanov" a
écrit :
ContextVar
--
The ``ContextVar`` class has the following constructor signature:
``ContextVar(name, *, default=_NO_DEFAULT)``. The ``name`` parameter
is used only for introspection and debug purposes, and is exposed
as a read-only ``ContextVar.name`` attribute. The ``default``
parameter is optional. Example::
# Declare a context variable 'var' with the default value 42.
var = ContextVar('var', default=42)
In term of API, "default" parameter name is strange. Why not simply calling
it "value"?
var = ContextVar('var', default=42)
and:
var = ContextVar('var')
var.set (42)
behaves the same, no?
The implementation explains where the "default" name comes from, but IMHO
"value" is a better name.
(The ``_NO_DEFAULT`` is an internal sentinel object used to
detect if the default value was provided.)
I would call it _NOT_SET.
* a read-only attribute ``Token.old_value`` set to the value the
variable had before the ``set()`` call, or to ``Token.MISSING``
if the variable wasn't set before.
Hum, I also suggest to rename Token.MISSING to Token.NOT_SET. It would be
more conistent with the last sentence.
C API
-
Would it be possible to make this API private?
2. ``int PyContextVar_Get(PyContextVar *, PyObject *default_value,
PyObject **value)``:
(...) ``value`` is always a borrowed
reference.
I'm not sure that it's a good idea to add a new public C function which
returns a borrowed reference. I would prefer to only use (regular) strong
references in the public API.
I don't want to elaborate here. You may see:
http://vstinner.readthedocs.io/python_new_stable_api.html
Internally, I don't care, do whatever you want for best performances :-)
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 567 v2
On Thu, Dec 28, 2017 at 1:51 AM, Victor Stinner
wrote:
> var = ContextVar('var', default=42)
>
> and:
>
> var = ContextVar('var')
> var.set (42)
>
> behaves the same, no?
No, they're different. The second sets the value in the current
context. The first sets the value in all contexts that currently
exist, and all empty contexts created in the future.
-n
--
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 567 v2
I have a couple basic questions around how this API could be used in practice. Both of my questions are for the Python API as applied to Tasks in asyncio. 1) Would this API support looking up the value of a context variable for **another** Task? For example, if you're managing multiple tasks using asyncio.wait() and there is an exception in some task, you might want to examine and report the value of a context variable for that task. 2) Would an appropriate use of this API be to assign a unique task id to each task? Or can that be handled more simply? I'm wondering because I recently thought this would be useful, and it doesn't seem like asyncio means for one to subclass Task (though I could be wrong). Thanks, --Chris On Wed, Dec 27, 2017 at 10:08 PM, Yury Selivanov wrote: > This is a second version of PEP 567. > > A few things have changed: > > 1. I now have a reference implementation: > https://github.com/python/cpython/pull/5027 > > 2. The C API was updated to match the implementation. > > 3. The get_context() function was renamed to copy_context() to better > reflect what it is really doing. > > 4. Few clarifications/edits here and there to address earlier feedback. > > > Yury > > > PEP: 567 > Title: Context Variables > Version: $Revision$ > Last-Modified: $Date$ > Author: Yury Selivanov > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 12-Dec-2017 > Python-Version: 3.7 > Post-History: 12-Dec-2017, 28-Dec-2017 > > > Abstract > > > This PEP proposes a new ``contextvars`` module and a set of new > CPython C APIs to support context variables. This concept is > similar to thread-local storage (TLS), but, unlike TLS, it also allows > correctly keeping track of values per asynchronous task, e.g. > ``asyncio.Task``. > > This proposal is a simplified version of :pep:`550`. The key > difference is that this PEP is concerned only with solving the case > for asynchronous tasks, not for generators. There are no proposed > modifications to any built-in types or to the interpreter. > > This proposal is not strictly related to Python Context Managers. > Although it does provide a mechanism that can be used by Context > Managers to store their state. > > > Rationale > = > > Thread-local variables are insufficient for asynchronous tasks that > execute concurrently in the same OS thread. Any context manager that > saves and restores a context value using ``threading.local()`` will > have its context values bleed to other code unexpectedly when used > in async/await code. > > A few examples where having a working context local storage for > asynchronous code is desirable: > > * Context managers like ``decimal`` contexts and ``numpy.errstate``. > > * Request-related data, such as security tokens and request > data in web applications, language context for ``gettext``, etc. > > * Profiling, tracing, and logging in large code bases. > > > Introduction > > > The PEP proposes a new mechanism for managing context variables. > The key classes involved in this mechanism are ``contextvars.Context`` > and ``contextvars.ContextVar``. The PEP also proposes some policies > for using the mechanism around asynchronous tasks. > > The proposed mechanism for accessing context variables uses the > ``ContextVar`` class. A module (such as ``decimal``) that wishes to > store a context variable should: > > * declare a module-global variable holding a ``ContextVar`` to > serve as a key; > > * access the current value via the ``get()`` method on the > key variable; > > * modify the current value via the ``set()`` method on the > key variable. > > The notion of "current value" deserves special consideration: > different asynchronous tasks that exist and execute concurrently > may have different values for the same key. This idea is well-known > from thread-local storage but in this case the locality of the value is > not necessarily bound to a thread. Instead, there is the notion of the > "current ``Context``" which is stored in thread-local storage, and > is accessed via ``contextvars.copy_context()`` function. > Manipulation of the current ``Context`` is the responsibility of the > task framework, e.g. asyncio. > > A ``Context`` is conceptually a read-only mapping, implemented using > an immutable dictionary. The ``ContextVar.get()`` method does a > lookup in the current ``Context`` with ``self`` as a key, raising a > ``LookupError`` or returning a default value specified in > the constructor. > > The ``ContextVar.set(value)`` method clones the current ``Context``, > assigns the ``value`` to it with ``self`` as a key, and sets the > new ``Context`` as the new current ``Context``. > > > Specification > = > > A new standard library module ``contextvars`` is added with the > following APIs: > > 1. ``copy_context() -> Context`` function is used to get a copy of >the current ``Context`` object for the current OS thread. > > 2. ``ContextVar`` class to
[Python-Dev] 'continue'/'break'/'return' inside 'finally' clause
Currently 'continue' is prohibited inside 'finally' clause, but 'break' and 'return' are allowed. What is the r 'continue' was prohibited in https://bugs.python.org/issue1542451. Should we prohibit also 'break' and 'return' or allow 'continue'? 'break' and 'return' are never used inside 'finally' clause in the stdlib. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Heap allocate type structs in native extension modules?
On Tue, Dec 26, 2017 at 3:00 PM, Benjamin Peterson wrote: > I imagine Cython already takes care of this? This appears to have a distinct purpose, albeit not unrelated to Cython. The OP's program would generate boilerplate C code for extension types the rest of which would perhaps be implemented by hand in C. Cython does this as well to an extent, but the generated code contains quite a bit of Cython-specific cruft and is not really meant to be edited by hand or read by humans in most cases. Anyways I don't think this answers the OP's question. > On Tue, Dec 26, 2017, at 02:16, Hugh Fisher wrote: >> I have a Python program which generates the boilerplate code for >> native extension modules from a Python source definition. >> (http://bitbucket.org/hugh_fisher/fullofeels if interested.) >> >> The examples in the Python doco and the "Python Essential Reference" >> book all use a statically declared PyTypeObject struct and >> PyType_Ready in the module init func, so I'm doing the same. Then >> Python 3.5 added a check for statically allocated types inheriting >> from heap types, which broke a couple of my classes. And now I'm >> trying to add a __dict__ to native classes so end users can add their >> own attributes, and this is turning out to be painful with static >> PyTypeObject structs >> >> Would it be better to use dynamically allocated type structs in native >> modules? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause
Looks to me the prohibition was to prevent a crash. It makes more sense to fix it. On Dec 28, 2017 03:39, "Serhiy Storchaka" wrote: Currently 'continue' is prohibited inside 'finally' clause, but 'break' and 'return' are allowed. What is the r 'continue' was prohibited in https://bugs.python.org/issue1542451. Should we prohibit also 'break' and 'return' or allow 'continue'? 'break' and 'return' are never used inside 'finally' clause in the stdlib. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% 40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Documenting types outside of typing
On Dec 27, 2017, at 18:59, Ivan Levkivskyi wrote: > > FWIW the same problem was discussed a year ago when documenting typing. At > that time the discussion was not conclusive, > so that some types use class:: directive while other use data:: directive. At > that time Guido was in favour of data:: and now in view of > PEP 560 many types in typing will stop being class objects, and will be just > (compact) objects. Therefore, my understanding is that > all special forms like Union, Any, ClassVar, etc. will use data:: in the docs. Thanks. I see that typing.rst has been updated to use data:: so I’ll change my branch accordingly. > Concerning the question whether it makes to document types, I think it makes > sense if it is a publicly available type (or type alias) > that will be useful to annotate user code. Thanks, that’s my feeling about it too. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 567 v2
On Thu, Dec 28, 2017 at 5:28 AM, Chris Jerdonek wrote: > I have a couple basic questions around how this API could be used in > practice. Both of my questions are for the Python API as applied to Tasks in > asyncio. > > 1) Would this API support looking up the value of a context variable for > **another** Task? For example, if you're managing multiple tasks using > asyncio.wait() and there is an exception in some task, you might want to > examine and report the value of a context variable for that task. No, unless that another Task explicitly shares the value or captures its context and shares it. Same as with threading.local. > > 2) Would an appropriate use of this API be to assign a unique task id to > each task? Or can that be handled more simply? I'm wondering because I > recently thought this would be useful, and it doesn't seem like asyncio > means for one to subclass Task (though I could be wrong). The API should be used to share one ID between a Task and tasks it creates. You can use it to store individual Task IDs, but a combination of a WeakKeyDictionary and Task.current_task() seems to be a better/easier option. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 567 v2
On Thu, Dec 28, 2017 at 4:51 AM, Victor Stinner wrote: > Hi, > > I like the new version of the PEP using "read only mapping" and > copy_context(). It's easier to understand. Thanks, Victor! > > I'm ok with seeing a context as a mapping, but I am confused about a context > variable considered as a mapping item. I still see a context variable as a > variable, so something which has a value or not. I just propose to rename > the default parameter of the ContextVar constructor. As Nathaniel already explained, a 'default' for ContextVars is literally a default -- default value returned when a ContextVar hasn't been assigned a value in a context. So my opinion on this is that 'default' is the less ambiguous name here. [..] > > * a read-only attribute ``Token.old_value`` set to the value the > variable had before the ``set()`` call, or to ``Token.MISSING`` > if the variable wasn't set before. > > > Hum, I also suggest to rename Token.MISSING to Token.NOT_SET. It would be > more conistent with the last sentence. I like MISSING more than NOT_SET, but this is very subjective, of course. If Guido wants to rename it I rename it. > C API > - > > > Would it be possible to make this API private? We want _decimal and numpy to use the new API, and they will call ContextVar.get() on basically all operations, so it needs to be as fast as possible. asyncio/uvloop also want the fastest copy_context() and Context.run() possible, as they use them for *every* callback. So I think it's OK for us to add new C APIs here. > > 2. ``int PyContextVar_Get(PyContextVar *, PyObject *default_value, > PyObject **value)``: > (...) ``value`` is always a borrowed >reference. > > > I'm not sure that it's a good idea to add a new public C function which > returns a borrowed reference. I would prefer to only use (regular) strong > references in the public API. Sure, I'll change it. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 567 v2
Keep MISSING. On Dec 28, 2017 8:44 AM, "Yury Selivanov" wrote: > On Thu, Dec 28, 2017 at 4:51 AM, Victor Stinner > wrote: > > Hi, > > > > I like the new version of the PEP using "read only mapping" and > > copy_context(). It's easier to understand. > > Thanks, Victor! > > > > > I'm ok with seeing a context as a mapping, but I am confused about a > context > > variable considered as a mapping item. I still see a context variable as > a > > variable, so something which has a value or not. I just propose to rename > > the default parameter of the ContextVar constructor. > > As Nathaniel already explained, a 'default' for ContextVars is > literally a default -- default value returned when a ContextVar hasn't > been assigned a value in a context. So my opinion on this is that > 'default' is the less ambiguous name here. > > [..] > > > > * a read-only attribute ``Token.old_value`` set to the value the > > variable had before the ``set()`` call, or to ``Token.MISSING`` > > if the variable wasn't set before. > > > > > > Hum, I also suggest to rename Token.MISSING to Token.NOT_SET. It would be > > more conistent with the last sentence. > > I like MISSING more than NOT_SET, but this is very subjective, of > course. If Guido wants to rename it I rename it. > > > > C API > > - > > > > > > Would it be possible to make this API private? > > We want _decimal and numpy to use the new API, and they will call > ContextVar.get() on basically all operations, so it needs to be as > fast as possible. asyncio/uvloop also want the fastest copy_context() > and Context.run() possible, as they use them for *every* callback. So > I think it's OK for us to add new C APIs here. > > > > > > 2. ``int PyContextVar_Get(PyContextVar *, PyObject *default_value, > > PyObject **value)``: > > (...) ``value`` is always a borrowed > >reference. > > > > > > I'm not sure that it's a good idea to add a new public C function which > > returns a borrowed reference. I would prefer to only use (regular) strong > > references in the public API. > > Sure, I'll change it. > > Yury > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause
28.12.17 16:38, Guido van Rossum пише: Looks to me the prohibition was to prevent a crash. It makes more sense to fix it. The crash can be fixed by just removing the check after finishing issue17611. But is there any use case for 'continue'/'break'/'return' inside 'finally' clause? The code like try: return 1 finally: return 2 or try: continue finally: break looks at least confusing. Currently 'break' and 'return' are never used inside 'finally' clause in the stdlib. I would want to see a third-party code that uses them. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause
I don't think the language definition should be judgmental here. The semantics are unambiguous. On Dec 28, 2017 11:38 AM, "Serhiy Storchaka" wrote: > 28.12.17 16:38, Guido van Rossum пише: > >> Looks to me the prohibition was to prevent a crash. It makes more sense >> to fix it. >> > > The crash can be fixed by just removing the check after finishing > issue17611. > > But is there any use case for 'continue'/'break'/'return' inside 'finally' > clause? The code like > > try: > return 1 > finally: > return 2 > > or > > try: > continue > finally: > break > > looks at least confusing. Currently 'break' and 'return' are never used > inside 'finally' clause in the stdlib. I would want to see a third-party > code that uses them. > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 567 v2
NLe 28 déc. 2017 11:20 AM, "Nathaniel Smith" a écrit :
On Thu, Dec 28, 2017 at 1:51 AM, Victor Stinner
wrote:
> var = ContextVar('var', default=42)
>
> and:
>
> var = ContextVar('var')
> var.set (42)
>
> behaves the same, no?
No, they're different. The second sets the value in the current
context. The first sets the value in all contexts that currently
exist, and all empty contexts created in the future.
Oh, that's an important information. In this case, "default" is the best
name.
The PEP may be more explicit about the effect on all contexts. Proposition
of documentation:
"The optional *default* parameter is the default value in all contexts. If the
variable is not set in the current context, it is returned by by
context[var_name] and by var.get(), when get() is called without the
default parameter."
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is static typing still optional?
On Tue, 26 Dec 2017 at 21:00 Ned Batchelder wrote: > On 12/26/17 1:49 PM, Chris Barker wrote: > > On Sat, Dec 23, 2017 at 5:54 PM, Nick Coghlan wrote: > >> >> I still wonder about the "fields *must* be annotated" constraint though. >> I can understand a constraint that the style be *consistent* (i.e. all >> fields as annotations, or all fields as field instances), since that's >> needed to determine the field order, but I don't see the problem with the >> "no annotations" style otherwise. >> > > IIUC, without annotations, there is no way to set a field with no default. > > And supporting both approaches violates "only one way to do it" in, I > think, a confusing manner -- particularly if you can't mix and match them. > > Also, could does using class attributes without annotations make a mess > when subclassing? -- no I haven't thought that out yet. > > > > I have not been following the design of dataclasses, and maybe I'm > misunderstanding the state of the work. My impression is that attrs was a > thing, and lots of people loved it, so we wanted something like it in the > stdlib. > Yes. > Data Classes is that thing, but it is a new thing being designed from > scratch. There are still big questions about how it should work, but it is > already a part of 3.7. > I wouldn't characterize it as "big questions". For some people there's a question as to how to make them work without type hints, but otherwise how they function is settled. > > Often when people propose new things, we say, "Put it on PyPI first, and > let's see how people like it." Why isn't that the path for Data Classes? > Why are they already part of 3.7 when we have no practical experience with > them? Wouldn't it be better to let the design mature with real > experience? Especially since some of the questions being asked are about > how it interrelates with another large new feature with little practical > use yet (typing)? > The short answer: "Guido said so". :) The long answer (based on my understanding, which could be wrong :) : Guido liked the idea of an attrs-like thing in the stdlib, but not attrs itself as Guido was after a different API. Eric V. Smith volunteered to work on a solution, and so Guido, Hynek, and Eric got together and discussed things at PyCon US. A design was hashed out, Eric went away and implemented it, and that led to the current solution. The only thing left is some people don't like type hints and so they don't want a stdlib module that requires them to function (there's no issue with how they relate *to* type hints, just how to make dataclasses work *without* type hints). So right now we are trying to decide what should represent the "don't care" type hint. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Heap allocate type structs in native extension modules?
On Thu, Dec 28, 2017, at 03:29, Erik Bray wrote: > On Tue, Dec 26, 2017 at 3:00 PM, Benjamin Peterson > wrote: > > I imagine Cython already takes care of this? > > This appears to have a distinct purpose, albeit not unrelated to > Cython. The OP's program would generate boilerplate C code for > extension types the rest of which would perhaps be implemented by hand > in C. Cython does this as well to an extent, but the generated code > contains quite a bit of Cython-specific cruft and is not really meant > to be edited by hand or read by humans in most cases. It still seems the OP is likely to reinvent a lot of Cython. One option is to write a bunch of "pure" .c and then only have the Python bindings in Cython. > > Anyways I don't think this answers the OP's question. I think this belongs on python-list anyway. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
