[Python-Dev] Re: RFC on PEP 655: Required[] and NotRequired[] for TypedDict
Thanks for the feedback Petr, - if this PEP really only affects typing.py and external projects/tools, it should say so clearly (so e.g. a parser experts can skip reading the PEP with clear conscience, even though it "introduces two new syntaxes") Did propose a new paragraph in the Abstract section that should make this clear: "No Python grammar changes are made by this PEP. Correct usage of required and potentially-missing keys of TypedDicts is intended to be enforced only by static type checkers and in particular is not enforced by Python itself at runtime." - in Specification, clarify what "It is an error" means -- is it a Python runtime error, or an error type checkers should raise? Same for "It is valid". It is a *logical* error. To be explicit: (1) It's definitely an error that type checkers MUST raise. (2) The runtime implementations of Required[] and NotRequired[] MAY also choose to raise errors at runtime on a best effort basis. In particular the prohibition on nesting Required[] and NotRequired[] inside each other is likely to be enforced at runtime by the __getitem__ functions on the type form implementations, because it is easy to do so. All of the preceding clarifications have been proposed in a PR: https://github.com/python/peps/pull/2388 Best, -- David Foster | Seattle, WA, USA Contributor to Python's type system ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3TWAKPPEORI2VPEAVNCC4GIOR22TAILQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on PEP 655: Required[] and NotRequired[] for TypedDict
On 30/01/2022 05:15, David Foster wrote: This PEP [1] introduces syntax to mark individual keys of a TypedDict as either required or potentially-missing. Previously the only way to have a TypedDict with mixed required and non-required keys was to define two TypedDicts - one with total=True and one with total=False - and have one of those TypedDicts inherit from the other. This PEP introduces special forms Required[] and NotRequired[] to the "typing" module that can be used to directly mark individual items of a TypedDict as required or not-required. We have implementations for mypy, pyright, and pyanalyze. Consensus on the PEP has been reached in typing-sig. My understanding of the PEP process as described in PEP 1 [2] is that content review should be requested of core developers here in python-dev. So consider this my official request for comments! ^_^ I believe that after review & feedback from python-dev the next step would be to submit this PEP to the Steering Council. However it's not clear to me from [2] where I should actually do that when the time comes. I think the names `Required` and `NotRequired` are too generic for something which can only be used in one context (`TypedDict`s), and it is not immediately obvious when reading code without context what the difference between `NotRequired` and `Optional` might be. Could they be put into a pseudo-module like `typing.io`, e.g. `TypedDict.Required`? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HSVVKLSFJAAONQXRFDZ4UUHRYKBM6GTU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] PSA: Linux vulnerability
https://arstechnica.com/information-technology/2022/03/linux-has-been-bitten-by-its-most-high-severity-vulnerability-in-years/ -- ~Ethan~ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FUJJ2MYILKOT2OFKBY44BZU7VMLAPCPE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] PEP 684: A Per-Interpreter GIL
I'd really appreciate feedback on this new PEP about making the GIL per-interpreter. The PEP targets 3.11, but we'll see if that is too close. I don't mind waiting one more release, though I'd prefer 3.11 (obviously). Regardless, I have no intention of rushing this through at the expense of cutting corners. Hence, we'll see how it goes. The PEP text is included inline below. Thanks! -eric === PEP: 684 Title: A Per-Interpreter GIL Author: Eric Snow Discussions-To: python-dev@python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 08-Mar-2022 Python-Version: 3.11 Post-History: 08-Mar-2022 Resolution: Abstract Since Python 1.5 (1997), CPython users can run multiple interpreters in the same process. However, interpreters in the same process have always shared a significant amount of global state. This is a source of bugs, with a growing impact as more and more people use the feature. Furthermore, sufficient isolation would facilitate true multi-core parallelism, where interpreters no longer share the GIL. The changes outlined in this proposal will result in that level of interpreter isolation. High-Level Summary == At a high level, this proposal changes CPython in the following ways: * stops sharing the GIL between interpreters, given sufficient isolation * adds several new interpreter config options for isolation settings * adds some public C-API for fine-grained control when creating interpreters * keeps incompatible extensions from causing problems The GIL --- The GIL protects concurrent access to most of CPython's runtime state. So all that GIL-protected global state must move to each interpreter before the GIL can. (In a handful of cases, other mechanisms can be used to ensure thread-safe sharing instead, such as locks or "immortal" objects.) CPython Runtime State - Properly isolating interpreters requires that most of CPython's runtime state be stored in the ``PyInterpreterState`` struct. Currently, only a portion of it is; the rest is found either in global variables or in ``_PyRuntimeState``. Most of that will have to be moved. This directly coincides with an ongoing effort (of many years) to greatly reduce internal use of C global variables and consolidate the runtime state into ``_PyRuntimeState`` and ``PyInterpreterState``. (See `Consolidating Runtime Global State`_ below.) That project has `significant merit on its own `_ and has faced little controversy. So, while a per-interpreter GIL relies on the completion of that effort, that project should not be considered a part of this proposal--only a dependency. Other Isolation Considerations -- CPython's interpreters must be strictly isolated from each other, with few exceptions. To a large extent they already are. Each interpreter has its own copy of all modules, classes, functions, and variables. The CPython C-API docs `explain further `_. .. _caveats: https://docs.python.org/3/c-api/init.html#bugs-and-caveats However, aside from what has already been mentioned (e.g. the GIL), there are a couple of ways in which interpreters still share some state. First of all, some process-global resources (e.g. memory, file descriptors, environment variables) are shared. There are no plans to change this. Second, some isolation is faulty due to bugs or implementations that did not take multiple interpreters into account. This includes CPython's runtime and the stdlib, as well as extension modules that rely on global variables. Bugs should be opened in these cases, as some already have been. Depending on Immortal Objects - :pep:`683` introduces immortal objects as a CPython-internal feature. With immortal objects, we can share any otherwise immutable global objects between all interpreters. Consequently, this PEP does not need to address how to deal with the various objects `exposed in the public C-API `_. It also simplifies the question of what to do about the builtin static types. (See `Global Objects`_ below.) Both issues have alternate solutions, but everything is simpler with immortal objects. If PEP 683 is not accepted then this one will be updated with the alternatives. This lets us reduce noise in this proposal. Motivation == The fundamental problem we're solving here is a lack of true multi-core parallelism (for Python code) in the CPython runtime. The GIL is the cause. While it usually isn't a problem in practice, at the very least it makes Python's multi-core story murky, which makes the GIL a consistent distraction. Isolated interpreters are also an effective mechanism to support certain concurrency models. :pep:`554` discusses this in more detail. Indirect Benefits - Most of the effort needed for a per-interpreter GIL has benefits that make those tasks worth doing anyway: * makes multiple-interpre
[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount" (round 3)
On Mon, Feb 28, 2022 at 6:01 PM Eric Snow wrote: > The updated PEP text is included below. The largest changes involve > either the focus of the PEP (internal mechanism to mark objects > immortal) or the possible ways that things can break on older 32-bit > stable ABI extensions. All other changes are smaller. In particular, I'm hoping to get your thoughts on the "Accidental De-Immortalizing" section. While I'm confident we will find a good solution, I'm not yet confident about the specific solution. So feedback would be appreciated. Thanks! -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2NNPKXRL6HY7IYUDMEQ6DS5RC3AYQKYQ/ Code of Conduct: http://python.org/psf/codeofconduct/
