[Python-Dev] Seeking review: Reinstate ability to run python.org locally with local PEPs repo
The PEPs repo [1] has a README [2] with instructions for configuring a local installation of the python.org website such that you can develop PEPs locally and see how they will appear on python.org locally. However the README's instructions reference a python.org configuration setting (PEP_REPO_PATH) that appears to no longer exist. So I reimplemented support for the PEP_REPO_PATH setting for local python.org installations and have a pull request out for review at [3] which has been seeking a reviewer for about 2 weeks. Is there someone on this list (or at some other particular location) who would be willing to give me a review? [1]: https://github.com/python/peps [2]: https://github.com/python/peps/blob/master/README.rst [3]: https://github.com/python/pythondotorg/pull/1735 -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VYX5NIW2EDA7ZCKCURTK74U6RTQ6RGT3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] RFC on PEP 655: Required[] and NotRequired[] for TypedDict
This PEP [1] introduces syntax to mark individual keys of a TypedDict as either required or potentially-missing. Previously the only way to have a TypedDict with mixed required and non-required keys was to define two TypedDicts - one with total=True and one with total=False - and have one of those TypedDicts inherit from the other. This PEP introduces special forms Required[] and NotRequired[] to the "typing" module that can be used to directly mark individual items of a TypedDict as required or not-required. We have implementations for mypy, pyright, and pyanalyze. Consensus on the PEP has been reached in typing-sig. My understanding of the PEP process as described in PEP 1 [2] is that content review should be requested of core developers here in python-dev. So consider this my official request for comments! ^_^ I believe that after review & feedback from python-dev the next step would be to submit this PEP to the Steering Council. However it's not clear to me from [2] where I should actually do that when the time comes. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy [1]: https://www.python.org/dev/peps/pep-0655/ [2]: https://www.python.org/dev/peps/pep-0001/#pep-review-resolution ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C7QCZ5J277O5WHDJF5PFTA453DY7DS7S/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on PEP 655: Required[] and NotRequired[] for TypedDict
Hi folks, PEP 655 (Required[] and NotRequired[] for TypedDict) is still
looking for feedback from core devs.
I've copied the latest PEP text at the bottom of this email to make it
easier to comment on.
Thank you for your time.
Best,
--
David Foster | Seattle, WA, USA
Contributor to Python's type system
### BEGIN PEP 655 ###
PEP: 655
Title: Marking individual TypedDict items as required or potentially-missing
Author: David Foster
Sponsor: Guido van Rossum
Discussions-To: typing-sig at python.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30-Jan-2021
Python-Version: 3.11
Post-History: 31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021,
17-Jan-2022, 28-Jan-2022
Abstract
:pep:`589` defines syntax
for declaring a TypedDict with all required keys and syntax for defining
a TypedDict with :pep:`all potentially-missing keys <589#totality>`
however it
does not provide any syntax to declare some keys as required and others
as potentially-missing. This PEP introduces two new syntaxes:
``Required[]`` which can be used on individual items of a
TypedDict to mark them as required, and
``NotRequired[]`` which can be used on individual items
to mark them as potentially-missing.
Motivation
==
It is not uncommon to want to define a TypedDict with some keys that are
required and others that are potentially-missing. Currently the only way
to define such a TypedDict is to declare one TypedDict with one value
for ``total`` and then inherit it from another TypedDict with a
different value for ``total``:
::
class _MovieBase(TypedDict): # implicitly total=True
title: str
class Movie(_MovieBase, total=False):
year: int
Having to declare two different TypedDict types for this purpose is
cumbersome.
This PEP introduces two new type qualifiers, ``typing.Required`` and
``typing.NotRequired``, which allow defining a *single* TypedDict with
a mix of both required and potentially-missing keys:
::
class Movie(TypedDict):
title: str
year: NotRequired[int]
Rationale
=
One might think it unusual to propose syntax that prioritizes marking
*required* keys rather than syntax for *potentially-missing* keys, as is
customary in other languages like TypeScript:
::
interface Movie {
title: string;
year?: number; // ? marks potentially-missing keys
}
The difficulty is that the best word for marking a potentially-missing
key, ``Optional[]``, is already used in Python for a completely
different purpose: marking values that could be either of a particular
type or ``None``. In particular the following does not work:
::
class Movie(TypedDict):
...
year: Optional[int] # means int|None, not potentially-missing!
Attempting to use any synonym of “optional” to mark potentially-missing
keys (like ``Missing[]``) would be too similar to ``Optional[]``
and be easy to confuse with it.
Thus it was decided to focus on positive-form phrasing for required keys
instead, which is straightforward to spell as ``Required[]``.
Nevertheless it is common for folks wanting to extend a regular
(``total=True``) TypedDict to only want to add a small number of
potentially-missing keys, which necessitates a way to mark keys that are
*not* required and potentially-missing, and so we also allow the
``NotRequired[]`` form for that case.
Specification
=
The ``typing.Required`` type qualifier is used to indicate that a
variable declared in a TypedDict definition is a required key:
::
class Movie(TypedDict, total=False):
title: Required[str]
year: int
Additionally the ``typing.NotRequired`` type qualifier is used to
indicate that a variable declared in a TypedDict definition is a
potentially-missing key:
::
class Movie(TypedDict): # implicitly total=True
title: str
year: NotRequired[int]
It is an error to use ``Required[]`` or ``NotRequired[]`` in any
location that is not an item of a TypedDict.
It is valid to use ``Required[]`` and ``NotRequired[]`` even for
items where it is redundant, to enable additional explicitness if desired:
::
class Movie(TypedDict):
title: Required[str] # redundant
year: NotRequired[int]
It is an error to use both ``Required[]`` and ``NotRequired[]`` at the
same time:
::
class Movie(TypedDict):
title: str
year: NotRequired[Required[int]] # ERROR
The :pep:`alternative syntax <589#alternative-syntax>`
for TypedDict also supports
``Required[]`` and ``NotRequired[]``:
::
Movie = TypedDict('Movie', {'name': str, 'year': NotRequired[int]})
Interaction with ``total=False``
Any :pep:`589`-style TypedDict declared with ``total=False`` is equivalent
to a TypedDict with an implicit ``total=True`` definition with all of its
keys marked as ``NotRequired[]``.
Therefore:
::
class _MovieBase(TypedDict): # imp
[Python-Dev] Re: PEP 638: Syntactic macros
quot;Implementation" section below, it is mentioned that nodes in the`_ast` module would be made immutable. That sounds like a backward-incompatible change to me. (10) > Currently, all AST nodes are allocated using an arena allocator. > Changing to use the standard allocator might slow compilation down a little, > but has advantages in terms of maintenance, as much code can be deleted. I presume the arena allocator was introduced in the first place for a reason. Perhaps to improve performance? By removing the arena allocator are there potential downsides other than a performance regression? (11) > Reference Implementation > '''''''''''''''''''''''' > > None as yet. Seems like you could get a prototype off the ground by implementing an initial version as a fake Python source file text encoding. Then you could put something like `# coding=macros` at the top of a source file to have it preprocessed by a prototype macro system. (12) Thanks for taking the time to read my comments. -- David Foster | Seattle, WA, USA Contributor to mypy, TypedDict, and Python's type system [1]: https://docs.python.org/3/reference/grammar.html ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AW6KOQ2V3TS6TQUHS4VGFIKSR3SOLJCJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on PEP 655: Required[] and NotRequired[] for TypedDict
Thanks for the feedback Petr, - if this PEP really only affects typing.py and external projects/tools, it should say so clearly (so e.g. a parser experts can skip reading the PEP with clear conscience, even though it "introduces two new syntaxes") Did propose a new paragraph in the Abstract section that should make this clear: "No Python grammar changes are made by this PEP. Correct usage of required and potentially-missing keys of TypedDicts is intended to be enforced only by static type checkers and in particular is not enforced by Python itself at runtime." - in Specification, clarify what "It is an error" means -- is it a Python runtime error, or an error type checkers should raise? Same for "It is valid". It is a *logical* error. To be explicit: (1) It's definitely an error that type checkers MUST raise. (2) The runtime implementations of Required[] and NotRequired[] MAY also choose to raise errors at runtime on a best effort basis. In particular the prohibition on nesting Required[] and NotRequired[] inside each other is likely to be enforced at runtime by the __getitem__ functions on the type form implementations, because it is easy to do so. All of the preceding clarifications have been proposed in a PR: https://github.com/python/peps/pull/2388 Best, -- David Foster | Seattle, WA, USA Contributor to Python's type system ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3TWAKPPEORI2VPEAVNCC4GIOR22TAILQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC on PEP 655: Required[] and NotRequired[] for TypedDict
On 3/8/22 8:42 AM, Patrick Reader wrote: I think the names `Required` and `NotRequired` are too generic for something which can only be used in one context (`TypedDict`s), and > Could they be > put into a pseudo-module like `typing.io`, e.g. `TypedDict.Required`? It sounds like you are proposing something like: ``` from typing import TypedDict from typing.TypedDict import NotRequired # 👈 class Movie(TypedDict): title: str stars: NotRequired[int] ``` Is that really much different than: ``` from typing import NotRequired, TypedDict # 👈 class Movie(TypedDict): title: str stars: NotRequired[int] ``` Only the imports differ. And now users now have to specially remember that the "typing-related" NotRequired marker needs to imported not from `typing`, as is usual practice, but instead needs to be imported from somewhere else. it is not immediately obvious when reading code without context what the difference between `NotRequired` and `Optional` might be. Indeed PEP 655 §"How to Teach This" recommends not using both Optional and NotRequired in the same place, which might also mean the same file, and provides suggestions to avoid using Optional at all. Related: The word on the street is that "T|None" is likely to be a favored replacement for Optional in general going forward, since it's faster to type and doesn't require an extra import (of Optional from typing). Best, -- David Foster | Seattle, WA, USA Contributor to Python's type system ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZEO56ZSUVWHOPGN4MASUNVHBOHBINBG3/ Code of Conduct: http://python.org/psf/codeofconduct/
