[Python-Dev] Re: PEP 647 (type guards) -- final call for comments
I think it's a reasonable criticism that it's not obvious that a function annotated with a return type of `TypeGuard[x]` should return a bool. That said, the idea of a user-defined type guard comes from TypeScript, where the syntax is described [here](https://www.typescriptlang.org/docs/handbook/advanced-types.html#user-defined-type-guards). As you can see, the return type annotation here is also not a boolean. To my knowledge, that has not been a barrier for TypeScript developers. As Guido said, it's something that a developer can easily look up if they are confused about what it means. I'm open to alternative formulations that meet the following requirements: 1. It must be possible to express the type guard within the function signature. In other words, the implementation should not need to be present. This is important for compatibility with type stubs and to guarantee consistent behaviors between type checkers. 2. It must be possible to annotate the input parameter types _and_ the resulting (narrowed) type. It's not sufficient to annotate just one or the other. 3. It must be possible for a type checker to determine when narrowing can be applied and when it cannot. This implies the need for a bool response. 4. It should not require changes to the grammar because that would prevent this from being adopted in most code bases for many years. Mark, none of your suggestions meet these requirements. Gregory, one of your suggestions meets these requirements: ```python def is_str_list(val: Constrains[List[object]:List[str]) -> bool: ... ``` So, the question is whether this is more understandable and readable than this: ```python def is_str_list(val: List[object]) -> TypeGuard[List[str]]: ... ``` Both of these approaches would require a developer to do a search to understand the meaning. The former also introduces a novel use of the slice notation, which I find very unintuitive. Between these two, I find the latter to be clearly preferable, but that's admittedly a subjective opinion. As for choosing the name of the annotation... Most annotations in Python are nouns, for good reason. (There are a few exceptions like `Optional` that are adjectives.) For that reason, I'm not a fan of `Narrows`. I definitely wouldn't use `Constrains` because there's already a meaning in the Python type system for a "constrained type variable" (a TypeVar can constrained to two or more different types). `TypeGuard` is the term that is used in other languages to describe this notion, so it seems reasonable to me to adopt this term rather than making up a new term. Yes, it's introducing a new term that most Python users are not yet familiar with, but I can tell you from experience that very few Python developers know what "type narrowing" means. Some education will be required regardless of the formulation we choose. Steven, you said you'd like to explore a decorator-based formulation. Let's explore that. Here's what that it look like if we were to meet all of the above requirements. ```python @type_guard(List[str]) def is_str_list(val: List[object]) -> bool: ... ``` The problem here, as I mention in the "rejected ideas" section of the PEP, is that even with postponed type evaluations (as described in PEP 563), the interpreter cannot postpone the evaluation of an expression if it's used as the argument to a decorator. That's because it's not being used as a type annotation in this context. So while Mark is correct to point out that there has been a mechanism available for forward references since PEP 484, we've been trying to eliminate the use of quoted type expressions in favor of postponed evaluation. This would add a new case that can't be handled through postponed evaluation. Perhaps you still don't see that as a strong enough justification for rejecting the decorator-based formulation. I'm not entirely opposed to using a decorator here, but I think on balance that the `TypeGuard[x]` formulation is better. Once again, that's a subjective opinion. Paul said: >...to work around deficiencies in the current generation of Python typecheckers It sounds like you're implying that this functionality will be no longer needed at some point in the future when type checkers improve in some (unspecified) manner. If that's what you meant to convey, then I disagree. I think there will be an ongoing need for this functionality. There's good evidence for this in TypeScript, where user-defined type guards have been adopted widely by developers. -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.
[Python-Dev] Re: [EXTERNAL] PEP 647 Accepted
Hi Barry,
Thanks for the note. Apologies for the slow reply — your email got trapped in
Microsoft’s spam filters, and I just noticed it.
The idea of using a wrapper type was my first thought too. In fact, I
implemented that solution in prototype form. It was disliked by almost everyone
who tried to use the feature. The wrapper approach also got a negative reaction
on the typing-sig when I posted the initial proto-spec. A wrapper prevents some
common use cases (e.g. filter functions) and was found to be cumbersome and
confusing.
I understand your concern about the fact that type guards return bools but this
is not reflected in the return type. This was debated at length in the
typing-sig, and we considered various alternatives. In the end, we weren’t able
to come up with anything better. I’m somewhat comfited by the fact that
TypeScript’s formulation of this feature (which was the inspiration for the
idea and is generally a well-liked feature in that language) also does not
directly mention “boolean” in its return type annotation. Here’s an example of
the syntax in TypeScript:
```
function isNone(type: Type): type is NoneType {
return type.category === TypeCategory.None;
}
```
-Eric
On 4/6/21, 1:31 PM, "Barry Warsaw" wrote:
The Python Steering Council reviewed PEP 647 -- User-Defined Type Guards, and
is happy to accept the PEP for Python 3.10. Congratulations Eric!
We have one concern about the semantics of the PEP however. In a sense, the
PEP subverts the meaning of the return type defined in the signature of the
type guard, to express an attribute of the type guard function. Meaning, type
guard functions actually *do* return bools, but this is not reflected in the
return type:
"Using this new mechanism, the is_str_list function in the above example would
be modified slightly. Its return type would be changed from bool to
TypeGuard[List[str]]. This promises not merely that the return value is
boolean, but that a true indicates the input to the function was of the
specified type.”
In fact, the promise that it returns a bool is de-facto knowledge you must have
when you see “TypeGuard” in the return type. It is an implicit assumption.
Generally this might not be a problem, however when a type guard function is
used for multiple purposes (e.g. a type guard and a “regular” function), then
the return type is misleading, since a TypeGuard object is *not* returned.
It’s unclear what type checkers would do in this case.
The SC debated alternatives, including the decorator syntax specifically
mentioned in the Rejected Ideas. We also discussed making TypeGuard a
“wrapping” type defining an __bool__() so that e.g. is_str_list() would be
defined as such:
def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
"""Determines whether all objects in the list are strings"""
return TypeGuard(all(isinstance(x, str) for x in val))
but this also isn’t quite accurate, and we were concerned that this might be
highly inconvenient in practice. In a sense, the type guard-ness of the
function is an attribute about the function, not about the parameters or return
type, but there is no way to currently express that using Python or type
checking syntax.
I am not sure whether you considered and rejected this option, but if so,
perhaps you could add some language to the Rejected Ideas about it. Ultimately
we couldn’t come up with anything better, so we decided that the PEP as it
stands solves the problem in a practical manner, and that this is for the most
part a wart that users will just have to learn and internalize.
Cheers,
-Barry (on behalf of the Python Steering Council)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/LEXTUSWQOIH7P2XZ3OVXFAIIQC6NWX2E/
Code of Conduct: http://python.org/psf/codeofconduct/
