[Python-Dev] In support of PEP 649

[Samuel Colvin]

I've read the recent discussions

regarding PEP 649 and PEP 563 with interest, Larry Hastings recently
contacted me when prompted

to do so in a related discussion.

I maintain pydantic  which uses type
annotations to provide runtime data validation.

I'm extremely worried that if PEP 649 gets rejected, pydantic will not be
able to fully support python 3.10 and may even end up getting abandoned, at
best it will be much slower and more brittle.

Please, please put pragmatism over principle and accept PEP 649. As
discussed in this issue
, using
`typing.get_type_hints()` is not a good replacement for type annotations
that are accessible as python objects.

I know this is not a popularity contest, but I think it's worth explaining
how widely used pydantic is for those who don't use it or haven't heard of
it:

   - it's downloaded ~7m times a month and is the most popular standalone
   python data validation library by stars
   - it's used (in production and open source projects) by most of the big
   tech companies you've heard of - it's widely used both in web development
   and data science
   - it is one of the foundations of FastAPI
 which
   is by far the fastest growing python web framework and recently came third
   (behind django and flask) in web frameworks by usage in the python
   developer survey
   

Pydantic is not the only one, there are numerous other libraries that use
type annotations at runtime.

In short, you may not like it, but runtime inspection of type annotations
makes python better, and a massive and growing number of people rely on
that.

Rejecting PEP 649 would put that in peril, please accept it.

Samuel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7VMJWFGHVXDSRQFHMXVJKDDOVT47B54T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 563 in light of PEP 649

[Samuel Colvin]

Thanks so much for this, I think it makes a lot of sense. You've saved me
by explaining why PEP 563 semantics are problematic for pydantic far more
articulately than I could have done.

I can entirely see why PEP 563 made sense at the time, myself (and others
in the "runtime use of annotations" community) should have engaged with the
PEP before it was accepted, we might have been able to smooth out these
wrinkles then.

*One other route occurs to me:*

Switch from `from __future__ import annotations` to a per module feature
flag (say, something like `from __switches__ import postponed_annotations`).

I don't know if there's a precedent for this?, but I think it has a lot of
advantages:

* those who want type annotations as strings get their way (just a switch
of statement at the beginning of files)
* those who want runtime access to type annotations (pydantic etc.) get
their way
* if you have one module in a big code base that uses pydantic or similar,
you only need to pay the price of runtime type annotations in that file
* `from __future__ import annotations` can continue to prompt PEP 563
semantics (for now or indefinitely), independently it can also continue to
work for PEP 585 and PEP 604 as I believe it does now
* there's room for PEP 649 or similar in the future to improve forward refs
and performance when the switch is not set, without the rush to get it into
3.10
* we could even allow the default to be changed with an env variable or
command line flag like -O / PYTHONOPTIMIZE - but maybe this too complicated

I understand that adding another switch to python is not ideal, but given
that we are where we are, it seems like a pragmatic solution.

For me I'd be happy if there was anyway whatsoever to keep the current (or
PEP 649) behaviour in 3.10.

Samuel

--

Samuel Colvin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XCKF3JHD4APAJPL4QEYOGZFLKBCMBYSE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: In support of PEP 649

[Samuel Colvin]

Thank you everyone for your responses.

I entirely accept that I should have brought this up earlier, perhaps much 
earlier.

In my defence, when PEP 563 first came on my radar I assumed that 
get_type_hint() would be improved before it became the default behaviour, AFAIK 
it hasn't really changed. In particular the subtle changes to scope described 
in detail in PEP 649 [1] are still there and are still a big headache for 
pydantic.

As Paul Ganssle says, my timing was not great, but it could also have been 
worse.

I also never meant to create an "us versus them" style discussion, I was just 
worried about the narrow time window I had and the likelihood that my voice 
would not be heard, hence sounding as impassioned as I did.

For a fairly comprehensive description of why PEP 563 represents challenges to 
pydantic, I defer to Łukasz Langa (the instigator of PEP 563) who has explained 
it very clearly in another thread "[Python-Dev] PEP 563 in light of PEP 649" 
[2].

Samuel

[1] - https://www.python.org/dev/peps/pep-0649/#id9
[2] - 
https://mail.python.org/archives/list/python-dev@python.org/message/ZBJ7MD6CSGM6LZAOTET7GXAVBZB7O77O/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CJTAJJEXZXM5WKJ47POLK3YWWNPKK5NB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: In support of PEP 649

[Samuel Colvin]

There are a number of issues around recursive types, in general PEP 563
doesn't make a big difference either way in this case.

I think you mean something like

from pydantic import BaseModel
from typing import Optional

class Foo(BaseModel):
x: int
foo: Optional['Foo']

Foo.update_forward_refs()
print(Foo(x=1, foo={'x': 1}))

The only difference with "from __future__ import annotations" is you can
unquote Foo, e.g. Optional[Foo]. In all cases you still need
"Foo.update_forward_refs()" since Foo is not defined while creating Foo.

You could also use ForwardRef: "FooType =
ForwardRef('Foo')...Optional[FooType]", but that's slightly more verbose
and doesn't add anything.

(*Sidenote:* as I'm writing this, I realise you could know what Foo is
while creating Foo (same name -> reference to itself) and perhaps
automatically call update_forward_refs() at the end of
ModelMetaclass.__new__(), but you still need the
public update_forward_refs() for more complex cases; not least bodging
around PEP 563 scopes, see here

.)

Samuel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XSDPDMFRCW4726NN5ZN2UVXHWVZCNH7X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: In support of PEP 649

[Samuel Colvin]

>
> As far as I know, both Pydantic and marshmallow start using annotation
> for runtime type after PEP 563 is accepted. Am I right?


Not quite, pydantic was released in June 2017 (see HN post:
https://news.ycombinator.com/item?id=14477222) and always used annotations,
PEP 563 was created in September 2017, a few months later.

I can't speak about Marshmallow, back then I don't think it used
annotations at all, but maybe it does now.

When PEP 563 is accepted, there are some tools using runtime type in
> annotation, but they are not adopted widely yet.


Yes, that's completely fair.

Samuel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FVFL7FO4RPGSJPP5QJ2U4KBU6IWCT2YX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [python-committers] PEP 563 and Python 3.10.

[Samuel Colvin]

This is great news!

Thanks so much for hearing us and putting up with our last minute request.

I'm sure we can find a solution that, while not perfect, can satisfy most
of the people most of the time.

Samuel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I6XVK3AVNYVOSET3BXBC5SAJHRRPPU7G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] The cursed fourth Python 3.11 beta (3.11.0b4) is available

[Samuel Colvin]

Does anyone have the ear of someone at github?

It would make testing this beta much easier if they would merge the PR to
allow installing 3.11.b4 in github actions:

https://github.com/actions/python-versions/pull/177

Samuel

Also the GH bot is using DD/MM/ date format :-( whyy?

--

Samuel Colvin
07801160713


On Mon, 11 Jul 2022 at 20:32, Pablo Galindo Salgado 
wrote:

> BSD-style checksum format hashes for the release artefacts:
>
> SHA256 (python-3.11.0b4-embed-arm64.zip) =
> 272c6bb4948c597f6578f64c2b15a70466c5dfb49f9b84dba57a84e59e7bd4ef
> SHA256 (python-3.11.0b4-amd64.exe) =
> a3514b0401e6a85416f3e080586c86ccd9e2e62c8a54b9119d9e6415e3cadb62
> SHA256 (python-3.11.0b4-macos11.pkg) =
> 860647775d4e6cd1a8d71412233df5dbe3aa2886fc16d82a59ab2f625464f2d7
> SHA256 (python-3.11.0b4-embed-win32.zip) =
> 36b81da7986f8d59be61adb452681dbd3257ebb90bd89092b2fbbd9356e06425
> SHA256 (python-3.11.0b4-arm64.exe) =
> ad0d1429682ba1edc0c0cf87f68a3d1319b887b715da70a91db41d02be4997a4
> SHA256 (python-3.11.0b4-embed-amd64.zip) =
> 66e6bb44c36da36ecc1de64efdb92f52ba3a19221dba2a89e22e39f715bd205b
> SHA256 (Python-3.11.0b4.tar.xz) =
> 1d93b611607903e080417c1a9567f5fbbf5124cc5c86f4afbba1c8fd34c5f6fb
> SHA256 (python-3.11.0b4.exe) =
> 6febc152711840337f53e2fd5dc12bb2b1314766f591129282fd372c855fa877
> SHA256 (Python-3.11.0b4.tgz) =
> 257e753db2294794fa8dec072c228f3f53fd541a303de9418854b3c2512ccbec
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NOBNKLWKMJAHGLV6GVNITUBJ7OFJLIXY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QSB57XVNQ7F366ADCOZEAVSMC4TQALHW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] The cursed fourth Python 3.11 beta (3.11.0b4) is available

[Samuel Colvin]

Since a few people have reacted to my PS comment:

I suffered (committed) a very confusing typo; the github bot refers to
"07/13/2022" (e.g. MM/DD/) which drew my ire, I then confusingly
referenced a different format in my comment.

Overall, I agree we should be using ISO8601 for exactly this reason (at
least for dates, for datetimes ISO8601 gets pretty wacky
<https://ijmacd.github.io/rfc3339-iso8601/>)

Samuel

--

Samuel Colvin
07801160713


On Fri, 15 Jul 2022 at 05:39, Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> Alan G. Isaac writes:
>
>  > 4. It implements ISO 8601 (which exists for a reason):
>  > https://en.wikipedia.org/wiki/ISO_8601#Calendar_dates
>
> Yes!!!  "Standardization is my Valentine!" :-D
>
> --
> RIP WotR Bombshell
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/O7EKVHRG3GGAHYGQEPCUMPPYU5S67FGO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PQ2DVETX6TWQF4SVWLPGPU5VUL26MU5N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Switching to Discourse

[Samuel Colvin]

Hi, no I think "discord" refers to https://discord.com/

If discord is not finalised, we might also consider https://zulip.com/
which rust uses <https://rust-lang.zulipchat.com/> and would (based on a
very quick look) appear to be more appropriate for python development's use
case?

Samuel

--

Samuel Colvin


On Thu, 21 Jul 2022 at 18:06, Skip Montanaro 
wrote:

> I have a perhaps stupid question. Is Discord the same as
> discuss.python.org, just by another name? I find the similarity in
> names a bit confusing.
>
> Skip
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/B3WC35H3BMUNDVAOEJSGRAZOQCJ4KHD7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SNULJ757REYYBSKWC6E43TJVMK7CT4HY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Switching to Discourse

[Samuel Colvin]

Reading this thread and thinking about discuss.python.org/Discourse - I'm
surprised no one is advocating github discussions
<https://docs.github.com/en/discussions>.

In particular organisation discussions
<https://github.blog/changelog/2022-04-12-organization-discussions/> would
provide an obvious central place for discussions that would be easy to find
and use for everyone.

Advantages of github discussions:

   - Virtually all developers have a github account and are familiar with
   github & GFM
   - Github provides great support for participating or watching discussion
   via email - Discourse is really bad at this (at least by default)
   - GH discussions obviously integrate well with the rest of github -
   links to issues & pull requests (including other repos), discussions can be
   moved to other repos, issues can be created from discussions, issues can be
   converted to discussions - e.g. if someone creates a bug report which
   should really be a feature discussion
   - No extra service to maintain or pay for
   - GH discussions (unlike issues) provide good threading functionality
   without the full treeview madness of hackernews etc.

Before going "all in" with discuss.python.org/Discourse I think GH
discussions should be seriously considered.

Samuel

--

Samuel Colvin


On Fri, 15 Jul 2022 at 12:19, Petr Viktorin  wrote:

> Hello,
> Currently development discussions are split between multiple
> communication channels, for example:
> - python-dev and discuss.python.org for design discussions,
> - GitHub Issues and Pull Requests for specific changes,
> - IRC, Discord and private chats for real-time discussions,
> - Topic-specific channels like typing-sig.
>
> While most of these serve different needs, there is too much overlap
> between python-dev and discuss.python.org. It seems that for most
> people, this situation is worse than sticking to either one platform –
> even if we don't go with that person's favorite.
>
> The discuss.python.org experiment has been going on for quite a while,
> and while the platform is not without its issues, we consider it a
> success. The Core Development category is busier than python-dev.
> According to staff, discuss.python.org is much easier to moderate.. If
> you're following python-dev but not discuss.python.org, you're missing
> out.
>
> The Steering Council would like to switch from python-dev to
> discuss.python.org.
> Practically, this means:
> - Moving the required PEP announcements to discuss.python.org
> - Moving discuss.python.org up in the devguide communications page
> (https://devguide.python.org/communication/)
> - And that's it?
>
> I imagine that the mailing list will stay around for continuing past
> discussion threads and for announcements, eventually switching to
> auto-reject incoming messages with a pointer to discuss.python.org.
>
> To be clear, discuss.python.org allows editing posts, which is frankly
> handy for typos and clarifications. Editing alone should not be used for
> adding new info -- we should cultivate a culture of being friendly to
> mail users & notification watchers. This probably bears repeating in a
> few places.
>
> We're aware not everyone wants to use the discuss.python.org website,
> but there are some ways to avoid it:
>
> - For new PEPs, you can point your RSS client to
> https://www.python.org/dev/peps/peps.rss – it's not e-mail, but many
> email clients have RSS support. You can also watch the Steering Council
> issues on GitHub (https://github.com/python/steering-council/issues/)
> for important questions and discussions.
>
> - You can use discuss.python.org's “mailing list mode” (which subscribes
> you to all new posts), possibly with filtering and/or categorizing
> messages locally.
>
> However, we would like to know if this will pose an undue burden to
> anyone, if there are workflows or usage problems that we are not aware
> of. As mentioned, this is something the Steering Council thinks is a
> good idea, but we want to make sure we're aware of all the impact when
> we make the final decision.
>
>
>
> – Petr, on behalf of the Steering Council
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VHFLDK43DSSLHACT67X4QA3UZU73WYYJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RNLXOVCSVA63YIGTQRONBXBY4PROWNMJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Switching to Discourse

[Samuel Colvin]

Hi Brett, I understand your points.

I think the main point of difference is the gap in usability between GitHub
discussions and Discourse - I think it's massive, but I understand others
will be less enamoured by GitHub and less frustrated by Discourse than me.

One correction:

but that does make the discussion specific to the repo


With Organisation Discussions
<https://github.blog/changelog/2022-04-12-organization-discussions/>,
discussions are attached to the organisation, not a repr.

Samuel

--

Samuel Colvin


On Fri, 22 Jul 2022 at 19:45, Brett Cannon  wrote:

>
>
> On Fri, Jul 22, 2022 at 3:45 AM Samuel Colvin  wrote:
>
>> Reading this thread and thinking about discuss.python.org/Discourse -
>> I'm surprised no one is advocating github discussions
>> <https://docs.github.com/en/discussions>.
>>
>
> I think it's because discuss.python.org is what we decided to try years
> ago at the 2018 core dev sprint (so nearly 4 years ago), while GitHub
> Discussions would be a brand new thing to try and get people on board with.
>
>
>>
>> In particular organisation discussions
>> <https://github.blog/changelog/2022-04-12-organization-discussions/> would
>> provide an obvious central place for discussions that would be easy to find
>> and use for everyone.
>>
>> Advantages of github discussions:
>>
>>- Virtually all developers have a github account and are
>>familiar with github & GFM
>>
>>
> Discourse lets you log in via GitHub. I'm not sure if Discourse is
> straight Commonmark (probably is, though, since the co-creator of Discourse
> kicked off Commonmark because of Discourse).
>
>
>>
>>- Github provides great support for participating or watching
>>discussion via email - Discourse is really bad at this (at least by 
>> default)
>>- GH discussions obviously integrate well with the rest of github -
>>links to issues & pull requests (including other repos), discussions can 
>> be
>>moved to other repos, issues can be created from discussions, issues can 
>> be
>>converted to discussions - e.g. if someone creates a bug report which
>>should really be a feature discussion
>>
>>
> True, but that does make the discussion specific to the repo, which in
> this instance would be CPython and somewhat the language itself. This
> doesn't encompass something like packaging which has completely moved all
> development discussions over to discuss.python.org (and people have been
> generally happy with it). So I'm not sure if moving over to Discussions
> would actually lead to discuss.python.org going anywhere if you were
> trying to eliminate that need.
>
>
>>
>>- No extra service to maintain or pay for
>>
>> This is already true for discuss.python.org; Discourse is kindly
> donating the hosting on their SaaS platform.
>
>>
>>- GH discussions (unlike issues) provide good threading functionality
>>without the full treeview madness of hackernews etc.
>>
>> Before going "all in" with discuss.python.org/Discourse I think GH
>> discussions should be seriously considered.
>>
>
> If you can get people excited enough to say they are willing to give it a
> try, and the folks saying they are going to stop participating if/when we
> move to Discourse would actually stay if we moved to Discussions, then we
> can definitely talk about it.
>
> -Brett
>
>
>>
>> Samuel
>>
>> --
>>
>> Samuel Colvin
>>
>>
>> On Fri, 15 Jul 2022 at 12:19, Petr Viktorin  wrote:
>>
>>> Hello,
>>> Currently development discussions are split between multiple
>>> communication channels, for example:
>>> - python-dev and discuss.python.org for design discussions,
>>> - GitHub Issues and Pull Requests for specific changes,
>>> - IRC, Discord and private chats for real-time discussions,
>>> - Topic-specific channels like typing-sig.
>>>
>>> While most of these serve different needs, there is too much overlap
>>> between python-dev and discuss.python.org. It seems that for most
>>> people, this situation is worse than sticking to either one platform –
>>> even if we don't go with that person's favorite.
>>>
>>> The discuss.python.org experiment has been going on for quite a while,
>>> and while the platform is not without its issues, we consider it a
>>> success. The Core Development category is busier than python-dev.
>>> According to staff, discuss.python.org is much easier to moderate..