[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Steve Holden]

On Mon, Nov 15, 2021 at 8:42 AM Kyle Stanley  wrote:

> On Sat, Nov 13, 2021 at 5:04 PM  wrote:
>
>>
>>
>> def 𝚑𝓮𝖑𝒍𝑜():
>>
> [... Python code it's easy to believe isn't grammatical ...]

> return ₛ
>>
>
> 0_o color me impressed, I did not think that would be legal syntax. Would
> be interesting to include in a textbook, if for nothing else other than to
> academically demonstrate that it is possible, as I suspect many are not
> aware.
>

I'm afraid the best Paul, Alex, Anna and I can hope to do is bring it to
the attention of readers of Python in a Nutshell's fourth edition (on
current plans, hitting the shelves about the same time as 3.11, please tell
your friends ;-) ). Sadly, I'm not aware of any academic classes that use
the Nutshell as a course text, so it seems unlikely to gain the
attention of academic communities.

Given the wider reach of this list, however, one might hope that by the
time the next edition comes out this will be old news due to the
publication of blogs and the like. With luck, a small fraction of the
programming community will become better-informed about Unicode and the
design or programming languages. It's interesting that the egalitarian wish
to allow use of native "alphabetics" has turned out to be such a viper's
nest.

Particular thanks to Stephen J. Turnbull for his thoughtful and
well-informed contribution above.

Kind regards,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FNSI6EXCWMMCXEJNYWVVR5LMFOM6M5ZB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Mark Shannon]

Hi,

I am surprised by the insistence on this thread for excluding comparisons from 
constant folding.
Why should we special case comparisons? Am I missing something here?

We already constant fold a variety of expressions

0 * 7
'' * 7
True - True
True * False

(All the above are falsey)

Excluding  1 < 2 seems inconsistent.

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BRQHDHZKNIMMXCA5SSAW2N3D57V2ZDZJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Antoine Pitrou]

On Mon, 29 Nov 2021 12:32:19 +
Mark Shannon  wrote:
> Hi,
> 
> I am surprised by the insistence on this thread for excluding comparisons 
> from constant folding.
> Why should we special case comparisons? Am I missing something here?

Is it actually special-cased or is it just not implemented?

IMHO at compile-time there's not much potential (for Python, which
doesn't have "true" constants).  At run-time and based on
specialization, there could be.

Regards

Antoine.


> 
> We already constant fold a variety of expressions
> 
> 0 * 7
> '' * 7
> True - True
> True * False
> 
> (All the above are falsey)
> 
> Excluding  1 < 2 seems inconsistent.
> 
> Cheers,
> Mark.



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/253RQ2BTTVUEUBFAMULQBBUTMQB5EH2W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Steven D'Aprano]

On Mon, Nov 29, 2021 at 12:32:19PM +, Mark Shannon wrote:
> Hi,
> 
> I am surprised by the insistence on this thread for excluding comparisons 
> from constant folding.
> Why should we special case comparisons? Am I missing something here?

We[1] are worried that the benefit gained will not be worth the 
maintenance burden of constant folding comparisons. Unlike 
constant-folding arithmetic expressions, the benefit for comparisons is 
small: code that compares two literals e.g. `3 < 5` is probably very 
rare, outside of tests. So this will help almost nobody, but still 
require maintenance.

(And the tests will need to be changed, if we add this, otherwise they 
will only be testing the keyhole optimizer, not the runtime comparison!)

I have no idea of how much maintenance the keyhole optimizer requires. 

We have a volunteer willing to do the work (Jeremy). Do we have a core 
developer willing to review their work, and mentor them if if it is not 
up to standard? If not, then this conversation will go nowhere.



[1] That's an editorial "we". Personally, I don't have an opinion one 
way or another :-)


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MUL7KTCPUGBTYZF4AXYI5UDE7UIC5ANR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Serhiy Storchaka]

29.11.21 14:32, Mark Shannon пише:
> Excluding  1 < 2 seems inconsistent.

It is not excluded, it is just not included. There should be reasons for
adding any feature, and the benefit should exceed the cost.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/53Q5J2MHMQRIJL4KD5FSWRAQICDQXDUT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Victor Stinner]

You should consider "no longer have to justify why it's not optimized"
as a clear benefit of making this change :-) This optimization is
proposed once a year for many years...

For me, any possible compilation-ahead optimization (which doesn't
break the Python semantics) is worth it ;-) It's done once, possible
even before the program is run for the first time, and then makes the
code faster.

Victor
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HEFGUVWHSN276YXEKEBOKYLUPFNLILTF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Serhiy Storchaka]

29.11.21 18:36, Victor Stinner пише:
> You should consider "no longer have to justify why it's not optimized"
> as a clear benefit of making this change :-) This optimization is
> proposed once a year for many years...
> 
> For me, any possible compilation-ahead optimization (which doesn't
> break the Python semantics) is worth it ;-) It's done once, possible
> even before the program is run for the first time, and then makes the
> code faster.

I consider the cost of rejecting such idea (this is actually the first
time it was pushed so persistent) less than the cost of implementing and
maintaining it. The proposed PR adds around 200 lines of code. It is
almost 20% of the current size of ast_opt.c, it is not small. And just
at first look I see a flaw in it -- it will emit a BytesWarning when
compare strings and bytes. After fixing this the size will grow even
more. And there may be other issues with this code which will be found
months later. Later, when we rewrite the optimizer or change the
structure of AST we will need to update this code too, with possibility
to introduce new bugs.

On other hand, the benefit of this optimization is exact zero. It does
not make code faster, because it optimizes the code not used in real
programs.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LOIK6ZX43QHQSNWWUSE6YE3O5YBVGJOM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Pablo Galindo Salgado]

I agree with Serhiy's analysis.

On Mon, 29 Nov 2021 at 17:10, Serhiy Storchaka  wrote:

> 29.11.21 18:36, Victor Stinner пише:
> > You should consider "no longer have to justify why it's not optimized"
> > as a clear benefit of making this change :-) This optimization is
> > proposed once a year for many years...
> >
> > For me, any possible compilation-ahead optimization (which doesn't
> > break the Python semantics) is worth it ;-) It's done once, possible
> > even before the program is run for the first time, and then makes the
> > code faster.
>
> I consider the cost of rejecting such idea (this is actually the first
> time it was pushed so persistent) less than the cost of implementing and
> maintaining it. The proposed PR adds around 200 lines of code. It is
> almost 20% of the current size of ast_opt.c, it is not small. And just
> at first look I see a flaw in it -- it will emit a BytesWarning when
> compare strings and bytes. After fixing this the size will grow even
> more. And there may be other issues with this code which will be found
> months later. Later, when we rewrite the optimizer or change the
> structure of AST we will need to update this code too, with possibility
> to introduce new bugs.
>
> On other hand, the benefit of this optimization is exact zero. It does
> not make code faster, because it optimizes the code not used in real
> programs.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LOIK6ZX43QHQSNWWUSE6YE3O5YBVGJOM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LSDOY5TLG3AEAPBGR7BAPI65LEMZ3QZ2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Christopher Barker]

On Mon, Nov 29, 2021 at 1:21 AM Steve Holden  wrote:

> It's interesting that the egalitarian wish to allow use of native
> "alphabetics" has turned out to be such a viper's nest.
>

Indeed.

However, is there no way to restrict identifiers at least to the alphabets
of natural languages? Maybe it wouldn’t help much, but does anyone need to
use letter-like symbols designed for math expressions? I would say maybe,
but certainly not have them auto-converted to the “normal” letter?

For that matter, why have any auto-conversion all?

The answer may be that it’s too late to change now, but I don’t think I’ve
seen a compelling (or any?) use case for that conversion.

-CHB


Particular thanks to Stephen J. Turnbull for his thoughtful and
> well-informed contribution above.
>
> Kind regards,
> Steve
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/FNSI6EXCWMMCXEJNYWVVR5LMFOM6M5ZB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H4QOXCLPBMGNGQHNFFQDYJQANUH6323C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Victor Stinner]

I guess that you're talking about
https://github.com/python/cpython/pull/29810/files

To be honest, I never looked into Python/ast_opt.c. I expected a
shorter implementation, a copy/paste or a few lines additions of "x ==
y" optimization. But it seems like "1 == 1" is not optimized neither.
Oh. I made the assumptions that other operations were already
optimized.

Victor

On Mon, Nov 29, 2021 at 6:11 PM Serhiy Storchaka  wrote:
>
> 29.11.21 18:36, Victor Stinner пише:
> > You should consider "no longer have to justify why it's not optimized"
> > as a clear benefit of making this change :-) This optimization is
> > proposed once a year for many years...
> >
> > For me, any possible compilation-ahead optimization (which doesn't
> > break the Python semantics) is worth it ;-) It's done once, possible
> > even before the program is run for the first time, and then makes the
> > code faster.
>
> I consider the cost of rejecting such idea (this is actually the first
> time it was pushed so persistent) less than the cost of implementing and
> maintaining it. The proposed PR adds around 200 lines of code. It is
> almost 20% of the current size of ast_opt.c, it is not small. And just
> at first look I see a flaw in it -- it will emit a BytesWarning when
> compare strings and bytes. After fixing this the size will grow even
> more. And there may be other issues with this code which will be found
> months later. Later, when we rewrite the optimizer or change the
> structure of AST we will need to update this code too, with possibility
> to introduce new bugs.
>
> On other hand, the benefit of this optimization is exact zero. It does
> not make code faster, because it optimizes the code not used in real
> programs.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/LOIK6ZX43QHQSNWWUSE6YE3O5YBVGJOM/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7LEERBQSXAFRFZKRGHUDO4RMDI6GODLE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Optimizing literal comparisons and contains

[Barry Warsaw]

I also agree with this analysis.  I’ll just add that any discussion about 
optimization needs to also show some data, for let’s say both contrived 
examples (like this one IMHO) and as real-world examples as possible.  
Sometimes “obvious” optimizations fool you and things actually get slower.  Or 
they don’t speed things up as much as you think they will.

Optimizations usually involve adding complexity.  I would like some sense of 
what we’d gain for the added complexity.

Cheers,
-Barry

> On Nov 29, 2021, at 09:04, Serhiy Storchaka  wrote:
> 
> 29.11.21 18:36, Victor Stinner пише:
>> You should consider "no longer have to justify why it's not optimized"
>> as a clear benefit of making this change :-) This optimization is
>> proposed once a year for many years...
>> 
>> For me, any possible compilation-ahead optimization (which doesn't
>> break the Python semantics) is worth it ;-) It's done once, possible
>> even before the program is run for the first time, and then makes the
>> code faster.
> 
> I consider the cost of rejecting such idea (this is actually the first
> time it was pushed so persistent) less than the cost of implementing and
> maintaining it. The proposed PR adds around 200 lines of code. It is
> almost 20% of the current size of ast_opt.c, it is not small. And just
> at first look I see a flaw in it -- it will emit a BytesWarning when
> compare strings and bytes. After fixing this the size will grow even
> more. And there may be other issues with this code which will be found
> months later. Later, when we rewrite the optimizer or change the
> structure of AST we will need to update this code too, with possibility
> to introduce new bugs.
> 
> On other hand, the benefit of this optimization is exact zero. It does
> not make code faster, because it optimizes the code not used in real
> programs.
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/LOIK6ZX43QHQSNWWUSE6YE3O5YBVGJOM/
> Code of Conduct: http://python.org/psf/codeofconduct/



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TZTA3NVDEKUMGZTK6QDD4R6F36E4NVMM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Barry Warsaw]

On Nov 26, 2021, at 01:13, Paul Moore  wrote:

> I'd therefore interpret Barry's plea as being for *anyone* with a use
> for annotations to provide their feedback (at least, anyone who
> accepts that annotations are types), with particular emphasis on
> people who want to use the types declared in annotations to affect
> runtime behaviour, as that's the most under-represented group at the
> moment (and it's not clear whether it's under-represented because
> there aren't many such uses, or because the users aren't being heard
> from).

Spot on.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4522JDS2ZOYXXFGPVXRGJ67IB3YLCEFM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Barry Warsaw]

I hope my understanding of where the SC’s debate about this currently sits 
isn’t a misrepresentation, but since I sent the email on behalf of the SC, let 
me try to clarify what I was trying to say.

Aside: A little insight into how the SC works.  For communications like this, 
after whatever debate we have in our live weekly meetings (and sometimes 
off-line discussions in Slack and email), one of us volunteers to write a draft 
of the email.  We then all comment and wordsmith until we’re in unanimous 
agreement that it’s ready to go out.  Then usually the person who wrote it will 
send the email.  We all have slightly different styles, but I usually sign it 
with my name “on behalf of the Steering Council”.  That’s my way of saying that 
the communication comes from all of us and represents the SC’s opinion, but it 
generally has the written style of the original drafter.

> On Nov 25, 2021, at 13:41, Christopher Barker  wrote:
> From Barry's email:
> """
> This is also a call to you, the folks who both care deeply about typing in 
> Python, and have a solid understanding of the subject matter (or willingness 
> to learn) to help us!  There are use cases we don’t know about, and unclear 
> requirements from both the static and dynamic typing users.
> """
> I don't care deeply about typing, and I'm not a static or dynamic typing 
> user. So this implies that a decision has been made that annotations are for 
> typing, and typing only.

Practically speaking, I think that’s current reality: annotations are for 
typing, be they static type checking or dynamic runtime uses.  I was 
inelegantly saying that even with that narrower interpretation of annotations, 
there are use cases, interactions, side-effects, semantics, and implications 
that we do not fully understand.  PEP 563 and 649 have visible effects that 
even within that domain can have important side effects.  For example, PEP 
563’s loss of local scope, which even “de-stringify-ing” can’t recover.  This 
is what we need help with.

> (not that you're not allowed to use for anything else, of course you are, but 
> that other uses won;t be taken into account when designing the new interface)
> 
> But I have never seen that clearly stated anywhere. The closest is from PEP 
> 563, where it says:
> 
> """
> With this in mind, uses for annotations incompatible with the aforementioned 
> PEPs should be considered deprecated.
> """
> 
> Which pretty much makes the point, but it's a bit subtle -- what does 
> "incompatible' mean?

You make a good point.  I agree that while all the signs are there for 
“annotations are for typing”, this has never been explicitly or sufficiently 
codified, and I’ve been proposing that Someone write a PEP that makes this an 
official pronouncement.  That someone may very well be a person on the SC, but 
given the timing of elections, it’s likely to fall to the next SC, if they 
still agree with that position! :-D

My recollection of the history of annotations falls somewhere between Greg’s 
and Guido’s.  Annotations as a feature were inspired by the typing use case 
(with no decision at the time whether those were to be static or runtime 
checks), but at the same time allowing for experimentation for other use cases. 
 Over time, annotations-for-typing clearly won the mindset and became the 
predominant use case.  Personally, I was strongly against type annotations 
because of my experience in other languages, but over time I was also won over. 
 From library documentation, to complex code bases, to the transient nature of 
contributors, type annotations are pretty compelling, and most (but not all) of 
my worries really didn’t come to fruition.

We can lament the non-typing use of annotations, but I think that horse is out 
of the barn and I don’t know how you would resolve conflicts of use for typing 
and non-typing annotations.  It’s been a slow boil, with no definitive 
pronouncement, and that needs to be fixed, but I think it's just acknowledging 
reality.  That’s my personal opinion.

> well, yes. The issue is that, intended or not, typing is making it's way into 
> Python culture. As an instructor of beginning python users, I am unsure at 
> this point when to introduce type annotations.
> 
> What is their role? Up to today, I have treated them as an advanced feature, 
> useful for "complex codebases". But there are any number of examples 
> springing up on the internet, to the point where many students now think they 
> are "best practice", if not actually required.

This is an important observation.  As much as I’m in the "type annotations are 
good” crowd now, I still think they should always be optional.  Python’s use is 
so broad these days, I for one don’t want to have to add annotations to every 
bit of Python I write.

-Barry




signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to p

[Python-Dev] Re: The current state of typing PEPs

[Larry Hastings]

On 11/29/21 2:56 PM, Barry Warsaw wrote:

PEP 563 and 649 have visible effects that even within that domain can have 
important side effects.  For example, PEP 563’s loss of local scope, which even 
“de-stringify-ing” can’t recover.  This is what we need help with.


Well, sure.  If PEP 563 and 649 didn't have visible effects, there'd be 
no point in doing them.  That said, I suggest 649 does a lovely job of 
avoiding /undesirable/ side-effects.


Sure, 649 has observable side effects.  For example, you can detect 
whether or not 649 is active by rebinding a name after it's used in an 
annotation but before examining that annotation at runtime.  This seems 
harmless--unlikely to happen in production code, and easily remedied if 
someone did trip over it.


A more credible side effect: if you use an undefined name in an 
annotation, you won't notice at compile-time.  Now, this is actually 
649's major feature!  But there are also scenarios where this feature 
could cover up a bug, like if you misspell a name--you won't notice 
until you examine the annotation at runtime (or, more likely, until you 
run your static analyzer).  563 has this same behavior--and it wasn't 
enough to prevent 563 being accepted.  So I assume this wouldn't be 
enough to prevent accepting 649 either.


649 has effects on memory usage and performance, but honestly I'm not 
worried about these.  I don't think the memory usage and performance of 
the prototype were particularly bad.  Anyway, as I've said many times: 
we should figure out the semantics we want first, and then we can worry 
about optimization.  The Python core dev community has no end of smart 
people who love optimizing things--I'm sure if 649 was accepted and 
merged, the optimizations would start rolling in.


Then of course there are also things 649 simply doesn't do, e.g. resolve 
the "if TYPE_CHECKING" situation.  But it's not appropriate to call that 
a "side effect" per se.


And that's my list.  If anybody knows of other visible side effects from 
649, naturally you should contact the SC.  And/or me, if you think we 
could change 649 to mitigate it without losing its major features.



Happy holidays,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XD6A3GG3BUW7V57FS36B5JPBH5OLN347/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Eric V. Smith]

Here's a use case for runtime type annotations: dataclasses use 
annotations to define fields. With the exception of ClassVar, the actual 
type is ignored. There's code in dataclasses.py to deal with stringized 
annotations, specifically just looking for ClassVar. I'd like to see 
this special case go away, assuming 649 is accepted.


And while dataclasses itself doesn't care about the types, apparently 
users do, and want to be able to call get_type_hints on the annotations. 
Many people actually want dataclasses to call this for them, but I've 
resisted. See https://bugs.python.org/issue36643


There are plenty of bugs around this issue, some of which have been 
fixed. But there are also at least a couple that haven't been fixed, and 
I'm not sure they will be. I'm hoping that 649 gets accepted and I don't 
have to worry about the problem at all. Here's an example of an open bug 
around get_type_hints with dataclasses: https://bugs.python.org/issue45524


Eric

On 11/29/2021 6:00 PM, Barry Warsaw wrote:

On Nov 26, 2021, at 01:13, Paul Moore  wrote:


I'd therefore interpret Barry's plea as being for *anyone* with a use
for annotations to provide their feedback (at least, anyone who
accepts that annotations are types), with particular emphasis on
people who want to use the types declared in annotations to affect
runtime behaviour, as that's the most under-represented group at the
moment (and it's not clear whether it's under-represented because
there aren't many such uses, or because the users aren't being heard
from).

Spot on.

-Barry


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4522JDS2ZOYXXFGPVXRGJ67IB3YLCEFM/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ENVEVQUEMEDKT2YUI2LQOBZDJC2YMNQ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Terry Reedy]

On 11/29/2021 5:56 PM, Barry Warsaw wrote:


On Nov 25, 2021, at 13:41, Christopher Barker  wrote:

What is their role? Up to today, I have treated them as an advanced feature, useful for 
"complex codebases". But there are any number of examples springing up on the internet, 
to the point where many students now think they are "best practice", if not actually 
required.



This is an important observation.  As much as I’m in the "type annotations are 
good” crowd now, I still think they should always be optional.  Python’s use is so 
broad these days, I for one don’t want to have to add annotations to every bit of 
Python I write.


Maybe it should be reiterated with whatever decision comes forth that

>>> def muladd(x, y, z):
... return x * (y+z)
...
>>> muladd(3.1459, 87.33, 2.7e2)
1124.124447
>>> muladd(3, 5, 7)
36
>>> from fractions import Fraction as Fr
>>> muladd(Fr(22, 7), 87, Fr(2714, 100))
Fraction(62777, 175)
>>> muladd(3, 'hel', 'lo ')
'hello hello hello '

and other duck-typed code will always be legal, idiomatic, and even 
expected as good practice for beginner, informal, exploratory, and 
similar python code.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R2HLTW5DZQGKCMJTTFTAKHX7EYQWNROG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Steven D'Aprano]

On Mon, Nov 29, 2021 at 07:56:16PM -0500, Terry Reedy wrote:

> Maybe it should be reiterated with whatever decision comes forth that
> 
> >>> def muladd(x, y, z):
> ... return x * (y+z)
> ...

[...]

> and other duck-typed code will always be legal, idiomatic, and even 
> expected as good practice for beginner, informal, exploratory, and 
> similar python code.

Many of the typing-related PEP comes with such a disclaimer, listed as 
"Non-goals". For example:

https://www.python.org/dev/peps/pep-0484/#non-goals

Łukasz's stringified annotations PEP has a non-goals section:

https://www.python.org/dev/peps/pep-0563/#non-goals

Larry's deferred evaluation PEP does not:

https://www.python.org/dev/peps/pep-0649/

but I don't think we should hold it against PEP-649. It's not trying to 
sneakily sneak mandatory static typechecking in by the back door like 
some sort of sneaking sneak :-)

-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XRBNYBEK6ZBZUC2KIDBGEYKYXZ2CBF4C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Guido van Rossum]

On Mon, Nov 29, 2021 at 5:01 PM Terry Reedy  wrote:

> On 11/29/2021 5:56 PM, Barry Warsaw wrote:
>
> > On Nov 25, 2021, at 13:41, Christopher Barker 
> wrote:
> >> What is their role? Up to today, I have treated them as an advanced
> feature, useful for "complex codebases". But there are any number of
> examples springing up on the internet, to the point where many students now
> think they are "best practice", if not actually required.
>
> > This is an important observation.  As much as I’m in the "type
> annotations are good” crowd now, I still think they should always be
> optional.  Python’s use is so broad these days, I for one don’t want to
> have to add annotations to every bit of Python I write.
>
> Maybe it should be reiterated with whatever decision comes forth that
>
>  >>> def muladd(x, y, z):
> ... return x * (y+z)
> ...
>  >>> muladd(3.1459, 87.33, 2.7e2)
> 1124.124447
>  >>> muladd(3, 5, 7)
> 36
>  >>> from fractions import Fraction as Fr
>  >>> muladd(Fr(22, 7), 87, Fr(2714, 100))
> Fraction(62777, 175)
>  >>> muladd(3, 'hel', 'lo ')
> 'hello hello hello '
>
> and other duck-typed code will always be legal, idiomatic, and even
> expected as good practice for beginner, informal, exploratory, and
> similar python code.
>

Why would it need to be reiterated? Are there really people who believe
that such code would become invalid? AFAIK *everybody* here agrees that
this should stay valid. So who would we be reiterating it for?

(Yes, several static type checkers have options that cause the checker to
complain about unannotated code. But that's not the default behavior and
running a static checker is a choice, like running a linter. Three-space
indents or capitalized function names are never going to be disallowed
either, even though PEP 8 says that's not how you ought to code.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5O2GJKTCQ7QUWLAUXKVWIGPTYZZNJ6EA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Terry Reedy]

On 11/29/2021 8:16 PM, Guido van Rossum wrote:
On Mon, Nov 29, 2021 at 5:01 PM Terry Reedy > wrote:


On 11/29/2021 5:56 PM, Barry Warsaw wrote:

 > On Nov 25, 2021, at 13:41, Christopher Barker
mailto:python...@gmail.com>> wrote:
 >> What is their role? Up to today, I have treated them as an
advanced feature, useful for "complex codebases".




But there are any
number of examples springing up on the internet, to the point where
many students now think they are "best practice", if not actually
required.


Chris, can you say anything more about why people would get such a 
mis-impression?



 > This is an important observation.  As much as I’m in the "type
annotations are good” crowd now, I still think they should always be
optional.  Python’s use is so broad these days, I for one don’t want
to have to add annotations to every bit of Python I write.

Maybe it should be reiterated with whatever decision comes forth that

  >>> def muladd(x, y, z):
...     return x * (y+z)
...
  >>> muladd(3.1459, 87.33, 2.7e2)
1124.124447
  >>> muladd(3, 5, 7)
36
  >>> from fractions import Fraction as Fr
  >>> muladd(Fr(22, 7), 87, Fr(2714, 100))
Fraction(62777, 175)
  >>> muladd(3, 'hel', 'lo ')
'hello hello hello '

and other duck-typed code will always be legal, idiomatic, and even
expected as good practice for beginner, informal, exploratory, and
similar python code.



Why would it need to be reiterated? Are there really people who believe 
that such code would become invalid? AFAIK *everybody* here agrees that 
this should stay valid. So who would we be reiterating it for?


The people that Chris B. referred to, and their 'sources'.  But we need 
more info to know how to best counter such wrong beliefs.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KLJ7UCGAWQ7X5EW6WPHBIH2XC3DIVEFZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Barry Warsaw]

On Nov 29, 2021, at 15:57, Larry Hastings  wrote:
> 
> On 11/29/21 2:56 PM, Barry Warsaw wrote:
>> PEP 563 and 649 have visible effects that even within that domain can have 
>> important side effects.  For example, PEP 563’s loss of local scope, which 
>> even “de-stringify-ing” can’t recover.  This is what we need help with.
> Well, sure.  If PEP 563 and 649 didn't have visible effects, there'd be no 
> point in doing them.  That said, I suggest 649 does a lovely job of avoiding 
> undesirable side-effects.
> 
> Sure, 649 has observable side effects.  For example, you can detect whether 
> or not 649 is active by rebinding a name after it's used in an annotation but 
> before examining that annotation at runtime.  This seems harmless--unlikely 
> to happen in production code, and easily remedied if someone did trip over it.

PEP 563 also has visible side effects, but the real implication of them wasn’t 
well understood at the time the PEP was approved.  Or maybe it was and we 
underestimated the impact on certain important users and use cases.  I’m 
particularly wary about getting into that same (um) pickle again.

> A more credible side effect: if you use an undefined name in an annotation, 
> you won't notice at compile-time.  Now, this is actually 649's major feature! 
>  But there are also scenarios where this feature could cover up a bug, like 
> if you misspell a name--you won't notice until you examine the annotation at 
> runtime (or, more likely, until you run your static analyzer).  563 has this 
> same behavior--and it wasn't enough to prevent 563 being accepted.  So I 
> assume this wouldn't be enough to prevent accepting 649 either.

How common is it for code to examine annotations at run time, outside of 
specialized libraries that have to be more defensive anyway (especially given 
they already have to be defensive in light of 563)?

> 649 has effects on memory usage and performance, but honestly I'm not worried 
> about these.  I don't think the memory usage and performance of the prototype 
> were particularly bad.  Anyway, as I've said many times: we should figure out 
> the semantics we want first, and then we can worry about optimization.  The 
> Python core dev community has no end of smart people who love optimizing 
> things--I'm sure if 649 was accepted and merged, the optimizations would 
> start rolling in.

I agree about that priority (semantics first, optimization later), as long as 
those semantics don’t make such future optimizations impossible, or severely 
degrade important performance constraints such as import times.

> Then of course there are also things 649 simply doesn't do, e.g. resolve the 
> "if TYPE_CHECKING" situation.  But it's not appropriate to call that a "side 
> effect" per se.

Although, it would be nice to have a comprehensive solution to this and other 
situations too, even if they happen outside of PEP 563/649 discussion…

> And that's my list.  If anybody knows of other visible side effects from 649, 
> naturally you should contact the SC.  And/or me, if you think we could change 
> 649 to mitigate it without losing its major features.

That list is exactly what I'm (we’re?) looking for.

-Barry




signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EJS5Z3LLIDHXEBJ6KEXWV2A3LVXEAAV5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Steve Dower]

On 11/30/2021 1:16 AM, Guido van Rossum wrote:
Why would it need to be reiterated? Are there really people who believe 
that such code would become invalid? AFAIK *everybody* here agrees that 
this should stay valid. So who would we be reiterating it for?


(Yes, several static type checkers have options that cause the checker 
to complain about unannotated code. But that's not the default behavior 
and running a static checker is a choice, like running a linter. 
Three-space indents or capitalized function names are never going to be 
disallowed either, even though PEP 8 says that's not how you ought to code.)


I know it sounds like a straw-man argument, but I don't think it is. I 
just think the non-straw-men making the argument aren't "important" 
enough to call out specifically - they are "random individual 
programmers who have gotten the wrong impression from another language 
and applied it to Python" rather than "influencers trying to change how 
we code".[1]


And we can always say that the *language* does not require running the 
type checker, but the reality for many developers is that the projects 
and teams they are part of will require it. And often, they require it 
because it's some kind of global "Best Practice" rather than because it 
makes sense for their particular situation.


For example, I saw a snippet fly by on Twitter the other day (again, not 
the fault of the individual who posted it, so I'm not 
naming-and-shaming) showing how to use Black to format a big directory 
of Jupyter notebooks. Convenient, perhaps, but the vast majority of 
people with "big directories of Jupyter notebooks" are going to be much 
happier if you just leave them alone! And yet, "The Community" is 
suggesting we should format all of them in bulk.


THAT'S the kind of thing that also has been happening with typing, and 
why some of us feel the need to publicly re-state things that are all 
agreed upon within this group, but are struggling to be heard over the 
public discourse on the topics. And this kind of reiteration is easier 
when our official documents (PEPs, etc.) state it explicitly.


Cheers,
Steve


[1]: Okay, I will call out the Python Bytes podcast - which I've been on 
a couple of times - as one that regularly opines on topics based on very 
little information and says things more definitively than they've been 
said anywhere else. They have a much bigger (and more easily influenced) 
audience than python-dev.


I'll also say that what I see from LWN is generally very good in this 
respect, their writers handle things well. Though I don't tend to read 
or listen to either of these that often.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SFLYLQO4YVQGV3BUKQO7KS6N367HNG62/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Inada Naoki]

On Thu, Nov 18, 2021 at 8:00 AM Barry Warsaw  wrote:
>
> Does PEP 563 or 649 satisfy static and dynamic typing needs?
>
> In the interest of full transparency, we want to let the Python community 
> know that the Steering Council continues to discuss PEP 563 (Postponed 
> Evaluation of Annotations) and PEP 649 (Deferred Evaluation Of Annotations 
> Using Descriptors).  We have concluded that we lack enough detailed 
> information to make a decision in favor of either PEP.  As we see it, 
> adopting either PEP 563 or PEP 649 as the default would be insufficient. They 
> will not fully resolve the existing problems these PEPs intend to fix, will 
> break some existing code, and likely don’t address all the use cases and 
> requirements across the static and dynamic typing constituents.  We are also 
> uncertain as to the best migration path from the current state of affairs.
>
> Defer decision on PEP 563 and 649 in 3.11
>
> As such, at this time, the only reasonable path forward that the SC sees is 
> to defer the decision in Python 3.11 again, essentially keeping the 3.10 
> status quo.  We know that this is far from ideal, but it’s also the safest 
> path since we can’t clearly make the situation better, and we don’t have 
> confidence that either PEP solves the problems once and for all.  
> Pragmatically, we don’t want to make the situation worse, and we really don’t 
> want to find ourselves back here again in a couple of releases because we 
> overlooked an important requirement for a set of users.
>

In my opinion, we can agree that we can not make PEP 563 default in
the future because it will break too many use cases.
Anyone against making a statement that "PEP 563 will never be the
default behavior"?

Then, we do not need to decide "PEP 563 or 649".
We can focus on whether we can replace "stock semantics + opt-in PEP
563" with PEP 649 or not.

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KCZKREGT66VE67MPT7NNJ5UPCACY6X7A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[David Mertz, Ph.D.]

On Mon, Nov 29, 2021 at 8:17 PM Guido van Rossum  wrote:

> Why would it need to be reiterated? Are there really people who believe
> that such code would become invalid? AFAIK *everybody* here agrees that
> this should stay valid. So who would we be reiterating it for?
>

I'm certainly not alone, among people on this list, in regard to the
following. But I teach a lot of people who are coders, but not necessarily
senior (in Python, or otherwise).  I also mentor/lead such junior
programmers.

I find it quite common when people who haven't known Python for 20 years
see type annotations, they have trouble getting the optional and gradual
nature of them.  Explaining that is certainly not impossible, nor even all
that difficult.  However, it DOES need to be explained.  Especially when
less experienced Pythonistas have worked with statically typed languages,
they often see code with type annotations and overgeneralize to their
requirement.

I don't know exactly what that means about *where* this needs to be
explained.  Maybe just by trainers and authors.  But maybe things in
official Python documentation as well, which seem more definitive.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H2P7KUV6IRZLKZ3G73RJGJN5U6NZOCYQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Larry Hastings]

On 11/29/21 7:10 PM, Inada Naoki wrote:

Anyone against making a statement that "PEP 563 will never be the
default behavior"?


I think only the SC is empowered to make such a statement.



Then, we do not need to decide "PEP 563 or 649".
We can focus on whether we can replace "stock semantics + opt-in PEP
563" with PEP 649 or not.


I doubt the current status quo--keeping PEP 563 as optional behavior, 
permanently--is viable long-term.  It causes problems for most runtime 
uses of annotations, which we were collectively basically ignoring until 
now.  As runtime use of annotations /and/ typing both become more 
pervasive, these problems are only going to grow.



//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JHOJWKZZWW4TB6TS7GX26WERKAEAC7VH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Christopher Barker]

I don't think the issue of how typing is making its way intot he community
is particular relevant to this thread, but I was directly asked a question,
so I'll answer it:

I don't know exactly where the impression is coming from that typing is a
best practice, but it's certainly creeping into example code, etc. So folks
will see examples of code that have nothing to do with typing, and probably
have no need for it, and there will be type annotations. Many (most?) folks
learn from examples more than targeted instruction, so as those examples
grow, so will the impression that it's an inherent part of Python.

David Mertz' experience is similar to mine as an instructor.

Another anecdotal example: I've been teaching an intro to Python class for
years, based on the same material, that we try to keep up to date. But
typing is not yet included. However, about a year ago, one of the other
instructors (that had not previously developed materials for the class)
added a new example / exercise to the "intro to classes" lesson. And that
example had type annotations. I ended up removing them, as I thought it
could be pretty confusing to students to see them for the first time with
no explanation (would they think it was something specific to classes?). I
wish I'd discussed it with the other instructor, I don't know why that was
done -- was it standard practice in their workplace? Seemed like a best
practice? I have no idea.

Another observation is that more and more library authors are being asked
to add type annotations -- maybe that IS a best practice for widely used
libraries, but that's one more piece of data.

Also: I know this isn't intentional, but Guido has an enormous influence on
the Python community -- so I suspect that the fact that he's taken an
active role in developing static typing has influenced how it's being
perceived.

I do think this is a topic for the community to grapple with, but not
really a Python-dev responsibility.

-CHB


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PTOBKJ6ILORMRORXLTL2WXADIOMO3AXX/
Code of Conduct: http://python.org/psf/codeofconduct/