[Python-Dev] Re: Concerns about PEP 634

[Chris Angelico]

On Sun, Feb 7, 2021 at 6:25 PM Paul Sokolovsky  wrote:
>
> Hello,
>
> On Sat, 6 Feb 2021 23:05:19 -0800
> Guido van Rossum  wrote:
>
> > That’s incorrect. __future__ is used when something new and
> > *incompatible* is being introduced (and the old way is being
> > deprecated at the same time). For experimental modules we use the
> > term provisional. There’s no official term for experimental syntax
> > (since we’ve never had any), but we could call that provisional as
> > well.
>
> Thanks, I'm aware. Thus the nature of my proposal is: redefine (extend)
> cases when __future__ is used, e.g. when a PEP itself says something
> like "There're many more could be done, but good things come in
> pieces, and wise men know when to stand back and relax before
> continuing. So, see ya next time!"
>
> My argument that such usage of __future__ would correspond more to the
> expectations of the Python end users ("feature is not fully developed
> yet, and there could be small incompatible changes going forward").

But future directives are not for things that aren't fully developed
yet. What gives people this idea?

For example, Python 2 code can say "from __future__ import
print_function" or "division". Was the print function in a provisional
state, with incompatible changes coming? No. Was the division operator
redefined? No. The directives cause compilation changes (removing a
keyword, using a different division operator) that were backward
incompatible *at launch* but they haven't changed since. In fact, part
of the promise of __future__ is that it WILL remain compatible - you
can write "from __future__ import nested_scopes" in a modern Python
script, and the behaviour will be exactly correct.

If you want a directive to put at the top of a script that says that
new features are being used, I'd suggest something like this:

#!/usr/bin/env python3.9

or whatever version you require.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KJ2X6P2BZOW6W4Y7S6SOULWCKYQUAIGL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Concerns about PEP 634

[Paul Sokolovsky]

Hello,

On Sun, 7 Feb 2021 19:09:14 +1100
Chris Angelico  wrote:

> On Sun, Feb 7, 2021 at 6:25 PM Paul Sokolovsky 
> wrote:
> >
> > Hello,
> >
> > On Sat, 6 Feb 2021 23:05:19 -0800
> > Guido van Rossum  wrote:
> >  
> > > That’s incorrect. __future__ is used when something new and
> > > *incompatible* is being introduced (and the old way is being
> > > deprecated at the same time). For experimental modules we use the
> > > term provisional. There’s no official term for experimental syntax
> > > (since we’ve never had any), but we could call that provisional as
> > > well.  
> >
> > Thanks, I'm aware. Thus the nature of my proposal is: redefine
> > (extend) cases when __future__ is used, e.g. when a PEP itself says
> > something like "There're many more could be done, but good things
> > come in pieces, and wise men know when to stand back and relax
> > before continuing. So, see ya next time!"
> >
> > My argument that such usage of __future__ would correspond more to
> > the expectations of the Python end users ("feature is not fully
> > developed yet, and there could be small incompatible changes going
> > forward").  
> 
> But future directives are not for things that aren't fully developed
> yet. What gives people this idea?
> 
> For example, Python 2 code can say "from __future__ import
> print_function" or "division". Was the print function in a provisional
> state, with incompatible changes coming? No. 

So, you're saying that, by the benevolence of divine providence,
most (can you truly vouch for "all" and provide evidence?) features so
far added to __future__ never were changed (enough).

From that, you derive the conclusion that only things that can never
change can be added to __future__. (You can even point to a yellowish
paper where something like that is written).

But that vision doesn't much correspond to reality. The world is dynamic
and ever-changing. It's good luck that simple print() function never
changed beyond its original definition and "/" for ints wasn't cast
back to return ints. But pattern matching is much more complex than
that, and knowing that there were definitely bugfixes for both print()
and "/", we can estimate that pattern matching will need only more,
including some externally-visible fixes. 

So, for as long as there was (and is!) "from __future__ import
with_statement", there can also be "from __future__ import
match_statement".

> Was the division operator
> redefined? No. The directives cause compilation changes (removing a
> keyword, using a different division operator) that were backward
> incompatible *at launch* but they haven't changed since. In fact, part
> of the promise of __future__ is that it WILL remain compatible - you
> can write "from __future__ import nested_scopes" in a modern Python
> script, and the behaviour will be exactly correct.

All that would point that we need something like "from __experimental__
import ...". But I don't go that far. I think that existing __future__
is good enough for the purpose.

> If you want a directive to put at the top of a script that says that
> new features are being used, I'd suggest something like this:
> 
> #!/usr/bin/env python3.9
> 
> or whatever version you require.

That doesn't say that this particular module uses experimental pattern
matching feature. It also has a side effect of pinning a script to a
particular executable, which may not yet exist on some users' systems,
or already not exists on other users' systems.

> 
> ChrisA


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UCCTGZJ6TCICLX77YXPY7XIIK6PC3STR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Concerns about PEP 634

[Chris Angelico]

On Sun, Feb 7, 2021 at 7:54 PM Paul Sokolovsky  wrote:
>
> So, you're saying that, by the benevolence of divine providence,
> most (can you truly vouch for "all" and provide evidence?) features so
> far added to __future__ never were changed (enough).

No, I'm saying that the __future__ directive is not for the purpose of
allowing subsequent changes. It is completely unrelated to any future
changes, it is a matter of bringing the future *now*.

> From that, you derive the conclusion that only things that can never
> change can be added to __future__. (You can even point to a yellowish
> paper where something like that is written).

Uhh well, that's not quite right; everything is subject to change. But
those changes are the domain of the subsequent proposals, not the
__future__ directive. The statement in a Py2 program "from __future__
import unicode_literals" was not for the purpose of permitting PEP 393
or PEP 414, both of which changed Unicode strings in Python 3.

> But that vision doesn't much correspond to reality. The world is dynamic
> and ever-changing. It's good luck that simple print() function never
> changed beyond its original definition and "/" for ints wasn't cast
> back to return ints. But pattern matching is much more complex than
> that, and knowing that there were definitely bugfixes for both print()
> and "/", we can estimate that pattern matching will need only more,
> including some externally-visible fixes.

So? If there are changes to be made, then let a subsequent proposal
make those changes.

> All that would point that we need something like "from __experimental__
> import ...". But I don't go that far. I think that existing __future__
> is good enough for the purpose.

Maybe that would be useful, if the match statement were deemed to be
an experimental feature. That's not the current proposal. Are you
proposing that this be done, and if so, why?

> > If you want a directive to put at the top of a script that says that
> > new features are being used, I'd suggest something like this:
> >
> > #!/usr/bin/env python3.9
> >
> > or whatever version you require.
>
> That doesn't say that this particular module uses experimental pattern
> matching feature. It also has a side effect of pinning a script to a
> particular executable, which may not yet exist on some users' systems,
> or already not exists on other users' systems.
>

Yes. If that's not what you want, then what IS it you want? Do you
need a comment saying "# this code uses the match statement" or "#
this code uses str.removeprefix" or "# this code uses positional-only
parameters"? What is gained by such directives?

I'm done arguing about this. Every time this PEP comes up, it gets yet
another round of the same arguments and I keep getting suckered into
answering them again. When will I learn...

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XJUTUE27XA5RH4EO2IYLJHUYVDNVKCGJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Concerns about PEP 634

[Paul Sokolovsky]

Hello,

On Sun, 7 Feb 2021 20:02:48 +1100
Chris Angelico  wrote:

> On Sun, Feb 7, 2021 at 7:54 PM Paul Sokolovsky 
> wrote:
> >
> > So, you're saying that, by the benevolence of divine providence,
> > most (can you truly vouch for "all" and provide evidence?) features
> > so far added to __future__ never were changed (enough).  
> 
> No, I'm saying that the __future__ directive is not for the purpose of
> allowing subsequent changes. It is completely unrelated to any future
> changes, it is a matter of bringing the future *now*.

And nobody says that there will be *incompatible* changes to the
pattern matching, just that previous experience, and discussion of the
PEP622/PEP634 shows that there's a probability of that (higher than
usual).

In that regard, it may be due to warn users about what pattern matching
is - a novelty. And among mechanisms Python already has for that,
__future__ seems like the best fit (because users already treat it like
that, regardless of what format definition is).

> > From that, you derive the conclusion that only things that can never
> > change can be added to __future__. (You can even point to a
> > yellowish paper where something like that is written).  
> 
> Uhh well, that's not quite right; everything is subject to change. But
> those changes are the domain of the subsequent proposals, not the
> __future__ directive. The statement in a Py2 program "from __future__
> import unicode_literals" was not for the purpose of permitting PEP 393
> or PEP 414, both of which changed Unicode strings in Python 3.

If everything is subject to change, then perhaps we can accept usage of
__future__ to introduce initial pattern matching implementation. But
its purpose is not to enable pattern matching per se, just to keep early
adopters in loop that it's a novel feature, subject to change.

> > But that vision doesn't much correspond to reality. The world is
> > dynamic and ever-changing. It's good luck that simple print()
> > function never changed beyond its original definition and "/" for
> > ints wasn't cast back to return ints. But pattern matching is much
> > more complex than that, and knowing that there were definitely
> > bugfixes for both print() and "/", we can estimate that pattern
> > matching will need only more, including some externally-visible
> > fixes.  
> 
> So? If there are changes to be made, then let a subsequent proposal
> make those changes.

Yes, and let's communicate the possibility that future proposals will
very likely be there (there're already, e.g. PEP642), and they may make
slight incompatible changes - by requiring early adopters to use "from
__future__ import ...".

> > All that would point that we need something like "from
> > __experimental__ import ...". But I don't go that far. I think that
> > existing __future__ is good enough for the purpose.  
> 
> Maybe that would be useful, if the match statement were deemed to be
> an experimental feature. That's not the current proposal. Are you
> proposing that this be done, and if so, why?

Me - not. Someone else proposed to use __future__. I jumped on that
idea because I think it's a good compromise with both people who
say "patmatching misses some things, like merge it with them already"
and who say "there's seem to be discord, let's not merge until it's
cleared". Needless to say, user who like/want patmatching, shouldn't
feel upset about __future__ either. And it's a responsible act overall
to warn early adopters, again.

So, now let me ask you - I guess you're not against patmatching, but
seem to be concerned with __future__. Why, beyond pure
language-lawyering matters ("there's a county bill of 1873 which says
__future__ isn't used like that").

> > > If you want a directive to put at the top of a script that says
> > > that new features are being used, I'd suggest something like this:
> > >
> > > #!/usr/bin/env python3.9
> > >
> > > or whatever version you require.  
> >
> > That doesn't say that this particular module uses experimental
> > pattern matching feature. It also has a side effect of pinning a
> > script to a particular executable, which may not yet exist on some
> > users' systems, or already not exists on other users' systems.
> >  
> 
> Yes. If that's not what you want, then what IS it you want? Do you
> need a comment saying "# this code uses the match statement" or "#
> this code uses str.removeprefix" or "# this code uses positional-only
> parameters"? What is gained by such directives?

A more formal'ish "contract" between developers and users which says
"We ship you a new cool feature! But, there can be some changes going
forward yet." __future__ (among existing means) satisfies that, a
comment - no.

> I'm done arguing about this. Every time this PEP comes up, it gets yet
> another round of the same arguments and I keep getting suckered into
> answering them again. When will I learn...

Don't take that personally, the issue is developers vs endusers gap,
and the discussion happens 

[Python-Dev] Re: Concerns about PEP 634

[Mark Shannon]

Hi Daniel,

On 06/02/2021 7:47 pm, Daniel Moisset wrote:

Hi Mark,

I think some of these issues have already been raised and replied (even 
if no agreement has been reached). but this is a good summary, so let me 
reply with a summary of responses for this. >
On Sat, 6 Feb 2021 at 15:51, Mark Shannon > wrote:


Hi,

Since a decision on PEP 634 is imminent, I'd like to reiterate some
concerns that I voiced last year.

I am worried about the semantics and implementation of PEP 634.
I don't want to comment on the merits of pattern matching in
general, or
the proposed syntax in PEP 634 (or PEP 640 or PEP 642).

Semantics
-

1. PEP 634 eschews the object model, in favour of adhoc instance
checks,
length checks and attribute accesses.

This is in contrast to almost all of the the rest of the language,
which
uses special (dunder) methods:
    All operators,
    subscripting,
    attribute lookup,
    iteration,
    calls,
    tests,
    object creation,
    conversions,
    and the with statement

AFAICT, no justification is given for this.
Why can't pattern matching use the object model?


No one has said that "we can't". It's just that "we don't have to". The 
underlying mechanisms used by pattern matching (instance check, length, 
attribute access) already have their defined protocols and support 
within the object model. It's analogous as the way in which  iterable 
unpacking didn't need to define it's own object model special methods, 
because the existing iteration mechanism used in for loops was sufficient.


You seem to be jumping on my use of the word "can't".
I should have said
"Why *doesn't* PEP 634 use the object model?"

As for unpacking, it builds on iteration in a way that is clear and precise.

For example, I can desugar:

a, b = t

into:

try:
__tmp = iter(t)
except TypeError:
raise TypeError("cannot unpack non-iterable ...")
try:
__tmp_a = next(__tmp)
__tmp_b = next(__tmp)
except StopIteration:
raise ValueError("not enough values ...")
try:
next(__tmp)
except StopIteration:
pass
else:
raise raise ValueError("too many values ...")
a = __tmp_a; b = __tmp_b

Noting that variables starting "__tmp" are invisible.

Why not do something similar for PEP 634?




This does not exclude possible future extensions to the object model to 
include a richer protocol like described in 
https://www.python.org/dev/peps/pep-0622/#custom-matching-protocol 
(where it also describes why we're not proposing that *now*, why it can 
be done later, and why we think it's best to do it later)


I don't see how this is relevant. It is the semantics of PEP 634 as 
proposed that concerns me.





PEP 343 (the "with" statement) added the __enter__ and __exit__ methods
to the object model, and that works very well.


2. PEP 634 deliberately introduces a large area of undefined behaviour
into Python.


https://www.python.org/dev/peps/pep-0634/#side-effects-and-undefined-behavior

Python has, in general, been strict about not having undefined
behaviour.
Having defined semantics means that you can reason about programs, even
non-idiomatic ones.
[This is not unique to Python, it is really only C and C++ that have
areas of undefined behaviour]


The C standard uses a very peculiar definition of "undefined behaviour" 
(I'm not familiar with the C++ one to assert anything, I'll assume it's 
the same), where for certain set of programs, any resulting behaviour is 
valid, even at compile time (so a compiler that deletes all your files 
when trying to compile "void f() {int a[10]; a[10]=0;}" is standard 
compliant). Comparing that with the use of the term "undefined 
behaviour" in the PEP is not very useful, because even if they are the 
same words, they don't have the same meaning


If you want to compare it with the C standards, the term we'd use would 
be "implementation defined behaviour". Python has a lot of those. For 
example, the output of all these python programs can change between 
implementations (And some of these even change between versions of cpython):


  * print({3,2,1})
  * print(hash("hello"))
  * if id(1) == id(1): print("hello")
  * import sys; print(sys.getsizeof([]))

Some order of operations is also implementation dependent for example in
def foo():
    print(open("/etc/passwd")).read()
foo()

The moment where file closing happens is implementation dependent.


The existence of a few cases of undefined behaviour (e.g. race 
conditions) does not excuse adding a whole lot more. Especially when 
there is no good reason.




The section you linked to introduces behaviour in similar lines to all 
of the above.



I can see no good reason for adding undefined behaviour. It doesn't
help
anyone.


It helps for the point 3 that you mention (See below)


No it doesn't.
It is an impediment to performanc

[Python-Dev] PR Review request - bpo-41928: Add support for Unicode Path Extra Field in ZipFile

[Andrea Giudiceandrea via Python-Dev]

Hi Python Dev team,
I submitted a PR https://github.com/python/cpython/pull/23736 two months ago.

The PR, which fixes an issue https://bugs.python.org/issue41928 in ZipFile, is 
"awaiting core review".

Best regards.

Andrea Giudiceandrea
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/72TDFT6DXN2IMVBYLWWHKBXHXEV4CAAU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: What's up with assignment expression and tuples?

[Guido van Rossum]

Hi Paul,

I suggest that you just go straight to the PEP phase.

--Guido

On Thu, Feb 4, 2021 at 11:54 PM Paul Sokolovsky  wrote:

> Hello,
>
> Everyone knows how hard to find a compelling usecase for the assignment
> expression operator (":=", colloquially "walrus operator").
> https://www.python.org/dev/peps/pep-0572/ examples never felt
> compelling and we all remember the split it caused.
>
> I finally found a usecase where *not* using assignment expression is
> *much* worse than using it. I'm working on SSA (Static Single
> Assignment,
> https://en.wikipedia.org/wiki/Static_single_assignment_form) conversion
> of Python programs, where there's a need to "join" dataflow of values
> from different control flow paths using a special function (Phi
> function). This "joining" itself creates a new variable, and of course,
> the original variable was used in an expression. We've got
> assignment in expression, assignment expression operator to the rescue!
>
> With it, a simple loop like:
>
> 
> a = 0
> while a < 5:
> a += 1
> 
>
> becomes:
>
> 
> a0 = 0
> while (a1 := phi(a0, a2)) < 5:
> a2 = a1 + 1
> 
>
> So far, so good. But semantics of Phi function is parallel assignment.
> No problem with Python either, "a, b = b, c" is exactly parallel
> assignment. So, let's try example with 2 variables:
>
> 
> a = 0
> b = 10
> while a < 5:
> a += 1
> b += 1
> 
>
> becomes:
>
> 
> a0 = 0
> b0 = 10
> while ((a1, b1) := phi([a0, a2], [b0, b2]))[0] < 5:
> a2 = a1 + 1
> b2 = b1 + 1
> 
>
> But oops:
>
> > SyntaxError: cannot use assignment expressions with tuple
>
> To reproduce in the REPL:
>
> 
> >>> ((a, b) := (1, 2))
>   File "", line 1
> SyntaxError: cannot use assignment expressions with tuple
> 
>
> Why this accidental syntactic gap? Why assignment statement can do
> parallel assignment with a tuple on LHS, and assignment operator
> suddenly can't?
>
> Why the adhoc naming and conceptual shift on the AST level, when PEP572
> explicitly talks about *assignment operator*, but corresponding node on
> the AST level is called NamedExpr? Why look at assignment expression as
> "name of expression" instead of assignment expression per se?
>
> It's of course not a problem to recast:
>
> NamedExpr(expr target, expr value)
>
> to
>
> NamedExpr(expr* target, expr value)
>
> in the ASDL (and it works out of the box), the point is that it should
> have been ExprAssign from the start (folloing the AugAssign and
> AnnAssign tradition).
>
>
> --
> Best regards,
>  Paul  mailto:pmis...@gmail.com
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/6IFDG7PAFPHVPGULANOQDAHP2X743HCE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CATW4D6SP6ZYHBQYBIXGT7UWJCQ6F2GQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Concerns about PEP 634

[Steven D'Aprano]

On Sun, Feb 07, 2021 at 09:58:03AM +0300, Paul Sokolovsky wrote:

> > A feature doesn't need to be a `__future__` import for you to just
> > not use it.
> 
> Right, and the talk is about the opposite - early adopters of
> PEP634-style pattern matching should mark the modules in which they
> *use* PEP-634-style pattern matching, so it will be easy in the future
> to spot such modules, if we get updated pattern matching style.

Because `grep "from __future__ import match"` is easier than 
`grep "match:"` ?


> All features added via __future__ were new, and thus experimental.

They were new, they were not experimental.


> For
> example, they contained long trail of bugs and issues, which were
> haunting their users for a long time, but at least the users knew what
> to expect, seeing the __future__ import.

Paul, that is bullshit. I use that in the technical sense:

http://www2.csudh.edu/ccauthen/576f12/frankfurt__harry_-_on_bullshit.pdf

Please just stop.

There was no "long trial of bugs and issues" involving (e.g.) unicode 
literals, true division, nested scopes, print as a function etc; there 
may have been a few enhancements for the with statement, e.g. the long 
running desire to be able to parenthesise context managers.

In the context of the Python community, "experimental" has a specific 
meaning: features which are intentionally documented as having an 
unstable API which may be subject to change. But I'm pretty sure you 
know that.

It absolutely does not refer to the addition of a new stable feature, or 
the normal process of discovering impementation bugs, enhancement 
requests, or even the occasional unforeseen problem which may need 
resolving. But I'm pretty sure you know that too.

I don't know what users you have in mind if you think that every new 
feature needs to go through a redundant and pointless future import. 
Users who get paid by the line, and adding that one extra line of code 
earns them an extra dollar? Users who love unnecessary boilerplate at 
the top of every file? Users who don't interact with beginners who don't 
understand why they can't use the new feature?

Users who want to impose an extra burden on the entire community, 
starting with the core devs who have to implement the unnecessary future 
import, and going on all the way down to every single coder who says to 
themselves "Oh, I forgot the future import!" when their code fails.

Future imports were designed carefully and thoughtfully as a mechanism 
to gradually introduce **backwards incompatible** syntactic changes. 
They are not there just to add friction to every single new syntactic 
feature.



-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AGXHWKSHCOH5ZCSIWYU4KFKKZVS4CIBV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Concerns about PEP 634

[Steven D'Aprano]

On Sun, Feb 07, 2021 at 12:32:40PM +0300, Paul Sokolovsky wrote:

> And nobody says that there will be *incompatible* changes to the
> pattern matching, just that previous experience, and discussion of the
> PEP622/PEP634 shows that there's a probability of that (higher than
> usual).

Fine. The time to involve `__future__` is if and when such incompatible 
changes are to be made, not for every new feature "just in case".


> In that regard, it may be due to warn users about what pattern matching
> is - a novelty.

Every new feature is a novelty. Who are these users that you have in 
mind that are sophisticated enough to be using pattern matching and 
future imports, but not sophisticated enough to realise that it is new 
in version 3.10 (or whatever version it ends up being)?

We don't need to "warn users" that new features are new features. They 
will already know it is new, until such time that it's no longer new.


> If everything is subject to change, then perhaps we can accept usage of
> __future__ to introduce initial pattern matching implementation. But
> its purpose is not to enable pattern matching per se, just to keep early
> adopters in loop that it's a novel feature, subject to change.

I don't believe that there is any proposal to make pattern matching 
subject to change beyond the usual evolution of the language. I don't 
think that there is anything provisional about the feature.

But even if there is, that still doesn't make it necessary to "warn" 
uses of the feature by use of future imports.

Paul, your argument is that "early adopters" of new features need to be 
protected from hypothetical future changes that don't exist yet. Why 
should only early adopters get that benefit?

Don't the rest of us deserve protection from hypothetical future changes 
that don't exist yet?

from __past__ import for_loops, import

"just in case" some hypothetical future change affects for loops and 
imports. How else will we know that our code uses for loops and 
imports? /s

-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CI66GDT6SGVWOP7CWMQZSD5XTEMTUNU3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Concerns about PEP 634

[Oscar Benjamin]

On Sat, 6 Feb 2021 at 19:54, Daniel Moisset  wrote:
>
> Hi Mark,
>
> I think some of these issues have already been raised and replied (even if no 
> agreement has been reached). but this is a good summary, so let me reply with 
> a summary of responses for this.
>
> On Sat, 6 Feb 2021 at 15:51, Mark Shannon  wrote:
>>
>> Hi,
>>
>> Since a decision on PEP 634 is imminent, I'd like to reiterate some
>> concerns that I voiced last year.
>>
>> I am worried about the semantics and implementation of PEP 634.
>> I don't want to comment on the merits of pattern matching in general, or
>> the proposed syntax in PEP 634 (or PEP 640 or PEP 642).
>>
>> Semantics
>> -
>>
>> 1. PEP 634 eschews the object model, in favour of adhoc instance checks,
>> length checks and attribute accesses.
>>
>> This is in contrast to almost all of the the rest of the language, which
>> uses special (dunder) methods:
>>All operators,
>>subscripting,
>>attribute lookup,
>>iteration,
>>calls,
>>tests,
>>object creation,
>>conversions,
>>and the with statement
>>
>> AFAICT, no justification is given for this.
>> Why can't pattern matching use the object model?
>
>
> No one has said that "we can't". It's just that "we don't have to". The 
> underlying mechanisms used by pattern matching (instance check, length, 
> attribute access) already have their defined protocols and support within the 
> object model. It's analogous as the way in which  iterable unpacking didn't 
> need to define it's own object model special methods, because the existing 
> iteration mechanism used in for loops was sufficient.
>
> This does not exclude possible future extensions to the object model to 
> include a richer protocol like described in 
> https://www.python.org/dev/peps/pep-0622/#custom-matching-protocol (where it 
> also describes why we're not proposing that *now*, why it can be done later, 
> and why we think it's best to do it later)

I find that particular section of the PEP unsatisfying. The
illustrated protocol seems to be much more complex than anything I
would have imagined.

The thing that I find jarring about this PEP is that it seems to make
positional arguments second class to keyword arguments and attributes
at the same time as suggesting an equivalence that doesn't really
exist between keyword arguments and attributes. I raised something
related previously:
https://mail.python.org/archives/list/python-dev@python.org/thread/V6UC7QEG4WLQY6JBC4MEIK5UGF7X2GSD/

If I have a class that takes positional arguments like p = Point2D(x,
y) (example from the PEP) then in order to match it I have to define
attributes for x and y and the matcher will use p.x and p.y to match
against the "arguments" in `case Point2D(a, b)`. The PEP proposes to
use __match_args__ == ['x', 'y'] in order to translate the argument
positions 0 and 1 (of a and b) into attributes x and y and then use
getattr to compare them in the matching protocol. It is very likely
though that the Point2D class can efficiently return a tuple of
positional arguments like (x, y) that could be compared more directly
against (a, b) without any need for *attributes* to be involved. I
would have expected that __match_args__ would be used in reverse to
the way the PEP proposes: to translate keyword arguments from the
pattern to positional arguments on the object rather than positional
arguments in the pattern to attributes on the object.

The PEP says:

> Late in the design process, however, it was realized that the need for a 
> custom matching protocol was much less than anticipated. Virtually all the 
> realistic (as opposed to fanciful) uses cases brought up could be handled by 
> the built-in matching behavior, although in a few cases an extra guard 
> condition was required to get the desired effect.

The PEP introduces many special cases for builtin types rather than a
general extensible mechanism that can work for builtin types as well
as user-defined types. That means it can handle various common cases
but misses out on others. Referring to the thread I linked above
mentioning sympy if I have a class like Add with instances like Add(x,
y, z), Add(x, y, z, t) etc. It would seem very natural to me that I
should be able to match against that but it isn't possible to do that
directly because there is no way in PEP 622 to handle variadic
arguments when matching against a user defined class.

The PEP makes it possible to match variadic-ish "arguments" with
builtins like tuple, list, etc but that is done with special casing
rather than with a protocol that can be applied equally to user
defined types. Personally I would be happier with the PEP if it had a
clear protocol such that builtin types could support the protocol in
an efficient and natural way. If a protocol works well for builtin
types then that suggests that it can work well in general. If the only
way to make this work for builtin types is to give them special-case
support then that points to a d

[Python-Dev] Re: Dusting off PEP 533?

[Richard Levasseur]

I, too, think I ran into basically the same situation as Paul: An outer
async generator calls an inner async generator. The outer doesn't fully
consume the inner. The inner uses an async context manager. The inner CM
isn't exited when expected and resources aren't released when
required/expected.

FWIW, the analysis of PEP 533 seems spot on -- for synchronous code, while
still possible, it's harder to trigger thanks to the GC tricks. For
asynchronous code, it easily (and subtly) results in resources not being
released when expected. Or perhaps never at all? IIUC, a long-lived server
might just accumulate unfinished async generators indefinitely, or at least
until some OS resource was exhausted.

In my particular case, I have thousands of network requests to make, each
of which is potentially large and/or slow to complete in its entirety, so
I'm taking extra care to break early to minimize resource usage to both
client and servers. I spent about an entire day trying to figure why the
synchronous version of my code Just Worked, while the asynchronous version
simply refused to close things when I was done with them. It was only after
I started looking at third party libraries and stumbled upon a reference to
PEP 533 and a detailed blog post from 5 years ago

that
I had a clue what was going on. My point being, the feature PEP 533
describes would have made things Just Work in line with my expectations.


On Tue, Jan 5, 2021 at 8:35 AM Stestagg  wrote:

> For what it's worth, this recent issue: https://bugs.python.org/issue42762
> is somewhat related to the non-async aspect of 533. (generator func getting
> closed during __del__ and causing odd side-effects at seemingly random
> points in the execution flow)
>
> The example code provided in that issue is contrived, and involves
> catching unconditional exceptions and recursing, but following the code
> flow in that situation feels unreasonably hard, and deciding if it's a
> python bug or not is even harder.
>
> I'm not sure how easy an implementation of 533 would be that doesn't break
> a lot of existing code, but if one could be made, it might help make
> generator lifecycles more predictable.
>
>
>
>
> On Tue, Jan 5, 2021 at 1:08 PM Senthil Kumaran 
> wrote:
>
>> Hi Paul,
>>
>> > Per PEP 525, I can call aclose coroutine method to cleanup the
>> generator, but
>> > it requires the code iterating to be aware that that closing the
>> generator is
>> > necessary.
>>
>> How about treating this as a bug for the specific use case that you
>> mentioned,
>> rather than a complete addition of " PEP 533 -- Deterministic cleanup for
>> iterators". I am not certain why PEP 533 was deffered.
>>
>> I'd also suggest a documentation update if the details of aclose
>> requirement is
>> mentioned only in a PEP and not in the documentation.
>>
>> Thank you,
>> Senthil
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/EPKMYH7DH5G6YR3AQHZTQKHBQF46YXLC/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VB64QJAYU2CCLOHRCBUIQOHMS2DIQQCC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D7CO5ZMIPT3R4YO3OCWGAYG5PFC5S53I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dusting off PEP 533?

[Paul Bryan]

For what it's worth, I've increasingly considered creating a context in
a generator (sync or async) to be an antipattern. To this end, in my
case, I moved creation of the context outside of the generator. This
resulted in a more explicit contract, which actually simplified
reasoning about its behavior. 

On Sun, 2021-02-07 at 16:18 -0800, Richard Levasseur wrote:
> I, too, think I ran into basically the same situation as Paul: An
> outer async generator calls an inner async generator. The outer
> doesn't fully consume the inner. The inner uses an async context
> manager. The inner CM isn't exited when expected and resources aren't
> released when required/expected.
> 
> FWIW, the analysis of PEP 533 seems spot on -- for synchronous code,
> while still possible, it's harder to trigger thanks to the GC tricks.
> For asynchronous code, it easily (and subtly) results in resources
> not being released when expected. Or perhaps never at all? IIUC, a
> long-lived server might just accumulate unfinished async generators
> indefinitely, or at least until some OS resource was exhausted.
> 
> In my particular case, I have thousands of network requests to make,
> each of which is potentially large and/or slow to complete in its
> entirety, so I'm taking extra care to break early to minimize
> resource usage to both client and servers. I spent about an entire
> day trying to figure why the synchronous version of my code Just
> Worked, while the asynchronous version simply refused to close things
> when I was done with them. It was only after I started looking at
> third party libraries and stumbled upon a reference to PEP 533 and a
> detailed blog post from 5 years ago that I had a clue what was going
> on. My point being, the feature PEP 533 describes would have made
> things Just Work in line with my expectations.
> 
> 
> On Tue, Jan 5, 2021 at 8:35 AM Stestagg  wrote:
> > For what it's worth, this recent issue:
> > https://bugs.python.org/issue42762  is somewhat related to the non-
> > async aspect of 533. (generator func getting closed during __del__
> > and causing odd side-effects at seemingly random points in the
> > execution flow)
> > 
> > The example code provided in that issue is contrived, and involves
> > catching unconditional exceptions and recursing, but following the
> > code flow in that situation feels unreasonably hard, and deciding
> > if it's a python bug or not is even harder.
> > 
> > I'm not sure how easy an implementation of 533 would be that
> > doesn't break a lot of existing code, but if one could be made, it
> > might help make generator lifecycles more predictable.
> > 
> > 
> > 
> > 
> > On Tue, Jan 5, 2021 at 1:08 PM Senthil Kumaran
> >  wrote:
> > > Hi Paul,
> > > 
> > > > Per PEP 525, I can call aclose coroutine method to cleanup the
> > > generator, but
> > > > it requires the code iterating to be aware that that closing
> > > the generator is
> > > > necessary.
> > > 
> > > How about treating this as a bug for the specific use case that
> > > you mentioned,
> > > rather than a complete addition of " PEP 533 -- Deterministic
> > > cleanup for
> > > iterators". I am not certain why PEP 533 was deffered.
> > > 
> > > I'd also suggest a documentation update if the details of aclose
> > > requirement is
> > > mentioned only in a PEP and not in the documentation.
> > > 
> > > Thank you,
> > > Senthil
> > > ___
> > > Python-Dev mailing list -- python-dev@python.org
> > > To unsubscribe send an email to python-dev-le...@python.org
> > > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > > Message archived at
> > >
> >
> https://mail.python.org/archives/list/python-dev@python.org/message/EPKMYH7DH5G6YR3AQHZTQKHBQF46YXLC/
> > > Code of Conduct: http://python.org/psf/codeofconduct/
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> >
> https://mail.python.org/archives/list/python-dev@python.org/message/VB64QJAYU2CCLOHRCBUIQOHMS2DIQQCC/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/D7CO5ZMIPT3R4YO3OCWGAYG5PFC5S53I/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4FCYB63LCWI5D7AGRRDHEEXE5IIB2SAZ/