Re: [Python-Dev] PEP 580 (C call protocol) draft implementation
Thanks, Jeroen. I haven't review your code yet, but benchmark shows no significant slowdown. It's good start! $ ./python -m perf compare_to master.json pep580.json -G --min-speed=5 Slower (6): - scimark_fft: 398 ms +- 20 ms -> 442 ms +- 42 ms: 1.11x slower (+11%) - xml_etree_process: 99.6 ms +- 5.2 ms -> 109 ms +- 16 ms: 1.10x slower (+10%) - crypto_pyaes: 138 ms +- 1 ms -> 149 ms +- 13 ms: 1.09x slower (+9%) - pathlib: 24.8 ms +- 1.8 ms -> 27.0 ms +- 3.8 ms: 1.09x slower (+9%) - spectral_norm: 155 ms +- 8 ms -> 165 ms +- 17 ms: 1.06x slower (+6%) - django_template: 151 ms +- 5 ms -> 160 ms +- 8 ms: 1.06x slower (+6%) Faster (6): - pickle_list: 5.37 us +- 0.74 us -> 4.80 us +- 0.34 us: 1.12x faster (-11%) - regex_v8: 29.5 ms +- 3.3 ms -> 27.1 ms +- 0.1 ms: 1.09x faster (-8%) - telco: 8.08 ms +- 1.19 ms -> 7.45 ms +- 0.16 ms: 1.09x faster (-8%) - regex_effbot: 3.84 ms +- 0.36 ms -> 3.56 ms +- 0.05 ms: 1.08x faster (-7%) - sqlite_synth: 3.98 us +- 0.53 us -> 3.72 us +- 0.07 us: 1.07x faster (-6%) - richards: 89.3 ms +- 9.9 ms -> 84.6 ms +- 5.7 ms: 1.06x faster (-5%) Benchmark hidden because not significant (48) Regards, On Sat, Jun 23, 2018 at 12:32 AM Jeroen Demeyer wrote: > Hello all, > > I have a first draft implementation of PEP 580 (introducing the C call > protocol): > > https://github.com/jdemeyer/cpython/tree/pep580 > > Almost all tests pass, only test_gdb and test_pydoc fail for me. I still > have to fix those. > > > Jeroen. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com > -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 02:24, Guido van Rossum wrote:
> A quick follow-up: PEP 572 currently has two ideas: (a) introduce := for
> inline assignment, (b) when := is used in a comprehension, set the scope for
> the target as if the assignment occurred outside any comprehensions. It
> seems we have more support for (a) than for (b) -- at least Nick and Greg
> seem to be +0 or better for (a)
Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
'NAME := EXPR'?" bothers me a bit, but it's the implementation
implications of parent local scoping that I fear will create a
semantic tar pit we can't get out of later.
> but -1 for (b). IIRC (b) originated with
> Tim. But his essay on the topic, included as Appendix A
> (https://www.python.org/dev/peps/pep-0572/#appendix-a-tim-peters-s-findings)
> does not even mention comprehensions. However, he did post his motivation
> for (b) on python-ideas, IIRC a bit before PyCon; and the main text of the
> PEP gives a strong motivation
> (https://www.python.org/dev/peps/pep-0572/#scope-of-the-target).
> Nevertheless, maybe we should compromise and drop (b)?
Unfortunately, I think the key rationale for (b) is that if you
*don't* do something along those lines, then there's a different
strange scoping discrepancy that arises between the non-comprehension
forms of container displays and the comprehension forms:
(NAME := EXPR,) # Binds a local
tuple(NAME := EXPR for __ in range(1)) # Doesn't bind a local
[NAME := EXPR] # Binds a local
[NAME := EXPR for __ in range(1)] # Doesn't bind a local
list(NAME := EXPR for __ in range(1)) # Doesn't bind a local
{NAME := EXPR} # Binds a local
{NAME := EXPR for __ in range(1)} # Doesn't bind a local
set(NAME := EXPR for __ in range(1)) # Doesn't bind a local
{NAME := EXPR : EXPR2} # Binds a local
{NAME := EXPR : EXPR2 for __ in range(1)} # Doesn't bind a local
set((NAME := EXPR, EXPR2) for __ in range(1)) # Doesn't bind a local
Those scoping inconsistencies aren't *new*, but provoking them
currently involves either class scopes, or messing about with
locals().
The one virtue that choosing this particular set of discrepancies has
is that the explanation for why they happen is the same as the
explanation for how the iteration variable gets hidden from the
containing scope: because "(EXPR for )" et al create an implicitly
nested scope, and that nested scope behaves the same way as an
explicitly nested scope as far as name binding and name resolution is
concerned.
Parent local scoping tries to mitigate the surface inconsistency by
changing how write semantics are defined for implicitly nested scopes,
but that comes at the cost of making those semantics inconsistent with
explicitly nested scopes and with the read semantics of implicitly
nested scopes.
The early iterations of PEP 572 tried to duck this whole realm of
potential semantic inconsistencies by introducing sublocal scoping
instead, such that the scoping for assignment expression targets would
be unusual, but they'd be consistently unusual regardless of where
they appeared, and their quirks would clearly be the result of how
assignment expressions were defined, rather than only showing up in
how they interacted with other scoping design decisions made years
ago.
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 09:25, Guido van Rossum wrote: > A "neutral" argument about (b) is that despite the "horrified" reactions > that Nick saw, in practice it's going to confuse very few people (again, due > to my point about Python's scope rules). I'd wager that the people who might > be most horrified about it would be people who feel strongly that the change > to the comprehension scope rules in Python 3 is a big improvement, and who > are familiar with the difference in implementation of comprehensions (though > not generator expressions) in Python 2 vs. 3. FWIW, the most cryptic parent local scoping related exception I've been able to devise so far still exhibits PEP 572's desired "Omitting the comprehension scope entirely would give you the same name lookup behaviour" semantics: >>> def outer(x=1): ... def middle(): ... return [x := x + i for i in range(10)] ... return middle() ... >>> outer() Traceback (most recent call last): ... NameError: free variable 'x' referenced before assignment in enclosing scope It isn't the parent local scoping, or even the assignment expression, that's at fault there, since you'd get exactly the same exception for: def outer(x=1): def middle(): x = x +1 return x return middle() Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 12:44, Nick Coghlan wrote:
> Unfortunately, I think the key rationale for (b) is that if you
> *don't* do something along those lines, then there's a different
> strange scoping discrepancy that arises between the non-comprehension
> forms of container displays and the comprehension forms:
I've been mostly ignoring this proposal for a while now, so I'm going
to respond here in the context of someone with a bit of an idea of the
underlying complexities, but otherwise coming at it as a new proposal.
>
> (NAME := EXPR,) # Binds a local
> tuple(NAME := EXPR for __ in range(1)) # Doesn't bind a local
>
> [NAME := EXPR] # Binds a local
> [NAME := EXPR for __ in range(1)] # Doesn't bind a local
> list(NAME := EXPR for __ in range(1)) # Doesn't bind a local
>
> {NAME := EXPR} # Binds a local
> {NAME := EXPR for __ in range(1)} # Doesn't bind a local
> set(NAME := EXPR for __ in range(1)) # Doesn't bind a local
>
> {NAME := EXPR : EXPR2} # Binds a local
> {NAME := EXPR : EXPR2 for __ in range(1)} # Doesn't bind a local
> set((NAME := EXPR, EXPR2) for __ in range(1)) # Doesn't bind a local
None of those "discrepancies" bother me in the slightest, when taken
in isolation as you present them here. I suspect you could lead me
through a chain of logic that left me understanding why you describe
them as discrepancies, but without that explanation, I'm fine with all
of them.
I'd also say that they seem contrived (not just in the use of
artificial names, but also in the sense that I'm not sure why I'd want
to use this *pattern*) so I'd happily say "well, don't do that then"
if things started behaving non-intuitively.
> Those scoping inconsistencies aren't *new*, but provoking them
> currently involves either class scopes, or messing about with
> locals().
And to reinforce my point above, I already consider putting
significant code in class scopes, or using locals() to be techniques
that should only be used sparingly and with a clear understanding of
the subtleties. I'm sure you could say "but the examples above would
be much more common" in response to which I'd like to see real use
cases that behave non-intuitively in the way you're concerned about.
> The one virtue that choosing this particular set of discrepancies has
> is that the explanation for why they happen is the same as the
> explanation for how the iteration variable gets hidden from the
> containing scope: because "(EXPR for )" et al create an implicitly
> nested scope, and that nested scope behaves the same way as an
> explicitly nested scope as far as name binding and name resolution is
> concerned.
But that's precisely why I find the behaviour intuitive - the nested
scope is the *reason* things behave this way, not some sort of
easily-overlooked way the "problem" can be explained away.
> Parent local scoping tries to mitigate the surface inconsistency by
> changing how write semantics are defined for implicitly nested scopes,
> but that comes at the cost of making those semantics inconsistent with
> explicitly nested scopes and with the read semantics of implicitly
> nested scopes.
>
> The early iterations of PEP 572 tried to duck this whole realm of
> potential semantic inconsistencies by introducing sublocal scoping
> instead, such that the scoping for assignment expression targets would
> be unusual, but they'd be consistently unusual regardless of where
> they appeared, and their quirks would clearly be the result of how
> assignment expressions were defined, rather than only showing up in
> how they interacted with other scoping design decisions made years
> ago.
Those last two paragraphs made my head explode, as far as I can see by
virtue of the fact that they try to over-analyze the fairly simple
intuition I have that "there's a nested scope involved".
Disclaimer: I may well have got a *lot* of subtleties wrong here, and
it's quite likely that my impressions don't stand up to the harsh
reality of how the implementation works. But my comments are on the
basis of my *intuition*, whether that's right or wrong. And if the
reality violates my intuition, it's *other* constructs that I find
non-intuitive, not this one. (I'm perfectly happy to concede that it's
not possible to avoid *any* non-intuitive behaviour - all I'm trying
to say is that my intuition doesn't balk at this one, unlike yours).
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 13:17, Nick Coghlan wrote: > On 25 June 2018 at 09:25, Guido van Rossum wrote: >> A "neutral" argument about (b) is that despite the "horrified" reactions >> that Nick saw, in practice it's going to confuse very few people (again, due >> to my point about Python's scope rules). I'd wager that the people who might >> be most horrified about it would be people who feel strongly that the change >> to the comprehension scope rules in Python 3 is a big improvement, and who >> are familiar with the difference in implementation of comprehensions (though >> not generator expressions) in Python 2 vs. 3. > > FWIW, the most cryptic parent local scoping related exception I've > been able to devise so far still exhibits PEP 572's desired "Omitting > the comprehension scope entirely would give you the same name lookup > behaviour" semantics: > > >>> def outer(x=1): > ... def middle(): > ... return [x := x + i for i in range(10)] > ... return middle() > ... > >>> outer() > Traceback (most recent call last): > ... > NameError: free variable 'x' referenced before assignment in enclosing > scope > > It isn't the parent local scoping, or even the assignment expression, > that's at fault there, since you'd get exactly the same exception for: > > def outer(x=1): > def middle(): > x = x +1 > return x > return middle() > Once again offering an "intuition" based response: 1. That definition of outer() is very complicated, I don't *expect* to understand it without checking the details. So the NameError is simply "hmm, wonder what triggered that?" not "OMG that's not what I'd expect!" :-) 2. Given that your version with no assignment expression or comprehension exhibits the same behaviour, I'm not sure what your argument is here anyway... Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 22:17, Nick Coghlan wrote: > FWIW, the most cryptic parent local scoping related exception I've > been able to devise so far still exhibits PEP 572's desired "Omitting > the comprehension scope entirely would give you the same name lookup > behaviour" semantics: > > >>> def outer(x=1): > ... def middle(): > ... return [x := x + i for i in range(10)] > ... return middle() > ... > >>> outer() > Traceback (most recent call last): > ... > NameError: free variable 'x' referenced before assignment in enclosing > scope > > It isn't the parent local scoping, or even the assignment expression, > that's at fault there, since you'd get exactly the same exception for: > > def outer(x=1): > def middle(): > x = x +1 > return x > return middle() Oops, I didn't mean to say "exactly the same exception" here, as the whole reason I'd settled on this example as the most cryptic one I'd found so far was the fact that the doubly nested version *doesn't* give you the same exception as the singly nested version: the version without the comprehension throws UnboundLocalError instead. However, the resolution is the same either way: either 'x' has to be declared as 'nonlocal x' in 'middle', or else it has to be passed in to 'middle' as a parameter. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 09:02, Guido van Rossum wrote: > On Sun, Jun 24, 2018 at 11:50 AM Steven D'Aprano > wrote: >> I will have more to say about the whole "comprehensions are their own >> scope" issue later. But I'd like to see Nick's proposed PEP, or at least >> a draft of it, before making any final decisions. > > > Agreed, though I assume it's just `given` again. While I still have some TODO notes of my own to resolve before posting it to python-ideas, the examples section at https://github.com/ncoghlan/peps/pull/2/files#diff-7a25ca1769914c1141cb5c63dc781f32R202 already gives a pretty good idea of the differences relative to PEP 572: rebinding existing names is unchanged from PEP 572, but introducing new names requires a bit of "Yes, I really do want to introduce this new name here" repetition. The big difference from previous iterations of the "given" idea is that it doesn't try to completely replace the proposed inline assignments, it just supplements them by providing a way to do inline name *declarations* (which may include declaring targets as global or nonlocal, just as regular function level declarations can). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25 June 2018 at 13:24, Nick Coghlan wrote: > On 25 June 2018 at 22:17, Nick Coghlan wrote: >> FWIW, the most cryptic parent local scoping related exception I've >> been able to devise so far still exhibits PEP 572's desired "Omitting >> the comprehension scope entirely would give you the same name lookup >> behaviour" semantics: >> >> >>> def outer(x=1): >> ... def middle(): >> ... return [x := x + i for i in range(10)] >> ... return middle() >> ... >> >>> outer() >> Traceback (most recent call last): >> ... >> NameError: free variable 'x' referenced before assignment in enclosing >> scope >> >> It isn't the parent local scoping, or even the assignment expression, >> that's at fault there, since you'd get exactly the same exception for: >> >> def outer(x=1): >> def middle(): >> x = x +1 >> return x >> return middle() > > Oops, I didn't mean to say "exactly the same exception" here, as the > whole reason I'd settled on this example as the most cryptic one I'd > found so far was the fact that the doubly nested version *doesn't* > give you the same exception as the singly nested version: the version > without the comprehension throws UnboundLocalError instead. At the level of "what my intuition says" the result is the same in both cases - "it throws an exception". I have no intuition on *which* exception would be raised and would experiment (or look up the details) if I cared. > However, the resolution is the same either way: either 'x' has to be > declared as 'nonlocal x' in 'middle', or else it has to be passed in > to 'middle' as a parameter. Once someone told me that's what I needed, it's sufficiently obvious that I'm fine with that. If no-one was able to tell me what to do, I'd simply rewrite the code to be less obfuscated :-) I've probably explained my intuition enough here. If we debate any further I'll just end up knowing what's going on, destroying my value as an "uninformed user" :-) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25.06.2018 2:30, Greg Ewing wrote: Guido van Rossum wrote: Greg seem to be +0 or better for (a) Actually, I'm closer to -1 on (a) as well. I don't like := as a way of getting assignment in an expression. The only thing I would give a non-negative rating is some form of "where" or "given". "as" was suggested even before is became a keyword in `with'. ( if (re.match(regex,line) as m) is not None: ) The only objective objection I've heard is it's already used in `import' and `with' -- but that's perfectly refutable. Brief summary of reasons for disliking ":=": * Cryptic use of punctuation * Too much overlap in functionality with "=" * Asymmetry between first and subsequent uses of the bound value * Makes expressions cluttered and hard to read to my eyes -- Regards, Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 25.06.2018 14:44, Nick Coghlan wrote:
On 25 June 2018 at 02:24, Guido van Rossum wrote:
A quick follow-up: PEP 572 currently has two ideas: (a) introduce := for
inline assignment, (b) when := is used in a comprehension, set the scope for
the target as if the assignment occurred outside any comprehensions. It
seems we have more support for (a) than for (b) -- at least Nick and Greg
seem to be +0 or better for (a)
Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
'NAME := EXPR'?" bothers me a bit, but it's the implementation
implications of parent local scoping that I fear will create a
semantic tar pit we can't get out of later.
but -1 for (b). IIRC (b) originated with
Tim. But his essay on the topic, included as Appendix A
(https://www.python.org/dev/peps/pep-0572/#appendix-a-tim-peters-s-findings)
does not even mention comprehensions. However, he did post his motivation
for (b) on python-ideas, IIRC a bit before PyCon; and the main text of the
PEP gives a strong motivation
(https://www.python.org/dev/peps/pep-0572/#scope-of-the-target).
Nevertheless, maybe we should compromise and drop (b)?
Unfortunately, I think the key rationale for (b) is that if you
*don't* do something along those lines, then there's a different
strange scoping discrepancy that arises between the non-comprehension
forms of container displays and the comprehension forms:
(NAME := EXPR,) # Binds a local
tuple(NAME := EXPR for __ in range(1)) # Doesn't bind a local
[NAME := EXPR] # Binds a local
[NAME := EXPR for __ in range(1)] # Doesn't bind a local
list(NAME := EXPR for __ in range(1)) # Doesn't bind a local
{NAME := EXPR} # Binds a local
{NAME := EXPR for __ in range(1)} # Doesn't bind a local
set(NAME := EXPR for __ in range(1)) # Doesn't bind a local
{NAME := EXPR : EXPR2} # Binds a local
{NAME := EXPR : EXPR2 for __ in range(1)} # Doesn't bind a local
set((NAME := EXPR, EXPR2) for __ in range(1)) # Doesn't bind a local
Those scoping inconsistencies aren't *new*, but provoking them
currently involves either class scopes, or messing about with
locals().
I've got an idea about this.
The fact is, assignments don't make much sense in an arbitrary part of a
comprehension:
`for' variables are assigned every iteration, so when the result is
returned, only the final value will be seen.
(And if you need a value every iteration, just go the explicit way and
add it to the returned tuple.)
Contrary to that, the "feeder" expression is only evaluated once at the
start -- there, assignments do make sense.
Effectively, it's equivalent to an additional line:
seq = range(calculate_b() as bottom, calculate_t() as top)
results = [calculate_r(bottom,r,top) for r in seq]
So, I suggest to evaluate the "feeder" expression in a local scope but
expressions that are evaluated every iteration in a private scope.
--
Regards,
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Not giving a vote, as I'm just a lurker, but: Le 25/06/2018 à 01:30, Greg Ewing a écrit : > > Actually, I'm closer to -1 on (a) as well. I don't like := as a > way of getting assignment in an expression. The only thing I would > give a non-negative rating is some form of "where" or "given". This resonates with me for a yet different reason: expressing the feature with a new operator makes it feel very important and fundamental, so that beginners would feel compelled to learn it early, and old-timers tend to have a strong gut reaction to it. Using merely a keyword makes it less prominent. Baptiste ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
[This is my one reply in this thread today. I am trying to limit the amount of time I spend to avoid another overheated escalation.] On Mon, Jun 25, 2018 at 4:44 AM Nick Coghlan wrote: > Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or > 'NAME := EXPR'?" bothers me a bit, but it's the implementation > implications of parent local scoping that I fear will create a > semantic tar pit we can't get out of later. > Others have remarked this too, but it really bother me that you are focusing so much on the implementation of parent local scoping rather than on the "intuitive" behavior which is super easy to explain -- especially to someone who isn't all that familiar (or interested) with the implicit scope created for the loop control variable(s). According to Steven (who noticed that this is barely mentioned in most tutorials about comprehensions) that is most people, however very few of them read python-dev. It's not that much work for the compiler, since it just needs to do a little bit of (new) static analysis and then it can generate the bytecode to manipulate closure(s). The runtime proper doesn't need any new implementation effort. The fact that sometimes a closure must be introduced where no explicit initialization exists is irrelevant to the runtime -- this only affects the static analysis, at runtime it's no different than if the explicit initialization was inside `if 0`. Unfortunately, I think the key rationale for (b) is that if you > *don't* do something along those lines, then there's a different > strange scoping discrepancy that arises between the non-comprehension > forms of container displays and the comprehension forms: > > (NAME := EXPR,) # Binds a local > tuple(NAME := EXPR for __ in range(1)) # Doesn't bind a local > [...] > Those scoping inconsistencies aren't *new*, but provoking them > currently involves either class scopes, or messing about with > locals(). > In what sense are they not new? This syntax doesn't exist yet. > The one virtue that choosing this particular set of discrepancies has > is that the explanation for why they happen is the same as the > explanation for how the iteration variable gets hidden from the > containing scope: because "(EXPR for )" et al create an implicitly > nested scope, and that nested scope behaves the same way as an > explicitly nested scope as far as name binding and name resolution is > concerned. > Yeah, but most people don't think much about that explanation. You left out another discrepancy, which is more likely to hit people in the face: according to your doctrine, := used in the "outermost iterable" would create a local in the containing scope, since that's where the outermost iterable is evaluated. So in this example a = [x := i+1 for i in range(y := 2)] the scope of x would be the implicit function (i.e. it wouldn't leak) while the scope of y would be the same as that of a. (And there's an even more cryptic example, where the same name is assigned in both places.) This is another detail of comprehensions that I assume tutorials (rightly, IMO) gloss over because it's so rarely relevant. But it would make the explanation of how := works in comprehensions more cumbersome: you'd have to draw attention to the outermost iterable, otherwise "inline assignment in comprehensions has the same scope as the comprehension's loop control variable(s)" would lead one to believe that y's scope above would also be that of the implicit function. > Parent local scoping tries to mitigate the surface inconsistency by > changing how write semantics are defined for implicitly nested scopes, > but that comes at the cost of making those semantics inconsistent with > explicitly nested scopes and with the read semantics of implicitly > nested scopes. > Nobody thinks about write semantics though -- it's simply not the right abstraction to use here, you've introduced it because that's how *you* think about this. > The early iterations of PEP 572 tried to duck this whole realm of > potential semantic inconsistencies by introducing sublocal scoping > instead, such that the scoping for assignment expression targets would > be unusual, but they'd be consistently unusual regardless of where > they appeared, and their quirks would clearly be the result of how > assignment expressions were defined, rather than only showing up in > how they interacted with other scoping design decisions made years > ago. > There was also another variant in some iteration or PEP 572, after sublocal scopes were already eliminated -- a change to comprehensions that would evaluate the innermost iterable in the implicit function. This would make the explanation of inline assignment in comprehensions consistent again (they were always local to the comprehension in that iteration of the PEP), at the cost of a backward incompatibility that was ultimately withdrawn. -- --Guido van Rossum (python.org/~guido) __
Re: [Python-Dev] Intent to accept PEP 561 -- Distributing and Packaging Type Information
OK, last call! I'll accept the current draft tomorrow unless someone pushes back. On Fri, Jun 22, 2018 at 8:37 AM Nick Coghlan wrote: > On 23 June 2018 at 01:16, Guido van Rossum wrote: > > That sounds like you're supporting PEP 561 as is, right? > > Aye, I'm personally fine with it - we do need to do something about > automatically reserving the derived names on PyPI, but I don't think > that's a blocker for the initial PEP acceptance (instead, it will go > the other way: PEP acceptance will drive Warehouse getting updated to > handle the convention already being adopted by the client tools). > > > Excuse my > > ignorance, but where are API testing stub interfaces described or used? > > They're not - it's just the context for Donald referring to "stubs" as > being a general technical term with other meanings beyond the "type > hinting stub file" one. > > As such, there's three parts to explaining why we're not worried about > the terminology clash: > > - Ethan searched for projects called "*-stubs" or "*_stubs" and didn't > find any, so the practical impact of any terminology clash will be low > - there isn't an established need to automatically find testing stub > libraries based on an existing project name the way there is for type > hints > - even if such a need did arise in the future, the "py.typed" marker > file and the different file extension for stub files within a package > still gives us an enormous amount of design flexibility > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 6/24/2018 7:25 PM, Guido van Rossum wrote: I'd wager that the people who might be most horrified about it the (b) scoping rule change would be people who feel strongly that the change to the comprehension scope rules in Python 3 is a big improvement, I might not be one of those 'most horrified' by (b), but I increasingly don't like it, and I was at best -0 on the comprehension scope change. To me, iteration variable assignment in the current scope is a non-problem. So to me the change was mostly useless churn. Little benefit, little harm. And not worth fighting when others saw a benefit. However, having made the change to nested scopes, I think we should stick with them. Or repeal them. (I believe there is another way to isolate iteration names -- see below). To me, (b) amounts to half repealing the nested scope change, making comprehensions half-fowl, half-fish chimeras. and who are familiar with the difference in implementation of comprehensions (though not generator expressions) in Python 2 vs. 3. That I pretty much am, I think. In Python 2, comprehensions (the fish) were, at least in effect, expanded in-line to a normal for loop. Generator expressions (the fowls) were different. They were, and still are, expanded into a temporary generator function whose return value is dropped back into the original namespace. Python 3 turned comprehensions (with 2 news varieties thereof) into fowls also, temporary functions whose return value is dropped back in the original namespace. The result is that a list comprehension is equivalent to list(generator_ expression), even though, for efficiency, it is not implemented that way. (To me, this unification is more a benefit than name hiding.) (b) proposes to add extra hidden code in and around the temporary function to partly undo the isolation. list comprehensions would no longer be equivalent to list(generator_expression), unless generator_expressions got the same treatment, in which case they would no longer be equivalent to calling the obvious generator function. Breaking either equivalence might break someone's code. --- How loop variables might be isolated without a nested scope: After a comprehension is parsed, so that names become strings, rename the loop variables to something otherwise illegal. For instance, i could become '', just as lambda becomes '' as the name of the resulting function. Expand the comprehension as in Python 2, except for deleting the loop names along with the temporary result name. Assignment expressions within a comprehension would become assignment expressions within the for loop expansion and would automatically add or replace values in the namespace containing the comprehension. In other words, I am suggesting that if we want name expressions in comprehensions to act as they would in Python 2, then we should consider reverting to an altered version of the Python 2 expansion. --- In any case, I think (b) should be a separate PEP linked to a PEP for (a). The decision for (a) could be reject (making (b) moot), accept with (b), or accept unconditionally (but still consider (b)). -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Tue, Jun 26, 2018 at 5:37 AM, Terry Reedy wrote: > How loop variables might be isolated without a nested scope: After a > comprehension is parsed, so that names become strings, rename the loop > variables to something otherwise illegal. For instance, i could become > '', just as lambda becomes '' as the name of the resulting > function. Expand the comprehension as in Python 2, except for deleting the > loop names along with the temporary result name. > > Assignment expressions within a comprehension would become assignment > expressions within the for loop expansion and would automatically add or > replace values in the namespace containing the comprehension. In other > words, I am suggesting that if we want name expressions in comprehensions to > act as they would in Python 2, then we should consider reverting to an > altered version of the Python 2 expansion. So. sublocal scopes, like in the earliest versions of PEP 572? The wheel turns round and round, and the same spokes come up. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 6/25/2018 8:25 AM, Paul Moore wrote:
On 25 June 2018 at 12:44, Nick Coghlan wrote:
Unfortunately, I think the key rationale for (b) is that if you
*don't* do something along those lines, then there's a different
strange scoping discrepancy that arises between the non-comprehension
forms of container displays and the comprehension forms:
I've been mostly ignoring this proposal for a while now, so I'm going
to respond here in the context of someone with a bit of an idea of the
underlying complexities, but otherwise coming at it as a new proposal.
(NAME := EXPR,) # Binds a local
tuple(NAME := EXPR for __ in range(1)) # Doesn't bind a local
Of course not, in local scopes where is it not executed. But it would,
in the nested function where the assignment *is* executed. Ditto for
all of the following.
[NAME := EXPR] # Binds a local
[NAME := EXPR for __ in range(1)] # Doesn't bind a local
list(NAME := EXPR for __ in range(1)) # Doesn't bind a local
{NAME := EXPR} # Binds a local
{NAME := EXPR for __ in range(1)} # Doesn't bind a local
set(NAME := EXPR for __ in range(1)) # Doesn't bind a local
{NAME := EXPR : EXPR2} # Binds a local
{NAME := EXPR : EXPR2 for __ in range(1)} # Doesn't bind a local
set((NAME := EXPR, EXPR2) for __ in range(1)) # Doesn't bind a local
None of those "discrepancies" bother me in the slightest,
Me neither. I pretty much agree with the rest of what Paul said. If we
don't want comprehensions to execute in a nested scope, then we should
not create one. See my response to Guido for a possible alternative.
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Mon, Jun 25, 2018 at 8:37 PM, Terry Reedy wrote: > On 6/24/2018 7:25 PM, Guido van Rossum wrote: > >> I'd wager that the people who might be most horrified about it >> > > the (b) scoping rule change > > would be people who feel strongly that the change to the >> comprehension scope rules in Python 3 is a big improvement, >> > > I might not be one of those 'most horrified' by (b), but I increasingly > don't like it, and I was at best -0 on the comprehension scope change. To > me, iteration variable assignment in the current scope is a non-problem. > So to me the change was mostly useless churn. Little benefit, little > harm. And not worth fighting when others saw a benefit. > > However, having made the change to nested scopes, I think we should stick > with them. Or repeal them. (I believe there is another way to isolate > iteration names -- see below). To me, (b) amounts to half repealing the > nested scope change, making comprehensions half-fowl, half-fish chimeras. > [...] > -- > Terry Jan Reedy > > I'd like to ask: how many readers of this email have ever deliberately taken advantage of the limited Python 3 scope in comprehensions and generator expressions to use what would otherwise be a conflicting local variable name? I appreciate that the scope limitation can sidestep accidental naming errors, which is a good thing. Unfortunately, unless we anticipate Python 4 (or whatever) also making for loops have an implicit scope, I am left wondering whether it's not too large a price to pay. After all, special cases aren't special enough to break the rules, and unless the language is headed towards implicit scope for all uses of "for" one could argue that the scope limitation is a special case too far. It certainly threatens to be yet another confusion for learners, and while that isn't the only consideration, it should be given due weight. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Mon, Jun 25, 2018 at 2:16 PM Steve Holden wrote: > I'd like to ask: how many readers of > > this email have ever deliberately taken advantage of the limited Python 3 > scope in comprehensions and generator expressions to use what would > otherwise be a conflicting local variable name? > No, never, but the opposite has bitten me in production code (as I related several months back, a class-level variable was being used on the lhs of a comprehension and that failed when it was run in Py3). The caveat is that our code base is Py2+Py3, so we have the mindset that comprehension variables always leak. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Mon, Jun 25, 2018 at 5:14 PM Steve Holden wrote: > I'd like to ask: how many readers of > > this email have ever deliberately taken advantage of the limited Python 3 > scope in comprehensions and generator expressions to use what would > otherwise be a conflicting local variable name? > I have never once *deliberately* utilized the Python 3 local scoping in comprehensions. There were a few times in Python 2 where I made an error of overwriting a surrounding name by using it in a comprehension, and probably Python 3 has saved me from that a handful of times. Where I ever made such an error, it was with names like 'x' and 'i' and 'n'. They are useful for quick use, but "more important" variables always get more distinctive names anyway. Had the Python 2 behavior remained, I would have been very little inconvenienced; and I suppose comprehensions would have been slightly less "magic" (but less functional-programming). > > > I appreciate that the scope limitation can sidestep accidental naming > errors, which is a good thing. > > Unfortunately, unless we anticipate Python 4 (or whatever) also making for > loops have an implicit scope, I am left wondering whether it's not too > large a price to pay. After all, special cases aren't special enough to > break the rules, and unless the language is headed towards implicit scope > for all uses of "for" one could argue that the scope limitation is a > special case too far. It certainly threatens to be yet another confusion > for learners, and while that isn't the only consideration, it should be > given due weight. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Ivan Pozdeev via Python-Dev wrote: "as" was suggested even before is became a keyword in `with'. ( if (re.match(regex,line) as m) is not None: ) That's not equivalent where/given, though, since it still has the asymmetry problem. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Terry Reedy wrote: How loop variables might be isolated without a nested scope: After a comprehension is parsed, so that names become strings, rename the loop variables to something otherwise illegal. This doesn't change the situation conceptually, though, since the question arises of why not do the same mangling for names assigned within the comprehension. A decision still needs to be made about whether we *want* semantics that leak some things but not others. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Chris Angelico wrote: The wheel turns round and round, and the same spokes come up. A discussion long past, and a discussion yet to come. There are no beginnings or endings in the Wheel of Python... -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 3.7.0 / 3.6.6 Update: all systems go for final releases!
A quick update: after many months we are at the finish line. We are on track (mixing metaphors) to release 3.7.0 (and 3.6.6) this week on 2018-06-27. Since 3.7.0rc1 shipped 2 weeks ago, I am aware of only two noteworthy regressions that have been identified and now fixed. Since the issues for both have the potential to impact some (but small) subsets of 3.7.0 users and the fixes for both are straightforward and appear to be low-risk, I am planning to cherry-pick the fixes for them into 3.7.0 final without either another release candidate cycle or waiting for 3.7.1. There may be some doc fixes that get cherry-picked as well. At the moment, there are no plans for any bug cherry-picks for 3.6.6 final. As you know, a new feature release is a big deal and something for all of us to be proud of. A new feature release also has various, mostly minor, impacts to lots of different parts of our development infrastructure: to multiple branches of the cpython repo, to documentation builds, to different parts of the python.org web site, etc. You will start to see some of the changes roll out over the next 24 to 36 hours and it may take some time until everything is in place. So please be patient until the official release announcement goes out before reporting release-related issues. Also be advised that over the same period, there may be a few brief periods where commit access to various cpython branches is blocked in order to do the necessary release engineering. If you run into this, for example when trying to merge a PR, please try again in a few hours. Thanks and more later! https://bugs.python.org/issue33851 https://bugs.python.org/issue33932 -- Ned Deily n...@python.org -- [] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
