Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 24 June 2018 at 16:53, Chris Angelico wrote: > On Sun, Jun 24, 2018 at 4:33 PM, Nick Coghlan wrote: >> On 24 June 2018 at 15:56, Steven D'Aprano wrote: >>> On Sun, Jun 24, 2018 at 02:33:59PM +1000, Nick Coghlan wrote: >>> Given that PEP 572 *is* proposing implicit comprehension state export, >>> >>> "Implicit" and "explicit" are two terms which often get misused to mean >>> "I don't like it" and "I do like it". >>> >>> Making the intentional choice to use an assignment expression is not >>> really "implicit" in any meaningful sense. >> >> No, it's actually implicit: there's an extra "global NAME" or >> "nonlocal NAME" in the equivalent code for a comprehension that isn't >> there in the as-written source code, and doesn't get emitted for a >> regular assignment expression or for the iteration variable in a >> comprehension - it only shows up due to the defined interaction >> between comprehensions and assignment expressions. > > The implicit "nonlocal NAME" is only because there is an equally > implicit function boundary. Why is there a function boundary marked by > square brackets? It's not saying "def" or "lambda", which obviously > create functions. It's a 'for' loop wrapped inside a list display. > What part of that says "hey, I'm a nested function"? Nothing - that's why I refer to them as implicitly nested scopes (vs the explicitly nested scopes in functions and lambda expressions, where the scope is introduced via keyword). However, there's still a major behavioural tell at runtime that they're running in a nested scope: the iteration variables don't leak. (There are other tells as well, but not ones that most folks are likely to encounter) > So if there's an implicit function, with implicit declaration of a > magical parameter called ".0", why can't it have an equally implicit > declaration that "spam" is a nonlocal name? Because comprehensions don't do that for their iteration variables, because assignment expressions don't do that when used in explicitly nested scopes, because the required implicit scope declarations are context dependent, and because even such gyrations still can't hide the existence of the comprehension's implicitly nested scope when dealing with classes and the two-argument form of exec(). Since the implicitly nested scopes can't be hidden, it makes far more sense to me to just admit that they're there, and provide explicit syntax for cases where folks decide they really do want name bindings to leak out of that scope (whether those name bindings are assignment expression targets or the iteration variables themselves). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 03:56:47PM +1000, Steven D'Aprano wrote: > There is no consensus that the change to comprehensions was a good thing > or justified. On re-reading that, I think its wrong -- it wasn't really what I intended to say. What I intended to say was, in hindsight, more like: *Despite the consensus to change comprehension scope*, there's a contingent of people who are not convinced that the change was a good thing or justified. Sorry for the inaccurate comment. Mea culpa. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 24.06.2018 9:53, Chris Angelico wrote: On Sun, Jun 24, 2018 at 4:33 PM, Nick Coghlan wrote: On 24 June 2018 at 15:56, Steven D'Aprano wrote: On Sun, Jun 24, 2018 at 02:33:59PM +1000, Nick Coghlan wrote: Given that PEP 572 *is* proposing implicit comprehension state export, "Implicit" and "explicit" are two terms which often get misused to mean "I don't like it" and "I do like it". Making the intentional choice to use an assignment expression is not really "implicit" in any meaningful sense. My 2c. An expression is intuitively thought to be self-contained i.e. without side effects. if I write `a=b+1`, I'm not expecting it to do anything except assigning `a'. Expressions with side effects has long since proven to be problematic because of the implicit (thus hard to see and track) links they create (and because the result depends on the order of evaluation). Moreover, Python's other design elements have been consistently discouraging expressions with side effects, too (e.g. mutator methods intentionally return None instead of the new value, making them useless in expressions), so the proposition is in direct conflict with the language's design. Assignment expressions are a grey area: they carry the full implications of expressions with side effects described above, but their side effect is their only effect, i.e. they are explicit and prominent about the "evil" they do. No, it's actually implicit: there's an extra "global NAME" or "nonlocal NAME" in the equivalent code for a comprehension that isn't there in the as-written source code, and doesn't get emitted for a regular assignment expression or for the iteration variable in a comprehension - it only shows up due to the defined interaction between comprehensions and assignment expressions. The implicit "nonlocal NAME" is only because there is an equally implicit function boundary. Why is there a function boundary marked by square brackets? It's not saying "def" or "lambda", which obviously create functions. It's a 'for' loop wrapped inside a list display. What part of that says "hey, I'm a nested function"? So if there's an implicit function, with implicit declaration of a magical parameter called ".0", why can't it have an equally implicit declaration that "spam" is a nonlocal name? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru -- Regards, Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 05:24:12PM +0300, Ivan Pozdeev via Python-Dev wrote: > An expression is intuitively thought to be self-contained i.e. without > side effects. > if I write `a=b+1`, I'm not expecting it to do anything except assigning > `a'. a = d.pop(1) a = d.setdefault(key, 0) chars_written = file.write(text) > Expressions with side effects has long since proven to be problematic > because of the implicit (thus hard to see and track) links they create > (and because the result depends on the order of evaluation). If you're going to take a hard-core functional approach to side-effects, I think you are using the wrong language. Nearly everything in Python *could* have side-effects (even if usually it won't). Even your own example of "b+1" (depending on what b.__add__ does). > Moreover, Python's other design elements have been consistently > discouraging expressions with side effects, too (e.g. mutator methods > intentionally return None instead of the new value, making them useless > in expressions), I don't think that's the reason why mutator methods return None. They return None rather than self to avoid confusion over whether they return a copy or not. https://docs.python.org/3/faq/design.html#why-doesn-t-list-sort-return-the-sorted-list > so the proposition is in direct conflict with the > language's design. Python is full of operations with side-effects. Besides, they're not quite useless: (alist.append() or alist) is functionally equivalent to alist.append returning self. Just a bit more verbose. Methods (and functions) all return a value, even if that value is None, so they can be used in expressions. If Guido wanted Pascal style procedures, which cannot be used in expressions, we would have them by now :-) -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
> Is it possible, given that we are not paying for those reports, to > customize the 'exclude_lines' definitions? Do you want to exclude python code or C code? For C code you can mark sections that exclude coverage in lcov with comments like "LCOV_EXCL_START" http://ltp.sourceforge.net/coverage/lcov/geninfo.1.php For Python code, coverage.py also has some comments you can put down to exclude lines: http://coverage.readthedocs.io/en/coverage-4.2/excluding.html If you don't need line level granularity, you can always add files to ignore in our codecov.yml file: https://docs.codecov.io/docs/ignoring-paths ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
A quick follow-up: PEP 572 currently has two ideas: (a) introduce := for inline assignment, (b) when := is used in a comprehension, set the scope for the target as if the assignment occurred outside any comprehensions. It seems we have more support for (a) than for (b) -- at least Nick and Greg seem to be +0 or better for (a) but -1 for (b). IIRC (b) originated with Tim. But his essay on the topic, included as Appendix A ( https://www.python.org/dev/peps/pep-0572/#appendix-a-tim-peters-s-findings) does not even mention comprehensions. However, he did post his motivation for (b) on python-ideas, IIRC a bit before PyCon; and the main text of the PEP gives a strong motivation ( https://www.python.org/dev/peps/pep-0572/#scope-of-the-target). Nevertheless, maybe we should compromise and drop (b)? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 04:33:38PM +1000, Nick Coghlan wrote: [...] > > Making the intentional choice to use an assignment expression is not > > really "implicit" in any meaningful sense. > > No, it's actually implicit: there's an extra "global NAME" or > "nonlocal NAME" in the equivalent code for a comprehension that isn't > there in the as-written source code, and doesn't get emitted for a > regular assignment expression or for the iteration variable in a > comprehension - it only shows up due to the defined interaction > between comprehensions and assignment expressions. You seem to be talking about an implementation which could change in the future. I'm talking semantics of the proposed language feature. As a programmer writing Python code, I have no visibility into the implementation. The implementation could change ten times a day for all I care, so long as the semantics remain the same. I want the desired semantics to drive the implementation, not the other way around. You seem to want the implementation to drive the semantics, by eliminating the proposed feature because it doesn't match your deep understanding of the implementation as a nested function. I want this feature because its useful, and without it the use-cases for assignment expressions are significantly reduced. As far as "implicit", for the sake of the discussion, I'll grant you that one. Okay, the proposed behaviour will implicitly enable comprehensions to export their state. Now what? Is that a good thing or a bad thing? If "implicit" (with or without the scare quotes) is such a bad thing to be avoided, why are comprehensions implemented using an implicit function? > The problem I have with PEP 572 is that it proposes *breaking that > otherwise universal pattern* - instead of having assignment > expressions in comprehensions implicitly declare the name as local in > the nested comprehension scope, it instead has them: You talk about "nested comprehension scope", and that's a critical point, but I'm going to skip answering that for now. I have a draft email responding to another of your posts on that topic, which I hope to polish in the next day. > 1. implicitly declare the name as global or as nonlocal in the > comprehension (or else raise an exception), depending on the nature of > the parent scope where the comprehension is used > 2. in the nonlocal reference case, amend the symbol table analysis to > act like there was a preceding "if 0:\n for NAME in ():\n pass" in the > parent scope (so the compiler knows which outer function scope to > target) If it is okay for you to amend the list comprehension to behave as if it were wrapped in an implicit nested function, why shouldn't it be okay to behave as if assignments inside the comprehension included an implicit nonlocal declaration? > The rationale being given for why that is OK is: > > 1. "Everyone" thinks comprehensions are just a for loop (even though > that hasn't been true for the better part of a decade, and was never > true for generator expressions) Obviously "everyone" is an exaggeration, but, yes, I stand by that -- most people don't even give comprehension scope a thought until they get bitten by it. Either because (Python 2) they don't realise the loop variable is local to their current scope: http://www.librador.com/2014/07/10/Variable-scope-in-list-comprehension-vs-generator-expression/ or (Python 3) they get bitten by the change: https://old.reddit.com/r/Python/comments/425qmb/strange_python_27_34_difference/ (As is so often the case, whatever behaviour we choose, we're going to surprise somebody.) It is hardly surprising that people don't think too hard about scoping of comprehensions. Without a way to perform assignments inside comprehensions, aside from the loop variables themselves, there's nothing going on inside a comprehension where it makes a visible difference whether it is a local scope or a sublocal scope. *IF* assignment expressions are introduced, that is going to change. We have some choices: 1. Keep assignment expressions encapsulated in their implicit function, and be prepared for people to be annoyed because (with no way to declare them global or non-local inside an expression), they can't use them to get data in and out of the comprehension. 2. Allow assignment expressions to be exported out of the comprehension, and be prepared for people to be annoyed because they clobbered a local. (But for the reasons Tim Peters has already set out, I doubt this will happen often.) 3. Allow some sort of extra comprehension syntax to allow global/nonlocal declarations inside comprehensions. x = 1 [nonlocal x := x+i for i in sequence] (Hmmm... I thought I would hate that more than I actually do.) 4. Have some sort of cunning plan whereby if the variable in question exists in the local scope, it is implicitly local inside the comprehension: x = 1 [x := i+1 for i in (1, 2)] ass
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 09:24:39AM -0700, Guido van Rossum wrote: > A quick follow-up: PEP 572 currently has two ideas: (a) introduce := for > inline assignment, (b) when := is used in a comprehension, set the scope > for the target as if the assignment occurred outside any comprehensions. It > seems we have more support for (a) than for (b) -- at least Nick and Greg > seem to be +0 or better for (a) but -1 for (b). IIRC (b) originated with > Tim. I'm not sure who came up with the idea first, but as I remember it, the first mention of this came in a separate thread on Python-Ideas: https://mail.python.org/pipermail/python-ideas/2018-April/049631.html so possibly I'm to blame :-) That thread starts here: https://mail.python.org/pipermail/python-ideas/2018-April/049622.html If I did get the idea from Tim, I don't remember doing so. > But his essay on the topic, included as Appendix A ( > https://www.python.org/dev/peps/pep-0572/#appendix-a-tim-peters-s-findings) > does not even mention comprehensions. However, he did post his motivation > for (b) on python-ideas, IIRC a bit before PyCon; and the main text of the > PEP gives a strong motivation ( > https://www.python.org/dev/peps/pep-0572/#scope-of-the-target). > Nevertheless, maybe we should compromise and drop (b)? I will have more to say about the whole "comprehensions are their own scope" issue later. But I'd like to see Nick's proposed PEP, or at least a draft of it, before making any final decisions. If it came down to it, I'd be happy with the ability to declare an assignment target nonlocal in the comprehension if that's what it takes. What do you think of this syntax? [global|nonlocal] simple_target := expression Inside a comprehension, without a declaration, the target would be sublocal (comprehension scope); that should make Nick happier :-) -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
[Guido] > A quick follow-up: PEP 572 currently has two ideas: (a) introduce := for inline > assignment, (b) when := is used in a comprehension, set the scope for the > target as if the assignment occurred outside any comprehensions. It seems > we have more support for (a) than for (b) -- at least Nick and Greg seem to > be +0 or better for (a) but -1 for (b). IIRC (b) originated with Tim. But his > essay on the topic, included as Appendix A > ( https://www.python.org/dev/peps/pep-0572/#appendix-a-tim-peters-s-findings) > does not even mention comprehensions. I was writing up my observations about simple changes to existing code. Since there's nothing sanely akin to binding non-for-targets possible in comprehensions now, comprehensions were out of scope for that effort (which was limited to staring at existing code already doing bindings). :> However, he did post his motivation for (b) on python-ideas, IIRC a bit > before PyCon; and the main text of the PEP gives a strong motivation > (https://www.python.org/dev/peps/pep-0572/#scope-of-the-target). Nevertheless, > maybe we should compromise and drop (b)? Two things to say about that. First, the original example I gave would be approximately as well addressed by allowing to declare intended scopes in magically synthesized functions; like (say) p = None # to establish the intended scope of `p` while any( # split across lines just for readability n % p == 0 for p in small_primes): n //= p It didn't really require an inline assignment, just a way to override the unwanted (in this case) "all `for` targets are local to the invisible function" rigid consequence of the implementation du jour. Second, if it's dropped, then the PEP needs more words to define what happens in cases like the following, because different textual parts of a comprehension execute in different scopes, and that can become visible when bindings can be embedded: def f(): y = -1 ys = [y for _ in range(y := 5)] print(y, ys) Here `range(y := 5)` is executed in f's scope. Presumably the `y` in `y for` also refers to f's scope, despite that `y` textually _appears_ to be assigned to in the body of the listcomp, and so would - for that reason - expected to be local to the synthesized function, and so raise `UnboundLocalError` when referenced. It's incoherent without detailed knowledge of the implementation. def g(): y = -1 ys = [y for y in range(y := 5)] print(y, ys) And here the `y` in `y for y` is local to the synthesized function, and presumably has nothing to do with the `y` in the `range()` call. That's incoherent in its own way. Under the current PEP, all instances of `y` in `f` refer to the f-local `y`, and the listcomp in `g` is a compile-time error. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Mon, Jun 25, 2018 at 4:06 AM, Steven D'Aprano wrote: > > Remember, the driving use-case which started this (ever-so-long) > discussion was the ability to push data into a comprehension and then > update it on each iteration, something like this: > > x = initial_value() > results = [x := transform(x, i) for i in sequence] Which means there is another option. 5. Have the assignment be local to the comprehension, but the initial value of ANY variable is looked up from the surrounding scopes. That is: you will NEVER get UnboundLocalError from a comprehension/genexp; instead, you will look up the name as if it were in the surrounding scope, either getting a value or bombing with regular old NameError. Or possibly variations on this such as "the immediately surrounding scope only", rather than full name lookups. It'd have an awkward boundary somewhere, whichever way you do it. This isn't able to send information *out* of a comprehension, but it is able to send information *in*. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
This thread started with a request for educator feedback, which I took to
mean observations of student reactions. I've only had the chance to test
the proposal on ~20 students so far, but I'd like the chance to gather more
data for your consideration before the PEP is accepted or rejected.
On Sun, Jun 24, 2018 at 11:09 AM Steven D'Aprano
wrote:
> Remember, the driving use-case which started this (ever-so-long)
> discussion was the ability to push data into a comprehension and then
> update it on each iteration, something like this:
>
> x = initial_value()
> results = [x := transform(x, i) for i in sequence]
>
If that is the driving use-case, then the proposal should be rejected. The
``itertools.accumulate`` function has been available for a little while now
and it handles this exact case. The accumulate function may even be more
readable, as it explains the purpose explicitly, not merely the algorithm.
And heck, it's a one-liner.
results = accumulate(sequence, transform)
The benefits for ``any`` and ``all`` seem useful. Itertools has
"first_seen" in the recipes section. While it feels intuitively useful, I
can't recall ever writing something similar myself. For some reason, I
(almost?) always want to find all (counter-)examples and aggregate them in
some way -- min or max, perhaps -- rather than just get the first.
Even so, if it turns out those uses are quite prevalent, wouldn't a new
itertool be better than a new operator? It's good to solve the general
problem, but so far the in-comprehension usage seems to have only a handful
of cases.
On Fri, Jun 22, 2018 at 9:14 PM Chris Barker via Python-Dev <
python-dev@python.org> wrote:
> again, not a huge deal, just a little bit more complexity
>
I worry that Python may experience something of a "death by a thousand
cuts" along the lines of the "Remember the Vasa" warning. Python's greatest
strength is its appeal to beginners. Little bits of added complexity have a
non-linear effect. One day, we may wake up and Python won't be recommended
as a beginner's language.
On Fri, Jun 22, 2018 at 7:48 PM Steven D'Aprano wrote:
> On Fri, Jun 22, 2018 at 10:59:43AM -0700, Michael Selik wrote:
>
> Of course they do -- they're less fluent at reading code. They don't
> have the experience to judge good code from bad.
>
On the other hand, an "expert" may be so steeped in a particular subculture
that he no longer can distinguish esoteric from intuitive. Don't be so fast
to reject the wisdom of the inexperienced.
> The question we should be asking is, do we only add features to Python
> if they are easy for beginners? It's not that I especially want to add
> features which *aren't* easy for beginners, but Python isn't Scratch and
> "easy for beginners" should only be a peripheral concern.
>
On the contrary, I believe that "easy for beginners" should be a major
concern. Ease of use has been and is a, or even the main reason for
Python's success. When some other language becomes a better teaching
language, it will eventually take over in business and science as well.
Right now, Python is Scratch for adults. That's a great thing. Given the
growth of the field, there are far more beginner programmers working today
than there ever have been experts.
Mozilla's array comprehensions are almost identical to Python's, aside
> from a couple of trivial differences:
>
I can't prove it, but I think the phrase ordering difference is not trivial.
> Students who are completely new to programming can see the similarity of
> > [Python] list comprehensions to spoken language.
>
> I've been using comprehensions for something like a decade, and I can't
>
Python: any(line.startswith('#') for line in file)
English: Any line starts with "#" in the file?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 11:50 AM Steven D'Aprano wrote: > [Guido] > > [...] IIRC (b) originated with Tim. > > I'm not sure who came up with the idea first, but as I remember it, the > first mention of this came in a separate thread on Python-Ideas: > > https://mail.python.org/pipermail/python-ideas/2018-April/049631.html > > so possibly I'm to blame :-) > Actually that post sounds like the OP of that thread (Peter O'Connor) is to blame -- he proposed a similar thing using `=` for the assignment and custom syntax (`from `) to specify the initial value, and it looks like that inspired you. > > But his essay on the topic, included as Appendix A ( > > > https://www.python.org/dev/peps/pep-0572/#appendix-a-tim-peters-s-findings > ) > > does not even mention comprehensions. However, he did post his motivation > > for (b) on python-ideas, IIRC a bit before PyCon; and the main text of > the > > PEP gives a strong motivation ( > > https://www.python.org/dev/peps/pep-0572/#scope-of-the-target). > > Nevertheless, maybe we should compromise and drop (b)? > > I will have more to say about the whole "comprehensions are their own > scope" issue later. But I'd like to see Nick's proposed PEP, or at least > a draft of it, before making any final decisions. > Agreed, though I assume it's just `given` again. > If it came down to it, I'd be happy with the ability to declare an > assignment target nonlocal in the comprehension if that's what it takes. > What do you think of this syntax? > > [global|nonlocal] simple_target := expression > > Inside a comprehension, without a declaration, the target would be > sublocal (comprehension scope); that should make Nick happier :-) > It's more special syntax. Just taking part (a) of PEP 572 would make most people happy enough. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 12:03 PM Tim Peters wrote:
> [Guido]
> :> However, [Tim] did post his motivation for (b) on python-ideas, IIRC a
> bit
> > before PyCon; and the main text of the PEP gives a strong motivation
> > (https://www.python.org/dev/peps/pep-0572/#scope-of-the-target).
> Nevertheless,
> > maybe we should compromise and drop (b)?
>
> Two things to say about that. First, the original example I gave would be
> approximately as well addressed by allowing to declare intended scopes in
> magically synthesized functions; like (say)
>
> p = None # to establish the intended scope of `p`
> while any( # split across lines just for readability
> n % p == 0 for p in small_primes):
> n //= p
>
> It didn't really require an inline assignment, just a way to override the
> unwanted (in this case) "all `for` targets are local to the invisible
> function" rigid consequence of the implementation du jour.
>
Hm, that's more special syntax. The nice bit about (b) as currently
specified is that it adds no syntax -- it adds a scope rule, but (as IIRC
Steven has convincingly argued) few people care about those. Python's scope
rules, when fully specified, are intricate to the point of being arcane
(e.g. for class scopes) but all that has a purpose -- to make them so DWIM
("Do what I Mean") that in practice you almost never have to worry about
them, *especially* when reading non-obfuscated code (and also when writing,
except for a few well-known patterns).
> Second, if it's dropped, then the PEP needs more words to define what
> happens in cases like the following, because different textual parts of a
> comprehension execute in different scopes, and that can become visible
> when bindings can be embedded:
>
> def f():
> y = -1
> ys = [y for _ in range(y := 5)]
> print(y, ys)
>
> Here `range(y := 5)` is executed in f's scope. Presumably the `y` in `y
> for` also refers to f's scope, despite that `y` textually _appears_ to be
> assigned to in the body of the listcomp, and so would - for that reason -
> expected to be local to the synthesized function, and so raise
> `UnboundLocalError` when referenced. It's incoherent without detailed
> knowledge of the implementation.
>
That code should have the same meaning regardless of whether we accept (b)
or not -- there is only one `y`, in f's scope. I don't mind if we have to
add more words to the PEP's scope rules to make this explicit, though I
doubt it -- the existing weirdness (in the comprehension spec) about the
"outermost iterable" being evaluated in the surrounding scope specifies
this. I wouldn't call it incoherent -- I think what I said about scope
rules above applies here, it just does what you expect.
> def g():
> y = -1
> ys = [y for y in range(y := 5)]
> print(y, ys)
>
> And here the `y` in `y for y` is local to the synthesized function, and
> presumably has nothing to do with the `y` in the `range()` call. That's
> incoherent in its own way.
>
> Under the current PEP, all instances of `y` in `f` refer to the f-local
> `y`, and the listcomp in `g` is a compile-time error.
>
And under the (b)-less proposal, `g` would interpret `y for y` as both
referring to a new variable created just for the comprehension, and `y :=
5` as referring to g's scope. Again I don't think it needs extra words in
the spec. And the end user docs might just say "don't do that" (with a link
to the reference manual's rule about the "outermost iterable").
Even if in the end we did find a case where we'd have to write an explicit
rule to make what happens here a consequence of the spec rather than the
implementation, that doesn't count as an argument for keeping (b) to me.
In favor of (b) we have a few examples (see
https://www.python.org/dev/peps/pep-0572/#scope-of-the-target) that require
it, and more that you described on python-ideas (and also the motivating
use case from the thread that Steven dug up, starting here:
https://mail.python.org/pipermail/python-ideas/2018-April/049622.html).
A "neutral" argument about (b) is that despite the "horrified" reactions
that Nick saw, in practice it's going to confuse very few people (again,
due to my point about Python's scope rules). I'd wager that the people who
might be most horrified about it would be people who feel strongly that the
change to the comprehension scope rules in Python 3 is a big improvement,
and who are familiar with the difference in implementation of
comprehensions (though not generator expressions) in Python 2 vs. 3.
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Guido van Rossum wrote: Greg seem to be +0 or better for (a) Actually, I'm closer to -1 on (a) as well. I don't like := as a way of getting assignment in an expression. The only thing I would give a non-negative rating is some form of "where" or "given". Brief summary of reasons for disliking ":=": * Cryptic use of punctuation * Too much overlap in functionality with "=" * Asymmetry between first and subsequent uses of the bound value * Makes expressions cluttered and hard to read to my eyes -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 2:10 PM Chris Angelico wrote: > On Mon, Jun 25, 2018 at 4:06 AM, Steven D'Aprano > wrote: > > > > Remember, the driving use-case which started this (ever-so-long) > > discussion was the ability to push data into a comprehension and then > > update it on each iteration, something like this: > > > > x = initial_value() > > results = [x := transform(x, i) for i in sequence] > > Which means there is another option. > > 5. Have the assignment be local to the comprehension, but the initial > value of ANY variable is looked up from the surrounding scopes. > > That is: you will NEVER get UnboundLocalError from a > comprehension/genexp; instead, you will look up the name as if it were > in the surrounding scope, either getting a value or bombing with > regular old NameError. > > Or possibly variations on this such as "the immediately surrounding > scope only", rather than full name lookups. It'd have an awkward > boundary somewhere, whichever way you do it. > > This isn't able to send information *out* of a comprehension, but it > is able to send information *in*. > But this "horrifies" me for a slightly different reason: it effectively introduces a new case of dynamic scoping, which Python used to do everywhere but has long switched away from, with the exception of class scopes (whose difference with function scopes sometimes confuses people -- usually people who put too much code in their class scope). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Steven D'Aprano wrote: You seem to be talking about an implementation which could change in the future. I'm talking semantics of the proposed language feature. The way I see it, it's not about implementation details, it's about having a mental model that's easy to reason about. "Comprehensions run in their own scope, like a def or lambda" is a clear and simple mental model. It's easy to explain and keep in your head. The proposed semantics are much more complicated, and as far as I can see, are only motivated by use cases that you shouldn't really be doing in the first place. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
Greg Ewing writes: > Actually, I'm closer to -1 on (a) as well. I don't like := as a > way of getting assignment in an expression. The only thing I would > give a non-negative rating is some form of "where" or "given". +1 to this; ‘:=’ doesn't convey the meaning well. Python's syntax typically eschews cryptic punctuation in faviour of existing words that convey an appropriate meaning, and I agree with Greg that would be a better way to get this effect. -- \ “Self-respect: The secure feeling that no one, as yet, is | `\suspicious.” —Henry L. Mencken | _o__) | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 2:41 PM Michael Selik wrote: > This thread started with a request for educator feedback, which I took to > mean observations of student reactions. I've only had the chance to test > the proposal on ~20 students so far, but I'd like the chance to gather more > data for your consideration before the PEP is accepted or rejected. > Sure. Since the target for the PEP is Python 3.8 I am in no particular hurry. It would be important to know how you present it to your students. > On Sun, Jun 24, 2018 at 11:09 AM Steven D'Aprano > wrote: > >> Remember, the driving use-case which started this (ever-so-long) >> discussion was the ability to push data into a comprehension and then >> update it on each iteration, something like this: >> >> x = initial_value() >> results = [x := transform(x, i) for i in sequence] >> > > If that is the driving use-case, then the proposal should be rejected. The > ``itertools.accumulate`` function has been available for a little while now > and it handles this exact case. The accumulate function may even be more > readable, as it explains the purpose explicitly, not merely the algorithm. > And heck, it's a one-liner. > > results = accumulate(sequence, transform) > I think that's a misunderstanding. At the very least the typical use case is *not* using an existing transform function which is readily passed to accumulate -- instead, it's typically written as a simple expression (e.g. `total := total + v` in the PEP) which would require a lambda. Plus, I don't know what kind of students you are teaching, but for me, whenever the solution requires a higher-order function (like accumulate), this implies a significant speed bump -- both when writing and when reading code. (Honestly, whenever I read code that uses itertools, I end up making a trip to StackOverflow :-). > The benefits for ``any`` and ``all`` seem useful. Itertools has > "first_seen" in the recipes section. > (I think you mean first_true().) > While it feels intuitively useful, I can't recall ever writing something > similar myself. For some reason, I (almost?) always want to find all > (counter-)examples and aggregate them in some way -- min or max, perhaps -- > rather than just get the first. > I trust Tim's intuition here, he's written about this. Also, Python's predecessor, ABC, had quantifiers (essentially any() and all()) built into the language, and the semantics included making the first (counter-)example available ( https://homepages.cwi.nl/~steven/abc/qr.html#TESTS). Essentially IF SOME x IN values HAS x < 0: WRITE "Found a negative x:", x equivalently IF EACH x IN values HAS x >= 0: # ... ELSE: WRITE "Found a negative x:", x and even IF NO x IN values HAS x < 0: # ... ELSE: WRITE "Found a negative x:", x > Even so, if it turns out those uses are quite prevalent, wouldn't a new > itertool be better than a new operator? It's good to solve the general > problem, but so far the in-comprehension usage seems to have only a handful > of cases. > Perhaps, but IMO the new itertool would be much less useful than the new syntax. > On Fri, Jun 22, 2018 at 9:14 PM Chris Barker via Python-Dev < > python-dev@python.org> wrote: > >> again, not a huge deal, just a little bit more complexity >> > > I worry that Python may experience something of a "death by a thousand > cuts" along the lines of the "Remember the Vasa" warning. Python's greatest > strength is its appeal to beginners. Little bits of added complexity have a > non-linear effect. One day, we may wake up and Python won't be recommended > as a beginner's language. > I don't think that appeal to beginners is Python's greatest strength. I'd rather say that it is its appeal to both beginners and experts (and everyone in between). If true appeal to beginners is needed, Scratch or Processing would probably be better. > On Fri, Jun 22, 2018 at 7:48 PM Steven D'Aprano > wrote: > >> On Fri, Jun 22, 2018 at 10:59:43AM -0700, Michael Selik wrote: >> >> Of course they do -- they're less fluent at reading code. They don't >> have the experience to judge good code from bad. >> > > On the other hand, an "expert" may be so steeped in a particular > subculture that [they] no longer can distinguish esoteric from intuitive. > Don't be so fast to reject the wisdom of the inexperienced. > Nor should we cater to them excessively though. While the user is indeed king, it's also well known that most users when they are asking for a feature don't know what they want (same for kings, actually, that's why they have advisors :-). > The question we should be asking is, do we only add features to Python >> if they are easy for beginners? It's not that I especially want to add >> features which *aren't* easy for beginners, but Python isn't Scratch and >> "easy for beginners" should only be a peripheral concern. >> > > On the contrary, I believe that "easy for beginners" should be a major > concern. Ease of use has been
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 4:57 PM Guido van Rossum wrote: > On Sun, Jun 24, 2018 at 2:41 PM Michael Selik wrote: > >> This thread started with a request for educator feedback, which I took to >> mean observations of student reactions. I've only had the chance to test >> the proposal on ~20 students so far, but I'd like the chance to gather more >> data for your consideration before the PEP is accepted or rejected. >> > > Sure. Since the target for the PEP is Python 3.8 I am in no particular > hurry. It would be important to know how you present it to your students. > Absolutely. Since this has come up, I'll make an effort to be more systematic in data collection. > On Sun, Jun 24, 2018 at 11:09 AM Steven D'Aprano >> wrote: >> >>> Remember, the driving use-case which started this (ever-so-long) >>> discussion was the ability to push data into a comprehension and then >>> update it on each iteration, something like this: >>> >>> x = initial_value() >>> results = [x := transform(x, i) for i in sequence] >>> >> >> If that is the driving use-case, then the proposal should be rejected. >> The ``itertools.accumulate`` function has been available for a little while >> now and it handles this exact case. The accumulate function may even be >> more readable, as it explains the purpose explicitly, not merely the >> algorithm. And heck, it's a one-liner. >> >> results = accumulate(sequence, transform) >> > > I think that's a misunderstanding. At the very least the typical use case > is *not* using an existing transform function which is readily passed to > accumulate -- instead, it's typically written as a simple expression (e.g. > `total := total + v` in the PEP) which would require a lambda. > Plus, I don't know what kind of students you are teaching, but for me, > whenever the solution requires a higher-order function (like accumulate), > this implies a significant speed bump -- both when writing and when reading > code. (Honestly, whenever I read code that uses itertools, I end up making > a trip to StackOverflow :-). > Mostly mid-career professionals, of highly varying backgrounds. The higher-order functions do require some cushioning getting into, but I have some tricks I've learned over the years to make it go over pretty well. On Fri, Jun 22, 2018 at 7:48 PM Steven D'Aprano wrote: >> > On Fri, Jun 22, 2018 at 10:59:43AM -0700, Michael Selik wrote: >>> >>> Of course they do -- they're less fluent at reading code. They don't >>> have the experience to judge good code from bad. >>> >> >> On the other hand, an "expert" may be so steeped in a particular >> subculture that [they] no longer can distinguish esoteric from intuitive. >> Don't be so fast to reject the wisdom of the inexperienced. >> > > Nor should we cater to them excessively though. While the user is indeed > king, it's also well known that most users when they are asking for a > feature don't know what they want (same for kings, actually, that's why > they have advisors :-). > > >> The question we should be asking is, do we only add features to Python >>> if they are easy for beginners? It's not that I especially want to add >>> features which *aren't* easy for beginners, but Python isn't Scratch and >>> "easy for beginners" should only be a peripheral concern. >>> >> >> On the contrary, I believe that "easy for beginners" should be a major >> concern. Ease of use has been and is a, or even the main reason for >> Python's success. When some other language becomes a better teaching >> language, it will eventually take over in business and science as well. >> Right now, Python is Scratch for adults. That's a great thing. Given the >> growth of the field, there are far more beginner programmers working today >> than there ever have been experts. >> > > I'm sorry, but this offends me, and I don't believe it's true at all. > Python is *not* a beginners language, and you are mixing ease of use and > ease of learning. Python turns beginners into experts at an unprecedented > rate, and that's the big difference with Scratch. > By saying "Scratch for adults" I meant that Python is a language that can be adopted by beginners and rapidly make them professionals, not that it's exclusively a beginner's language. Also, Scratch and similar languages, like NetLogo, have some interesting features that allow beginners to write some sophisticated parallelism. I don't mean "beginner's language" in that it's overly simplistic, but that it enables what would be complex in other languages. I realize that my phrasing was likely to be misunderstood without knowing the context that I teach working professionals who are asked to be immediately productive at high-value tasks. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
>
> [Tim]
>>
> . First, the original example I gave would be approximately as well
>> addressed by allowing to declare intended scopes in magically synthesized
>> functions; like (say)
>>
>> p = None # to establish the intended scope of `p`
>> while any( # split across lines just for readability
>> n % p == 0 for p in small_primes):
>> n //= p
>>
>> It didn't really require an inline assignment, just a way to override the
>> unwanted (in this case) "all `for` targets are local to the invisible
>> function" rigid consequence of the implementation du jour.
>>
>
[Guido]
> Hm, that's more special syntax.
>
Of course - I'm anticipating that the PEP will be changed to throw out
useful assignment expressions in comprehensions, but I still want a way to
"export" comprehension for-targets at times ;-)
> The nice bit about (b) as currently specified is that it adds no syntax --
> it adds a scope rule, but (as IIRC Steven has convincingly argued) few
> people care about those. Python's scope rules, when fully specified, are
> intricate to the point of being arcane (e.g. for class scopes) but all that
> has a purpose -- to make them so DWIM ("Do what I Mean") that in practice
> you almost never have to worry about them, *especially* when reading
> non-obfuscated code (and also when writing, except for a few well-known
> patterns).
>
You and Steven and i appear to be on the same page here - but it's in a
book nobody else seems to own :-( To me it's just screamingly obvious that
total = 0
cumsums = [total := total + value for value in data]
"should do" what it obviously intends to do - and that the only thing
stopping that is a bass-ackwards focus on what most trivially falls out of
the current implementation.
...
def f():
>> y = -1
>> ys = [y for _ in range(y := 5)]
>> print(y, ys)
>>
>> Here `range(y := 5)` is executed in f's scope. Presumably the `y` in `y
>> for` also refers to f's scope, despite that `y` textually _appears_ to be
>> assigned to in the body of the listcomp, and so would - for that reason -
>> expected to be local to the synthesized function, and so raise
>> `UnboundLocalError` when referenced. It's incoherent without detailed
>> knowledge of the implementation.
>>
>
> That code should have the same meaning regardless of whether we accept (b)
> or not -- there is only one `y`, in f's scope. I don't mind if we have to
> add more words to the PEP's scope rules to make this explicit, though I
> doubt it -- the existing weirdness (in the comprehension spec) about the
> "outermost iterable" being evaluated in the surrounding scope specifies
> this. I wouldn't call it incoherent -- I think what I said about scope
> rules above applies here, it just does what you expect.
>
Remove "y = -1" and - voila! - we have the dreaded "parent local scoping"
Nick finds so baffling to explain (or so he claims). That is, "has exactly
the same scope in the comprehension as in the parent block, and will create
a local in the latter if the name is otherwise unknown in the parent" comes
with assignment expressions, regardless of whether _all_ such targets
"leak" (the current PEP) or only targets in the expression defining the
iterable of the outermost `for` (the PEP without leaking assignment
expressions in comprehensions).
As to whether it "does what you expect", no, not really! In a world where
_all_ binding targets in a comprehension are claimed to be local to the
comprehension, I _expect_ that `y := 5` appearing inside the listcomp means
`y` is local to the listcomp. "Oh - unless the binding appears in the
expression defining the iterable of the outermost `for`" comes from Mars.
Not that it really matters much, but (b) provides consistent semantics in
these cases. No need to search Mars for weird exceptions ;-)
...
> A "neutral" argument about (b) is that despite the "horrified" reactions
> that Nick saw, in practice it's going to confuse very few people (again,
> due to my point about Python's scope rules). I'd wager that the people who
> might be most horrified about it would be people who feel strongly that the
> change to the comprehension scope rules in Python 3 is a big improvement,
> and who are familiar with the difference in implementation of
> comprehensions (though not generator expressions) in Python 2 vs. 3.
>
I also doubt it will generally confuse people in practice (to the contrary,
I expect they _will_ be confused if things like the cumulative sums example
blow up with UnboundLocalError).
But I still don't get the source of the "horror". Assignment expression
semantics are wholly consistent with ordinary nested lexical scoping, with
or without (b). The only difference is in the scopes picked for assignment
expression target names (except for those appearing in the expression
defining the iterable yadda yadda yadda).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/l
