Re: [Python-Dev] Python 3.7.0 is now available! (and so is 3.6.6)

2018-06-28 Thread Victor Stinner
Hi,

I updated my list of Python known vulnerabilities and the good news is
that Python 3.6.6 and 3.7.0 have no known vulnerability :-)

Python 3.7.0 comes with fixes for:

* CVE-2018-1000117: Buffer overflow vulnerability in os.symlink on Windows
* CVE-2018-1060: difflib and poplib catastrophic backtracking
* Expat 2.2.3 (ex: CVE-2017-11742)
* urllib FTP protocol stream injection
* update zlib to 1.2.11 (CVE-2016-9840, CVE-2016-9841, CVE-2016-9842,
CVE-2016-9843; only Windows and macOS are impacted)

More information at: http://python-security.readthedocs.io/vulnerabilities.html

Victor

2018-06-28 2:58 GMT+02:00 Ned Deily :
> On behalf of the Python development community and the Python 3.7 release
> team, we are pleased to announce the availability of Python 3.7.0.
> Python 3.7.0 is the newest feature release of the Python language, and
> it contains many new features and optimizations. You can find Python
> 3.7.0 here:
>
> https://www.python.org/downloads/release/python-370/
>
> Most third-party distributors of Python should be making 3.7.0 packages
> available soon.
>
> See the "What’s New In Python 3.7" document
> (https://docs.python.org/3.7/whatsnew/3.7.html) for more information
> about features included in the 3.7 series. Detailed information about
> the changes made in 3.7.0 can be found in its change log. Maintenance
> releases for the 3.7 series will follow at regular intervals starting in
> July of 2018.
>
> We hope you enjoy Python 3.7!
>
> P.S. We are also happy to announce the availability of Python 3.6.6, the
> next maintenance release of Python 3.6:
>
> https://www.python.org/downloads/release/python-366/
>
> Thanks to all of the many volunteers who help make Python Development
> and these releases possible! Please consider supporting our efforts by
> volunteering yourself or through organization contributions to the
> Python Software Foundation.
>
> https://www.python.org/psf/
>
> --
>   Ned Deily
>   n...@python.org -- []
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Victor Stinner
✨ Congrats Nick on your 100 emails thread 😍! ✨ You won a virtual
piece of cake: 🍰

2018-06-22 16:22 GMT+02:00 Nick Coghlan :
> On 22 June 2018 at 02:26, Antoine Pitrou  wrote:
>> Indeed.  But, for a syntax addition such as PEP 572, I think it would be
>> a good idea to ask their opinion to teaching/education specialists.
>>
>> As far as I'm concerned, if teachers and/or education specialists were
>> to say PEP 572 is not a problem, my position would shift from negative
>> towards neutral.
>
> I asked a handful of folks at the Education Summit the next day about it:
>
> * for the basic notion of allowing expression level name binding using
> the "NAME := EXPR" notation, the reactions ranged from mildly negative
> (I read it as only a "-0" rather than a "-1") to outright positive.
> * for the reactions to my description of the currently proposed parent
> local scoping behaviour in comprehensions, I'd use the word
> "horrified", and feel I wasn't overstating the response :)
>
> While I try to account for the fact that I implemented the current
> comprehension semantics for the 3.x series, and am hence biased
> towards considering them the now obvious interpretation, it's also the
> case that generator expressions have worked like nested functions
> since they were introduced in Python 2.4 (more than 13 years ago now),
> and comprehensions have worked the same way as generator expressions
> since Python 3.0 (which has its 10th birthday coming up in December
> this year).
>
> This means that I take any claims that the legacy Python 2.x
> interpretation of comprehension behaviour is intuitively obvious with
> an enormous grain of salt - for the better part of a decade now, every
> tool at a Python 3 user's disposal (the fact that the iteration
> variable is hidden from the current scope, reading the language
> reference [1], printing out locals(), using the dis module, stepping
> through code in a debugger, writing their own tracing function, and
> even observing the quirky interaction with class scopes) will have
> nudged them towards the "it's a hidden nested function" interpretation
> of expected comprehension behaviour.
>
> Acquiring the old mental model for the way comprehensions work pretty
> much requires a developer to have started with Python 2.x themselves
> (perhaps even before comprehensions and lexical closures were part of
> the language), or else have been taught the Python 2 comprehension
> model by someone else - there's nothing in Python 3's behaviour to
> encourage that point of view, and plenty of
> functional-language-inspired documentation to instead encourage folks
> to view comprehensions as tightly encapsulated declarative container
> construction syntax.
>
> I'm currently working on a concept proposal at
> https://github.com/ncoghlan/peps/pull/2 that's much closer to PEP 572
> than any of my previous `given` based suggestions: for already
> declared locals, it devolves to being the same as PEP 572 (except that
> expressions are allowed as top level statements), but for any names
> that haven't been previously introduced, it prohibits assigning to a
> name that doesn't already have a defined scope, and instead relies on
> a new `given` clause on various constructs that allows new target
> declarations to be introduced into the current scope (such that "if
> x:= f():" implies "x" is already defined as a target somewhere else in
> the current scope, while "if x := f() given x:" potentially introduces
> "x" as a new local target the same way a regular assignment statement
> does).
>
> One of the nicer features of the draft proposal is that if all you
> want to do is export the iteration variable from a comprehension, you
> don't need to use an assignment expression at all: you can just append
> "... given global x" or "... given nonlocal x" and export the
> iteration variable directly to the desired outer scope, the same way
> you can in the fully spelled out nested function equivalent.
>
> Cheers,
> Nick.
>
> [1] From 
> https://docs.python.org/3.0/reference/expressions.html#displays-for-lists-sets-and-dictionaries:
> 'Note that the comprehension is executed in a separate scope, so names
> assigned to in the target list don’t “leak” in the enclosing scope.'
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Baptiste Carvello
Le 28/06/2018 à 01:31, Greg Ewing a écrit :
> Well, I remain profoundly unconvinced that writing comprehensions
> with side effects is ever a good idea, and Tim's examples did
> nothing to change that.

Comprehensions with side effects feel scary indeed. But I could see
myself using some variant of the "cumsum" example (for scientific work
at the command prompt):

>>> x=0; [x:=x+i for i in range(5)]

Here the side effects are irrelevant, the "x" variable won't be reused.
But it needs to be initialized at the start of the comprehension.

I would happily get rid of the side-effects, but then what would be a
non-cryptic alternative to the above example?

Baptiste
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Chris Barker - NOAA Federal via Python-Dev
Sent from my iPhone

> > So what about:
> >
> > l = [x:=i for i in range(3)]
> >
> > vs
> >
> > g = (x:=i for i in range(3))
> >
> > Is there any way to keep these consistent if the "x" is in the regular 
> > local scope?
>
> I'm not clear on what the question is.  The list comprehension would bind ` l 
> ` to [0, 1, 2] and leave the local `x` bound to 2.  The second example binds 
> `g` to a generator object, which just sits there unexecuted.  That has 
> nothing to do with the PEP, though.
>
> If you go on to do, e.g.,
>
> l = list(g)
>
> then, same as the listcomp, `l` will be bound to [0, 1, 2] and the local `x` 
> will be left bound to 2.

OK, it has been said that the priority is that

list(a_gen_expression)

Behave the same as

[the_same_expression]

So we’re good there. And maybe it’s  correct that leaving the running
of the gen_exp ‘till later is pretty uncommon, particularly for
newbies, but:

If the execution of the gen_exp is put off, it really confuses things
— that name being changed would happen at some arbitrary tone, and at
least in theory, the gen_exp could be passed off to somewhere else in
the code, and be run or not run completely remotely from where the
name is used.

So while this is technically the same as the comprehension, it is not
the same as a generator function which does get its own scope.

And we should be clear how it will work — after all, in py2, the
handling of the looping name was handled differently in gen_exp vs
comprehensions.

So I think a local scope for all comprehension-like things would be
the way to go.

But getting back to the original thread topic — python has a number of
places that you can only use expressions — adding the ability to bind
a name in all these places complicates the language significantly.

>   Put a body B in a listcomp and any side effects due to executing B

Maybe it’s just me, but re-binding a name seems like a whole new
category of side effect.

-CHB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Tim Peters
[Chris Barker]
>>> So what about:
>>>
>>> l = [x:=i for i in range(3)]
>>>
>>> vs
>>>
>>> g = (x:=i for i in range(3))
>>>
>>> Is there any way to keep these consistent if the "x" is in the regular
local scope?

[Tim]
>> I'm not clear on what the question is.  The list comprehension would
>> bind ` l ` to [0, 1, 2] and leave the local `x` bound to 2.  The second
>> example binds `g` to a generator object, which just sits there
>> unexecuted.  That has nothing to do with the PEP, though.
>>
>> If you go on to do, e.g.,
>>
>> l = list(g)
>>
>> then, same as the listcomp, `l` will be bound to [0, 1, 2] and the local
`x` will
>> be left bound to 2.

[Chris]
> OK, it has been said that the priority is that
>
> list(a_gen_expression)
>
> Behave the same as
>
> [the_same_expression]

That's certainly desirable.


> So we’re good there. And maybe it’s  correct that leaving the running
> of the gen_exp ‘till later is pretty uncommon, particularly for newbies,

Common or not, I have no idea why anyone would write a genexp like the one
you gave, except to contrive an example of silly behavior exhibited by
silly code ;-)

It's really not interesting to me to make up code as goofy as you can
conceive of - the interesting questions are about plausible code (including
plausible coding errors).

>  but:
>
> If the execution of the gen_exp is put off, it really confuses things
> — that name being changed would happen at some arbitrary tone, and at
> least in theory, the gen_exp could be passed off to somewhere else in
> the code, and be run or not run completely remotely from where the
> name is used.

Sure.

> So while this is technically the same as the comprehension, it is not
> the same as a generator function which does get its own scope.

It is the same as a generator function with appropriate scope declarations
- a generator expression is, after all, implemented _by_ a nested generator
function.  You can write a workalike to your code above today, but nobody
worries about that because nobody does that ;-)

def f():
def bashx(outermost):
nonlocal x
for i in outermost:
x = i
yield i

x = 12
g = bashx(range(3))
print("x before", x)
L = list(g)
print("L", L)
print("x after", x)

Then calling `f()` prints:

x before 12
L [0, 1, 2]
x after 2

> And we should be clear how it will work — after all, in py2, the
> handling of the looping name was handled differently in gen_exp vs
> comprehensions.

The PEP specifies the semantics.  If it's accepted, that will be folded
into the docs.

> So I think a local scope for all comprehension-like things would be
> the way to go.
>
> But getting back to the original thread topic — python has a number of
> places that you can only use expressions — adding the ability to bind
> a name in all these places complicates the language significantly.

Did adding ternary `if` (truepart if expression else falsepart) complicate
the language significantly?  Python has rarely expanded the number of
expression forms, but whenever it has the sky didn't actually fall despite
earnest warnings that disaster was inevitable ;-)

>>   Put a body B in a listcomp and any side effects due to executing B

> Maybe it’s just me, but re-binding a name seems like a whole new
> category of side effect.

With no trickery at all, you've always been able to rebind attributes, and
mutate containers, in comprehensions and genexps.  Because `for` targets
aren't limited to plain names; e.g.,

g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))

is already "legal", and will stomp all over the complex `for` targets when
executed - there's nothing "local" about them.  But nobody worries about
that because nobody does stuff like that.

And as in my goofy code above, mucking with binding of plain names is also
possible today.  Indeed, straightforward if that's what you _want_ to do.
But nobody does.

It's just not one of Python's goals to make it impossible to write useless
code ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Terry Reedy

On 6/28/2018 8:05 AM, Baptiste Carvello wrote:

Le 28/06/2018 à 01:31, Greg Ewing a écrit :

Well, I remain profoundly unconvinced that writing comprehensions
with side effects is ever a good idea, and Tim's examples did
nothing to change that.


Comprehensions with side effects feel scary indeed. But I could see
myself using some variant of the "cumsum" example (for scientific work
at the command prompt):


x=0; [x:=x+i for i in range(5)]


Creating an unneeded list with a comprehension purely for side effects 
is considered a bad idea by many.


x = 0
for i in range(5): x += i


Here the side effects are irrelevant, the "x" variable won't be reused.


If we ignore the side effect on x, the above is equivalent to 'pass' ;-)

Perhaps you meant

x = 0
cum = [x:=x+i for i in range(5)]

which is equivalent to

x, cum = 0, []
for i in range(5): x += i; cum.append(x)


But it needs to be initialized at the start of the comprehension.

I would happily get rid of the side-effects, but then what would be a
non-cryptic alternative to the above example?


The above as likely intended can also be written

import itertools as it
cum = list(it.accumulate(range(5)))

We have two good existing alternatives to the proposed innovation.

--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Chris Barker via Python-Dev
On Thu, Jun 28, 2018 at 9:28 AM, Tim Peters  wrote:

> >>> g = (x:=i for i in range(3))
> Common or not, I have no idea why anyone would write a genexp like the one
> you gave, except to contrive an example of silly behavior exhibited by
> silly code ;-)
>

yes, it was a contrived example, but the simplest one I could think of off
the top of my head that re-bound a name in the loop -- which was what I
thought was the entire point of this discussion?

If we think hardly anyone is ever going to do that -- then I guess it
doesn't matter how it's handled.

> So while this is technically the same as the comprehension, it is not
> > the same as a generator function which does get its own scope.
>
> It is the same as a generator function with appropriate scope declarations
> - a generator expression is, after all, implemented _by_ a nested generator
> function.  You can write a workalike to your code above today, but nobody
> worries about that because nobody does that ;-)
>
> def f():
> def bashx(outermost):
> nonlocal x
> for i in outermost:
> x = i
> yield i
>

but here the keyword "nonlocal" is used -- you are clearly declaring that
you are messing with a nonlocal name here -- that is a lot more obvious
than simply using a :=

And "nonlocal" is not used that often, and when it is it's for careful
closure trickery -- I'm guessing := will be far more common. And, of
course, when a newbie encounters it, they can google it and see what it
means -- far different that seeing a := in a comprehension and
understanding (by osmosis??) that it might make changes in the local scope.

And  I don't think you can even do that with generator expressions now --
as they can only contain expressions. Which is my point -- this would allow
the local namespace to be manipulated in places it never could before.

Maybe it's only comprehensions, and maybe it'll be rare to have a confusing
version of those, so it'll be no big deal, but this thread started talking
about educators' take on this -- and as an educator, I think this really
does complicate the language.

Python got much of it's "fame" by being "executable pseudo code" -- its
been moving farther and farther away from those roots. That's generally a
good thing, as we've gain expressiveness in exchangel, but we shouldn't
pretend it isn't happening, or that this proposal doesn't contribute to
that trend.

Did adding ternary `if` (truepart if expression else falsepart) complicate
> the language significantly?
>

I don't think so -- no. For two reasons:

1) the final chosen form is kind of verbose, but therefor more like
"executable pseudo code" :-) As apposed to the C version, for instance.

2) it added one new construct, that if, when someone sees it for the first
(or twenty fifth) time and doesn't understand it, they can look it up, and
find out. and it only effects that line of code.

So adding ANYTHING does complicate the language, by simply making it a bit
larger, but some things are far more complicating than others.

Python has rarely expanded the number of expression forms, but whenever it
> has the sky didn't actually fall despite earnest warnings that disaster was
> inevitable ;-)
>

Well, I've been surprised by what confused students before, and I will
again. But I dont hink there is any doubt that Python 3.7 is a notably
harder to learn that Python 1.5 was...


> > Maybe it’s just me, but re-binding a name seems like a whole new
> > category of side effect.
>
> With no trickery at all, you've always been able to rebind attributes, and
> mutate containers, in comprehensions and genexps.  Because `for` targets
> aren't limited to plain names; e.g.,
>
> g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))
>

sure, but you are explicitly using the names "object" and "a" here -- so
while side effects in comprehension are discouraged, it's not really a
surprised that namespaces specifically named are changed.

and this:

In [*55*]: x = 0


In [*56*]: [x *for* x *in* range(3)]

Out[*56*]: [0, 1, 2]


In [*57*]: x

Out[*57*]: 0

doesn't change x in the local scope -- if that was a good idea, why is a
good idea to have := in a comprehension effect the local scope??

But maybe it is just me.


> And as in my goofy code above, mucking with binding of plain names is also
> possible today.  Indeed, straightforward if that's what you _want_ to do.
> But nobody does.
>
> It's just not one of Python's goals to make it impossible to write useless
> code ;-)
>

 I suppose we need to go back and look at the "real" examples of where/how
folks think they'll use := in comprehensions, and see how confusing it may
be.

One of these conversations was started with an example something like this:

[(f(x), g(f(x))) for x in an_iterable]

The OP didn't like having to call f() twice. So that would become:

[ (temp:=f(x), g(temp)) for x in an_iterable]

so now the question is: should "temp" be created / changed i

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Greg Ewing

Baptiste Carvello wrote:

x=0; [x:=x+i for i in range(5)]


what would be a
non-cryptic alternative to the above example?


Personally I wouldn't insist on trying to do it with a
comprehension at all, but if forced to come up with a
readable syntax for that, it would probably be something
like

   [x for i in range(5) letting x = x + 1 given x = 0]

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] We now have C code coverage!

2018-06-28 Thread Terry Reedy

On 6/24/2018 5:03 AM, Ammar Askar wrote:

Is it possible, given that we are not paying for those reports, to
customize the 'exclude_lines' definitions?


Do you want to exclude python code or C code?


Python code.


For Python code, coverage.py also has some comments you can
put down to exclude lines:
http://coverage.readthedocs.io/en/coverage-4.2/excluding.html


Yes, by default, one can use '# pragma: no cover' and if one uses the 
--branch flag, '# pragma: no branch'.  For more 'advanced exclusion', 
one can use the following, normally in .coveragerc.

[report]
exclude_lines = ...
"This is useful if you have often-used constructs to exclude that can be 
matched with a regex. You can exclude them all at once without littering 
your code with exclusion pragmas."


For IDLE's test suite, I use a customized .coveragerc.  I strongly 
prefer to not abandon that and litter the code with # pragmas.


In order to make sense of the coverage report and have it be truthful, 
one needs to know what options are being used.

Is the --branch flag set?
Is .coveragerc or some other configuration file in use?
If so, what is the content?
Do we have any control over the use and content of exclusion settings?

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Tim Peters
[Chris]
> yes, it was a contrived example, but the simplest one I could think of off
> the top of my head that re-bound a name in the loop -- which was what I
> thought was the entire point of this discussion?

But why off the top of your head?  There are literally hundreds & hundreds
of prior messages about this PEP, not to mention that you could also find
examples in the PEP.  Why make up a senseless example?

> If we think hardly anyone is ever going to do that -- then I guess it
doesn't matter
> how it's handled.

So look at real examples.  One that's been repeated at least a hundred
times wants a local to "leak into" a listcomp:

total = 0
cumsums = [total ::= total + value for value in data]

As an educator, how are you going to explain that blowing up with
UnboundLocalError instead?  Do you currently teach that comprehensions and
genexps are implemented via invisible magically generated lexically nested
functions?  If not, you're going to have to start for people to even begin
to make sense of UnboundLocalError if `total` _doesn't_ "leak into" that
example.  My belief is that just about everyone who doesn't know "too much"
about the current implementation will be astonished & baffled if that
example doesn't "just work".

In other cases it's desired that targets "leak out":

while any(n % (divisor := p) == 0 for p in small_primes):
n //= divisor

And in still other cases no leaking (neither in nor out) is desired.

Same as `for` targets in that way,. but in the opposite direction:  they
don't leak and there's no way to make them leak, not even when that's
wanted.  Which _is_ wanted in the last example above, which would be
clearer still written as:

while any(n % p == 0 for p in small_primes):
n //= p

But that ship has sailed.


> ...
> And "nonlocal" is not used that often, and when it is it's for careful
closure
> trickery -- I'm guessing := will be far more common.

My guess (recorded in the PEP's Appendix A) is that assignment expressions
_overall_ will be used more often than ternary `if` but significantly less
often than augmented assignment.  I expect their use in genexps and
comprehensions will be minimal.  There are real use cases for them, but the
vast majority of genexps and comprehensions apparently have no use for them
at all.


> And, of course, when a newbie encounters it, they can google it and see
what
> it means -- far different that seeing a := in a comprehension and
understanding
> (by osmosis??) that it might make changes in the local scope.

Which relates to the above:  how do you teach these things?  The idea that
"a newbie" even _suspects_ that genexps and listcomps have something to do
with lexically nested scopes and invisible nested functions strikes me as
hilarious ;-)

Regardless of how assignment expressions work in listcomps and genexps,
this example (which uses neither) _will_ rebind the containing block's `x`:

[x := 1]

How then are you going to explain that this seemingly trivial variation
_doesn't_?

[x := 1 for ignore in "a"]

For all the world they both appear to be binding `x` in the code block
containing the brackets.  So let them.

Even worse,

[x for ignore in range(x := 1)]

will rebind `x` in the containing block _regardless_ of how assignment
expression targets are treated in "most of" a comprehension, because the
expression defining the iterable of the outermost "for" _is_ evaluated in
the containing block (it is _not_ evaluated in the scope of the synthetic
function).

That's not a special case for targets if they all "leak", but is if they
don't.


> And  I don't think you can even do that with generator expressions now --
as
> they can only contain expressions.

Expressions can invoke arbitrary functions, which in turn can do anything
whatsoever.

> Which is my point -- this would allow the local namespace to be
manipulated
> in places it never could before.

As above, not true.  However, it would make it _easier_ to write senseless
code mucking with the local namespace - if that's what you want to do.


> Maybe it's only comprehensions, and maybe it'll be rare to have a
confusing
> version of those, so it'll be no big deal, but this thread started
talking about
> educators' take on this -- and as an educator, I think this really does
> complicate the language.

I'll grant that it certainly doesn't simplify the language ;-)


> Python got much of it's "fame" by being "executable pseudo code" -- its
been
> moving farther and farther away from those roots. That's generally a good
thing,
> as we've gain expressiveness in exchangel, but we shouldn't pretend it
isn't
> happening, or that this proposal doesn't contribute to that trend.

I didn't say a word about that one way or the other.  I mostly agree, but
at the start Guido was aiming to fill a niche between shell scripting
languages and C.  It was a very "clean" language from the start, but not
aimed at beginners.  Thanks to his experience working on ABC, it carried
over some key ideas that wer

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

2018-06-28 Thread Terry Reedy

On 6/28/2018 11:21 PM, Tim Peters wrote:

[somewhere below] this is the last time I'm going to repeat it all again ;-)

For me, this is your most convincing exposition and summary of why the 
proposal is at least ok.  Thank you.




[Chris]
 > yes, it was a contrived example, but the simplest one I could think 
of off

 > the top of my head that re-bound a name in the loop -- which was what I
 > thought was the entire point of this discussion?

But why off the top of your head?  There are literally hundreds & 
hundreds of prior messages about this PEP, not to mention that you could 
also find examples in the PEP.  Why make up a senseless example?


 > If we think hardly anyone is ever going to do that -- then I guess it 
doesn't matter

 > how it's handled.

So look at real examples.  One that's been repeated at least a hundred 
times wants a local to "leak into" a listcomp:


total = 0
cumsums = [total ::= total + value for value in data]

As an educator, how are you going to explain that blowing up with 
UnboundLocalError instead?  Do you currently teach that comprehensions 
and genexps are implemented via invisible magically generated lexically 
nested functions?  If not, you're going to have to start for people to 
even begin to make sense of UnboundLocalError if `total` _doesn't_ "leak 
into" that example.  My belief is that just about everyone who doesn't 
know "too much" about the current implementation will be astonished & 
baffled if that example doesn't "just work".


In other cases it's desired that targets "leak out":

while any(n % (divisor := p) == 0 for p in small_primes):
     n //= divisor

And in still other cases no leaking (neither in nor out) is desired.

Same as `for` targets in that way,. but in the opposite direction:  they 
don't leak and there's no way to make them leak, not even when that's 
wanted.  Which _is_ wanted in the last example above, which would be 
clearer still written as:


while any(n % p == 0 for p in small_primes):
     n //= p

But that ship has sailed.


 > ...
 > And "nonlocal" is not used that often, and when it is it's for 
careful closure

 > trickery -- I'm guessing := will be far more common.

My guess (recorded in the PEP's Appendix A) is that assignment 
expressions _overall_ will be used more often than ternary `if` but 
significantly less often than augmented assignment.  I expect their use 
in genexps and comprehensions will be minimal.  There are real use cases 
for them, but the vast majority of genexps and comprehensions apparently 
have no use for them at all.



 > And, of course, when a newbie encounters it, they can google it and 
see what
 > it means -- far different that seeing a := in a comprehension and 
understanding

 > (by osmosis??) that it might make changes in the local scope.

Which relates to the above:  how do you teach these things?  The idea 
that "a newbie" even _suspects_ that genexps and listcomps have 
something to do with lexically nested scopes and invisible nested 
functions strikes me as hilarious ;-)


Regardless of how assignment expressions work in listcomps and genexps, 
this example (which uses neither) _will_ rebind the containing block's `x`:


[x := 1]

How then are you going to explain that this seemingly trivial variation 
_doesn't_?


[x := 1 for ignore in "a"]

For all the world they both appear to be binding `x` in the code block 
containing the brackets.  So let them.


Even worse,

[x for ignore in range(x := 1)]

will rebind `x` in the containing block _regardless_ of how assignment 
expression targets are treated in "most of" a comprehension, because the 
expression defining the iterable of the outermost "for" _is_ evaluated 
in the containing block (it is _not_ evaluated in the scope of the 
synthetic function).


That's not a special case for targets if they all "leak", but is if they 
don't.



 > And  I don't think you can even do that with generator expressions 
now -- as

 > they can only contain expressions.

Expressions can invoke arbitrary functions, which in turn can do 
anything whatsoever.


 > Which is my point -- this would allow the local namespace to be 
manipulated

 > in places it never could before.

As above, not true.  However, it would make it _easier_ to write 
senseless code mucking with the local namespace - if that's what you 
want to do.



 > Maybe it's only comprehensions, and maybe it'll be rare to have a 
confusing
 > version of those, so it'll be no big deal, but this thread started 
talking about

 > educators' take on this -- and as an educator, I think this really does
 > complicate the language.

I'll grant that it certainly doesn't simplify the language ;-)


 > Python got much of it's "fame" by being "executable pseudo code" -- 
its been
 > moving farther and farther away from those roots. That's generally a 
good thing,
 > as we've gain expressiveness in exchangel, but we shouldn't pretend 
it isn't

 > happening, or that this proposal doesn't contribute to that tr