Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Stefan Behnel]

Stephen J. Turnbull schrieb am 23.02.2018 um 03:31:
> Barry Warsaw writes:
>  > rather than having to pause to reason about what that 1-element
>  > list-like syntax actually means, and 2) will this encourage even
>  > more complicated comprehensions that are less readable than just
>  > expanding the code into a for-loop?
> 
> Of course it will encourage more complicated comprehensions, and we
> know that complexity is less readable.  On the other hand, a for loop
> with a temporary variable will take up at least 3 statements vs. a
> one-statement comprehension.

IMHO, any complex comprehension should be split across multiple lines,
definitely if it uses multiple for-loops, as in the discussed example. So
the "space win" of a complex comprehension that requires temporary values
over a multi-line for-statement is actually not big in these cases.

There are certainly cases where a comprehension still looks better, but I'm
all for not encouraging a hacky idiom to stuff more into a comprehension.
Comprehensions should be used to *improve* readabilty, not to reduce it.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Serhiy Storchaka]

22.02.18 23:33, Barry Warsaw пише:

On Feb 22, 2018, at 11:04, Serhiy Storchaka  wrote:


Stephan Houben proposed an idiom which looks similar to new hypothetic syntax:

result = [y + g(y) for x in range(10) for y in [f(x)]]

`for y in [expr]` in a comprehension means just assigning expr to y. I never 
seen this idiom before, but it can be a good replacement for a hypothetic 
syntax for assignment in comprehensions. It changes the original comprehension 
less than other approaches, just adds yet one element in a sequence of for-s 
and if-s. I think that after using it more widely it will become pretty 
idiomatic.


My questions are 1) will this become idiomatic enough to be able to understand 
at a glance what is going on, rather than having to pause to reason about what 
that 1-element list-like syntax actually means, and 2) will this encourage even 
more complicated comprehensions that are less readable than just expanding the 
code into a for-loop?


I think everyone will have to pause when encounter this idiom the first 
time. Next time it will look more common. But the same is happened with 
other idioms like "lambda x=x:", "'...' % (x,)", "x = x or {}", etc. 
This is a correct Python syntax, and you don't need to know anything 
special, besides learned from the tutorial, for understanding it.


All other alternatives (except the first one, which looks to me less 
readable than iterating a 1-element list) can't be used as an 
expression. Then require several statements. At least four statements in 
the case of a for-loop.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Paul Moore]

On 23 February 2018 at 09:12, Stefan Behnel  wrote:
> Stephen J. Turnbull schrieb am 23.02.2018 um 03:31:
>> Barry Warsaw writes:
>>  > rather than having to pause to reason about what that 1-element
>>  > list-like syntax actually means, and 2) will this encourage even
>>  > more complicated comprehensions that are less readable than just
>>  > expanding the code into a for-loop?
>>
>> Of course it will encourage more complicated comprehensions, and we
>> know that complexity is less readable.  On the other hand, a for loop
>> with a temporary variable will take up at least 3 statements vs. a
>> one-statement comprehension.
>
> IMHO, any complex comprehension should be split across multiple lines,
> definitely if it uses multiple for-loops, as in the discussed example. So
> the "space win" of a complex comprehension that requires temporary values
> over a multi-line for-statement is actually not big in these cases.
>
> There are certainly cases where a comprehension still looks better, but I'm
> all for not encouraging a hacky idiom to stuff more into a comprehension.
> Comprehensions should be used to *improve* readabilty, not to reduce it.

In my view:

1. The proposal is for an optimisation, not a change to the language.
So anything bad that can be done after the change, can be done now.
2. I doubt many people avoid this construct at the moment because it's
slow, it's more likely they do so because they hadn't thought of it,
or because it harms readability.
3. Announcing that this construct is no longer slow might encourage
some extra people to use it (because they now know about it, and they
assume that the fact that we've optimised it implies we think it's a
good idea).
4. Ultimately, readability will be the main factor here. And
readability is subjective, so we sort of have to trust people to use
their common sense.

This could easily be a premature optimisation. But on the other hand,
it's a case of not making things unexpectedly slow, so I'm fine with
that.

If Serihy doesn't feel that the optimisation code is a major
maintenance burden, I'd say go for it. It's a minor quality of life
improvement for a niche case, let's not view (or promote) it as
anything more than that.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Nick Coghlan]

On 23 February 2018 at 03:09, Eric Snow  wrote:
> So, coupled with slow CI, linting failures were
> particularly onerous.  We all got in the habit of locally running the
> linter frequently.  IIRC, I even set up VIM to run the linter whenever
> I saved.

For reindent.py, we used to have instructions for setting that up as a
local pre-commit hook in Mercurial. It may make sense to create
similar instructions for git (if we don't already have those
somewhere).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Wes Turner]

On Friday, February 23, 2018, Nick Coghlan  wrote:

> On 23 February 2018 at 03:09, Eric Snow 
> wrote:
> > So, coupled with slow CI, linting failures were
> > particularly onerous.  We all got in the habit of locally running the
> > linter frequently.  IIRC, I even set up VIM to run the linter whenever
> > I saved.
>
> For reindent.py, we used to have instructions for setting that up as a
> local pre-commit hook in Mercurial. It may make sense to create
> similar instructions for git (if we don't already have those
> somewhere).


A pre-commit hook with flake8, clang-format, and the requisite CPython
style configs would be great.

https://git-scm.com/book/gr/v2/Customizing-Git-Git-Hooks

Batch and ad-hoc CLN commits to cleanup for style really muddle the git
blame history.


>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

[Eric V. Smith]

On 2/22/18 9:43 PM, Nick Coghlan wrote:

On 22 February 2018 at 20:55, Eric V. Smith  wrote:

A related issue is that dataclasses derived from frozen dataclasses are
automatically "promoted" to being frozen.


@dataclass(frozen=True)

... class A:
... i: int
...

@dataclass

... class B(A):
... j: int
...

b = B(1, 2)
b.j = 3

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\home\eric\local\python\cpython\lib\dataclasses.py", line 452, in
_frozen_setattr
raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'j'

Maybe it should be an error to declare B as non-frozen?


It would be nice to avoid that, as a mutable subclass of a frozen base
class could be a nice way to model hashable-but-mutable types:

>>> @dataclass(frozen=True) # This is the immutable/hashable bit
... class A:
... i: int
...
>>> @dataclass # This adds the mutable-but-comparable parts
... class B(A):
... j: int
... __hash__ = A.__hash__


However, disallowing this case for now *would* be a reasonable way to
postpone making a decision until 3.8 - in the meantime, folks would
still be able to experiment by overriding __setattr__ and __delattr__
after the dataclass decorator sets them.


Yes, that's what I was thinking. If we can't come up with a good 
solution for this now, make it an error and think about it for 3.8.


I'm doing some experimenting on this and Raymond's case and I'll post 
some ideas this weekend.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Wes Turner]

On Wednesday, February 21, 2018, Nick Coghlan  wrote:

> On 22 February 2018 at 08:19, Barry Warsaw  wrote:
> > As for the bug tracker, I still do like Roundup, and we have a huge
> investment in it, not just in resources expended to make it rock, but also
> in all the history in it and everything that integrates with it.  I
> wouldn’t stop anybody who’s motivated to spearhead a move to GH issues, but
> I also don’t think that can be taken up lightly.
>
> [...]
>

> Personally, the only things I really miss on Roundup vs GitHub issues
> are usability tweaks like Markdown support in the comment editor, and
> inline dropdowns for @-mentions of other users and #-mentions of other
> issues, so if someone is motivated to work on issue tracking
> enhancements, that seems like a more fruitful endeavour than trying to
> migrate wholesale to a proprietary third party service.


Additional Roundup ENHancements:

- [ ] ENH: Issue mention 'trackbacks'
  Optionally quoting the full issue title inline instead of just #123 would
be great: "#123: Issue title"

- [ ] ENH: GitHub OAuth integration; to make it easy to login to Roundup
with GitHub credentials

  http://python-social-auth.readthedocs.io/en/latest/backends/github.html


>
> Cheers,
> Nick.
>
> P.S. That said, one tracker that I think absolutely *would* be worth
> migrating to GitHub is the meta-tracker at
> http://psf.upfronthosting.co.za/roundup/meta. We haven't customised
> that instance the way we have bugs.python.org, and consolidating it
> and the source repo at http://hg.python.org/tracker/roundup/ into a
> single https://github.com/python/bugs.python.org repo would better
> align tracker development with development on other parts of the
> infrastructure.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Oleg Broytman]

On Fri, Feb 23, 2018 at 08:12:13AM -0500, Wes Turner  
wrote:
> A pre-commit hook with flake8

   The problem now is not how to configure git with flake8 (et al) but
how to configure flake8 (and other tools) to minimize the noise with the
current codebase.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2018-02-16 - 2018-02-23)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6485 (+29)
  closed 38190 (+37)
  total  44675 (+66)

Open issues with patches: 2530 


Issues opened (54)
==

#31355: Remove Travis CI macOS job: rely on buildbots
https://bugs.python.org/issue31355  reopened by pitrou

#32008: Example suggest to use a TLSv1 socket
https://bugs.python.org/issue32008  reopened by christian.heimes

#32861: urllib.robotparser: incomplete __str__ methods
https://bugs.python.org/issue32861  opened by michael-lazar

#32862: os.dup2(fd, fd, inheritable=False) behaves inconsistently
https://bugs.python.org/issue32862  opened by izbyshev

#32865: os.pipe  creates inheritable FDs with a bad internal state on 
https://bugs.python.org/issue32865  opened by eryksun

#32866: zipimport loader.get_data() requires absolute zip file path
https://bugs.python.org/issue32866  opened by barry

#32867: argparse assertion failure with multiline metavars
https://bugs.python.org/issue32867  opened by MaT1g3R

#32871: Interrupt .communicate() on SIGTERM/INT
https://bugs.python.org/issue32871  opened by porton

#32872: backport of #32305 causes regressions in various packages
https://bugs.python.org/issue32872  opened by doko

#32873: Pickling of typing types
https://bugs.python.org/issue32873  opened by serhiy.storchaka

#32874: IDLE: Add tests for pyparse
https://bugs.python.org/issue32874  opened by csabella

#32875: Add __exit__() method to event loops
https://bugs.python.org/issue32875  opened by porton

#32876: HTMLParser raises exception on some inputs
https://bugs.python.org/issue32876  opened by hanno

#32877: Login to bugs.python.org with Google account NOT working
https://bugs.python.org/issue32877  opened by ruffsl

#32878: Document value of st_ino on Windows
https://bugs.python.org/issue32878  opened by gvanrossum

#32879: Race condition in multiprocessing Queue
https://bugs.python.org/issue32879  opened by TwistedSim

#32880: IDLE: Fix and update and cleanup pyparse
https://bugs.python.org/issue32880  opened by terry.reedy

#32881: pycapsule:PyObject * is NULL pointer
https://bugs.python.org/issue32881  opened by zhaoya

#32882: SSLContext.set_ecdh_curve() not accepting x25519
https://bugs.python.org/issue32882  opened by sruester

#32883: Key agreement parameters not accessible
https://bugs.python.org/issue32883  opened by sruester

#32884: Adding the ability for getpass to print asterisks when passowr
https://bugs.python.org/issue32884  opened by matanya.stroh

#32885: Tools/scripts/pathfix.py leaves bunch of ~ suffixed files arou
https://bugs.python.org/issue32885  opened by hroncok

#32886: new Boolean ABC in numbers module
https://bugs.python.org/issue32886  opened by smarie

#32887: os: Users of path_converter don't handle fd == -1 properly
https://bugs.python.org/issue32887  opened by izbyshev

#32888: Improve exception message in ast.literal_eval
https://bugs.python.org/issue32888  opened by Rosuav

#32890: os: Some functions may report bogus errors on Windows
https://bugs.python.org/issue32890  opened by izbyshev

#32891: Add 'Integer' as synonym for 'Integral' in numbers module.
https://bugs.python.org/issue32891  opened by terry.reedy

#32892: Remove specific constant AST types in favor of ast.Constant
https://bugs.python.org/issue32892  opened by serhiy.storchaka

#32893: ast.literal_eval() shouldn't accept booleans as numbers in AST
https://bugs.python.org/issue32893  opened by serhiy.storchaka

#32894: AST unparsing of infinity numbers
https://bugs.python.org/issue32894  opened by serhiy.storchaka

#32896: Error when subclassing a dataclass with a field that uses a de
https://bugs.python.org/issue32896  opened by John Didion

#32897: test_gdb failed on Fedora 27
https://bugs.python.org/issue32897  opened by amitg-b14

#32900: Teach pdb to step through asyncio et al.
https://bugs.python.org/issue32900  opened by smurfix

#32901: Update 3.7 and 3.8 Windows and macOS installer builds to tcl/t
https://bugs.python.org/issue32901  opened by terry.reedy

#32902: FileInput "inplace" redirects output of other threads
https://bugs.python.org/issue32902  opened by mj4int

#32903: os.chdir() may leak memory on Windows
https://bugs.python.org/issue32903  opened by izbyshev

#32904: os.chdir(), os.getcwd() may crash on Windows in presence of ra
https://bugs.python.org/issue32904  opened by izbyshev

#32907: pathlib: test_resolve_common fails on Windows
https://bugs.python.org/issue32907  opened by izbyshev

#32909: ApplePersistenceIgnoreState warning on macOS
https://bugs.python.org/issue32909  opened by cbrnr

#32910: venv: Deactivate.ps1 is not created when Activate.ps1 was used
https://bugs.python.org/issue32910  opened by godaygo

#32911: Doc strings no longer stored in body of AST
https://bugs.python.org/issue32911  opened by Mark.Shannon

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Guido van Rossum]

On Thu, Feb 22, 2018 at 11:04 AM, Serhiy Storchaka 
wrote:

> Yet one discussion about reusing common subexpressions in comprehensions
> took place last week on the Python-ideas maillist (see topic "Temporary
> variables in comprehensions" [1]). The problem is that in comprehension
> like `[f(x) + g(f(x)) for x in range(10)]` the subexpression `f(x)` is
> evaluated twice. In normal loop you can introduce a temporary variable for
> `f(x)`. The OP wanted to add a special syntax for introducing temporary
> variables in comprehensions. This idea already was discussed multiple times
> in the past.
>
> There are several ways of resolving this problem with existing syntax.
>
> 1. Inner generator expression:
>
> result = [y + g(y) for y in (f(x) for x in range(10))]
>
> 2. The same, but with extracting the inner generator expression as a
> variable:
>
> f_samples = (f(x) for x in range(10))
> result = [y+g(y) for y in f_samples]
>
> 3. Extracting the expression with repeated subexpressions as a function
> with local variables:
>
> def func(x):
> y = f(x)
> return y + g(y)
> result = [func(x) for x in range(10)]
>
> 4. Implementing the whole comprehension as a generator function:
>
> def gen():
> for x in range(10):
> y = f(x)
> yield y + g(y)
> result = list(gen())
>
> 5. Using a normal loop instead of a comprehension:
>
> result = []
> for x in range(10):
> y = f(x)
> result.append(y + g(y))
>
> And maybe there are other ways.
>
> Stephan Houben proposed an idiom which looks similar to new hypothetic
> syntax:
>
> result = [y + g(y) for x in range(10) for y in [f(x)]]
>
> `for y in [expr]` in a comprehension means just assigning expr to y. I
> never seen this idiom before, but it can be a good replacement for a
> hypothetic syntax for assignment in comprehensions. It changes the original
> comprehension less than other approaches, just adds yet one element in a
> sequence of for-s and if-s. I think that after using it more widely it will
> become pretty idiomatic.
>
> I have created a patch that optimizes this idiom, making it as fast as a
> normal assignment. [2] Yury suggested to ask Guido on the mailing list if
> he agrees that this language patten is worth optimizing/promoting.
>
> [1] https://mail.python.org/pipermail/python-ideas/2018-February
> /048971.html
> [2] https://bugs.python.org/issue32856
>

I'm not saying anything new here, but since you asked specifically for my
opinion: I don't care for the idiom; it's never occurred to me before, and
it smells of cleverness. If I saw it in a code review I would probably ask
for a regular for-loop to make the code more maintainable.

But if you say it's useful for some class of users and it would be more
useful if it was faster, I'm fine with the optimization. The optimization
is also clever, and here I appreciate cleverness!

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Chris Barker - NOAA Federal]

> Is it similar enough to
>
>def f(x=[0]):

No, not at all — it’s a very different use case.

When I first saw this on the original thread, I needed to stare at it
a good while, and then whip up some code to experiment with it to know
what it did.

And not because I don’t know what a single element list means, or what
it means to iterate over a single element list, or what two fors mean
in a comprehension.

I was confused by the ‘x’ in the second iterable. I guess I’m (still)
not really clear on the scope(s) inside a comprehension, and when the
elements get evaluated in a list.

I expected that the list would be created once, with the value x had
initially, rather than getting the-evaluated each time through the
outer loop.

So I think that it is a very confusing use of comprehensions, and
always will be. I’m still surprised it’s legal. Anyone know if this
being allowed was deliberate or just kind of fell out of the
implementation?

So no, I don’t think it should be promoted as idiomatic.

All that being said, it’s valid Python, so why not optimize it?

-CHB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Guido van Rossum]

As to the validity or legality of this code, it's both, and working as
intended.

A list comprehension of the form

[STUFF for VAR1 in SEQ1 for VAR2 in SEQ2 for VAR3 in SEQ3]

should be seen (informally) as

for VAR1 in SEQ1:
for VAR2 in SEQ2:
for VAR3 in SEQ3:
"put STUFF in the result"

(If there are `if COND` phrases too those get inserted into the nested set
of blocks where they occur in the sequence.)


On Fri, Feb 23, 2018 at 9:41 AM, Chris Barker - NOAA Federal <
chris.bar...@noaa.gov> wrote:

> > Is it similar enough to
> >
> >def f(x=[0]):
>
> No, not at all — it’s a very different use case.
>
> When I first saw this on the original thread, I needed to stare at it
> a good while, and then whip up some code to experiment with it to know
> what it did.
>
> And not because I don’t know what a single element list means, or what
> it means to iterate over a single element list, or what two fors mean
> in a comprehension.
>
> I was confused by the ‘x’ in the second iterable. I guess I’m (still)
> not really clear on the scope(s) inside a comprehension, and when the
> elements get evaluated in a list.
>
> I expected that the list would be created once, with the value x had
> initially, rather than getting the-evaluated each time through the
> outer loop.
>
> So I think that it is a very confusing use of comprehensions, and
> always will be. I’m still surprised it’s legal. Anyone know if this
> being allowed was deliberate or just kind of fell out of the
> implementation?
>
> So no, I don’t think it should be promoted as idiomatic.
>
> All that being said, it’s valid Python, so why not optimize it?
>
> -CHB
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Chris Barker]

On Fri, Feb 23, 2018 at 9:51 AM, Guido van Rossum  wrote:

> As to the validity or legality of this code, it's both, and working as
> intended.
>
> A list comprehension of the form
>
> [STUFF for VAR1 in SEQ1 for VAR2 in SEQ2 for VAR3 in SEQ3]
>
> should be seen (informally) as
>
> for VAR1 in SEQ1:
> for VAR2 in SEQ2:
> for VAR3 in SEQ3:
> "put STUFF in the result"
>

Thanks -- right after posting, I realized that was the way to unpack it to
understand it. I think my confusion came from two things:

1) I usually don't care in which order the loops are ordered -- i.e., that
could be:

for VAR3 in SEQ3:
for VAR2 in SEQ2:
for VAR1 in SEQ1:
"put STUFF in the result"

As I usually don't care, I have to think about it (and maybe experiment to
be sure). (this is the old Fortran vs C order thing :-)

2) since it's a single expression, I wasn't sure of the evaluation order,
so maybe (in my head) it could have been (optimized) to be:

[STUFF for  VAR1 in Expression_that_evaluates_to_an_iterable1 for VAR2 in
Expression_that_evaluates_to_an_iterable2]

and that could translate to:

IT1 = Expression_that_evaluates_to_an_iterable1
IT2 = Expression_that_evaluates_to_an_iterable2
for VAR1 in IT1:
for VAR2 in IT2:
"put STUFF in the result"

In which case, VAR1 would not be available to
Expression_that_evaluates_to_an_iterable2.

Maybe that was very wrong headed -- but that's where my head went -- and
I'm not a Python newbie (maybe an oddity, though :-) )

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Guido van Rossum]

There are useful things you can only do with comprehensions if the second
for-loop can use the variable in the first for-loop. E.g.

[(i, j) for i in range(10) for j in range(i)]

On Fri, Feb 23, 2018 at 10:16 AM, Chris Barker 
wrote:

> On Fri, Feb 23, 2018 at 9:51 AM, Guido van Rossum 
> wrote:
>
>> As to the validity or legality of this code, it's both, and working as
>> intended.
>>
>> A list comprehension of the form
>>
>> [STUFF for VAR1 in SEQ1 for VAR2 in SEQ2 for VAR3 in SEQ3]
>>
>> should be seen (informally) as
>>
>> for VAR1 in SEQ1:
>> for VAR2 in SEQ2:
>> for VAR3 in SEQ3:
>> "put STUFF in the result"
>>
>
> Thanks -- right after posting, I realized that was the way to unpack it to
> understand it. I think my confusion came from two things:
>
> 1) I usually don't care in which order the loops are ordered -- i.e., that
> could be:
>
> for VAR3 in SEQ3:
> for VAR2 in SEQ2:
> for VAR1 in SEQ1:
> "put STUFF in the result"
>
> As I usually don't care, I have to think about it (and maybe experiment to
> be sure). (this is the old Fortran vs C order thing :-)
>
> 2) since it's a single expression, I wasn't sure of the evaluation order,
> so maybe (in my head) it could have been (optimized) to be:
>
> [STUFF for  VAR1 in Expression_that_evaluates_to_an_iterable1 for VAR2 in
> Expression_that_evaluates_to_an_iterable2]
>
> and that could translate to:
>
> IT1 = Expression_that_evaluates_to_an_iterable1
> IT2 = Expression_that_evaluates_to_an_iterable2
> for VAR1 in IT1:
> for VAR2 in IT2:
> "put STUFF in the result"
>
> In which case, VAR1 would not be available to Expression_that_evaluates_
> to_an_iterable2.
>
> Maybe that was very wrong headed -- but that's where my head went -- and
> I'm not a Python newbie (maybe an oddity, though :-) )
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE
> 
>   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Chris Barker]

On Fri, Feb 23, 2018 at 10:45 AM, Guido van Rossum  wrote:

> There are useful things you can only do with comprehensions if the second
> for-loop can use the variable in the first for-loop. E.g.
>
> [(i, j) for i in range(10) for j in range(i)]
>

indeed -- and that is fairly common use-case in nested for loops -- so good
to preserve this.

But I still think the original:

[g(y) for x in range(5) for y in [f(x)]]

Is always going to be confusing to read. Though I do agree that it's not
too bad when you unpack it into for loops:

In [89]: for x in range(5):
...: for y in [f(x)]:
...: l.append(g(y))

BTW, would it be even a tiny bit more efficient to use a tuple in the inner
loop?

[g(y) for x in range(5) for y in (f(x),)]

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Stefan Behnel]

Chris Barker schrieb am 23.02.2018 um 20:23:
> BTW, would it be even a tiny bit more efficient to use a tuple in the inner
> loop?
> 
> [g(y) for x in range(5) for y in (f(x),)]

Serhiy's optimisation does not use a loop at all anymore and folds it into
a direct assignment "y=f(x)" instead.

But in general, yes, changing a list iterable into a tuple is an
improvement as tuples are more efficient to allocate. Haven't tried it in
CPython (*), but it might make a slight difference for very short
iterables, which are probably common. Although the execution of the loop
body will likely dominate the initial allocation by far.

Stefan



(*) I implemented this list->tuple transformation in Cython a while ago,
but seeing Serhiy's change now got me thinking that this could be further
improved into a stack allocated C array, to let the C compiler unroll the
loop at will. I'll probably try that at some point...

https://github.com/cython/cython/issues/2117

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Stephen J. Turnbull]

Chris Barker writes:

 > But I still think the original:
 > 
 > [g(y) for x in range(5) for y in [f(x)]]
 > 
 > Is always going to be confusing to read.

But the point I was making with "def f(x=[0]):" was this: you have a
situation where your desired semantics is "value of some type"[1], but
the language's syntax doesn't permit a value of that type there, while
"singleton sequence of that type" works fine.

In fact, "singleton as value" is baked into Python in the form of
str.__getitem__ and bytes.__getitem__.  So we now have four use cases
for singleton as value: two stringish actual types, and the two idioms
"mutable default argument" and "local variable in comprehension".  The
horse is long since out of the barn.

Steve

Footnotes: 
[1]  Both "value" and "type" are used rather loosely here.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Steven D'Aprano]

On Fri, Feb 23, 2018 at 11:23:04AM -0800, Chris Barker wrote:

> But I still think the original:
> 
> [g(y) for x in range(5) for y in [f(x)]]
> 
> Is always going to be confusing to read. Though I do agree that it's not
> too bad when you unpack it into for loops:
> 
> In [89]: for x in range(5):
> ...: for y in [f(x)]:
> ...: l.append(g(y))


I think we should distinguish between: 

* actively confusing; and 
* merely not obvious at a glance.

I acknowledge that this idiom is not obvious at a glance, but I don't 
think this comes even close to actively confusing. As you say, once you 
mentally unpack the loops it becomes clear.

Given a potentially expensive DRY violation like:

[(function(x), function(x)+1) for x in sequence]

there are at least five ways to solve it.

* Perhaps it doesn't need solving; if the DRY violation is trivial 
enough, and the cost low enough, who cares?

* Re-write as a for-loop instead of a comprehension;

* Use a helper function:

def helper(x)
tmp = function(x)
return (tmp, tmp+1)

[helper(x) for x in sequence]

* Chain the operations:

[(a, a+1) for a in (function(x) for x in sequence)]

[(a, a+1) for a in map(function, sequence)]

* Use a second loop to get an assignment:

[(a, a+1) for x in sequence for a in [function(x)]]


I don't think we need to promote any one of the above as the One True 
idiom. They're all simple, more-or-less obvious or at least 
understandable, and deciding between them should be a matter of personal 
choice and in-house style guides.

 
> BTW, would it be even a tiny bit more efficient to use a tuple in the inner
> loop?
> 
> [g(y) for x in range(5) for y in (f(x),)]

The suggested patch will recognise both `y in (a,)` and `y in [a]` and 
treat them the same as a direct assignment `y=a`.

But if you're writing cross-interpreter code which might run on older 
versions of Python, or implementations which may not have this 
optimization, you might prefer to micro-optimize by using a tuple.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[David Mertz]

On Feb 23, 2018 9:26 PM, "Steven D'Aprano"  wrote:

Given a potentially expensive DRY violation like:

[(function(x), function(x)+1) for x in sequence]

there are at least five ways to solve it.


A 6th way is to wrap the expensive function in @lru_cache() to make it
non-expensive.


[(a, a+1) for x in sequence for a in [function(x)]]


It's funny to me how many people, even the BDFL, have said this is tricky
to reason about or recognize. I didn't think of it all by myself, but saw
it somewhere years ago. It seemed obvious once I saw it. Since then it's
something I do occasionally in my code without much need for thought.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Nick Coghlan]

On 24 February 2018 at 06:50, Stefan Behnel  wrote:

> But in general, yes, changing a list iterable into a tuple is an
> improvement as tuples are more efficient to allocate. Haven't tried it in
> CPython (*), but it might make a slight difference for very short
> iterables, which are probably common.


CPython has included the list->tuple conversion for lists of literals for
quite some time, and Serhiy just posted a patch to extend that to all
inline lists where it's a safe change to make:
https://bugs.python.org/issue32925

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[David Mertz]

FWIW, the nested loop over a single item is already in the language for 15
years or something. It's not that ugly, certainly not enough to need a new
'let' or 'where' keyword that basically does exactly the same thing with 3
fewer characters.

On Feb 23, 2018 10:04 PM, David Mertz  wrote:


On Feb 23, 2018 9:26 PM, "Steven D'Aprano"  wrote:

Given a potentially expensive DRY violation like:

[(function(x), function(x)+1) for x in sequence]

there are at least five ways to solve it.


A 6th way is to wrap the expensive function in @lru_cache() to make it
non-expensive.


[(a, a+1) for x in sequence for a in [function(x)]]


It's funny to me how many people, even the BDFL, have said this is tricky
to reason about or recognize. I didn't think of it all by myself, but saw
it somewhere years ago. It seemed obvious once I saw it. Since then it's
something I do occasionally in my code without much need for thought.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Barry Warsaw]

On Feb 21, 2018, at 19:03, Dan Stromberg  wrote:
> 
> Is flake8 that much better than pylint, that pylint wouldn't even be 
> discussed?

I honestly don’t use pylint all that much these days, but when I was evaluating 
them years ago, I found pylint to be too noisy about not-incorrect code.  I 
also liked how flake8 has a very nice plugin system, and I use that on my 
personal projects to enforce a consistent style.  It’s certainly possible that 
both tools have advanced significantly since I last made my personal decision.

I agree with Guido in general though, that we’d need to do some actual analysis 
on the stdlib and see what the results are.  I do think that a plugin system, 
or really good configurability, will help us tailor whatever tool we decide to 
use, so that it’s only warning about the things we care about and ignoring the 
rest, without requiring lots of comment pragmas in the code.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Barry Warsaw]

On Feb 22, 2018, at 02:12, Antoine Pitrou  wrote:

> Everytime I contribute to a project which has automatic style checks in
> CI, I find myself having to push pointless "cleanup" commits because
> the CI is barking at me for some ridiculous reason (such as putting two
> linefeeds instead of one between two Python functions).  Then I have to
> wait some more until CI finishes, especially if builds take long or
> build queues are clogged.
> 
> Overall it makes contributing more of a PITA than it needs be.  Do
> automatic style *fixes* if you want, but please don't annoy me with
> automatic style checks that ask me to do tedious grunt work on my spare
> time.

Look at it from the other side though.  If I’m reviewing someone’s PR, and the 
style is so inconsistent with the existing code base, then I’m distracted by 
constantly asking the contributor to clean up little stylistic things.  This 
wastes my time as a reviewer, where I’d rather be thinking about the 
substantive content of the change.  To have no style checks can leave you with 
a chaotic jumbled ness that is much harder to navigate around, understand, and 
contribute to by others, and once you’ve gone down that path, you just build up 
technical debt to eventually try to clean it up.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Barry Warsaw]

> On Feb 22, 2018, at 08:08, Antoine Pitrou  wrote:
> 
> That's a fair point I hadn't considered.  OTOH the style issues I
> usually comment on as a reviewer aren't the kind that would be caught
> by an automated style check (I tend to ask for comments or docstrings,
> or be nitpicky about some variable or function name).  YMMV :-)

Those are aesthetic comments that are important, and are also difficult to 
automate.  What I really don’t want to comment on are things like whitespace 
(vertical, horizontal, trailing), import order or format, consistent 
indentation, etc.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Barry Warsaw]

On Feb 22, 2018, at 09:09, Eric Snow  wrote:
> 
> I had exactly that experience on one particularly large Go project (on
> GitHub, with slow CI, driven by bots).

One thing that’s nice to set up if you can is multiple, parallel, independent 
CI runs.  E.g. if your full test suite takes a long time to run, but you can 
run your code style tests pretty quickly (e.g. flake8 and mypy), then you can 
get more immediate feedback about your PR code style, and it’s not so painful 
if your test suite is slow.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com