Re: [Python-Dev] Fix Unicode-disabled build of Python 2.7

[Terry Reedy]

On 6/24/2014 4:22 AM, Serhiy Storchaka wrote:

I submitted a number of patches which fixes currently broken
Unicode-disabled build of Python 2.7 (built with --disable-unicode
configure option). I suppose this was broken in 2.7 when C
implementation of the io module was introduced.

http://bugs.python.org/issue21833 -- main patch which fixes the io
module and adds helpers for testing.

http://bugs.python.org/issue21834 -- a lot of minor fixes for tests.

Following issues fix different modules and related tests:


This list and more to follow suggests that --disable-unicode was 
somewhat broken long before 2.7 and the introduction of _io.



http://bugs.python.org/issue21854 -- cookielib
http://bugs.python.org/issue21838 -- ctypes
http://bugs.python.org/issue21855 -- decimal
http://bugs.python.org/issue21839 -- distutils
http://bugs.python.org/issue21843 -- doctest
http://bugs.python.org/issue21851 -- gettext
http://bugs.python.org/issue21844 -- HTMLParser
http://bugs.python.org/issue21850 -- httplib and SimpleHTTPServer
http://bugs.python.org/issue21842 -- IDLE
http://bugs.python.org/issue21853 -- inspect
http://bugs.python.org/issue21848 -- logging
http://bugs.python.org/issue21849 -- multiprocessing
http://bugs.python.org/issue21852 -- optparse
http://bugs.python.org/issue21840 -- os.path
http://bugs.python.org/issue21845 -- plistlib
http://bugs.python.org/issue21836 -- sqlite3
http://bugs.python.org/issue21837 -- tarfile
http://bugs.python.org/issue21835 -- Tkinter
http://bugs.python.org/issue21847 -- xmlrpc
http://bugs.python.org/issue21841 -- xml.sax
http://bugs.python.org/issue21846 -- zipfile

Most fixes are trivial and are only several lines of a code.




--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 471 -- os.scandir() function -- a better and faster directory iterator

[Terry Reedy]

On 6/30/2014 9:44 PM, Ethan Furman wrote:

On 06/30/2014 06:28 PM, Ben Hoyt wrote:

I suppose the exact behavior is still under discussion, as there are
only
two or three fields one gets "for free" on Windows (I think...),
where as an
os.stat call would get everything available for the platform.


No, Windows is nice enough to give you all the same stat_result fields
during scandir (via FindFirstFile/FindNextFile) as a regular
os.stat().


Very nice.  Even less reason then to throw it away.  :)


I agree.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] buildbot.python.org down again?

[Terry Reedy]

On 7/6/2014 7:54 PM, Ned Deily wrote:

As of the moment, buildbot.python.org seems to be down again.


Several hours later, back up.

> Where is the best place to report problems like this?

We should have, if not already, an automatic system to detect down 
servers and report (email) to appropriate persons.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] == on object tests identity in 3.x

[Terry Reedy]

On 7/7/2014 7:22 AM, Andreas Maier wrote:

While discussing Python issue #12067
(http://bugs.python.org/issue12067#msg222442), I learned that Python 3.4
implements '==' and '!=' on the object type such that if no special
equality test operations are implemented in derived classes, there is a
default implementation that tests for identity (as opposed to equality
of the values).

The relevant code is in function do_richcompare() in Objects/object.c.

IMHO, that default implementation contradicts the definition that '=='
and '!=' test for equality of the values of an object.


A discrepancy between code and doc can be solved by changing either the 
code or doc. This is a case where the code should not change (for back 
compatibility with long standing behavior, if nothing else) and the doc 
should.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bluetooth 4.0 support in "socket" module

[Terry Reedy]

On 7/14/2014 9:57 AM, Tim Tisdall wrote:

2 questions not answered yet.


Also, is there a method to test changes against all the different *nix
variations?


We have a set of buildbots.
https://www.python.org/dev/buildbot/


Is Bluez the standard across the different *nix variations?


No idea.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] sum(...) limitation

[Terry Reedy]

On 8/2/2014 1:57 AM, Allen Li wrote:

On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:

No. We just can't put all possible use cases in the docstring. :-)


On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini  wrote:

 help(sum) tells clearly that it should be used to sum numbers and not
 strings, and with strings actually fails.

 However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.

 Is this to be considered a bug?


Can you explain the rationale behind this design decision?  It seems
terribly inconsistent.  Why are only strings explicitly restricted from
being sum()ed?  sum() should either ban everything except numbers or
accept everything that implements addition (duck typing).


O(n**2) behavior, ''.join(strings) alternative.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] class Foo(object) vs class Foo: should be clearly explained in python 2 and 3 doc

[Terry Reedy]

On 8/9/2014 2:44 PM, John Yeuk Hon Wong wrote:

Hi.

Referring to my discussion on [1] and then on #python this afternoon.

A little background would help people to understand where this was
coming from.

1. I write Python 2 code and have done zero Python-3 specific code.
2. I have always been using class Foo(object) so I do not know the new
style is no longer required in Python 3. I feel "stupid" and "wrong" by
thinking (object) is still a convention in Python 3.


If someone else tried to make you feel that way, they are Code of 
Conduct violators who should be ignored. If you are beating yourself on 
the head, stop.



3. Many Python 2 tutorials do not use object as the base class whether
for historical reason, or lack of information/education,


Probably both. Either way, the result is a disservice to readers.


and can cause confusing to newcomers searching for answers

> when they consult the official documentation.

I and some other people STRONGLY recommend that newcomers start with 
Python 3 and Python 3 docs and completely ignore Python 2 unless they 
cannot.



While Python 3 code no longer requires object be the base class for the
new-style class definition, I believe (object) is still required if one
has to write a 2-3 compatible code. But this was not explained or warned
anywhere in Python 2 and Python 3 code, AFAIK. (if I am wrong, please
correct me)

I propose the followings:

* It is desirable to state boldly to users that (object) is no longer
needed in Python-3 **only** code and warn users to revert to (object)
style if the code needs to be 2 and 3 compatible.


I think 'boldly' and 'warn' are a bit overstated.


* In addition, Python 2 doc [2] should be fixed by introducing the
new-style classes.


Definitely. The 2.x tutorial start with class x: and continues that way 
half way through the chapter.  I think it should start with class 
x(object): and at the end of the first half, briefly mention that class 
x in 2.x gets something slightly different that beginners can mostly 
ignore, while class x: in 3.x == class x(object): and that the latter 
works the same for both.


The 3.x tutorial, in the same place could *briefly* mention that class 
x: == class x(object): and the the latter is usually only used in code 
that also runs on 2.x or has been converted without removing the extra 
code.  The 3.x tutorial should *not* mention old style classes.


> This problem was noted a long long time ago according to [4].

The opening statement "Unfortunately, new-style classes have not yet 
been integrated into Python's standard documention." is perhaps a decade 
out of date.  That page should not have been included in the new site 
design without being modified.



[1]: https://news.ycombinator.com/item?id=8154471

[2]: https://docs.python.org/2/tutorial/classes.html
https://docs.python.org/3/tutorial/classes.html

[3]: https://docs.python.org/3/tutorial/classes.html

[4]: https://www.python.org/doc/newstyle/


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python2.7 infinite recursion when loading pickled object

[Terry Reedy]

On 8/11/2014 5:10 AM, Schmitt Uwe (ID SIS) wrote:

Python usage questions should be directed to python-list, for instance.


I discovered a problem using cPickle.loads from CPython 2.7.6.


The problem is your code having infinite recursion. You only discovered 
it with pickle.




The last line in the following code raises an infinite recursion

 class T(object):

 def __init__(self):
 self.item = list()

 def __getattr__(self, name):
 return getattr(self.item, name)


This is a (common) bug in your program.  __getattr__ should call 
self.__dict__(name) to avoid the recursion.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] sum(...) limitation

[Terry Reedy]

On 8/11/2014 8:26 AM, Ben Hoyt wrote:

It seems to me this is something of a pointless discussion -- I highly
doubt the current situation is going to change, and it works very well.
Even if not perfect, sum() is for numbers, sep.join() for strings.
However, I will add one comment:

I'm overall -1 on trying to change the current situation (except for
adding a join() builtin or str.join class method).


Did you know there actually is a str.join "class method"?


A 'method' is a function accessed as an attribute of a class.
An 'instance method' is a method whose first parameter is an instance of 
the class. str.join is an instance method.  A 'class method', wrapped as 
such with classmether(), usually by decorating it with @classmethod, 
would take the class as a parameter.



I've never
actually seen it used this way, but for people who just can't stand
sep.join(seq), you can always call str.join(sep, seq) -- works in Python
2 and 3:

 >>> str.join('.', ['abc', 'def', 'ghi'])
'abc.def.ghi'


One could even put 'join = str.join' at the top of a file.

All this is true of *every* instance method.  For instance

int.__add__(1, 2) == 1 .__add__(2) == 1 + 2

True

However, your point that people who cannot stand the abbreviation 
*could* use the full form that is being abbreviated.



In ancient Python, when strings did not have methods, the current string 
methods were functions in the string module. The functions were removed 
in 3.0.  Their continued use in 2.x code is bad for 3.x compatibility, 
so I would not encourage it.


>>> help(string.join)  # 2.7.8
Help on function join in module string:

join(words, sep=' ')
join(list [,sep]) -> string

Return a string composed of the words in list, with
intervening occurrences of sep.  The default separator is a
single space.

'List' is obsolete.  Since sometime before 2.7, 'words' meant an 
iterable of strings.


>>> def digits():
for i in range(10):
yield str(i)

>>> string.join(digits(), '')
'0123456789'

Of of the string functions, I believe the conversion of join (and its 
synonum 'joinfields') to a method has been the most contentious.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reviving restricted mode?

[Terry Reedy]

On 8/13/2014 12:19 PM, matsjoyce wrote:

Unless you remove all the things labelled "keep away from children". I wrote
this sandbox to allow python to be used as a "mods"/"add-ons" language for a
game I'm writing, hence the perhaps too strict nature.

About the crashers: as this is for games, its "fine" for the game to crash,
as long as the sandbox is not broken while crashing.

time and math can probably be allowed, but random imports a lot of
undesirable modules.

My sandbox doesn't use proxies, due to the introspection and complexity that
it involves. Instead it completely isolates the sandboxed globals, and checks
all arguments and globals for irregularities before passing control to non-
sandboxed functions.


pydev is for mainly for discussion of maintaining current versions and 
development of the next, and for discussion of PEPs which might apply to 
the one after next.


This discussion should be on python-list or perhaps python-ideas if 
there is a semi-concrete proposal for a future python.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray

[Terry Reedy]

On 8/18/2014 12:04 PM, Chris Barker wrote:

On Sun, Aug 17, 2014 at 2:41 PM, Barry Warsaw mailto:ba...@python.org>> wrote:

I think the biggest API "problem" is that default iteration returns
integers
instead of bytes.  That's a real pain.


what is really needed for this NOT to be a pain is a byte scalar.


The byte scalar is an int in range(256). Bytes is an array of such.


numpy has a scalar type for every type it supports -- this is a GOOD
THING (tm):

In [53]: a = np.array((3,4,5), dtype=np.uint8)

In [54]: a
Out[54]: array([3, 4, 5], dtype=uint8)

In [55]: a[1]
Out[55]: 4

In [56]: type(a[1])
Out[56]: numpy.uint8

In [57]: a[1].shape
Out[57]: ()


The lack of a  character type is a major source of "type errors" in
python (the whole list of strings vs a single string problem -- both
return a sequence when you index into them or iterate over them)


This is exactly what iterbytes would do  -- yields bytes of size 1.


Anyway, the character ship has long since sailed, but maybe a byte
scalar would be a good idea?

And FWIW, I think the proposal does make for a better, cleaner API.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)

[Terry Reedy]

Firefox does not want to connect to https:bugs.python.org. Plain 
bugs.python.org works fine. Has the certificate expired?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Terry Reedy]

On 8/20/2014 9:01 AM, Antoine Pitrou wrote:

Le 20/08/2014 07:08, Nick Coghlan a écrit :


It's not just the JVM that says text and binary APIs should be separate
- it's every widely used operating system services layer except POSIX.
The POSIX way works well *if* everyone reliably encodes things as UTF-8
or always uses encoding detection, but its failure mode is unfortunately
silent data corruption.

That said, there's a lot of Python software that is POSIX specific,
where bytes paths would be the least of the barriers to porting to
Windows or Jython. I'm personally +1 on consistently allowing binary
paths in lower level APIs, but disallowing them in higher level
explicitly cross platform abstractions like pathlib.


I fully agree with Nick's position here.

To elaborate specifically about pathlib, it doesn't handle bytes paths
but allows you to generate them if desired:
https://docs.python.org/3/library/pathlib.html#operators

Adding full bytes support to pathlib would have added a lot of
complication and fragility in the implementation *and* in the API (is it
allowed to combine str and bytes paths? should they have separate
classes?), for arguably little benefit.


I am glad you did not recreate the madness of pre 3.0 Python in that regard.


I think if you want low-level features (such as unconverted bytes paths
under POSIX), it is reasonable to point you to low-level APIs.


Do our docs somewhere explain the idea that files names are conceptually 
*names*, not arbitrary bytes; explain the concept of low-level versus 
high-level API' and point to the two types of APIs in Python?


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 4000 to explicitly declare we won't be doing a Py3k style compatibility break again?

[Terry Reedy]

On 8/20/2014 8:27 PM, Joseph Martinot-Lagarde wrote:


The pain was even bigger because in addition to the change in underlying
types, the names of the types were not compatible between the python
versions. I often try to write compatible code between python2 and 3,
and I can't use "str" because it has not the same meaning in both
versions, I can not use "unicode" because it disappeared in python3,


And bridge library should have the equivalent of
if 'py3': unicode = str


I can't use "byte" because it doesn't exist in python2.


2.7 (and 2.6?) already has
if 'py2': bytes = str
and I presume bridge libraries targeted before that was added include it 
also.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)

[Terry Reedy]

On 8/21/2014 10:41 AM, Armin Rigo wrote:

Hi,

On 18 August 2014 22:30, Oleg Broytman  wrote:

Aha, I see now -- the signing certificate is CAcert, which I've
installed manually.


I don't suppose anyone is particularly annoyed by this fact?


I noticed the issue, and started this thread, because someone posted an 
https::/bugs.python.org link. I ordinarily just go to bugs.python.org 
and get the http connection.  I have https-anywhere installed, but it 
must notice the dodgy certificate and silently not switch. So I never 
knew before tht there was an https connection available, and never 
thought to try it.


Given that we are shipping both login credentials and files over the 
connection, making https routine, with a proper certificate, might be a 
good idea.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)

[Terry Reedy]

On 8/21/2014 7:25 PM, Nick Coghlan wrote:


On 22 Aug 2014 04:45, "Benjamin Peterson" mailto:benja...@python.org>> wrote:
 >
 > Perhaps some board members could comment, but I hope the PSF could just
 > pay a few hundred a year for a proper certificate.

That's exactly what we're doing - MAL reminded me we reached the same
conclusion last time this came up, we'll just track it better this time
to make sure it doesn't slip through the cracks again.

(And yes, switching to forced HTTPS once this is addressed would also be
a good idea - we'll add it to the list)


I just switched from a 'low variety' short password of the sort almost 
crackable with brute force (today, though not several years ago) to a 
higher variety longer password. People with admin privileges on the 
tracker might be reminded to recheck.  What was adequate 10 years ago is 
not so now.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Terry Reedy]

On 8/26/2014 9:11 AM, R. David Murray wrote:

On Sun, 24 Aug 2014 13:27:55 +1000, Nick Coghlan  wrote:

As some examples of where bilingual computing breaks down:

* My NFS client and server may have different locale settings
* My FTP client and server may have different locale settings
* My SSH client and server may have different locale settings
* I save a file locally and send it to someone with a different locale setting
* I attempt to access a Windows share from a Linux client (or vice-versa)
* I clone my POSIX hosted git or Mercurial repository on a Windows client
* I have to connect my Linux client to a Windows Active Directory
domain (or vice-versa)
* I have to interoperate between native code and JVM code

The entire computing industry is currently struggling with this
monolingual (ASCII/Extended ASCII/EBCDIC/etc) -> bilingual (locale
encoding/code pages) -> multilingual (Unicode) transition. It's been
going on for decades, and it's still going to be quite some time
before we're done.

The POSIX world is slowly clawing its way towards a multilingual model
that actually works: UTF-8
Windows (including the CLR) and the JVM adopted a different
multilingual model, but still one that actually works: UTF-16-LE


Nick, I think the first half of your post is one of the clearest 
expositions yet of 'why Python 3' (in particular, the str to unicode 
change).  It is worthy of wider distribution and without much change, it 
would be a great blog post.



This kind of puts the "length" of the python2->python3 transition
period in perspective, doesn't it?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 477: selected ensurepip backports for Python 2.7

[Terry Reedy]

On 8/31/2014 6:00 PM, Nick Coghlan wrote:

Earlier versions of PEP 453 proposed bootstrapping pip into a Python 2.7
maintenance release in addition to including it with Python 3.4.

That part of the proposal proved to be controversial, so we dropped it
from the original PEP in order to focus on meeting the Python 3.4
specific release deadlines. This also had the benefit of working out the
kinks in the bootstrapping processing as part of the Python 3.4 release
cycle.

However, we still think we should start providing pip by default to
Python 2.7 users as well, at least as part of the Windows and Mac OS X
installers.


Having used pip for 3.4 on two windows computers, the only reason I see 
to *not* install it with Python would be 'push' people to 3.4+ by 
keeping 2.7 harder to use.


> One notable difference from PEP 453 is that because there is no venv
> module in 2.7, and hence no integration between venv and ensurepip,
> we can give redistributors the option of just disabling ensurepip
> entirely and redirecting users to platform specific installation
> tools.

This also suggests that ensurepip does not have to be left on a Win/OS 
install either, after installation is done.  Perhaps after ensuring that 
pip is installed, ensurepip could remove itself. This would remove any 
objection that its presence in /lib constitutes a new feature (which it 
is, even if not very useful).  It would also keep /lib uniform across 
platforms.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)

[Terry Reedy]

On 9/2/2014 1:49 AM, Oleg Broytman wrote:

Hi!

On Mon, Sep 01, 2014 at 08:32:27PM -0500, Skip Montanaro 
 wrote:

I got the same in Chrome on my Mac.

Skip
On Sep 1, 2014 8:00 PM, "John Wong"  wrote:


As of today I still am getting untrusted cert thought I would re-ping to
see if there is an ETA.


The signing certificate is still CAcert. One can install their root
certificate from http://www.cacert.org/index.php?id=3


This seems not to work for Firefox. "Windows installer package for 
browsers that use the Windows certificate store (for example Internet 
Explorer, Chrome on Windows and Safari on Windows)"


I installed it anyway, closed and reopened Firefox (but not rebooted) 
and https://bugs.python.org still gives Untrusted message.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Terry Reedy]

On 9/2/2014 7:47 PM, Glyph Lefkowitz wrote:


On Sep 2, 2014, at 4:28 PM, Nick Coghlan mailto:ncogh...@gmail.com>> wrote:


On 3 Sep 2014 09:08, "David Reid" mailto:dr...@dreid.org>> wrote:



> Clearly this change should be backported to Python2.

Proposing to break backwards compatibility in a maintenance release (...)


For code that depends on a bug, fixing the bug breaks compatibility 
between releases without and with the fix.  Nothing new.  We 
occasionally don't backpart disruptive bugfixes because of that. We also 
backport some security fixes to otherwise unmaintained versions.



As we keep saying, this is not a break in backwards compatibility, it's
a bug fix.


Disagreement on whether changing the default for certificate validation 
is a 'bugfix' or 'enhancement' (and in the colloquial or python tracker 
sense) the seems like the crux of the disagreement here.  The case for 
'bug', at least in the colloquial meaning, is at least plausible. On the 
other hand, the tracker meaning of 'behavior' issue (contrary to 
intention and doc) is more strict in including implementation bugs but 
not necessarily design bugs.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Terry Reedy]

On 9/26/2014 1:03 PM, Chris Barker wrote:

On Thu, Sep 25, 2014 at 5:38 PM, Donald Stufft mailto:don...@stufft.io>> wrote:


2) Switch to —user based on if the user has permission to
write to the
site-packages or not.


ouch -- no. Why not a clear error message if pip can't write to
site-packages -- something like:

I fairly strongly believe that the current default is doing a great
disservice
to users. I believe that for *most* people --user is the correct
option for
them to be using and the fact that it's not the default and requires
opt in
is a historical artifact more than anything else.


OK -- fine -- I think that history may be important -- at least for Py2,
but I agree that --user is a better default.

But what I'm objecting to is is switching the install mode based on the
permissions that the user happens to be running with at the time.

None of us should be routinely running as admin. So what I do (and I
image a LOT of people do) is try to do whatever I need to do, and only
if I get an error to I switch to admin mode (or add sudo, or ???). What
I'm suggesting is that folks that simply expect stuff to get installed
into the usual system python is not thinking ahead of time "oh, I need
be admin for this", but rather, simply do it, and then, if you get a
permission error, make the switch to either admin in mode, or add --user.

If we do switch the default to --user, then this issue goes away.

In short -- too much magic is bad.


Pip on Windows should act like a normal Windows program.  If I install 
Python for all users, I expect pipped packages to be installed for all 
users too, unless I specify otherwise.  If installation (for all users) 
requires admin privileges, I expect a UAC box to pop up and ask for the 
admin password.  This is pretty routine, at least with Win7.  Most every 
program I install does this either on installation or on first running. 
 Some Windows operations also pop up a box.  There are only a few 
things that require that I actually login as an admin user.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] 3.4.2rc1 online docs: cannot access tkinter, idle pages

[Terry Reedy]

I can access every page I have tried except for the tkinter and idle 
pages, which either give interminable 'Connecting...' or in one try, 404.

https://docs.python.org/3/library/tkinter.html
https://docs.python.org/3/library/idle.html

This is true from the index, previous/next from adjacent pages, and the 
index page

https://docs.python.org/3/library/tk.html

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.4.2rc1 online docs: cannot access tkinter, idle pages

[Terry Reedy]

On 10/5/2014 11:13 PM, Chris Angelico wrote:

On Mon, Oct 6, 2014 at 6:49 AM, Terry Reedy  wrote:

I can access every page I have tried except for the tkinter and idle pages,
which either give interminable 'Connecting...' or in one try, 404.
https://docs.python.org/3/library/tkinter.html
https://docs.python.org/3/library/idle.html


Is this still the case? I just tried them and they work for me.


Me too, now.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Review tool not detecting all changed files

[Terry Reedy]

On 10/14/2014 8:24 PM, Saimadhav Heblikar wrote:

Hi,

We were working on IDLE related issue [1] , when I noticed that the
review tool does not detect all affected files for the
cfg-ext-34-2.diff patch uploaded by Terry Reedy. Version 1 of the same
patch does not have this issue - the only difference between the two
files being line endings and time stamps. Also see Terry Reedy's
message in the same issue. [3]


Version 3 and 4 also works fine.


Could someone please let me know if this is normal behavior or not?


[1] - http://bugs.python.org/issue3068
[2] - http://bugs.python.org/file36904/cfg-ext-34-2.diff
[3] - http://bugs.python.org/issue3068#msg229315




--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Half the 'stable' buildbots are not running.

[Terry Reedy]

4 offline, 2 not compiling, 1 not compiling completely (no _ctypes == 
venv fail)


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] https://docs.python.org/3/using/index.html not linking correctly

[Terry Reedy]

If I go to https://docs.python.org/3/using/index.html and click on any 
of the TOC entries, I get 'connecting' indefinitely.  This problem seems 
unique to this file. I tried several other index files and clicking am 
entry brings up the corresponding page almost immediately.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https://docs.python.org/3/using/index.html not linking correctly

[Terry Reedy]

On 10/20/2014 7:29 PM, MRAB wrote:

On 2014-10-21 00:09, Eli Bendersky wrote:



On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote:

If I go to https://docs.python.org/3/using/index.html and click on
any of the TOC entries, I get 'connecting' indefinitely.  This
problem seems unique to this file. I tried several other index files
and clicking am entry brings up the corresponding page almost
immediately.


Works fine for me, Terry, in both Chrome and Firefox. Could be something
in your browser/caching? Try "private mode" or whatever that's called in
your browser of choice.


Just tried the first link:

https://docs.python.org/3/using/cmdline.html

I also get 'Connecting' indefinitely.

So I tested the link here:

http://www.downforeveryoneorjustme.com/

It said: "It's not just you! http://https looks down from here."


I am using Firefox, but the difference is that by happenstance, I was in 
privacy mode (perhaps for the first time with python.org -- I want the 
docs in the cache and history list -- because I accessed the docs to 
answer a StackOverflow question.) It is not just that document but also 
Execution model and Expression in Reference.  (I tried each 3 times 
minutes apart.)  I have not had a similiar problem with any Arstechnica 
or StackOverflow pages, or similar sites.


So the solution for me is to use normal mode, but something seems not 
quite right in the interaction between privacy mode Firefox and python.org.


No problem I could find with Internet Explorer's 'In Privacy' mode.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of C compilers for Python on Windows

[Terry Reedy]

On 10/25/2014 5:11 PM, Chris Angelico wrote:


It might fragment the community to have multiple different binary
distributions. But it ought to be possible for any person/organization
to say "We're going to make our own build of Python, with these
extension modules, built with this compiler, targeting this platform",
and do everything from source. That might mean they can no longer take
the short-cut of "download someone's MSVC-built extension and use it
as-is", but they should be able to take anyone's extension and build
it on their chosen compiler. Having MinGW as a formally supported
platform would make life a lot easier for people who want to test
CPython patches, for instance - my building and testing of PEP
463-enhanced Python was Linux-only, because I didn't want to try to
set up an entire new buildchain just to try to get a Windows binary
going. There's absolutely no need for that to be binary-compatible
with anything else; as long as it'll run the standard library, it'll
do.


David Murray's unanswered post laid out the path to move in the 
direction you want.  Either take it yourself or try to persuade other 
MinGW fans to do so.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

[Terry Reedy]

On 10/27/2014 12:23 PM, Stefan Richthofer wrote:



You mean Jython deletes instance attributes before calling __del__ ?


No. I think the term of "object resurrection" usually does not mean
bringing
back a deleted object in the sense that memory was already freed.
I think it rather means that nothing referred to an object, so it was on
the
"kill-list" of gc or zero-ref-count macro.


In either case, there is a final reference keeping the object alive, 
like an hospital patient kept alive by a final link with a life-support 
machine.  I think 'resuscitation' might be a better metaphor.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of C compilers for Python on Windows

[Terry Reedy]

On 10/29/2014 4:05 PM, Paul Moore wrote:

On 29 October 2014 15:31, Nathaniel Smith  wrote:

You can use Express editions of Visual Studio.


IIUC, the express edition compilers are 32-bit only, and what you actually
want are the "SDK compilers":
https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows

These are freely downloadable by anyone, no msdn subscription required, but
only if you know where to find them!

AFAICT the main obstacle to using MSVC to build python extensions (assuming
it can handle your code at all) is not anything technical, but rather that
there's no clear and correct tutorial on how to do it, and lots of confusion
and misinformation circulating.


Would it help if I wrote a document explaining how to set up the MS
compilers (free and paid for) to allow building of Python extensions?



There are a few provisos:

1. A lot of it will be pretty trivial: "If you have Vistal Studio
(full edition), install it. Done."
2. It will be out of date very fast as the situation for Python 3.5+
will be trivial across the board.
3. I don't have anywhere particularly authoritative to host it (maybe
the Python Wiki?) and it could easily get lost in the huge swamp of
variously outdated, over-complicated, or otherwise alternative
documents available. Ideally I'd like someone to suggest an "official"
location I could use.


There is already a section in the devguide for building Python itself, 
with mostly the same info, except it may not be up to date.



I don't want to do this if it won't be useful, as it'll take me a bit
of effort to confirm the process for the only non-trivial scenario
(64-bit Python 3.3/3.4 with free tools). But if people think it would
help, that's fine, I volunteer.


The devguide needs to be kept up to date.  If you open a tracker issue, 
put me as nosy to review and commit.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of C compilers for Python on Windows

[Terry Reedy]

On 10/29/2014 11:37 AM, Steve Dower wrote:


My ideal target (Utopia refined to be achievable) is for python-dev
to handle the Python binaries themselves (already done) and to make
them easy to bundle with applications/etc (I'm working on this for
3.5), and for PyPI to include pre-built wheels for Windows where
necessary.

Note that I don't require package developers to build their own
wheels, though I think that they are generally the right people to be
doing it. It would be nice to create a culture of delegation, so that
"someone" could be a trusted builder for a range of packages


Christoph Gohlke, Laboratory for Fluorescence Dynamics, University of 
California, Irvine now provides 32 and 64 bit builds for 2.6/7, 3.2/3/4 
(10 total) for about 300 'scientific open-source extension packages' an 
his site.

http://www.lfd.uci.edu/~gohlke/pythonlibs/

Perhaps the PyPI folks could discuss incorporating wheel versions on 
PyPI (I have no idea what is involved, but I presume it could be antomated.)


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of C compilers for Python on Windows

[Terry Reedy]

On 10/30/2014 8:59 AM, R. David Murray wrote:

On Wed, 29 Oct 2014 23:33:06 -0400, Terry Reedy  wrote:



The devguide needs to be kept up to date.  If you open a tracker issue,
put me as nosy to review and commit.


The devguide is about building python itself.  Paul is talking about
building extensions.  That should go in the packaging docs.  I don't
*think* Paul is volunteering to explain how to build python itself,
that's pretty much our job :)


I was thinking that building Python and extensions used the same tools, 
but then I read that the new compiler for 2.7 was extensions only and 
ditto for something else.  If VC Express 2010 is in a new location, that 
*should* be recorded in the devguide.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] problem with hg.python.org?

[Terry Reedy]

My normally dependable pull.bat script has 3 times given me this.

F:\Python\dev>hg --repository F:\Python\dev\5\py35 pull --verbose 
--config ui.merge=internal:merge ssh://h...@hg.python.org/cpython

pulling from ssh://h...@hg.python.org/cpython
searching for changes
all local heads known remotely
adding changesets
adding manifests
adding file changes
transaction abort!
rollback completed
abort: data/Doc/howto/webservers.rst.i@5f4ad429ae9f: unknown parent!
remote: Unable to write to standard output: The pipe is being closed.

Since I have no 'data/Doc/i@5f9f, I am guessing that the problem 
is not local.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] problem with hg.python.org?

[Terry Reedy]

On 11/1/2014 11:03 PM, Donald Stufft wrote:



On Nov 1, 2014, at 10:37 PM, Terry Reedy  wrote:



abort: data/Doc/howto/webservers.rst.i@5f4ad429ae9f: unknown parent!
remote: Unable to write to standard output: The pipe is being closed.



The internet suggests trying hg verify, does that do anything?


I should have though of that.  60+ integrity errors.  I am recloning.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] OneGet provider for Python

[Terry Reedy]

On 11/14/2014 7:12 PM, Vincent Povirk wrote:


If anyone has questions about OneGet generally, you should probably
ask them directly (see https://github.com/OneGet/oneget), as I am not
a definitive source of information on the project.


No subscription needed to follow a thread.  Python-dev is mirrored at 
news.gmane.net. One can either 'subscribe' to the newsgroup interface in 
a mai/news reader, as I do (OutlookExpress years ago, Thunderbird now), 
or use the webgroup interface.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Windows Dedicated Mailing List

[Terry Reedy]

On 11/15/2014 3:01 AM, Shorya Raj wrote:

Fair enough. Like I said, I wanted to see what the consensus was, and it
seems to be shifting towards not making another list.


Agreed.  The traffic here is not high enough to need a split.  Most OS 
specific issues that need a patch end up as a specific issue on the 
tracker with a limited subset of us as nosy.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem displaying the warning symbol

[Terry Reedy]

On 11/16/2014 11:49 AM, R. David Murray wrote:

On Sun, 16 Nov 2014 11:23:41 +0100, Stefano Borini 
 wrote:

I report a finding (bug?) about the warning unicode symbol, as reported here

http://stackoverflow.com/questions/26919799/cannot-make-warning-sign-visible-on-osx-terminal


Stefano, the right place to send an enquiry about a bug is python-list, 
not py-dev.



Bugs should be reported to bugs.python.org.


But only if one is fairly sure there is a bug.  The tracker is not for 
'is this a bug' questions.



I'm doubtful this is a Python problem though.  More likely it is an
issue with the curses library on OSX.


Warning signs display fine in Idle's tkinter Text widget with Lucida 
Console font, and python utf-8 encodes and decodes the char.


>>> s='\u26a0'
>>> s
'⚠'  # up-pointing triangle /_\ with ! inside
>>> b=s.encode('utf-8')
>>> b
b'\xe2\x9a\xa0'
>>> b.decode('utf-8')
'⚠'

Presuming that the bytes are correct, there is no bug here.

--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Static checker for common Python programming errors

[Terry Reedy]

On 11/17/2014 9:49 AM, Stefan Bucur wrote:

I'm developing a Python static analysis tool that flags common
programming errors in Python programs. The tool is meant to complement
other tools like Pylint (which perform checks at lexical and syntactic
level) by going deeper with the code analysis and keeping track of the
possible control flow paths in the program (path-sensitive analysis).

For instance, a path-sensitive analysis detects that the following
snippet of code would raise an AttributeError exception:

if object is None: # If the True branch is taken, we know the object is None
   object.doSomething() # ... so this statement would always fail

I'm writing first to the Python developers themselves to ask, in their
experience, what common pitfalls in the language & its standard library
such a static checker should look for. For instance, here [1] is a list
of static checks for the C++ language, as part of the Clang static
analyzer project.


You could also a) ask on python-list (new thread), or scan python 
questions on StackOverflow.  Todays's example: "Why does my function 
return None?"  Because there is no return statement.  Perhaps current 
checkers can note that, but what about if some branches have a return 
and others do not?  That is a likely bug.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators

[Terry Reedy]

On 11/20/2014 5:04 PM, Guido van Rossum wrote:

On Thu, Nov 20, 2014 at 12:13 PM, Serhiy Storchaka mailto:storch...@gmail.com>> wrote:

On 20.11.14 21:58, Antoine Pitrou wrote:

To me "generator_return" sounds like the addition to generator
syntax
allowing for return statements (which was done as part of the "yield
from" PEP). How about "generate_escape"?

Or may be "generator_stop_iteration"?

Or just "generator_stop"?


+1
About as sort as we could get for the informed user reader and neutrally 
cryptic so a naive reader will know to look it up rather than guess 
(wrongly)

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators

[Terry Reedy]

On 11/20/2014 2:36 PM, Guido van Rossum wrote:


There's still a lively discussion on python-ideas; Steven D'Aprano has
dug up quite a bit of evidence that StopIteration is used quite a bit in
ways that will break under the new behavior, and there also seems to be
quite a bit of third-party information that recommends StopIteration
over return to terminate a generator early.

However I don't see much evidence that the current behavior is *better*
than the proposal -- I see the current behavior as a definite wart, and
I have definitely seen people struggle to debug silent early loop
termination due to an "escaped" StopIteration.

That said, I think for most people the change won't matter, some people
will have to apply one of a few simple fixes, and a rare few will have
to rewrite their code in a non-trivial way (sometimes this will affect
"clever" libraries).

I wonder if the PEP needs a better transition plan, e.g.

- right now, start an education campaign


For StackOverflow, someone with a high-enough rep (you, Guido?) should 
add a tag for StopIteration with text like "The Python `StopIteration` 
exception."  (Copied from the current AttributeError tag, typeerror and 
NameError also exist.).  If and when the PEP is accepted, I would be 
willing to add comments and the new tag to existing and future 
questions/answers.



- with Python 3.5, introduce "from __future__ import generator_return",
and silent deprecation warnings
- with Python 3.6, start issuing non-silent deprecation warnings


I agree that two deprecation periods are needed.


- with Python 3.7, make the new behavior the default (subject to some
kind of review)

It would also be useful if we could extend the PEP with some examples of
the various categories of fixes that can be applied easily, e.g. a few
examples of "raise StopIteration" directly in a generator that can be
replaced with "return" (or omitted, if it's at the end); a few examples
of situations where "yield from" can supply an elegant fix (and an
alternative for code that needs to be backward compatible with Python
3.2 or 2.7); and finally (to be honest) an example of code that will
require being made more complicated.


Agree that explicit fixes are needed.  Deprecation message could refer 
to section of PEP.



Oh, and it would also be nice if the PEP included some suggested words
that 3rd party educators can use to explain the relationship between
StopIteration and generators in a healthier way (preferably a way that
also applies to older versions).

Chris, are you up to drafting these additions?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators

[Terry Reedy]

On 11/22/2014 2:49 PM, Ron Adam wrote:


On 11/22/2014 08:31 AM, Nick Coghlan wrote:



I'm definitely coming around to the point of view that, even if we
wouldn't
design it the way it currently works given a blank slate, the alternative
design doesn't provide sufficient benefit to justify the cost of changing
the behaviour & getting people to retrain their brains.


Me too.


Which gets us back to generator expressions and comprehensions.


> Comprehensions are used as a convenient way to create an object.  The

expression parts of the comprehension define the *body* of a loop, so a
StopIteration raised in it will bubble out.  As it would in any other
case where it is raised in the body of a loop.

Generator expressions on the other hand define the *iterator* to be used
in a for loop. A StopIteration raised in it is caught by the for loop.

So they both work as they are designed, but they look so similar, it
looks like one is broken.



It looks to me that there are three options...


OPTION 1:

Make comprehensions act more like generator expressions.


I was thinking about this also.


It would mean a while loop in the object creation point is converted to
a for loop. (or something equivalent.)


The code for [e(i) for i in it] already uses a for loop.  The issue is 
when e(i) raise StopIteration.  The solution to accomplish the above is 
to wrap the for loop in try: ... except StopIteration: pass.


def _list_comp_temp():
ret = []
try:
for i in it:
ret.append[e(i)]
except StopIteration:
pass
return ret

I believe this would make list(or set)(genexp) == [genexp] (or {genexp}) 
as desired.  In 2.x, there was a second difference, the leaking of 'i'. 
 3.x eliminated one difference, the leaking of the iteration variable, 
but not the other, the leaking of StopIteration.


The document about execution of comprehensions says "the comprehension 
is executed in a separate scope, so names assigned to in the target list 
don’t “leak”. What we would be adding for comprensions (but not genexps) 
is "and StopIteration raised in evaluating the expression" before 
'"leak"'.  We could then document the equality above.



Then both a comprehension and a generator expressions can be viewed as
defining iterators,


A comprehension is not an iterator.  The above would make a list or set 
comprehension the same as feeding a genexp to list() or set().



I think this fits with what Guido wants, but does so in a narrower
scope, only effecting comprehensions.  StopIteration is less likely to
leak out.


StopIteration would *not* leak out ever.  This is half of what Guido 
wants, the other half being the issue of obscure bugs with genrators in 
general.



But it also allows the use of the stop() hack to raise StopIteration in
comprehensions and terminate them early.


Yep. There is no good way to have next(it) work as intended, including 
in Raymond's example, where it should work, and not let stop() work.  I 
think this falls under our 'consenting adults' principle.



Currently it doesn't work as it does in generator expressions.
If the stop() hack works in both comprehensions and generator
expressions the same way, then maybe we can view it as less of a hack.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators

[Terry Reedy]

On 11/22/2014 5:23 PM, Chris Angelico wrote:

On Sun, Nov 23, 2014 at 8:03 AM, Ron Adam  wrote:

Making comprehensions work more like generator expressions
would, IMO, imply making the same change to all for loops: having a
StopIteration raised by the body of the loop quietly terminate the
loop.



I'm not suggesting making any changes to generator expressions or for loops
at all.  They would continue to work like they currently do.


But if you're suggesting making list comps react to StopIteration
raised by their primary expressions, then to maintain the
correspondence between a list comp and its expanded form, for loops
would have to change too. Or should that correspondence be broken, in
that single-expression loop constructs become semantically different
from statement loop constructs?


The 2.x correspondence between list comp and for loops was *already 
broken* in 3.0 by moving execution to a new function to prevent name 
binding in comprehension from leaking the way they did in 2.x.  We did 
not change for loops to match.


The following, which is an example of equivalence in 2.7, raises in 3.4 
because 'itertools' is not defined.


def binder(ns, name, ob):
ns[name] = ob

for mod in ['sys']:
binder(locals(), mod, __import__(mod))
print(sys)

[binder(locals(), mod, __import__(mod)) for mod in ['itertools']]
print(itertools)

Ceasing to leak *any* local bindings in 3.0 broke equivalence and code 
depending on such bindings.  Ceasing to leak StopIteration would break 
equivalence a bit more, but only a bit, and code dependent on such 
leakage, which I expect is extremely rare.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 2:33 PM, Donald Stufft wrote:


So a goal of mine here is to sort of use these as a bit of a test bed.
Moving CPython itself is a big and drastic change with a lot of
implications, but moving the “support” repositories is not nearly as
much, especially with a read only mirror on hg.python.org
 which would allow things like the PEP rendering
on www.python.org  to stay the same if we wanted
to. My hope was that we’d try this out, see how it works out, and if it
seems to be a good thing, then at a later time we can either look at
moving CPython itself or decide if it makes sense to do something
different. Maybe this should be spelled out in the PEP?

I’ve seen a few people say they were -1 because they didn’t want to
split between hg on the CPython side and git on the supporting repos
side.


Being only recently somewhat comfortable with hg, I do not really want 
to have to learn git at this time.



I’m not sure you can really get away from that because we’re
*already* in that situation, things like the docs building script is a
Git repo on Github,


As long as I can do test doc builds with the scripts in /Docs, this does 
not matter to me.  If the people who work with the site doc build script 
want it in git on Github, fine.



the python infrastructure itself is a git repo on Github,


Since I am not sure what you mean, its location does not matter to me.

> the new python.org  website is a git repo on

Github, the future PyPI is a git repo on GitHub.


Since I do not commit to either, ditto.  As far as I am concerned, the 
people involved with specialized repositories may as well have their 
preference.  This includes devinabox.


I have made a few commits to the PEP repository, but only to the one I 
co-authored, 434. Using the web form might make it easier to suggest 
changes to other peoples' PEPs.



IOW I’m not sure what the best way forward is. I think moving to these
tools for *all* repos is likely to be in the best interests of making
things better for both sides of that coin however I didn’t want to go
wholesale and try and make everything switch at all at once.


I agree with others that the current bottleneck is disposing of patches, 
especially those from non-core developers, not getting more patches to 
considier.  I am quite sure that if we significantly reduce the current 
backlog of 4600 issues and more expeditiously handles the contributions 
we do get, will will get more.  More than one person has said that they 
are disinclined to submit patches when they have multiple patches on the 
tracker already.


So I think the focus should be on making better use of developer time 
and having more of it.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 1:05 PM, Guido van Rossum wrote:

I don't feel it's my job to accept or reject this PEP, but I do have an
opinion.

...

- I am basically the only remaining active PEP editor, so I see most PEP
contributions by non-core-committers. Almost all of these uses github.
Not bitbucket, not some other git host, but github. I spend a fair
amount of time applying patches. It would most definitely be easier if I
could get them to send me pull requests.


The scope of the PEP is still apparently somewhat fluid.  I said 
elsewhere that I think the principal maintainers of a specialized 
single-branch repository should have the principal say in where it 
lives.  So I think you should be the one to decide on a PEP limited to 
moving the PEP repository.


My understanding is that if the peps were moved to github, then I would 
be able to suggest changes via an online web form that would generate a 
pull request from edited text.  If so, I would say go ahead and move 
them and see how it goes.


To me, the multibranch CPython repository is a very different issue. I 
think it should stay where it is for now, especially with 2.7 support 
extended.  I think for this we should better focus on better use of 
developer time and getting more developers active.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode decode exception

[Terry Reedy]

On 11/30/2014 3:07 AM, balaji marisetti wrote:

Hi,

When I try to iterate through the lines of a
file("openssl-1.0.1j/crypto/bn/asm/x86_64-gcc.c"), I get a
UnicodeDecodeError (in python 3.4.0 on Ubuntu 14.04). But there is no
such error with python 2.7.6. What could be the problem?


Questions about using current version should be directed to python-list 
or other support forums.  Py-dev is for development of future versions 
and releases.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] advice needed: best approach to enabling "metamodules"?

[Terry Reedy]

On 11/30/2014 2:27 PM, Ethan Furman wrote:

On 11/30/2014 11:15 AM, Guido van Rossum wrote:

On Sun, Nov 30, 2014 at 6:15 AM, Brett Cannon wrote:

On Sat, Nov 29, 2014, 21:55 Guido van Rossum wrote:


All the use cases seem to be about adding some kind of getattr hook
to modules. They all seem to involve modifying the CPython C code
anyway. So why not tackle that problem head-on and modify module_getattro()
to look for a global named __getattr__ and if it exists, call that instead
of raising AttributeError?


Not sure if anyone thought of it. :) Seems like a reasonable solution to me.
Be curious to know what the benchmark suite said the impact was.


Why would there be any impact? The __getattr__ hook would be similar to the
one on classes -- it's only invoked at the point where otherwise AttributeError
would be raised.


I think the bigger question is how do we support it back on 2.7?


I do not understand this question. We don't add new features to 2.7 and 
this definitely is one.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 4:45 PM, Donald Stufft wrote:

I think you are stimulating more heated discussion than is necessary by 
trying to do too much, both in terms of physical changes and in terms of 
opinion persuasion.


I am reminded of the integer division change.  The initial discussion 
was initially over-heated partly because the actual change to Python was 
coupled, quite unnecessarily, with a metaphysical opinion as the the 
ontological status of natural numbers versus integers -- an opinion that 
many, including me, considered to be wrong.  Once the metaphysical 
question was dropped, discussion went more smoothly.



Here’s another idea for an experiment that might be more generally useful.


I think a true experiment with one repository is easily justified.  The 
PEP repository is an obvious choice because a) the main editor is in 
favor, b) many but not all core committers interact with it, and c) it 
is not tied to the tracker.  The easier and more controversial option 
would be to move it completely to GitHub.  I expect part of the result 
would be pull requests from committers who would otherwise commit 
directly to hg.  The harder and, I think, more useful (generalizable) 
option would be to set up a mirror (or keep the hg version as a mirror 
;-) and experiment with coordinating the two mirrors.


Such an experiment should not preclude other experiments.  If Brett 
wants to similarly experiment with devinabox on another site, let him. 
If the horrible-to-Nick prospect of possibly moving CPython to GitHub, 
if nothing else is done, provokes Nick to improve the workflow 
otherwise, great.


If the mirror experiment is successful, the devguide might be the next 
experiment.  It does not have any one maintainer, and *is* tied to the 
tracker.  But herein lies the problem with the devguide.  There are 22 
issues, down just 1 from about a year ago.  All but 2 are more than a 
year old.  Many (most?) have patches, but enough consensus for anyone to 
push is hard.  As with other doc issues, there is no 'test' for when a 
non-trivial patch is 'good enough' and hence, in my opinion, too much 
bikeshedding and pursuit of the perfect.



As we've said there are two sides to the coin here, non-comitters and
comitters, a lot of the benefit of moving to Github is focused at
non-comitters although there are benefits for comitters themselves.


For maintaining Idle, I do not see the benefit.  Downloading patches 
from the tracker to my dev directory is trivial. I then apply to the 
current 3.x maintenance version, possibly with some hand editing, revise 
(always, that I can remember), and test.  Once satisfied, I backport to 2.7.



What if we focused an experiment on the benefits to non-comitters?


Users benefit by more patches being applied.  How do non-commiters 
benefit, really, by making it easier for them to submit patches that sit 
for years?  Ignore that.  Guido says that working with PEPs on GitHub 
would benefit him as a committer.



It's possible to maintain a git mirror of a Mercurial repository, in fact
we already have that at github.com/python/cpython
. What if we permit people
to make PRs against that repository, and then take those PRs and paste them
into roundup? Sort of like the "Remote hg repo" field. Then we can
create some integration that would post a comment to the ticket whenever

> that PR is updated

(sort of like the notification that happens when a new patch is uploaded).
The cannonical repository would still be hg.python.org
 and in order to actually
commit the PR commiters would need to turn the PR into a patch
(trivially easy, just add .diff or .patch to the PR URL).


This would be the focus of an experiment with the devguide, even if we 
have to generate some somewhat artificial pull requests for testing.  I 
really hope you try to make the above work.


The 3rd stage would be to expand on the above for doc patches. This is 
one area where we would get small ready-to-commit patches -- that do not 
need to be reported to the tracker.  Would it be possible to automate 
the following? Turn a doc PR into a patch, apply the patch to all 3 
branches (perhaps guided by the PR message), and generate a report, 
along with, currently, a 2.7 and 3.4 PR.  (I am thinking about how to do 
some of this doc patches with hg on windows.)


[snip premature discussion of moving cpython 'wholehog' to github.]

Summary research plan: 3 experiments, each depending of the preceding.
1. Link 2 repositories, one with pull requests
2. Link the PRs with the tracker
3. Make PRs work better with our multibranch, 2-head monster.
Report after each experiment (ending with 'success' or 'give-up').

--
Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 11/30/2014 8:19 PM, Pierre-Yves David wrote:


Mercurial have robust Windows support for a long time. This support is
native (not using cygwin) and handle properly all kind of strange corner
case. We have large scale ecosystem (http://unity3d.com/) using
Mercurial on windows.

We also have full featured GUI client http://tortoisehg.bitbucket.org/.


Tortoisehg comes with Hg Workbench, which I use for everything I can. 
The only thing I do not use it for is file (versus repository/changeset) 
commands, like file annotation. I do that with right click in Windows 
Explorer.  In my current physical state, having to do everything at a 
Windows command prompt would be more onerous.


There exists a TortoiseGit, but it lacks the unifying Workbench GUI. 
There exists at least one 3rd party workbench that purports to work with 
both hg and git. I have not looked at it yet.



It is actively developed by people who stay in touch with the Mercurial
upstream so new feature tend to land in the GUI really fast.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Terry Reedy]

On 12/1/2014 11:42 AM, Wes Turner wrote:

Here's a roundup of tools links, to make sure we're all on the same page:

Git HG Rosetta Stone
===
https://github.com/sympy/sympy/wiki/Git-hg-rosetta-stone#rosetta-stone

BugWarrior
===
BugWarrior works with many issue tracker APIs

https://warehouse.python.org/project/bugwarrior/

bugwarrior is a command line utility for updating your local
taskwarrior database from your forge issue trackers.
It currently supports the following remote resources:

  * github (api v3)
  * bitbucket
  * trac
  * bugzilla
  * megaplan
  * teamlab
  * redmine
  * jira
  * activecollab (2.x and 4.x)
  * phabricator

[...]


DVCS Interaction


Hg <-> Git

* https://warehouse.python.org/project/hg-git/ (dulwich)
* hg-github https://github.com/stephenmcd/hg-github

Git <-> Hg
--
* https://pypi.python.org/pypi/git-remote-hg/
* https://github.com/felipec/git-remote-hg

Python <-> Hg
---
| Wikipedia: https://en.wikipedia.org/wiki/Mercurial
| Homepage: http://hg.selenic.org/
| Docs: http://mercurial.selenic.com/guide
| Docs: http://hgbook.red-bean.com/
| Source: hg http://selenic.com/hg
| Source: hg http://hg.intevation.org/mercurial/crew

* http://evolution.experimentalworks.net/doc/user-guide.html
* (growing list of included extensions)

Python <-> Git
--
* GitPython, pygit2 (libgit2), dulwich
* https://github.com/libgit2/pygit2 (libgit2)
* https://pythonhosted.org/GitPython/ (Python)
* https://github.com/jelmer/dulwich (Python)
*
http://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html#installing-dependencies

GitHub -> BitBucket
-
* https://bitbucket.org/ZyX_I/gibiexport


Sphinx Documentation

* http://read-the-docs.readthedocs.org/en/latest/webhooks.html
* https://github.com/yoloseem/awesome-sphinxdoc
* changelogs, charts, csv, ipython, %doctest_mode


Is there an issue ticket or a wiki page that supports


https://wiki.python.org/moin/


Markdown/ReStructuredText,


whoops, I am not sure what moin uses.


where I could put this? Which URI do we assign to this artifact?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] WebM MIME type in mimetypes module

[Terry Reedy]

On 12/2/2014 7:07 PM, Chris Rebert wrote:

Hi all,

I'm seeking to move http://bugs.python.org/issue16329 towards conclusion.
Since the discussion on the issue itself seems to have petered out, I
thought I'd bring it up here.

To summarize the issue, it proposes adding an entry for WebM (
http://www.webmproject.org/docs/container/#naming ) to the mimetypes
standard library module's file-extension to MIME-type database.
(Specifically: .webm => video/webm )
Mozilla, Microsoft, Opera, and freedesktop.org (the de facto standard
*nix MIME type database package) all acknowledge the existence of a
video/webm MIME type (see the issue for relevant links), and this MIME
type is in WebM's documentation.
However, there is no official IANA registration for WebM's MIME type,
and none seems to be forthcoming/planned.

As R.D.M. said in the issue:

So we have two choices:
leave it to the platform mime types file to define because it is not even on 
track to be an official IANA standard,
or include it with a comment that it is a de-facto standard.

[...]

I guess I'd be OK with adding it as a de-facto standard, though I'm not 
entirely comfortable with it. But that would represent a change in policy, so 
others may want to weigh in.



Nobody has weighed in during the subsequent ~2 years, so I'm hoping a
few of y'all could weigh in one way or the other, and thus bring the
issue to a definitive conclusion.


If it has remained a defacto standard for the two years since your made 
that list, that would be a point in favor of recognizing it.  Have .webm 
files become more common in actual use?




--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Terry Reedy]

On 12/5/2014 3:04 PM, Brett Cannon wrote:


1. Contributor clones a repository from hg.python.org 
2. Contributor makes desired changes
3. Contributor generates a patch
4. Contributor creates account on bugs.python.org
 and signs the
[contributor
agreement](https://www.python.org/psf/contrib/contrib-form/)


I would like to have the process of requesting and enforcing the signing 
of CAs automated.



4. Contributor creates an issue on bugs.python.org
 (if one does not already exist) and uploads a patch


I would like to have patches rejected, or at least held up, until a CA 
is registered.  For this to work, a signed CA should be immediately 
registered on the tracker, at least as 'pending'.  It now can take a 
week or more to go through human processing.




5. Core developer evaluates patch, possibly leaving comments through our
[custom version of Rietveld](http://bugs.python.org/review/)
6. Contributor revises patch based on feedback and uploads new patch
7. Core developer downloads patch and applies it to a clean clone
8. Core developer runs the tests
9. Core developer does one last `hg pull -u` and then commits the
changes to various branches


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My thinking about the development process

[Terry Reedy]

On 12/6/2014 10:26 AM, Nick Coghlan wrote:

On 7 December 2014 at 01:07, Donald Stufft  wrote:

A likely solution is to use a pre-merge test runner for the systems that we
can isolate which will give a decent indication if the tests are going to
pass across the entire supported matrix or not and then continue to use the
current post-merge test runner to handle testing the esoteric systems that
we can’t work into the pre-merge testing.


Yep, that's exactly the approach I had in mind for this problem.


Most patches are tested on just one (major) system before being 
committed.  The buildbots confirm that there is no oddball failure 
elsewhere, and there is usually is not.  Testing user submissions on one 
system should usually be enough.


Committers should generally have an idea when wider testing is needed, 
and indeed it should be nice to be able to get wider testing on occasion 
*before* making a commit, without begging on the tracker.


What would be *REALLY* helpful for Idle development (and tkinter, 
turtle, and turtle demo testing) would be if there were a 
test.support.screenshot function that would take a screenshot and email 
to the tracker or developer.  There would also need to be at least one 
(stable) *nix test machine that actually runs tkinter code, and the 
ability to test on OSX with its different graphics options.  Properly 
testing Idle tkinter code that affects what users see is a real bottleneck.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

[Terry Reedy]

On 12/12/2014 1:24 PM, Mark Roberts wrote:


However, my point was that just because the core libraries by usage are
*starting* to roll out Python 3 support doesn't mean that things are
"easy" or "convenient" yet.

...

I suppose what I'm saying is that the long tail of libraries is far more
valuable than it seems the Python3 zealots are giving it credit for.
Please don't claim it's "easy" to move over just because merely most of
the top 20 libraries have been moved over. :-/


I agree that we should refrain from characterizing the difficulty of 
other peoples' work.  Conversions range from trivial to effectively 
impossible.  What we can say is that a) each library conversion make 
conversion easier for the users of that library and b) the number of 
conversions continue to increase.  I think some are trying to say that 
the number has reached a point where it is no longer fair to say that 
conversion is typically impossible.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEPs] Fwd: fixing broken link in pep 3

[Terry Reedy]

On 12/18/2014 4:19 PM, Chris Angelico wrote:

On Fri, Dec 19, 2014 at 5:39 AM, Guido van Rossum  wrote:

-- Forwarded message --
From: Victor Stinner 

Hi,

Yes, the link is dead. It looks like the following link contains the same
info:
https://docs.python.org/devguide/triaging.html

Dead page:
https://web.archive.org/web/20090704040931/http://www.python.org/dev/workflow/
"Core Development > Issue Workflow"

Victor


Edits made to PEP 3, link now updated.


PEP 3 is listed in PEP 0 under  Abandoned, Withdrawn, and Rejected PEPs
If this is proper, it does not make sense to update it.
If this is not, the header should be updated.


Noticed along the way that the
next link down (for people _submitting_ bugs) is pointing to the /2/
section of the docs; should that be updated to send people to /3/, or
are the two kept in sync?


The actual link in the doc is http://docs.python.org/bugs.html.  The 
site redirects that to http://docs.python.org/2/bugs.html.  To me, the 
redirection should be to http://docs.python.org/3/bugs.html, regardless 
of the PEP 3 status.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python 2.7.9 regression in argparse?

[Terry Reedy]

On 1/6/2015 7:39 AM, Victor Stinner wrote:

More context:

2014-12-19 12:43 GMT+01:00 anatoly techtonik :

https://github.com/nickstenning/honcho/pull/121


The link mentions the following changeset:
---
changeset:   93122:1a3143752db2
branch:  2.7
parent:  93112:927cca0b9337
user:R David Murray 
date:Fri Oct 17 20:07:08 2014 -0400
files:   Lib/argparse.py Lib/test/test_argparse.py Misc/NEWS
description:
#9351: set_defaults on subparser is no longer ignored if set on parent.

Before, if a default was set on the parent parser, any default for that
variable set via set_defaults on a subparser would be ignored.  Now
the subparser set_defaults is honored.

Patch by Jyrki Pullianinen.


diff -r 927cca0b9337 -r 1a3143752db2 Lib/argparse.py
--- a/Lib/argparse.py   Fri Oct 17 16:20:15 2014 -0500
+++ b/Lib/argparse.py   Fri Oct 17 20:07:08 2014 -0400
@@ -1089,7 +1089,14 @@ class _SubParsersAction(Action):
  # parse all the remaining options into the namespace
  # store any unrecognized options on the object, so that the top
  # level parser can decide what to do with them
-namespace, arg_strings = parser.parse_known_args(arg_strings,
namespace)
+
+# In case this subparser defines new defaults, we parse them
+# in a new namespace object and then update the original
+# namespace for the relevant parts.
+subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
+for key, value in vars(subnamespace).items():
+setattr(namespace, key, value)
+
  if arg_strings:
  vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
  getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)
---

Which is related to http://bugs.python.org/issue9351

Maybe argparse just became more strict? I don't understand the issue.


Steven Bethard, the argparse maintainer, defined the old behavior of 
ignoring subparser defaults (where there are also top level defaults) as 
a bug "counter to what people probably expect".  If the old behavior had 
been documented, changing it in a bug-fix release would have been a 
mistake.  But looking at the patch, the doc seems to have been silent on 
the issue.


This is not the first time someone considered a 'bug fix' to be a 
'regression', which it might be from their viewpoint.  The last comment 
on the github thread suggests that an easy fix was found.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] rst files

[Terry Reedy]

On 1/23/2015 7:15 PM, R. David Murray wrote:

On Fri, 23 Jan 2015 15:55:29 -0800, Guido van Rossum  wrote:

This adds entries to the index of the document -- similar to the index at
the end of a book. I think single vs. double refers to different types of
entries. Check out this page: https://docs.python.org/3/genindex.html

On Fri, Jan 23, 2015 at 3:43 PM, Ethan Furman  wrote:


Can somebody please explain this?

.. index::
single: formatting, string (%)
single: interpolation, string (%)
single: string; formatting
single: string; interpolation
single: printf-style formatting
single: sprintf-style formatting
single: % formatting
single: % interpolation

Specifically, what does index mean?  What does single vs double vs triple
mean?  Is there a reference somewhere I can read?


It is explained in the Sphinx documentation:
http://sphinx-doc.org/contents.html

Specifically:

http://sphinx-doc.org/markup/misc.html#directive-index


The devguide has a cpython doc oriented intro to .rst that includes
https://docs.python.org/devguide/documenting.html#index-generating-markup


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (no subject)

[Terry Reedy]

On 2/9/2015 7:29 PM, Neil Girdhar wrote:

For some reason I can't seem to reply using Google groups, which is is
telling "this is a read-only mirror" (anyone know why?)


I presume spam prevention.  Most spam on python-list comes from the 
read-write GG mirror.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Emit SyntaxWarning on unrecognized backslash escapes?

[Terry Reedy]

On 2/24/2015 1:14 PM, Guido van Rossum wrote:


And I'd weigh the needs of users who know what they are doing somewhat
higher than educating newbies through error messages. While newbies are
most likely to try out open() with a string literal, in "real" programs
that is rare, and filenames are typically taken from the command line or
from some other source where the peculiarities of Python string literals
are irrelevant.


I have been thinking about proposing a new how-to: Understanding Error 
Messages, with a section on tracebacks followed by an alphabetical 
listing of Exceptions that gives people problems, with possible 
solutions for each.  The following begins my first draft.


FileNotFoundError:

Possible cause [Windows]: You used a normal string literal to create a 
filename, you used '\' as the path separator, and you forgot that Python 
(like most languages) treats '\' as a case-sensitive escape character. 
For example: "C:\Test" is 7 chars and works as a file name, while 
'C:\test' is 6 chars, one a literal tab character.  The latter does not 
work as a file name and will raise FileNotFoundError.


Possible solutions: 1. Use raw string literals for Windows path names 
(r'C:\test'). 2 (recommended). Use '/' as the path separator 
('C:/test'), just as one does on other systems. This always works when 
passing file names from Python to Windows, even though it sometimes does 
not work in Windows Command Prompt console.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] how to inspect if something includes a bound first param

[Terry Reedy]

On 2/24/2015 8:56 PM, Gregory P. Smith wrote:

inspect.getargspec(method) and inspect.signature(method) both include
the 'self' parameter but how are we to figure out from method itself
that it is actually bound and that its first parameter is expected to be
a bound instance?


This seems like a python-list question.


So far I've come up with:
/inspect.ismethod(method) or inspect.ismethoddescriptor(method)/

But that is still not quite right as it remains False in 3.4 for the
simple case of:

 >>> class A:
...  def x(self):
...pass
...
 >>> inspect.ismethod(A.x)
False
 >>> inspect.ismethoddescriptor(A.x)
False


These are correct.  A.x is the function resulting from execution of the 
def statement.


>>> type(A.x)


This is different from 2.x.  If you call the function as a function, 
there is no requirement on its argument, and no binding.


>>> A.x(3)  # returns None
>>>

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] how to inspect if something includes a bound first param

[Terry Reedy]

On 2/24/2015 9:02 PM, Eric V. Smith wrote:


I'm not sure if it's correct, but deep in a library of mine I have:

elif type(fn) == types.MethodType:
 # bound method?
 if fn.im_self is None:
# no 'self'
 nskip = 0
 else:
# need to supply 'self'
 nskip = 1

This is also a 2.x library. No idea if it works with 3.x.


It will not.  3.x does not have 'unbound methods' in the above sense, 
and for bound methods, fn.im_self is now fn.__self__.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunning binary insertion sort algorithm in Timsort.

[Terry Reedy]

On 3/8/2015 8:17 PM, nha pham wrote:

We can optimize the TimSort algorithm by optimizing its binary insertion
sort.

The current version of binary insertion sort use this idea:

Use binary search to find a final position in sorted list for a new
element X. Then insert X to that location.

I suggest another idea:

Use binary search to find a final postion in sorted list for a new
element X. Before insert X to that location, compare X with its next
element.



For the next element, we already know if it is lower or bigger than X,
so we can reduce the search area to the left side or on the right side
of X in the sorted list.

I have applied my idea on java.util. ComparableTimSort.sort() and
testing. The execute time is reduced by 2%-6% with array of random integer.

Here is detail about algorithm and testing:
https://github.com/nhapq/Optimize_binary_insertion_sort


Apply the idea to timsort in python, and try the different cases 
discussed in Tim's documentation.  If it still looks good, open an issue 
with the patch and put tim.peters as nosy.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.7.9 regression in argparse

[Terry Reedy]

On 3/20/2015 9:31 AM, anatoly techtonik wrote:

Just a pointer for possible regression http://bugs.python.org/issue23058


I just added the argparse maintainer to the nosy list

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How to document functions with optional positional parameters?

[Terry Reedy]

On 3/21/2015 12:41 AM, Serhiy Storchaka wrote:

How to document functions with optional positional parameters?

For example binascii.crc32(). It has two positional parameters, one is
mandatory, and one is optional with default value 0. With Argument
Clinic its signature is crc32(data, crc=0, /). In the documentation it
is written as crc32(data[, crc]) (and noted that the default value of
the second parameter is 0). Both are not valid Python syntax. Can the
documentation be change to crc32(data, crc=0)?


I think the docs should be using / as well.


Related issues:

https://bugs.python.org/issue21488 (changed encode(obj,
encoding='ascii', errors='strict') to encode(obj, [encoding[, errors]]))

https://bugs.python.org/issue22832 (changed ioctl(fd, op[, arg[,
mutate_flag]]) to ioctl(fd, request, arg=0, mutate_flag=True))

https://bugs.python.org/issue22341 (discussed changing crc32(data[,
crc]) to crc32(data, crc=0))




--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PiCxx

[Terry Reedy]

On 3/24/2015 11:53 AM, π wrote:

Hello PythonDevvers,

I apologise in advance. This is slightly off topic. This will be my only
post on the subject.


pydev is about development *of* Python the language and CPython the 
implementation.  python-list is about development *with* Python.  This 
post would be on-topic for python-list and would have fit better there.



PyCXX (http://sourceforge.net/projects/cxx/) accomplishes roughly the
same as Boost::Python (C++ wrapper for CPython), only without requiring
Boost.

The original code has become tangled and bloated.
Over the winter I've rewritten it using C++11.  New C++11 features have
allowed for more direct/compact/concise/readable code.

I've called my rewrite PiCxx and put it up here:
https://github.com/p-i-/PiCxx
Improvements, suggestions, bug fixes, etc are all welcome.


Announcements and requests like this are fairly routine on python-list. 
 Sometimes they get responses and discussion on the list, sometimes not.


One can participate on python-list via news/web mirror 
gmane.comp.python.general at news.gmane.org.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Installing Python from the source code on Windows

[Terry Reedy]

On 4/3/2015 5:51 AM, Narsu wrote:

Hi Python
I'm working on a game project,using c++ as main
language, and using python as script.I've built
the Python from the source code on Windows, and when I
invoked method Py_Initialize my program exited. After some tests
I realized  as long as I move the Python-2.7.9/Lib file
to my program file, it works fine.
Please help me out .


Py-dev is for developing *future* versions of python.  Questions about 
using *current* versions of python should usually go to python-list.

PS: consider using 3.x for scripting.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

[Terry Reedy]

On 4/20/2015 9:07 PM, Chris Angelico wrote:

On Tue, Apr 21, 2015 at 10:52 AM, Ben Finney  wrote:



Jack is not complaining only about *writing* code. He's complaining
about the effect this will have on code that we all are expected to
*read*.


For reading, good function and parameter names and a good summary as the 
first line of the docstring are probably more useful *to humans* than 
formal type annotations.



Ahh. Yes, that's a concern. When you go digging into that library to
find out how it works, yes, you'd be face-to-face with their type
annotations.



Maybe it'd be of value to have a quick "code stripper" that takes away
all the annotations,


I was already thinking about that for Idle.  The file should then be 
either semi read-only or renamed.  Also options to merge annotations in 
from stub files for editing and to split back out when writing.



plus any other junk/framing that you're not
interested in, and gives you something you can browse in a text
editor? It could take away all the Sphinx adornments from docstrings,
any source control versioning markers, all that kind of thing.


Interesting idea.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

[Terry Reedy]

On 4/21/2015 6:50 PM, Chris Barker wrote:

On Tue, Apr 21, 2015 at 11:58 AM, Paul Sokolovsky mailto:pmis...@gmail.com>> wrote:

It does, and hope people won't be caught in "static typechecking"
loop and consider other usages too.


I an interested is using type hints for automatic or at least 
semi-automatic generation of tests, in particular for parameters hinted 
as generic and abstract.  A parameter "ints: Iterable(int)" should be 
work with empty, length 1, and longer tuples, lists, sets, dicts (with 
int keys) and a custom class.  Unless there is supplementary doc 
otherwise, it should handle negative, 0, and positive ints, small and 
large, in regular and random patterns.  (Thinking about this more, some 
of this could be done without the PEP being approved, but there has to 
be *some* syntax for test generator input information, and using the 
standard vocabulary of typing instead of inventing something would be 
easier.)



I"m confused -- from the bit I've been skimming the discussion, over on
python-ideas, and now here, is that this is all about "static typechecking".


The PEP is about a) defining type hints as the primary use of 
annotations* and b) adding a new module to standardize type hints 
instead of having multiple groups define multiple variations on the 
theme.  The urgency of developing a standard now is that there *are* at 
least two or three independent efforts.


* http://www.artima.com/weblogs/viewpost.jsp?thread=85551
(2004) makes it clear that Guido thought of static typing first and 
annotations second, in the form added to 3.0 years later, as an 
implementation thereof.


The primary motivation of the people working on the new module is static 
typechecking by 3rd party programs. This seems to be Guido's primary 
interest and what he believes will be the primary use.  It is also the 
primary promotional argument since type checking already exists.


However, this does not stop others from having other motivations and 
making other uses of the information once available.



It's not about run-time type checking.


The PEP does not prevent that, and will make it more feasible by 
standardizing on one system.



It's not about type-base performance optimization.


Nothing is planned for CPython, but type hints are already being used by 
MicroPython for this purpose.  (To me, this is one of the pluses of it 
having targeted 3.3.)



It's not about any use of annotations other than types.


Right, but there are multiple possible uses for type information.


What is it about other than static typechecking?


There is also documentation.  Type hints are partially an alternative to 
stylized entries in doc strings.  A program might combine the two or 
check consistency.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

[Terry Reedy]

On 4/21/2015 6:41 PM, Chris Barker wrote:


Well, it'll catch passing in a string instead of a sequence of strings
-- one of teh common and semi-insidious type errors I see a lot (at
least with newbies).

Oh wait, maybe it won't -- a string IS a sequence of strings. That's why
this is an insidioous bug in teh first place:

List[string]

will work, yes -- but now I'm locked down the user to an actual, list,
yes? So I try:

Iterable[string]

ah yes -- now users can use tuples of strings, or a custom class, or --
but wait, darn it, a string IS an iterable of strings.. SIGH.


I was just thinking today that for this, typing needs a subtraction 
(difference) operation in addition to an addition (union) operation: 
Difference(Iterable(str), str)


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

[Terry Reedy]

On 4/22/2015 8:45 PM, Guido van Rossum wrote:

On Wed, Apr 22, 2015 at 2:38 PM, Chris Barker mailto:chris.bar...@noaa.gov>> wrote:



On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote:

I was just thinking today that for this, typing needs a
subtraction (difference) operation in addition to an addition
(union) operation: Difference(Iterable(str), str)



Anyway, the point is that being able to say "all these types, except
this one" would solve this particular problem -- but would it solve
any others? Do we want this to work around a quirk in Pythons string
type?



More seriously, I doubt there are other important use cases for Difference.


I thought about Difference(numbers.Number, complex) to get ordered 
numbers, but numbers.Real should probably work.  I agree more real uses 
are needed before adding Difference.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492: async/await in Python; version 5

[Terry Reedy]

On 5/5/2015 6:25 PM, Yury Selivanov wrote:


Yes, there is no other popular event loop for 3.4 other
than asyncio,


There is the tk(inter) event loop which also ships with CPython, and 
which is commonly used.



that uses coroutines based on generators


Oh ;-) Tkinter event loop is callback based.  AFAIK, so is the asyncio 
event loop, but that is somehow masked by tasks that interface to 
coroutines.  Do you think the 'somehow' could be adapted to work with 
the tkinter loop?


What I do not understand is how io events become event loop Event 
instances.  For tk, keyboard and mouse actions seen by the OS become tk 
Events associated with a widget.  Some widgets generate events. User 
code can also generate (pseudo)events.


My specific use case is to be able to run a program in a separate 
process, but display the output in the gui process -- something like 
this (in Idle, for instance).  (Apologies if this misuses the new keywords.)


async def menu_handler()
ow = OutputWindow(args)  # tk Widget
proc = subprocess.Popen (or multiprocessing equivalent)
out = (stdout from process)
await for line in out:
ow.write(line)
finish()

I want the handler to not block event processing, and disappear after 
finishing.  Might 492 make this possible someday?  Or would having 'line 
in pipe' or just 'data in pipe' translated to a tk event likely require 
a patch to tk?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492: async/await in Python; version 5

[Terry Reedy]

On 5/5/2015 10:59 PM, Guido van Rossum wrote:

For this you should probably use an integration of asyncio (which can do
async subprocess output nicely) with Tkinter. Over in tulip-land there
is an demo of such an integration.


After redirection from googlecode tulip, I found
https://github.com/python/asyncio/tree/master/examples
None of the 4 *process*.py examples mention tkinter.

I also found "Create a Tkinter/Tulip integration"
https://github.com/python/asyncio/issues/21
with attachment tk_ayncio.zip
copied (with 'async' replacing 'tulip') to
https://bitbucket.org/haypo/asyncio_staging/src/bb76064d80b0a03bf3f7b13652e595dfe475c7f8/asyncio_tkinter/?at=default

None of the integration files mention subprocess, so I presume you are 
suggesting that I use a modification of one of the example subprocess 
coroutines with the integration framework.


If this works well, might it make sense to consider using an elaboration 
of examples/subprocess_shell.py to replace subprocess socket 
communication with pipe comminication?



On Tue, May 5, 2015 at 6:03 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote:



My specific use case is to be able to run a program in a separate
process, but display the output in the gui process -- something like
this (in Idle, for instance).  (Apologies if this misuses the new
keywords.)

async def menu_handler()
 ow = OutputWindow(args)  # tk Widget
 proc = subprocess.Popen (or multiprocessing equivalent)
 out = (stdout from process)
 await for line in out:
 ow.write(line)
 finish()

I want the handler to not block event processing, and disappear
after finishing.  Might 492 make this possible someday?  Or would
having 'line in pipe' or just 'data in pipe' translated to a tk
event likely require a patch to tk?


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492: async/await in Python; version 5

[Terry Reedy]

On 5/6/2015 5:39 PM, Guido van Rossum wrote:

Sorry to send you on such a wild goose chase! I did mean the issue you
found #21). I just updated it with a link to a thread that has more
news:
https://groups.google.com/forum/#!searchin/python-tulip/tkinter/python-tulip/TaSVW-pjWro/hCP6qS4eRnAJ

I wasn't able to verify the version by Luciano Ramalho. (And yes,
extending all this to working with a subprocess is left as an exercise.
It's all pretty academic IMO, given Tkinter's lack of popularity outside
IDLE.)


On Stackoverflow pyside has gotten 40 questions tagged in the last 30 
days, wxpython 70 in the last 30 days, pyqt 114 in 30 days, while 
tkinter has gotten 101 in the last week, which would project to about 
425 in the last 30 days.  So tkinter is being used at least by 
beginners. There have been a few tkinter and python-asyncio questions.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] anomaly

[Terry Reedy]

On 5/11/2015 3:40 AM, Florian Bruhin wrote:

[snip]

This trollish thread was cross-posted to python-list, where it was 
semi-ok, at least in the beginning, and pydev, where it is not.  It has 
continued on python-list with pydev removed. Please do not continue it 
here (on pydev).


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Tracker reviews look like spam

[Terry Reedy]

Gmail dumps patch review email in my junk box. The problem seems to be 
the spoofed From: header.


Received: from psf.upfronthosting.co.za ([2a01:4f8:131:2480::3])
by mx.google.com with ESMTP id 
m1si26039166wjy.52.2015.05.12.00.20.38

for ;
Tue, 12 May 2015 00:20:38 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning 
storch...@gmail.com does not designate 2a01:4f8:131:2480::3 as permitted 
sender) client-ip=2a01:4f8:131:2480::3;


Tracker reviews are the only false positives in my junk list. Otherwise, 
I might stop reviewing. Verizon does not even deliver mail that fails 
its junk test, so I would not be surprised if there are people who 
simply do not get emailed reviews.


Tracker posts are sent from Person Name 
Perhaps reviews could come 'from' Person Name 

Even direct tracker posts just get a neutral score.
Received-SPF: neutral (google.com: 2a01:4f8:131:2480::3 is neither 
permitted nor denied by best guess record for domain of 
roundup-ad...@psf.upfronthosting.co.za) client-ip=2a01:4f8:131:2480::3;


SPF is Sender Policy Framework
https://en.wikipedia.org/wiki/Sender_Policy_Framework

Checkins mail, for instance, gets an SPF 'pass' because python.org 
designates mail.python.org as a permitted sender.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Repository builds as 2.7.8+

[Terry Reedy]

Python 2.7.8+ (default, May 13 2015, 16:46:29) [MSC v.1500 32 bit 
(Intel)] on win32


Shouldn't this be 2.7.9+ or 2.7.10rc1?

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Repository builds as 2.7.8+

[Terry Reedy]

On 5/13/2015 5:45 PM, Zachary Ware wrote:

On Wed, May 13, 2015 at 4:05 PM, Terry Reedy  wrote:

Python 2.7.8+ (default, May 13 2015, 16:46:29) [MSC v.1500 32 bit (Intel)]
on win32

Shouldn't this be 2.7.9+ or 2.7.10rc1?


Make sure your repository is up to date, the patchlevel is correct at
the current tip of the 2.7 branch.


I had just installed tortoise/hg 3.4, pulled and updated twice.
Then ran external.bat on all branches.
2.7 tip is Kuchling's push at 15:25 utc

I deleted python_d.exe and rebuilt it.  External timestamp in Explorer 
is updated, internal one displayed on startup is not.


The only changes in my working directory are to idle files.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Repository builds as 2.7.8+

[Terry Reedy]

On 5/13/2015 6:22 PM, Antoine Pitrou wrote:

On Wed, 13 May 2015 18:15:10 -0400
Terry Reedy  wrote:


On 5/13/2015 5:45 PM, Zachary Ware wrote:

On Wed, May 13, 2015 at 4:05 PM, Terry Reedy  wrote:

Python 2.7.8+ (default, May 13 2015, 16:46:29) [MSC v.1500 32 bit (Intel)]
on win32

Shouldn't this be 2.7.9+ or 2.7.10rc1?


Make sure your repository is up to date, the patchlevel is correct at
the current tip of the 2.7 branch.


I had just installed tortoise/hg 3.4, pulled and updated twice.
Then ran external.bat on all branches.
2.7 tip is Kuchling's push at 15:25 utc


What does "hg sum" say?


F:\Python\dev\27>hg sum
parent: 96026:68d653f9a2c9
 #19934: fix mangled wording
branch: 2.7
commit: 6 modified
update: (current)

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython (merge 3.4 -> default): Added tests for more builtin types.

[Terry Reedy]

On 5/17/2015 1:57 AM, Serhiy Storchaka wrote:

On 17.05.15 02:44, Ned Deily wrote:

In article <20150516183940.21146.77...@psf.io>,
  serhiy.storchaka  wrote:

https://hg.python.org/cpython/rev/7b350f712c0e
changeset:   96099:7b350f712c0e
parent:  96096:f0c94892ac31
parent:  96098:955dffec3d94
user:Serhiy Storchaka 
date:Sat May 16 21:35:56 2015 +0300
summary:
   Added tests for more builtin types.
Made test_pprint discoverable.

files:
   Lib/test/test_pprint.py |  17 -
   1 files changed, 8 insertions(+), 9 deletions(-)


==
ERROR: test_coverage (test.test_trace.TestCoverage)
--
Traceback (most recent call last):
   File "/py/dev/3x/root/uxd/lib/python3.5/test/test_trace.py", line 312,
in test_coverage
 self._coverage(tracer)
   File "/py/dev/3x/root/uxd/lib/python3.5/test/test_trace.py", line 305,
in _coverage
 tracer.run(cmd)
   File "/py/dev/3x/root/uxd/lib/python3.5/trace.py", line 500, in run
 self.runctx(cmd, dict, dict)
   File "/py/dev/3x/root/uxd/lib/python3.5/trace.py", line 508, in runctx
 exec(cmd, globals, locals)
   File "", line 1, in 
AttributeError: module 'test.test_pprint' has no attribute 'test_main'

==
ERROR: test_coverage_ignore (test.test_trace.TestCoverage)
--
Traceback (most recent call last):
   File "/py/dev/3x/root/uxd/lib/python3.5/test/test_trace.py", line 327,
in test_coverage_ignore
 self._coverage(tracer)
   File "/py/dev/3x/root/uxd/lib/python3.5/test/test_trace.py", line 305,
in _coverage
 tracer.run(cmd)
   File "/py/dev/3x/root/uxd/lib/python3.5/trace.py", line 500, in run
 self.runctx(cmd, dict, dict)
   File "/py/dev/3x/root/uxd/lib/python3.5/trace.py", line 508, in runctx
 exec(cmd, globals, locals)
   File "", line 1, in 
AttributeError: module 'test.test_pprint' has no attribute 'test_main'

Also breaks 3.4.



Thank you Ned. Opened issue24215 for this because just restoring
test_main perhaps not the best way.


test_trace can be easily modified to use test_pprint as revised. See issue.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 553: Built-in debug()

[Terry Reedy]

On 9/7/2017 12:50 PM, Barry Warsaw wrote:

On Sep 6, 2017, at 23:10, Terry Reedy  wrote:


Environmental variables are set to strings, not objects.  It is not clear how 
you intend to handle the conversion.


The environment variable names a module import path.  Without quibbling about 
the details of the syntax (because honestly, I’m not convinced it’s a useful 
feature), it would work roughly like:

* The default value is equivalent to PYTHONBREAKPOINTHOOK=pdb.set_trace
* breakpoint() splits the value on the rightmost dot
* modules on the LHS are imported, then the RHS is getattr’d out of that
* That’s the callable breakpoint() calls


Environmental variables tend to be a pain on Windows and nigh unusable 
by beginners.  Leaving that aside, I see these problems with trying to 
use one for IDLE's *current* debugger.


pdb is universal, in the sense of working with any python run with 
actual or simulated stdin and stdout.  IDLE's idb is specific to working 
with IDLE. So one could not set an EV to 'idlelib.idb.start' and leave 
it while switching between IDLE and console.


pdb runs in one process, with communication between debugger and text ui 
handled by call and return.  It can be started on the fly.  IDLE runs 
code in a separate process.  The debugger has to run in the user 
process.  IDLE currently runs the GUI in the IDLE process.  So a 
complicated communication process has to be set up with rpc proxies and 
adaptors, and this is best done *before* code runs.  The on-the-fly 
function should not need an import and it might be better to not have one.


pdb.set_trace is a public and stable interface.  IDLE's is private and 
likely to be initially unstable.  I can imagine that the function that I 
would want to bind to sys.__breakpoint__ would be a bound method


Thinking about the above prompted me to rethink IDLE's debugger and 
consider rewriting it as an IDLE-independent gui debugger.  I'll will 
say more in response to Guido and you.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 553: Built-in debug()

[Terry Reedy]

On 9/7/2017 12:52 PM, Barry Warsaw wrote:

On Sep 7, 2017, at 07:46, Guido van Rossum  wrote:


Without following all this (or the PEP, yet) super exactly, does this mean you 
are satisfied with what the PEP currently proposes or would you like changes? 
It's a little unclear from what you wrote.


I’m also unsure whether Terry is good with the existing PEP or suggesting 
changes.


It seems to me that simplifying 'import pdb; pbd.set_trace()' to 
'breakpoint()' is not, in itself, justification for a new builtin. 
Rather, the justification is the possibility of invoking dbs other than 
pdb.  So I have been examining the feasibility of doing that, with 
IDLE's Idb + rpc + Debugger widget as a test case and example.  My 
conclusion: possible, but not as graceful as I would like.


In response to Barry's idea of PYTHONBREAKPOINTHOOK (default 
pdb.set_trace, and is 'HOOK' really needed), I examined further how 
IDLE's debugger is different and how the differences cause problems. 
Then a thought occurred to me: how about rewriting it as a pure tkinter 
GUI debugger, much more like pdb.  Call the module either gdb or tkdb. 
PYTHONBREAKPOINT=[package.].gdb.set_trace would then make more sense.


As with pdb, the UI would run in the same process as the debugger and 
user code.  No rpc setup needed.  No interference between the debug gui 
and the rest of IDLE (as sometimes happens now).  No need to pickle 
objects to display the bindings of global and local names. The reasons 
for separating user code from IDLE mostly do not apply to the debugger.


I would make it a single window with 2 panes, much like turtledemo, but 
with the turtle screen replaced with the GUI display.  (Try IDLE to see 
what that would look like now, or pull PR 2494 first to see it with one 
proposed patch.) Gdb should not be limited to running from IDLE but 
could be launched even when running code from the console.


After an initial version, the entry point could accept a list of 
breakpoint lines and a color mapping for syntax coloring.


I think breakpoint() should have a db= parameter so one can select a 
debugger in one removable line.  The sys interface is more useful for 
IDEs to change the default, possible with other args (like breakpoints 
and colors) bound to the callable.


Breakpoint() should pass on other args. In particular, people who invoke 
gdb from within a tkinter program should be able to pass in the root 
widget, to be the master of the debug window.  This has to be done from 
within the user program, and the addition of breakpoint allows that, and 
makes running the gui with user code more feasible.

---

A somewhat separate point: the name breakpoint() is slightly misleading, 
which has consequences if it is (improperly) called more than once. 
While breakpoint() acts as a breakpoint, what it does (at least in the 
default pdb case) is *initialize* and start a *new* debugger, possibly 
after an import.  Re-importing a module is no big deal.  Replacing an 
existing debugger with a *new* one, and tossing away all defined aliases 
and breakpoints and Bdb's internal caches, is.  It is likely not what 
most people would want or expect.  I think it more likely that people 
will call breakpoint() multiple times than they would, for instance, 
call pdb.set_trace() multiple times.


With a gui debugger, having one window go and another appear might be 
additionally annoying.  If the first window is not immediately GCed, 
having two windows would be confusing.  Perhaps breakpoint() could be 
made a no-op after the first call.



I think that's true for any IDE that has existing integrated debug 
capabilities. However for every IDE I would hope that calling breakpoint() will 
*also* break into the IDE's debugger. My use case is that sometimes I have a 
need for a *conditional* breakpoint where it's just much simpler to decide 
whether to break or not in Python code than it would be to use the IDE's 
conditional breakpoint facilities.


That certainly aligns with my own experience and expectations.  I guess I’m a 
fellow old dog. :)


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 553: Built-in debug()

[Terry Reedy]

On 9/7/2017 10:45 PM, Barry Warsaw wrote:

On Sep 7, 2017, at 16:19, Terry Reedy  wrote:


I think breakpoint() should have a db= parameter so one can select a debugger 
in one removable line.  The sys interface is more useful for IDEs to change the 
default, possible with other args (like breakpoints and colors) bound to the 
callable.


I’m skeptical about that.  I think any particular user is going to 
overwhelmingly use the same debugger, so having to repeat themselves every time 
they want to enter the debugger is going to get tedious fast.  I know it would 
annoy *me* if I had to tell it to use pdb every time I wrote `breakpoint()`, 
and I’m almost never going to use anything else.


OK


I’m also not sure what useful semantics for `db` would be.  E.g. what 
specifically would you set `db` to in order to invoke idle or tkdb (‘gdb’ would 
be an unfortunate name I think, given the popular existing GNU Debugger ;).


I will stick with tkdb until there is a shed to paint.


A somewhat separate point: the name breakpoint() is slightly misleading, which 
has consequences if it is (improperly) called more than once. While 
breakpoint() acts as a breakpoint, what it does (at least in the default pdb 
case) is *initialize* and start a *new* debugger, possibly after an import.  
Re-importing a module is no big deal.  Replacing an existing debugger with a 
*new* one, and tossing away all defined aliases and breakpoints and Bdb's 
internal caches, is.  It is likely not what most people would want or expect.  
I think it more likely that people will call breakpoint() multiple times than 
they would, for instance, call pdb.set_trace() multiple times.


Multiple calls to pdb.set_trace() is fairly common in practice today, so I’m 
not terribly concerned about it.


I am slightly surprised, but I guess if one does not set anything, one 
would not lose anything.



With a gui debugger, having one window go and another appear might be 
additionally annoying.  If the first window is not immediately GCed, having two 
windows would be confusing.  Perhaps breakpoint() could be made a no-op after 
the first call.


Your sys.breakpointhook could easily implement that, with a much better user 
experience than what built-in breakpoint() could do anyway.


Now that I know multiple breakpoint()s are likely, I definitely would.

--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 553, v3

[Terry Reedy]

On 9/13/2017 10:12 PM, Barry Warsaw wrote:

Here’s an update to PEP 553, which makes $PYTHONBREAKPOINT a first class 
feature.  I’ve also updated PR #3355 with the implementation to match.


Looks pretty good to me.  Reading the PR eliminated my remaining 
uncertainties.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 553

[Terry Reedy]

On 10/2/2017 12:44 AM, Guido van Rossum wrote:

- There's no rationale for the *args, **kwds part of the breakpoint() 
signature. (I vaguely recall someone on the mailing list asking for it 
but it seemed far-fetched at best.)


If IDLE's event-driven GUI debugger were rewritten to run in the user 
process, people wanting to debug a tkinter program should be able to 
pass in their root, with its mainloop, rather than having the debugger 
create its own, as it normally would.  Something else could come up.


- The explanation of the relationship between sys.breakpoint() and 
sys.__breakpointhook__ was unclear to me -- I had to go to the docs for 
__displayhook__ 
(https://docs.python.org/3/library/sys.html#sys.__displayhook__) to 
understand that sys.__breakpointhook__ is simply initialized to the same 
function as sys.breakpointhook, and the idea is that if you screw up you 
can restore sys.breakpointhook from the other rather than having to 
restart your process. Is that in fact it? The text 
"``sys.__breakpointhook__`` then stashes the default value of 
``sys.breakpointhook()`` to make it easy to reset" seems to imply some 
action that would happen... when? how?


- Some pseudo-code would be nice. It seems that it's like this:


This will be helpful to anyone implementing their own breakpointhook.


# in builtins
def breakpoint(*args, **kwds):
     import sys
     return sys.breakpointhook(*args, **kwds)

# in sys
def breakpointhook(*args, **kwds):
     import os
     hook = os.getenv('PYTHONBREAKPOINT')
     if hook == '0':
     return None
     if not hook:
         import pdb
     return pdb.set_trace(*args, **kwds)

     if '.' not in hook:
     import builtins
     mod = builtins
     funcname = hook
     else:
         modname, funcname = hook.rsplit('.', 1)
     __import__(modname)
     import sys
     mod = sys.modules[modname]
     func = getattr(mod, funcname)
     return func(*args, **kwds)

__breakpointhook__ = breakpointhook

Except that the error handling should be a bit better. (In particular 
the PEP specifies a try/except around most of the code in 
sys.breakpointhook() that issues a RuntimeWarning and returns None.)


- Not sure what the PEP's language around evaluation of PYTHONBREAKPOINT 
means for the above pseudo code. I *think* the PEP author's opinion is 
that the above pseudo-code is fine. Since programs can mutate their own 
environment, I think something like `os.environ['PYTHONBREAKPOINT'] = 
'foo.bar.baz'; breakpoint()` should result in foo.bar.baz() being 
imported and called, right?


- I'm not quite sure what sort of fast-tracking for PYTHONBREAKPOINT=0 
you had in mind beyond putting it first in the code above.


- Did you get confirmation from other debuggers? E.g. does it work for 
IDLE, Wing IDE, PyCharm, and VS 2015?


- I'm not sure what the point would be of making a call to breakpoint() 
a special opcode (it seems a lot of work for something of this nature). 
ISTM that if some IDE modifies bytecode it can do whatever it well 
please without a PEP.


- I don't see the point of calling `pdb.pm ()` at 
breakpoint time. But it could be done using the PEP with `import pdb; 
sys.breakpointhook = pdb.pm ` right? So this hardly 
deserves an open issue.


- I haven't read the actual implementation in the PR. A PEP should not 
depend on the actual proposed implementation for disambiguation of its 
specification (hence my proposal to add pseudo-code to the PEP).



--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Investigating time for `import requests`

[Terry Reedy]

On 10/2/2017 4:57 AM, Paul Moore wrote:


In practice, I don't think the fact that re.search() et al cache the
compiled expressions is that well known (it's mentioned in the
re.compile docs, but not in the re.search docs)


We could add redundant mentions in the functions ;-).


and so people often compile up front because they think it helps,
rather than actually measuring to check.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 553

[Terry Reedy]

On 10/2/2017 10:45 AM, Guido van Rossum wrote:
On Sun, Oct 1, 2017 at 11:15 PM, Terry Reedy <mailto:tjre...@udel.edu>> wrote:


On 10/2/2017 12:44 AM, Guido van Rossum wrote:

- There's no rationale for the *args, **kwds part of the
breakpoint() signature. (I vaguely recall someone on the mailing
list asking for it but it seemed far-fetched at best.)


If IDLE's event-driven GUI debugger were rewritten to run in the
user process, people wanting to debug a tkinter program should be
able to pass in their root, with its mainloop, rather than having
the debugger create its own, as it normally would.  Something else
could come up.


But if they care so much, they could also use a small wrapper as the 
sys.breakpointhook that retrieves the root and calls the IDLE debugger 
with that.


'They' include beginners that need the simplicity of breakpoint() the most.

Why is adding the root to the breakpoint() call better than 
that? To me, the main attraction for breakpoint is that there's 
something I can type quickly and insert at any point in the code.


I agree.


During a debugging session
I may try setting it in many different places. If I 
have to also pass the root each time I type "breakpoint()" that's just 
an unnecessary detail compared to having it done automatically by the hook.


Even though pdb.set_trace re-initializes each call, idb.whatever should 
*not*. So it should set something that can be queried. My idea was that 
a person should pass root only on the first call.  But that founders on 
the fact that 'first call' may not be deterministic.


if cond:
breakpoint()


breakpoint()

Besides which, someone might insert breakpoint() before creating a root.

So I will try instead initializing with
iroot = tk._default_root if tk._default_root else tk.Tk()
and stick with iroot.update() and avoid i.mainloop()

A revised tk-based debugger, closer to pdb than it is now, will require 
some experimentation.  I would like to be able to use it to debug IDLE 
run from a command line, and that will be a fairly severe test of 
compatibility with a tkinter application.


You could approve breakpoint() without args now and add them if and when 
there were more convincing need.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 530

[Terry Reedy]

On 10/27/2017 4:43 PM, London wrote:

can you help me get idol for my computer


Post questions about using python on python-list and include information 
about what OS you are running and what version of Python you want.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Migrate python-dev to Mailman 3?

[Terry Reedy]

On 11/1/2017 11:06 PM, Guido van Rossum wrote:
Another one is core-mentorship, which satisfies the same criteria; and 
in my view this has the added and useful property that its beneficiaries 
are non-core members. After that I'd do core-workflow. Honestly I'd 
leave python-committers alone for a while, we're a curmudgeonly group. :-)


As an idledev admin, I also volunteer it.  It has an archive but is 
currently dormant, so it could be shut down for even a couple of weeks 
to practice archive conversion and test messages.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reminder: 12 weeks to 3.7 feature code cutoff

[Terry Reedy]

On 11/2/2017 4:54 AM, Lele Gaifax wrote:


I'm not sure if the rebase should have been done on the original branch
instead of creating a new one, or instead if I should open a new PR (and close
the original one?).


It is normal to 'git merge upstream/master' after updating the master 
branch and checking out the pr branch, and then push the updated branch. 
 I sometimes have to do that when reviewing a pr.  Often, for me, the 
problem is not merge conflicts, but re version conflicts.  However, if 
there are merge conflicts that make it easier to reproduce the patch 
from the current master, closing and opening a new PR is ok too.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff

[Terry Reedy]

On 11/6/2017 9:47 PM, Nick Coghlan wrote:

On 7 November 2017 at 05:00, Alex Gaynor  wrote:

I also feel this decision was a mistake. If there's a consensus to revert,
I'm happy to draft a PEP.


Even without consensus to revert, I think it would be great to have a
PEP summarising some of the trade-offs between the different optio
I hope that there is consensus that neither the old nor current default 
is necessarily the best we can do.



And I do think there's openness to improving the situation, it's just
not clear yet that simple reversion of the previous change is the
right approach to attaining that improvement. In particular, the point
Georg Brandl raised about the poor interaction between the "-W"
command line option and Python code that calls
"warnings.filterwarnings" is still valid, and still a cause for
concern (whereby if code disables deprecation warnings, there's no
command line option or environment variable that can be used to
override that setting and force all the warnings to be shown despite
what the code says).

More generally, we seem to have (at least) 4 different usage models desired:

- "show my users nothing about legacy calls" (the app dev with a user
provided Python use case)


I believe that this is the current default.  But in practice, it often 
also means 'show the developer nothing about legacy calls', and therein 
lies the problem with the current default.



- "show me all legacy calls" (the testing use case, needs to be a
command line/environment setting)


I believe that this was the old default.  And I understand that it is 
the default when running the CPython test suite.  For testing the 
stdlib, it works because we control all the dependencies of each module. 
 But for other apps, the problem for both users and developers being 
overwhelmed with warnings from sources not in one's control.



- "only show me legacy calls in *my* code" (the "I trust my deps to
take care of themselves" use case)


Perhaps this should be the new default, where 'my code' means everything 
under the directory containing the startup file.  If an app developer 
either fixes or suppresses warnings from app code when they first 
appear, then users will seldom or never see warnings.  So for users, 
this would then be close to the current default.


Even for unmaintained code, the noise should be someone limited in most 
cases.



- "show me all legacy calls into a particular dependency" (the "I'm
considering upgrading this" use case, handled by the warn module on
PyPI)


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Terry Reedy]

On 11/9/2017 9:11 PM, Nick Coghlan wrote:

On 10 November 2017 at 05:51, Guido van Rossum  wrote:

If we have to change the name I'd vote for string_annotations -- "lazy" has
too many other connotations (e.g. it might cause people to think it's the
thunks). I find str_annotations too abbreviated, and stringify_annotations
is too hard to spell.


Aye, I'd be fine with "from __future__ import string_annotations" -
that's even more explicitly self-documenting than either of my
suggestions.


I think this is the best proposed so far.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560

[Terry Reedy]

On 11/14/2017 3:26 PM, Ivan Levkivskyi wrote:
After some discussion on python-ideas, see 
https://mail.python.org/pipermail/python-ideas/2017-September/047220.html, 
this PEP received positive comments. The updated version that takes into 
account the comments that appeared in the discussion so far is available 
at https://www.python.org/dev/peps/pep-0560/


Here I post the full text for convenience:

++

PEP: 560
Title: Core support for typing module and generic types
Author: Ivan Levkivskyi mailto:levkivs...@gmail.com>>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 03-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017

...

Suggested wording improvements:


Performance
---

The ``typing`` module is one of the heaviest and slowest modules in
the standard library even with all the optimizations made.



Mainly this is
because of subscripted generic types (see PEP 484 for definition of terms
used in this PEP) are class objects (see also [1]_).


Delete 'of' after 'because' to make this a proper sentence.


The three main ways how


"There are three ..." reads better to me.

the performance can be improved with the help of the proposed special 
methods:


- Creation of generic classes is slow since the ``GenericMeta.__new__`` is
   very slow; we will not need it anymore.

- Very long MROs for generic classes will be twice shorter;


I  believe by 'twice shorter', which is meaningless by itself, you mean 
'half as long'.  If so, please say the latter.

 > they are present

   because we duplicate the ``collections.abc`` inheritance chain
   in ``typing``.

- Time of instantiation of generic classes will be improved


Instantiation of generic classes will be faster.


   (this is minor however).


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Allow tuple unpacking in return and yield statements

[Terry Reedy]

On 11/25/2017 1:55 AM, David Cuthbert wrote:

First time contributing back -- if I should be filing a PEP or something like 
that for this, please let me know.


I don't think a PEP is needed.


Coming from https://bugs.python.org/issue32117, unparenthesized tuple unpacking 
is allowed in assignments:

 rest = (4, 5, 6)
 a = 1, 2, 3, *rest


Because except for (), it is ',', not '()' that makes a tuple a tuple.


but not in yield or return statements (these result in SyntaxErrors):

 return 1, 2, 3, *rest
 yield 1, 2, 3, *rest


To be crystal clear, a parenthesized tuple with unpacking *is* valid.

  return (1, 2, 3, *rest)
  yield (1, 2, 3, *rest)

So is an un-parenthesized tuple without unpacking.

Since return and yield are often the first half of a cross-namespace 
assignment, requiring the () is a bit surprising.  Perhaps someone else 
has a good reason for the difference.  Otherwise, +1 on the change.



The unpacking in assignments was enabled by a pre-3.2 commit that I haven't yet 
been able to track back to a discussion, but I suspect this asymmetry is 
unintentional. Here's the original commit:
https://github.com/python/cpython/commit/4905e80c3d2f6abb613d212f0313d1dfe09475dc

I've submitted a patch (CLA is signed and submitted, not yet processed), and 
Serihy said that since it changes the grammar I should have it reviewed here 
and have signoff by the BDFL.






While I haven't had a need for this myself, it was brought up by a user on 
StackOverflow 
(https://stackoverflow.com/questions/47272460/python-tuple-unpacking-in-return-statement/47326859).

Thanks!
Dave





--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Using async/await in place of yield expression

[Terry Reedy]

On 11/27/2017 5:05 PM, Guido van Rossum wrote:
On Mon, Nov 27, 2017 at 1:58 PM, Greg Ewing > wrote:


Guido van Rossum wrote:
The source for sleep() isn't very helpful -- e.g. @coroutine is
mostly a backwards compatibility thing.

So how are you supposed to write that *without* using @coroutine?

A simplified version using async def/await:


---

async def sleep(delay):
     f = Future()


This must be asyncio.Future as (by experiment) a 
concurrent.futures.Future cannot be awaited.  The a.F doc could add this 
as a difference.  Future needs an argument for the keyword-only loop 
parameter; as I remember, the default None gets replaced by the default 
asyncio loop.



     get_event_loop().call_later(delay, f.set_result)


A result is needed, such as None or delay, to pass to f.set_result.


     await f


I gather that
1. the value of the expression is the result set on the future, which 
would normally be needed, though not here;
2. the purpose of 'await f' here is simply to block exit from the 
coroutine, without blocking the loop, so that users of 'await sleep(n)' 
will actually pause (without blocking other code).


Since a coroutine must be awaited, and not just called, and await can 
only be used in a coroutine, there seems to be a problem of where to 
start.  The asyncio answer, in the PEP, is to wrap a coroutine call in a 
Task, which is, as I remember, done by the loop run methods.


Based on the notes above, and adding some prints, I got this to run:

---
import asyncio
import time

loop = asyncio.get_event_loop()

async def sleep(delay):
f = asyncio.Future(loop=loop)
loop.call_later(delay, f.set_result, delay)
print('start')
start = time.perf_counter()
d = await f
stop = time.perf_counter()
print(f'requested sleep = {d}; actual = {stop-start:f}')

loop.run_until_complete(sleep(1))
---
This produces:
start
requested sleep = 1; actual = .9--- [usually < 1]

---
Now, the question I've had since async and await were introduced, is how 
to drive async statements with tkinter.  With the help of the working 
example above, I make a start.


---
from asyncio import Future, Task
import tkinter as tk
import time


class ATk(tk.Tk):
"Enable tkinter program to use async def, etc, and await sleep."

def __init__(self):
super().__init__()

def task(self, coro):
"Connect async def coroutine to tk loop."
Task(coro, loop=self)

def get_debug(self):
"Internal method required by Future."
print('debug')
return False

def call_soon(self, callback, *args):
"Internal method required by Task and Future."
# TaskStep/Wakeup/MethWrapper has no .__name__ attribute.
# Tk.after requires callbacks to have one (bug, I think).
print('soon', callback, *args, hasattr(callback, '__name__'))
def wrap2(): callback(*args)
return self.after(0, wrap2)


root = ATk()

async def sleep(delay):
f = Future(loop=root)
def cb():
print('cb called')
f.set_result(delay)
root.after(int(delay*1000), cb)
print('start')
start = time.perf_counter()
d = await f
stop = time.perf_counter()
print(f'requested sleep = {d}; actual = {stop-start:f}')

root.task(sleep(1))
root.mainloop()
---

Output:
debug
soon  False
debug
start
cb called
soon  finished result=1> False

requested sleep = 1; actual = 1.01--- [always about 1.01]

Replacing the last two lines with

async def myloop(seconds):
while True:
print(f'*** {seconds} ***')
await sleep(seconds)

root.task(myloop(1.2))
root.task(myloop(.77))
root.mainloop()

prints interleaved 1.2 and .77 lines.

I will next work on animating tk widgets in the loops.

--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Zero-width matching in regexes

[Terry Reedy]

On 12/4/2017 6:21 PM, MRAB wrote:
I've finally come to a conclusion as to what the "correct" behaviour of 
zero-width matches should be: """always return the first match, but 
never a zero-width match that is joined to a previous zero-width match""".


Is this different from current re or regex?

If it's about to return a zero-width match that's joined to a previous 
zero-width match, then backtrack and keep on looking for a match.


Example:

 >>> print([m.span() for m in re.finditer(r'|.', 'a')])
[(0, 0), (0, 1), (1, 1)]

re.findall, re.split and re.sub should work accordingly.

If re.finditer finds n matches, then re.split should return a list of 
n+1 strings and re.sub should make n replacements (excepting maxsplit, 
etc.).



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] "CPython loves your Pull Requests" talk by Stéphane Wirtel

[Terry Reedy]

On 12/5/2017 10:25 AM, Mariatta Wijaya wrote:


* Time to merge a PR: 3 days in average, good!


Slide said 2.98 days, another said 4.4% by developers.

Regarding the average time to merge PR, I'm interested to know the 
average time to merge for PRs not made by Python Core Devs.


Trivially different: assume 0 days for all dev PRs, then average would 
be 2.96 / .956 = 3.12.


But any average that includes backports, which I suspect the above does, 
is skewed way down because backports are mostly merged soon after tests 
complete.  So I think 6 days average may be more realistic for master 
branch (3.7) PRs.


The average may be skewed down even more because some of the PRs that 
have been open a month or more will eventually be merged.


On the other hand, the mean, certainly by itself, is the *wrong* 
statistic for this data.  The argument for the value of using means 
depends on the data having a distribution that is at least roughly 
gaussian (misleadingly called 'normal').  The waiting times for merging 
appear to be negative exponential (slide 75).  Long waiting times have 
an oversized influence on the mean.


 So one should either calculate and report the median time or possibly 
the mean log(wait), converted back to waiting time.  In other words, 
exp(mean(map(log, waits))).  For the latter, one should probably start 
the clock after the initial CI tests finish, which is at least 1/2 hour.


--
Terry Jan Reedy (retired statistician)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 432 progress: Python initalization

[Terry Reedy]

On 12/14/2017 10:16 AM, Victor Stinner wrote:

Hi,

Serhiy Storchaka seems to be worried by the high numbers of commits in
https://bugs.python.org/issue32030 "PEP 432: Rewrite Py_Main()", so
let me explain the context of this work :-)


You could have (and still could) made that a master issue with multiple 
dependencies.  Last summer, I merged at least 20 patches for one idlelib 
file.  I split them up among 1 master issue and about 6 dependency 
issues.  That was essential because most of the patches were written by 
one of 3 new contributors and needed separate discussions about the 
strategy for a particular patch.


I completely agree with keeping PRs to a reviewable size.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Revisiting old enhancement requests

[Terry Reedy]

On 12/20/2017 12:36 AM, Steven D'Aprano wrote:

What is the best practice for revisiting old enhancement requests on the
tracker, if I believe that the time is right to revisit a rejected issue
from many years ago? (Nearly a decade.)


I have been thinking about the opposite: revisit old enhancement 
requests that have been open for a decade that I thing have no change 
ever and should be closed.



Should I raise a new enhancement request and link back to the old one,
or re-open the original?


I think the answer for both is to consider posting on python-ideas.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Terry Reedy]

On 12/21/2017 4:22 AM, Eric V. Smith wrote:

On 12/21/2017 1:46 AM, Chris Barker wrote:


I suggest that it be clear in the docs, and ideally in the PEP, that 
the dataclass decorator is using the *annotation" syntax, and that the 
the only relevant part it uses is that an annotation exists, but the 
value of the annotation is essentially (completely?) ignored. 


I think the PEP is very clear about this: "The dataclass decorator 
examines the class to find fields. A field is defined as any variable 
identified in __annotations__. That is, a variable that has a type 
annotation. With two exceptions described below, none of the Data Class 
machinery examines the type specified in the annotation."


This seems clear enough.  It could come after describing what a 
dataclass *is*.



I agree the docs should also be clear about this.




So we should have examples like:

@dataclass
class C:
 a: ...  # field with no default
 b: ... = 0 # filed with a default value

Then maybe:

@dataclass
class C:
 a: "the a parameter" # field with no default
 b: "another, different parameter" = 0.0 # field with a default

Then the docs can go to say that if the user wants to specify a type 
for use with a static type checking pre-processor, they can do it like 
so:


@dataclass
class C:
 a: int # integer field with no default
 b: float = 0.0 # float field with a default

And the types will be recognized by type checkers such as mypy.

And I think the non-typed examples should go first in the docs.


Module some bike-shedding, the above seems pretty good to me.


I'll leave this for others to decide. The docs, and how approachable 
they are to various audiences, isn't my area of expertise.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Terry Reedy]

On 12/21/2017 9:23 AM, Eric V. Smith wrote:



On 12/21/17 6:25 AM, Sven R. Kunze wrote:

On 21.12.2017 11:22, Terry Reedy wrote:



@dataclass
class C:
 a: int # integer field with no default
 b: float = 0.0 # float field with a default

And the types will be recognized by type checkers such as mypy.

And I think the non-typed examples should go first in the docs.




I still don't understand why "I don't care" can be defined by "leaving 
out"


@dataclass
class C:
 b = 0.0 # float field with a default


Because you can't know the order that x and y are defined in this example:

class C:
     x: int
     y = 0

'x' is not in C.__dict__, and 'y' is not in C.__annotations__.


I think the understanding problem with this feature arises from two 
factors: using annotations to define possibly un-initialized slots is 
non-obvious; a new use of annotations for something other than static 
typing is a bit of a reversal of the recent pronouncement 'annotations 
should only be used for static typing'.  Therefore, getting the 
permanent doc 'right' is important.


The following naively plausible alternative does not work and cannot 
sensibly be made to work because the bare 'x' in the class scope, as 
opposed to a similar error within a method, causes NameError before the 
class is created.


@dataclass
class C:
x
y = 0

I think the doc should explicitly say that uninitialized fields require 
annotation with something (anything, not necessarily a type) simply to 
avoid NameError during class creation.  It may not be obvious to some 
readers why x:'anything' does not also raise NameError, but that was a 
different PEP, and the dataclass doc could here link to wherever 
name:annotation in bodies is explained.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3

[Terry Reedy]

On 1/13/2018 3:02 PM, Brett Cannon wrote:



On Sat, Jan 13, 2018, 05:24 Antoine Pitrou, > wrote:


On Sat, 13 Jan 2018 13:54:33 +0100
Christian Heimes mailto:christ...@python.org>> wrote:
 >
 > If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I
can land
 > bunch of useful goodies like proper hostname verification [2], proper
 > fix for IP address in SNI TLS header [3], PEP 543 compatible
Certificate
 > and PrivateKey types (support loading certs and keys from file and
 > memory) [4], and simplified cipher suite configuration [5]. I can
 > finally clean up _ssl.c during the beta phase, too.

Given the annoyance of supporting old OpenSSL versions, I'd say +1 to
this.


+1 from me as well for the improved security.


FWIW, given that I will not be doing any of the work, +1 from me also.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com