[Python-Dev] EuroPython Language Summit report
EuroPython 2011 Language Summit === Here's a brief report on the EuroPython 2011 Language Summit, held on Sunday 19 June, 2011 in Florence. It was a fairly small meeting, with a lot of informal and loosely-focused discussion and few conclusions reached. I've outlined the topics that we discussed below. Present: Antonio Cuni Mark Dickinson Larry Hastings (chair) Marc-André Lemburg Ezio Melotti Antoine Pitrou Armin Ronacher Armin Rigo Mark Ramm Topics covered == Python 3 adoption - This was a long and wide-ranging discussion on the general state of Python 3 adoption. Some highlights: - Even amongst those present at the meeting, most are still using Python 2 for everyday work. - pypy: no definitive plans yet for PyPy supporting Python 3. - The web community is still very much focused on Python 2. - There's ongoing work (or perhaps just discussion?) on being able to generate Python 3 documentation using Sphinx running on Python 3. This would be particularly useful when using extensions like 'autodoc' for Python 3 code. - [Armin Ronacher] Python 3's Unicode support still has some dark areas. One example: when opening a text file for reading and writing, the default encoding used depends on the platform and on various environment variables. - Regular expression support in Python 3 needs improvement (support for Unicode character classes is a particular need). [Subtopic: what needs to be done to get the new regex module into Python? Should it replace the existing module? What about backwards compatibility issues?] - 2to3: It's still painful to upgrade a codebase from Python 2 to Python 3, and there doesn't seem to be a consensus on best practices yet. There was a long discussion that I can't hope to do justice to on whether 2to3 should be part of the standard library or not. (With side discussions on distutils and packaging in Python 3.) Armin Ronacher was one of the main participants in this discussion (and IIUC, the main opponent of having 2to3 in the standard library); Armin, do you want to summarize this? Another major topic of discussion was on how to manage Python 2 and Python 3 in plugin environments (e.g., Blender): is there some way that both Python 2 and Python 3 plugins could be used within the same system? This got a bit technical; perhaps someone else at the meeting would like to elaborate on this? State of Python 3 on the web Brief discussion. Summary: it's improving; better than before. We have CherryPy and WSGI. There are issues in WSGI that are going to become more apparent as more people move to Python 3 frameworks. Discussion of open PEPS --- PEP 3151: (Reworking the OS and IO exception hierarchy.) Antoine summarized this; no real discussion here. PEP 3118: (Not open, but ...) Revising the buffer protocol. The buffer protocol implementation is still buggy and incomplete. (Mostly okay for 1d buffers, not even close for higher-dimensional work.) PEP 397: Python launcher for Windows. (This was actually discussed later, after lunch; apologies for messing with the timeline here.) There was some confusion over the purpose of this PEP. Armin Ronacher had some objections to the PEP in its current form, but it's not clear to me what those objects are, or whether they still exist. Armin, care to elaborate? PEP 0380: Syntax for delegating to a subgenerator. PEP 3150: Statement local namespaces. PEP 3152: Cofunctions. For all three of the above PEPs, there was some feeling that additional syntax changes to the language are unnecessary and make it harder to learn; where possible, we should prefer using 3rd partly libraries. Issue 12326 (Linux 3) - From the issue report by Charles-François Natali:: Linus recently decided that the next Linux kernel version would 3.0 [1]. As a consequence, sys.platform (which is actually MACHDEP) will be 'linux3' on machines running Linux 3 kernels, and tests checking explicitely against 'linux2' will either break and won't run. There was significant discussion on how this should be solved. While there's a lot of discussion already on the tracker, the choice of solution may have significant ramifications, so it seems a good idea that this be discussed more widely on python-dev. Major questions:: - what should be done with Python 2.7? As it stands, as I understand it, sys.platform *will* become 'linux3' on Linux 3.x (it's a computed value somewhere) in Python 2.7.2, and since Python 2.7.2 is already out there we can't change that. Some of the proposed solutions to the issue would involve sys.platform changing between Python 2.7.2 and Python 2.7.3; is such a change acceptable for a bugfix release? - it's easy to fix up all uses of "== 'linux2' " in the sta
Re: [Python-Dev] EuroPython Language Summit report
Le vendredi 24 juin 2011 à 10:52 +0200, Mark Dickinson a écrit : > - [Armin Ronacher] Python 3's Unicode support still has some dark areas. What? Unicode support is perfect in Python 3! > One example: when opening a text file for reading and writing, the default > encoding used depends on the platform and on various environment > variables. ... oh, I agree. This choice is a big portability issue. Mac OS X, most Linux distro, BSD systems use UTF-8 local encoding, whereas Windows use legacy code pages like cp1252 (something like ISO-8859-1) or cp952 (shift jis). But sometimes, the locale is "C" (e.g. on our buildbots) and programs start to fail with Unicode errors... I see two options to improve the situation. (1) hard way: change open() API to make encoding a mandatory argument. Problem: it breaks compatibility with Python 3.0, 3.1 and 3.2 (ooops!); the encoding argument is the 4th argument, you have to use a keyword or choose a value for the buffering argument. I proposed to change open() API in Python 3.1 to make all arguments -except the filename and the mode- keyword-only arguments, but Guido rejected my idea: "Remember, for 3.0 we're trying to get a release out of the door, not cram in new features, no matter how small." http://bugs.python.org/issue4121 (2) soft way: add a warning if the encoding is implicit (locale encoding). I don't know what is the best warning type, and if it should be always displayed, only once, or not by default. Even if it is hidden by default, a careful developer will be able to use -Werror to fix bugs... I suspect that most tests fail if open() raises an exception if the encoding is not specified (e.g. see distutils/packaging issues about the file encoding). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EuroPython Language Summit report
On Fri, Jun 24, 2011 at 6:52 PM, Mark Dickinson wrote: > PEP 3118: (Not open, but ...) Revising the buffer protocol. The buffer > protocol implementation is still buggy and incomplete. (Mostly okay for 1d > buffers, not even close for higher-dimensional work.) Issue 10181 is the place to start for anyone that wants to help with this one! :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EuroPython Language Summit report
Mark Dickinson gmail.com> writes: > virtualenv in Python 3.3? > - > > Apparently there was some discussion at the last PyCon about the possibility > of virtualenv going into Python 3.3. As far as I know there's currently no > open tracker item or PEP for this. Larry Hastings knows more here. > > Update from Larry: """I'm supportive of putting it in in 3.3, and I'm > sitting with Raymond Hettinger right now and he supports it too. So I > think if we get a working implementation it'll go in. It's under heavy > discussion in c.l.p-d so I gather it's moving forward. Last I knew it was > Carl Meyer pushing it forward, but Vinay Sajip is the current standard- > bearer. I understand Windows support will be a bit tricky; I don't know if > they have someone who's going to handle it for them.""" Mark, thanks for the summary. Re. a PEP for virtual environments in Python, Carl is working on the PEP. The first draft by Carl with my initial comments is at [1]. There's still some work to do before we can actually present it as a PEP we're happy with. I'm pleased to report good progress with the proof of concept implementation. There are some issues still with packaging which I'm working through with Éric Araujo, but I've gone ahead and committed changes on my branch to get things working. It's a good exercising of the packaging functionality :-) What I have at the moment is pythonv [2]: A modified Python which supports virtual environments via changes in Modules/getpath.c, PC/getpathp.c, Lib/site.py, Lib/sysconfig.py, Lib/sysconfig.cfg and Lib/distutils/sysconfig.py. These changes basically detect if you're running in a virtual environment by looking for an env.cfg, and if found, fixing it up so sys.path is set as it should be. In addition, sys.site_prefix and sys.site_exec_prefix are created - if not in a virtual env, these have the same values as sys.prefix and sys.exec_prefix. With just these changes, the basics of a virtual environment are provided (in terms of isolation from other environments and the system Python). However, in order to be useful, the packaging tools have to work with sys.site_prefix and sys.site_exec_prefix, so changes have been made to sysconfig, distutils and packaging to do this. I'm presently working on a demonstration application which integrates the above work with Doug Hellmann's virtualenvwrapper and Guillermo López' port of it to Powershell to get a nice basic tool that'll support virtual environments with packaging as an easy-to-use CLI, as well as Distribute support while people migrate over to packaging, on Windows as well as Linux. I'm expecting to cover both the Linux and Windows aspects (though I won't say no to help ;-) and working through packaging issues relating to improved Windows support. The basic functionality is there now, both in Windows and Linux - the focus of work is in the ease-of-use CLI stuff which is not envisaged to be part of core Python, but nevertheless needs to be done to make virtual envs as painless as possible. BTW Of the 398 packages on PyPI having a Py3K trove classifier, I've tested installing all of them into a pythonv virtual env with Distribute, and 310 install without errors. Of the remaining 88, some are due to missing dependencies, others due to broken packages on PyPI. BTW of the Python regression test suite passes all tests in a pythonv virtualenv, other than those for which there already are tracker issues for the default branch in hg.python.org/cpython (test_lib2to3, test_packaging, test_sysconfig). Full test results are at [3]. All in all, it's looking reasonably good, and we hope to report more progress on the PEP and the proof of concept soon! Regards, Vinay Sajip [1] http://goo.gl/6u0S0 [2] https://bitbucket.org/vinay.sajip/pythonv [3] https://gist.github.com/1022705 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EuroPython Language Summit report
On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson wrote: > EuroPython 2011 Language Summit > === [...] > Unicode character classes is a particular need). [Subtopic: what needs > to be done to get the new regex module into Python? Should it replace > the existing module? What about backwards compatibility issues?] I'm pretty sure regex has backward compatibility as a goal for just this reason (so it can replace the current module). > PEP 0380: Syntax for delegating to a subgenerator. > > PEP 3150: Statement local namespaces. > > PEP 3152: Cofunctions. > > For all three of the above PEPs, there was some feeling that additional syntax > changes to the language are unnecessary and make it harder to learn; where > possible, we should prefer using 3rd partly libraries. I disagree with this with respect to 380. Haven't looked at the others. > Python 2.7 > -- > > How long will this be maintained? > Original decision was 5 years. As I recall, the actual decision was "at *least* 5 years". It's only been one so far...was the discussion that five years was too short or too long? As the code bases continue to drift apart, and we fix the fixable bugs, the number of patches going in to 2.7 will decrease over time, so I don't think the burden of continuing to support it will be too heavy. Currently about half of the non-feature-request issues (ie: bugs) in the tracker are tagged 2.7. I expect to see that percentage continue to decrease over time. -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2011-06-17 - 2011-06-24) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open2845 ( +2) closed 21335 (+43) total 24180 (+45) Open issues with patches: 1235 Issues opened (30) == #3067: setlocale error message is confusing http://bugs.python.org/issue3067 reopened by terry.reedy #12354: packaging.pypi.simple docs use both client and crawler variabl http://bugs.python.org/issue12354 opened by gruszczy #12355: Crawler doesn't follow redirection http://bugs.python.org/issue12355 opened by gruszczy #12361: Memory Leak in File Logging http://bugs.python.org/issue12361 opened by japerk #12364: Timeout (1 hour) in test_concurrent_futures.tearDown() on spar http://bugs.python.org/issue12364 opened by haypo #12365: URLopener should support context manager protocol http://bugs.python.org/issue12365 opened by mcjeff #12366: packaging.pypi.dist should abstract download errors. http://bugs.python.org/issue12366 opened by michael.mulich #12368: packaging.pypi.simple.Crawler assumes external download links http://bugs.python.org/issue12368 opened by michael.mulich #12372: semaphore errors on AIX 7.1 http://bugs.python.org/issue12372 opened by reshmi.george #12374: Execution model should explain compile vs definition vs execut http://bugs.python.org/issue12374 opened by ncoghlan #12375: Add packages_root to sys.path for hooks http://bugs.python.org/issue12375 opened by erik.bray #12376: unittest.TextTestResult.__init__ breaks under complex __mro__ http://bugs.python.org/issue12376 opened by branker #12377: Clean up use of packages_root/package_dir http://bugs.python.org/issue12377 opened by erik.bray #12378: smtplib.SMTP_SSL leaks socket connections on SSL error http://bugs.python.org/issue12378 opened by joeshaw #12379: build outside source fail in head http://bugs.python.org/issue12379 opened by rpetrov #12380: bytearray methods center, ljust, rjust don't accept a bytearra http://bugs.python.org/issue12380 opened by py.user #12381: bytearray methods count, find, index don't support None as in http://bugs.python.org/issue12381 opened by py.user #12382: [msilib] Obscure exception message when trying to open a non-e http://bugs.python.org/issue12382 opened by Robin.Jarry #12384: difflib.SequenceMatcher and Match: code and doc bugs http://bugs.python.org/issue12384 opened by terry.reedy #12385: the help for bytearray.maketrans describes bytes.maketrans http://bugs.python.org/issue12385 opened by py.user #12386: packaging fails in install_distinfo when writing RESOURCES http://bugs.python.org/issue12386 opened by vinay.sajip #12387: IDLE save keyboard shortcut problem http://bugs.python.org/issue12387 opened by Jacob.VB #12388: cannot specify recursive extra_files in packaging setup.cfg http://bugs.python.org/issue12388 opened by vinay.sajip #12391: packaging install fails to clean up temp files http://bugs.python.org/issue12391 opened by vinay.sajip #12392: pthread_kill() doesn't work on the main thread on FreeBSD6 http://bugs.python.org/issue12392 opened by haypo #12393: Packaging should provide support for extensible categories http://bugs.python.org/issue12393 opened by vinay.sajip #12394: packaging: generate scripts from callable (dotted paths) http://bugs.python.org/issue12394 opened by vinay.sajip #12395: packaging remove fails under Windows http://bugs.python.org/issue12395 opened by vinay.sajip #12397: re match object methods have no docstrings http://bugs.python.org/issue12397 opened by nedbat #12398: Sending binary data with a POST request in httplib can cause U http://bugs.python.org/issue12398 opened by sorin Most recent 15 issues with no replies (15) == #12398: Sending binary data with a POST request in httplib can cause U http://bugs.python.org/issue12398 #12397: re match object methods have no docstrings http://bugs.python.org/issue12397 #12391: packaging install fails to clean up temp files http://bugs.python.org/issue12391 #12376: unittest.TextTestResult.__init__ breaks under complex __mro__ http://bugs.python.org/issue12376 #12347: add an "extras" in packaging.pypi.simple.Crawler http://bugs.python.org/issue12347 #12336: tkinter.test.test_ttk.test_widgets.test_select() failure on Fr http://bugs.python.org/issue12336 #12323: ElementPath 1.3 expressions http://bugs.python.org/issue12323 #12322: ElementPath 1.3 expressions documentation http://bugs.python.org/issue12322 #12317: inspect.getabsfile() is not documented http://bugs.python.org/issue12317 #12297: Clarifications to atexit.register and unregister doc http://bugs.python.org/issue12297 #12295: Fix ResourceWarning in turtledemo help window http://bugs.python.org/issue12295 #12294: multiprocessing.Pool: Need a way to find out if work are finis http://bugs.python.org
Re: [Python-Dev] EuroPython Language Summit report
On 6/24/2011 7:18 AM, Victor Stinner wrote: Le vendredi 24 juin 2011 à 10:52 +0200, Mark Dickinson a écrit : One example: when opening a text file for reading and writing, the default encoding used depends on the platform and on various environment variables. ... oh, I agree. This choice is a big portability issue. Mac OS X, most Linux distro, BSD systems use UTF-8 local encoding, whereas Windows use legacy code pages like cp1252 (something like ISO-8859-1) or cp952 (shift jis). But sometimes, the locale is "C" (e.g. on our buildbots) and programs start to fail with Unicode errors... I see two options to improve the situation. The third is to make utf-8 the default. I believe this *is* the proper long term solution and both options are contrary to this. I believe that this is what I want for myself even on Windows. I believe utf-8 is the default or only storage by cross-platform international programs (certainly the ones I use). (1) hard way: change open() API to make encoding a mandatory argument. (2) soft way: add a warning if the encoding is implicit (locale encoding). (3) In 3.3, if the default is used and it is not utf-8, add a warning that the default will become utf-8 always in 3.4. Actually, I would like a PendingDeprecationWarning in 3.2.1 if possible. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EuroPython Language Summit report
On 6/24/2011 1:30 PM, Terry Reedy wrote: The third is to make utf-8 the default. I believe this *is* the proper long term solution and both options are contrary to this. +1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EuroPython Language Summit report
Le vendredi 24 juin 2011 à 16:30 -0400, Terry Reedy a écrit : > > I see two options to improve the situation. > > The third is to make utf-8 the default. I believe this *is* the proper > long term solution and both options are contrary to this. Oh yes, I also prefer this option, but I suspect that some people prefer to not break backward compatibility. Or should we consider this bad design choice as a bug? The UTF-8 encoder (of Python 3) raises an error if the text contains a surrogate character. The surrogatepass error handler should be used to encode surrogages. ... The surrogateescape can be used to encode back undecodable bytes (e.g. filename decoded by Python using the surrogateescape), but it is not a good idea to write illegal byte sequences (most programs will fail to open the file). > I believe that this is what I want for myself even on Windows. Can you open a UTF-8 files in all Windows program (and the text is displayed correctly)? I remember that notepad.exe writes an evil UTF-8 BOM, but I don't know if it requires this BOM to detect the UTF-8 encoding. Or do some program expect text files encoded to the ANSI code page? If you want to write files in the ANSI code page, you can use encoding="mbcs" (or use an explicit code page, like encoding="cp1252"). > (3) In 3.3, if the default is used and it is not utf-8, add a warning > that the default will become utf-8 always in 3.4. Actually, I would like > a PendingDeprecationWarning in 3.2.1 if possible. I'm not sure that the "and it is not utf-8" condition is a good idea. If you develop on Linux but your users are on Windows, you will not get the warning (even with -Werror) nor your users (users don't want to see warnings)... Or maybe an user using Windows and Linux will notice the bug without the warning :-) It doesn't mean that it is not possible to check your program: you can change your locale encoding (e.g. use LANG=C). At least, it will be possible to check test_distutils and test_packaging using LANG=C and -Werror :-) -- A fourth option is to use ASCII by default! Your program will work and be portable until you write the first non-ASCII character... Oh wait, it remembers me the Unicode nightmare of Python 2! Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EuroPython Language Summit report
On Sat, Jun 25, 2011 at 12:55 AM, R. David Murray wrote: [quoting VM summit notes] >> PEP 0380: Syntax for delegating to a subgenerator. >> >> PEP 3150: Statement local namespaces. >> >> PEP 3152: Cofunctions. >> >> For all three of the above PEPs, there was some feeling that additional >> syntax >> changes to the language are unnecessary and make it harder to learn; where >> possible, we should prefer using 3rd partly libraries. > > I disagree with this with respect to 380. Haven't looked at the others. Indeed, PEP 380 is *really* hard to do properly without language support. The language moratorium and lack of a Python 3 compatible patch are the only reasons it didn't go into 3.2 (and there's a patch porting the implementation to 3.3 up on bitbucket, although we've been tinkering with the compiler so it is likely out of date at this point). I'm the author PEP 3150 and *I* think it's a highly questionable and most likely terrible idea (hence the Deferred status). However, it's a good place to gather the related discussions, since proposals in that vein recur often on python-ideas. PEP 3152 definitely fits into the "let third parties experiment" bucket, though - once PEP 380 makes generators first class peers to functions in their support for modularisation, then we need to let the wider community play with the concept for a while before we embed anything more into the core language or standard library. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
