Re: [Python-Dev] [issue13703] Hash collision security issue
Glenn Linderman, 27.01.2012 07:47: > Can we have a tree type in 3.3, independent of dict? I'd be happy to see that happen, but I guess the usual requirements on stdlib extensions would apply here. I.e., someone has to write the code, make sure people actually use it to prove that it's worth being added, make sure it runs in different Python implementations, donate the code to the PSF asking for stdlib addition and agree to maintain it in the future. Such an addition is a totally separate issue from the hash collision attack issue. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
I'm curious why AVL tree rather than RB tree, simpler implementation? Somewhat arbitrary. AVL trees have a better performance than RB trees (1.44 log2(N) vs 2 log2(N) in the worst case). Wrt. implementation, I looked around for a trustworthy, reusable, free (as in speech), C-only implementation of both AVL and RB trees. The C++ std::map is out of question as it is C++, and many other free implementations are out of question as they are GPLed and LGPLed. Writing an implementation from scratch for a bugfix release is also out of the question. So I found Ian Piumarta's AVL tree 1.0 from 2006. I trust Ian Piumarta to get it right (plus I reviewed the code a little). There are some API glitches (such as assuming a single comparison function, whereas it would better be rewritten to directly invoke rich comparison, or such as node removal not returning the node that was removed). It gets most API decisions right, in particular wrt. memory management. The license is in the style of the MIT license. If somebody could propose an alternative implementation (e.g. one with an even more liberal license, or with a smaller per-node memory usage), I'd be open to change it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
mar...@v.loewis.de, 27.01.2012 09:55: > So I found Ian Piumarta's AVL tree 1.0 from 2006. I trust Ian Piumarta > to get it right (plus I reviewed the code a little). There are some > API glitches (such as assuming a single comparison function, whereas > it would better be rewritten to directly invoke rich comparison, or > such as node removal not returning the node that was removed). It > gets most API decisions right, in particular wrt. memory management. > The license is in the style of the MIT license. That sounds ok for internal use, and the implementation really looks short enough to allow the adaptations you propose and generic enough to be generally usable. However, note that my comment on Glenn's question regarding a stdlib addition of a tree type still applies - someone would have to write a suitable CPython wrapper for it as well as a separate pure Python implementation, and then offer both for inclusion and maintenance. I'm not sure it's a good idea to have multiple C tree implementations in CPython, i.e. one for internal use and one for the stdlib. Unless there's a serious interest in maintaining both, that is. After all, writing a Python wrapper for this may not be simpler than the work that went into one of the existing (C)Python tree implementations already. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
However, note that my comment on Glenn's question regarding a stdlib addition of a tree type still applies I agree with all that. Having a tree-based mapping type in the standard library is a different issue entirely. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 408 -- Standard library __preview__ package
Hello, Following an earlier discussion on python-ideas [1], we would like to propose the following PEP for review. Discussion is welcome. The PEP can also be viewed in HTML form at http://www.python.org/dev/peps/pep-0408/ [1] http://mail.python.org/pipermail/python-ideas/2012-January/013246.html Eli --- PEP: 408 Title: Standard library __preview__ package Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , Eli Bendersky Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2012-01-07 Python-Version: 3.3 Post-History: 2012-01-27 Abstract The process of including a new module into the Python standard library is hindered by the API lock-in and promise of backward compatibility implied by a module being formally part of Python. This PEP proposes a transitional state for modules - inclusion in a special ``__preview__`` package for the duration of a minor release (roughly 18 months) prior to full acceptance into the standard library. On one hand, this state provides the module with the benefits of being formally part of the Python distribution. On the other hand, the core development team explicitly states that no promises are made with regards to the module's eventual full inclusion into the standard library, or to the stability of its API, which may change for the next release. Proposal - the __preview__ package == Whenever the Python core development team decides that a new module should be included into the standard library, but isn't entirely sure about whether the module's API is optimal, the module can be placed in a special package named ``__preview__`` for a single minor release. In the next minor release, the module may either be "graduated" into the standard library (and occupy its natural place within its namespace, leaving the ``__preview__`` package), or be rejected and removed entirely from the Python source tree. If the module ends up graduating into the standard library after spending a minor release in ``__preview__``, its API may be changed according to accumulated feedback. The core development team explicitly makes no guarantees about API stability and backward compatibility of modules in ``__preview__``. Entry into the ``__preview__`` package marks the start of a transition of the module into the standard library. It means that the core development team assumes responsibility of the module, similarly to any other module in the standard library. Which modules should go through ``__preview__`` --- We expect most modules proposed for addition into the Python standard library to go through a minor release in ``__preview__``. There may, however, be some exceptions, such as modules that use a pre-defined API (for example ``lzma``, which generally follows the API of the existing ``bz2`` module), or modules with an API that has wide acceptance in the Python development community. In any case, modules that are proposed to be added to the standard library, whether via ``__preview__`` or directly, must fulfill the acceptance conditions set by PEP 2. It is important to stress that the aim of of this proposal is not to make the process of adding new modules to the standard library more difficult. On the contrary, it tries to provide a means to add *more* useful libraries. Modules which are obvious candidates for entry can be added as before. Modules which due to uncertainties about the API could be stalled for a long time now have a means to still be distributed with Python, via an incubation period in the ``__preview__`` package. Criteria for "graduation" - In principle, most modules in the ``__preview__`` package should eventually graduate to the stable standard library. Some reasons for not graduating are: * The module may prove to be unstable or fragile, without sufficient developer support to maintain it. * A much better alternative module may be found during the preview release Essentially, the decision will be made by the core developers on a per-case basis. The point to emphasize here is that a module's appearance in the ``__preview__`` package in some release does not guarantee it will continue being part of Python in the next release. Example --- Suppose the ``example`` module is a candidate for inclusion in the standard library, but some Python developers aren't convinced that it presents the best API for the problem it intends to solve. The module can then be added to the ``__preview__`` package in release ``3.X``, importable via:: from __preview__ import example Assuming the module is then promoted to the the standard library proper in release ``3.X+1``, it will be moved to a permanent location in the library:: import example And importing it from ``__preview__`` will no longer work. Rationale = Benefits for the core development team ---
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
+0. I think the idea is right, and will help to get good quality modules in at a faster rate. However it is compensating for a lack of interface and packaging standardization in the 3rd party module world. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Hi, A small comment from a user perspective. Since a package in preview is strongly linked to a given version of Python, any program taking advantage of it becomes strongly specific to a given version of Python. Such programs will of course break for any upgrade or downgrade of python version. To make the reason for the breakage more explicit, I believe that the PEP should provide examples of correct versionned usage of the module. Something along the lines of : if sys.version_info[:2] == (3, X): from __preview__ import example else: raise ImportError( 'Package example is only available as preview in Python version 3.X. Please check the documentation of your version of Python to see if and how you can get the package example.' ) cheers, Philippe ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Fri, 27 Jan 2012 15:21:33 +0200 Eli Bendersky wrote: > > Following an earlier discussion on python-ideas [1], we would like to > propose the following PEP for review. Discussion is welcome. The PEP > can also be viewed in HTML form at > http://www.python.org/dev/peps/pep-0408/ A big +1 from me. > Assuming the module is then promoted to the the standard library proper in > release ``3.X+1``, it will be moved to a permanent location in the library:: > > import example > > And importing it from ``__preview__`` will no longer work. Why not leave it accessible through __preview__ too? > Benefits for the core development team > -- > > Currently, the core developers are really reluctant to add new interfaces to > the standard library. A nit, but I think "reluctant" is enough and "really" makes the tone very defensive :) > Relationship with PEP 407 > = > > PEP 407 proposes a change to the core Python release cycle to permit interim > releases every 6 months (perhaps limited to standard library updates). If > such a change to the release cycle is made, the following policy for the > ``__preview__`` namespace is suggested: > > * For long term support releases, the ``__preview__`` namespace would always > be empty. > * New modules would be accepted into the ``__preview__`` namespace only in > interim releases that immediately follow a long term support release. Well this is all speculative (due to the status of PEP 407) but I think a simpler approach of having a __preview__ namespace in all releases (including LTS) would be easier to handler for both us and our users. People can refrain from using anything in __preview__ if that's what they prefer. The naming and the double underscores make it quite recognizable at the top of a source file :-) > Preserving pickle compatibility > --- > > A pickled class instance based on a module in ``__preview__`` in release 3.X > won't be unpickle-able in release 3.X+1, where the module won't be in > ``__preview__``. Special code may be added to make this work, but this goes > against the intent of this proposal, since it implies backward compatibility. > Therefore, this PEP does not propose to preserve pickle compatibility. Wouldn't it be a good argument to keep __preview__.XXX as an alias? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 14:37, Philippe Fremy wrote: Hi, A small comment from a user perspective. Since a package in preview is strongly linked to a given version of Python, any program taking advantage of it becomes strongly specific to a given version of Python. Such programs will of course break for any upgrade or downgrade of python version. To make the reason for the breakage more explicit, I believe that the PEP should provide examples of correct versionned usage of the module. Something along the lines of : if sys.version_info[:2] == (3, X): from __preview__ import example else: raise ImportError( 'Package example is only available as preview in Python version 3.X. Please check the documentation of your version of Python to see if and how you can get the package example.' ) A more normal incantation, as is often the way for packages that became parts of the standard library after first being a third party library (sometimes under a different name, e.g. simplejson -> json): try: from __preview__ import thing except ImportError: import thing So no need to target a very specific version of Python. Michael cheers, Philippe ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 15:09, Antoine Pitrou wrote: On Fri, 27 Jan 2012 15:21:33 +0200 Eli Bendersky wrote: Following an earlier discussion on python-ideas [1], we would like to propose the following PEP for review. Discussion is welcome. The PEP can also be viewed in HTML form at http://www.python.org/dev/peps/pep-0408/ A big +1 from me. Assuming the module is then promoted to the the standard library proper in release ``3.X+1``, it will be moved to a permanent location in the library:: import example And importing it from ``__preview__`` will no longer work. Why not leave it accessible through __preview__ too? +1 The point about pickling is one good reason, minimising code breakage (due to package name changing) is another. Michael Benefits for the core development team -- Currently, the core developers are really reluctant to add new interfaces to the standard library. A nit, but I think "reluctant" is enough and "really" makes the tone very defensive :) Relationship with PEP 407 = PEP 407 proposes a change to the core Python release cycle to permit interim releases every 6 months (perhaps limited to standard library updates). If such a change to the release cycle is made, the following policy for the ``__preview__`` namespace is suggested: * For long term support releases, the ``__preview__`` namespace would always be empty. * New modules would be accepted into the ``__preview__`` namespace only in interim releases that immediately follow a long term support release. Well this is all speculative (due to the status of PEP 407) but I think a simpler approach of having a __preview__ namespace in all releases (including LTS) would be easier to handler for both us and our users. People can refrain from using anything in __preview__ if that's what they prefer. The naming and the double underscores make it quite recognizable at the top of a source file :-) Preserving pickle compatibility --- A pickled class instance based on a module in ``__preview__`` in release 3.X won't be unpickle-able in release 3.X+1, where the module won't be in ``__preview__``. Special code may be added to make this work, but this goes against the intent of this proposal, since it implies backward compatibility. Therefore, this PEP does not propose to preserve pickle compatibility. Wouldn't it be a good argument to keep __preview__.XXX as an alias? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
2012/1/27 Eli Bendersky : > Criteria for "graduation" > - I think you also need "Criteria for being placed in __preview__". Do we just toss everything someone suggests in? -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
> A more normal incantation, as is often the way for packages that became
> parts of the standard library after first being a third party library
> (sometimes under a different name, e.g. simplejson -> json):
>
> try:
> from __preview__ import thing
> except ImportError:
> import thing
>
> So no need to target a very specific version of Python.
I think this is suboptimal, having to guess where modules are located,
you end up with this in every module:
try:
import cjson as json
except ImportError:
try:
import simplejson as json
except ImportError:
import json as json
Perhaps the versioned import stuff could be implemented (whatever the
syntax may be), in order that something like this can be done instead:
import regex('__preview__')
import regex('3.4')
Where clearly the __preview__ version makes no guarantees about
interface or implementation whatsoever.
etc.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 15:34, Benjamin Peterson wrote: 2012/1/27 Eli Bendersky: Criteria for "graduation" - I think you also need "Criteria for being placed in __preview__". Do we just toss everything someone suggests in? And given that permanently deleting something from __preview__ would be a big deal (deciding it didn't make the grade and should never graduate), the criteria shouldn't be much less strict than for adopting a package into the standard library. i.e. once something gets into __preview__ people are going to assume it will graduate at some point - __preview__ is a place for apis to stabilise and mature, not a place for dubious libraries that we may or may not want in the standard library at some point. Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 15:35, Matt Joiner wrote:
A more normal incantation, as is often the way for packages that became
parts of the standard library after first being a third party library
(sometimes under a different name, e.g. simplejson -> json):
try:
from __preview__ import thing
except ImportError:
import thing
So no need to target a very specific version of Python.
I think this is suboptimal, having to guess where modules are located,
you end up with this in every module:
try:
import cjson as json
except ImportError:
try:
import simplejson as json
except ImportError:
import json as json
It's trivial to wrap in a function though - or do the import in one
place and then import the package from there.
Michael
Perhaps the versioned import stuff could be implemented (whatever the
syntax may be), in order that something like this can be done instead:
import regex('__preview__')
import regex('3.4')
Where clearly the __preview__ version makes no guarantees about
interface or implementation whatsoever.
etc.
--
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
2012/1/26 Ethan Furman : > PEP: XXX Congratulations, you are now PEP 409. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 16:25, Michael Foord wrote: > On 27/01/2012 14:37, Philippe Fremy wrote: >> Hi, >> >> A small comment from a user perspective. >> >> Since a package in preview is strongly linked to a given version of >> Python, any program taking advantage of it becomes strongly specific to >> a given version of Python. >> >> Such programs will of course break for any upgrade or downgrade of >> python version. To make the reason for the breakage more explicit, I >> believe that the PEP should provide examples of correct versionned usage >> of the module. >> >> Something along the lines of : >> >> if sys.version_info[:2] == (3, X): >> from __preview__ import example >> else: >> raise ImportError( 'Package example is only available as preview in >> Python version 3.X. Please check the documentation of your version of >> Python to see if and how you can get the package example.' ) > > A more normal incantation, as is often the way for packages that became > parts of the standard library after first being a third party library > (sometimes under a different name, e.g. simplejson -> json): > > try: > from __preview__ import thing > except ImportError: > import thing > > So no need to target a very specific version of Python. > According to the PEP, the interface may change betweeen __preview__ and final inclusion in stdlib. It would be unwise as a developer to assume that a program written for the preview version will work correctly in the stdlib version, wouldn't it ? I would use your "normal" incantation only after checking that no significant API change have occured after stdlib integration. By the way, if as Antoine suggests, the package remain available in __preview__ even after it's accepted in the stdlib, how is the user supposed to deal with possible API changes ? cheers, Philippe ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Hello Philippe, On Fri, 27 Jan 2012 17:09:08 +0100 Philippe Fremy wrote: > > According to the PEP, the interface may change betweeen __preview__ and > final inclusion in stdlib. It would be unwise as a developer to assume > that a program written for the preview version will work correctly in > the stdlib version, wouldn't it ? > > I would use your "normal" incantation only after checking that no > significant API change have occured after stdlib integration. > > By the way, if as Antoine suggests, the package remain available in > __preview__ even after it's accepted in the stdlib, how is the user > supposed to deal with possible API changes ? The API *may* change but it would probably not change much anyway. Consider e.g. the "regex" module: it aims at compatibility with the standard "re" module; there may be additional APIs (e.g. new flags), but whoever uses it with the standard "re" API would not see any difference between the __preview__ version and the final version. cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
>> Assuming the module is then promoted to the the standard library proper in >> release ``3.X+1``, it will be moved to a permanent location in the library:: >> >> import example >> >> And importing it from ``__preview__`` will no longer work. > > Why not leave it accessible through __preview__ too? I guess there's no real problem with leaving it accessible, as long as it's clear that the API may have changed between releases. I.e. when a package "graduates" and is also left accessible through __preview__, it should obviously be just a pointer to the same package, so if the API changed, code that imported it from __preview__ in a previous release may stop working. > >> Benefits for the core development team >> -- >> >> Currently, the core developers are really reluctant to add new interfaces to >> the standard library. > > A nit, but I think "reluctant" is enough and "really" makes the > tone very defensive :) Agreed, I will change this > >> Relationship with PEP 407 >> = >> >> PEP 407 proposes a change to the core Python release cycle to permit interim >> releases every 6 months (perhaps limited to standard library updates). If >> such a change to the release cycle is made, the following policy for the >> ``__preview__`` namespace is suggested: >> >> * For long term support releases, the ``__preview__`` namespace would always >> be empty. >> * New modules would be accepted into the ``__preview__`` namespace only in >> interim releases that immediately follow a long term support release. > > Well this is all speculative (due to the status of PEP 407) but I think > a simpler approach of having a __preview__ namespace in all releases > (including LTS) would be easier to handler for both us and our users. > People can refrain from using anything in __preview__ if that's what > they prefer. The naming and the double underscores make it quite > recognizable at the top of a source file :-) I agree that it's speculative, and would recommend to decouple the two PEPs. They surely can live on their own and aren't tied. If PEP 407 gets accepted, this section can be reworded appropriately. > >> Preserving pickle compatibility >> --- >> >> A pickled class instance based on a module in ``__preview__`` in release 3.X >> won't be unpickle-able in release 3.X+1, where the module won't be in >> ``__preview__``. Special code may be added to make this work, but this goes >> against the intent of this proposal, since it implies backward compatibility. >> Therefore, this PEP does not propose to preserve pickle compatibility. > > Wouldn't it be a good argument to keep __preview__.XXX as an alias? Good point. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
>> Something along the lines of : >> >> if sys.version_info[:2] == (3, X): >> from __preview__ import example >> else: >> raise ImportError( 'Package example is only available as preview in >> Python version 3.X. Please check the documentation of your version of >> Python to see if and how you can get the package example.' ) > > > A more normal incantation, as is often the way for packages that became > parts of the standard library after first being a third party library > (sometimes under a different name, e.g. simplejson -> json): > > try: > from __preview__ import thing > except ImportError: > import thing > > So no need to target a very specific version of Python. > Yep, this is what I had in mind. And it appeared too trivial to place it in the PEP. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Fri, Jan 27, 2012 at 17:34, Benjamin Peterson wrote: > 2012/1/27 Eli Bendersky : >> Criteria for "graduation" >> - > > I think you also need "Criteria for being placed in __preview__". Do > we just toss everything someone suggests in? > I hoped to have this covered by: "In any case, modules that are proposed to be added to the standard library, whether via __preview__ or directly, must fulfill the acceptance conditions set by PEP 2." PEP 2 is quite detailed and I saw no need to repeat large chunks of it here. The idea is that all the same restrictions and caveats apply. The thing that goes away is promise for future API stability. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2012-01-20 - 2012-01-27)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open3234 (+25)
closed 22437 (+32)
total 25671 (+57)
Open issues with patches: 1391
Issues opened (44)
==
#6631: Disallow relative files paths in urllib*.open()
http://bugs.python.org/issue6631 reopened by amaury.forgeotdarc
#13829: exception error in _scproxy.so
http://bugs.python.org/issue13829 reopened by ned.deily
#13836: Define key failed
http://bugs.python.org/issue13836 opened by olivier57
#13837: test_shutil fails with symlinks enabled under Windows
http://bugs.python.org/issue13837 opened by pitrou
#13839: -m pstats should combine all the profiles given as arguments
http://bugs.python.org/issue13839 opened by anacrolix
#13841: multiprocessing should use sys.exit() where possible
http://bugs.python.org/issue13841 opened by brandj
#13842: Cannot pickle Ellipsis or NotImplemented
http://bugs.python.org/issue13842 opened by James.Sanders
#13843: Python doesn't compile anymore on our Solaris buildbot: undefi
http://bugs.python.org/issue13843 opened by haypo
#13845: Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on
http://bugs.python.org/issue13845 opened by haypo
#13846: Add time.monotonic() function
http://bugs.python.org/issue13846 opened by haypo
#13847: Catch time(), ftime(), localtime() and clock() errors
http://bugs.python.org/issue13847 opened by haypo
#13848: io.open() doesn't check for embedded NUL characters
http://bugs.python.org/issue13848 opened by pitrou
#13849: Add tests for NUL checking in certain strs
http://bugs.python.org/issue13849 opened by alex
#13850: Summary tables for argparse add_argument options
http://bugs.python.org/issue13850 opened by ncoghlan
#13851: Packaging distutils2 for Fedora
http://bugs.python.org/issue13851 opened by vikash
#13854: multiprocessing: SystemExit from child with non-int, non-str a
http://bugs.python.org/issue13854 opened by brandj
#13855: Add qualname support to types.FunctionType
http://bugs.python.org/issue13855 opened by meador.inge
#13856: xmlrpc / httplib changes to allow for certificate verification
http://bugs.python.org/issue13856 opened by Nathanael.Noblet
#13857: Add textwrap.indent() as counterpart to textwrap.dedent()
http://bugs.python.org/issue13857 opened by ncoghlan
#13860: PyBuffer_FillInfo() return value
http://bugs.python.org/issue13860 opened by skrah
#13861: test_pydoc failure
http://bugs.python.org/issue13861 opened by skrah
#13863: import.c sometimes generates incorrect timestamps on Windows +
http://bugs.python.org/issue13863 opened by mark.dickinson
#13865: distutils documentation says Extension has "optional" argument
http://bugs.python.org/issue13865 opened by tebeka
#13866: {urllib,urllib.parse}.urlencode should not use quote_plus
http://bugs.python.org/issue13866 opened by Stephen.Day
#13867: misleading comment in weakrefobject.h
http://bugs.python.org/issue13867 opened by Jim.Jewett
#13868: Add hyphen doc fix
http://bugs.python.org/issue13868 opened by Retro
#13869: CFLAGS="-UNDEBUG" build failure
http://bugs.python.org/issue13869 opened by skrah
#13871: namedtuple does not normalize field names when checking for du
http://bugs.python.org/issue13871 opened by Jim.Jewett
#13872: socket.detach doesn't mark socket._closed
http://bugs.python.org/issue13872 opened by anacrolix
#13873: SIGBUS in test_zlib on Debian bigmem buildbot
http://bugs.python.org/issue13873 opened by nadeem.vawda
#13874: test_faulthandler: read_null test fails with current clang
http://bugs.python.org/issue13874 opened by skrah
#13875: cmd: no user documentation
http://bugs.python.org/issue13875 opened by techtonik
#13876: Sporadic failure in test_socket
http://bugs.python.org/issue13876 opened by nadeem.vawda
#13878: test_sched failures on Windows buildbot
http://bugs.python.org/issue13878 opened by nadeem.vawda
#13879: Argparse does not support subparser aliases in 2.7
http://bugs.python.org/issue13879 opened by Tim.Willis
#13880: pydoc -k throws "AssertionError: distutils has already been pa
http://bugs.python.org/issue13880 opened by __KFL__
#13881: Stream encoder for zlib_codec doesn't use the incremental enco
http://bugs.python.org/issue13881 opened by amcnabb
#13882: Add format argument for time.time(), time.clock(), ... to get
http://bugs.python.org/issue13882 opened by haypo
#13884: IDLE 2.6.5 Recent Files undocks
http://bugs.python.org/issue13884 opened by mcgrete
#13886: readline-related test_builtin failure
http://bugs.python.org/issue13886 opened by nadeem.vawda
#13888: test_builtin failure when run after test_tk
http://bugs.python.org/issue13888 opened by nadeem.vawda
#13889: str(float) and round(float) issues with FPU precision
http://bugs.python.org/issue13889 opened by samuel.iseli
#13890: test
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Eli Bendersky gmail.com> writes: > > Hello, > > Following an earlier discussion on python-ideas [1], we would like to > propose the following PEP for review. Discussion is welcome. The PEP > can also be viewed in HTML form at > http://www.python.org/dev/peps/pep-0408/ > > [1] http://mail.python.org/pipermail/python-ideas/2012-January/013246.html > I'm -1 on this, for a pretty simple reason. Something goes into __preview__, instead of it's final destination directly because it needs feedback/possibly changes. However, given the release cycle of the stdlib (~18 months), any feedback it gets can't be seen by actual users until it's too late. Essentially you can only get one round of stdlib. I think a significantly healthier process (in terms of maximizing feedback and getting something into it's best shape) is to let a project evolve naturally on PyPi and in the ecosystem, give feedback to it from an inclusion perspective, and then include it when it becomes ready on it's own merits. The counter argument to this is that putting it in the stdlib gets you signficantly more eyeballs (and hopefully more feedback, therefore), my only response to this is: if it doesn't get eyeballs on PyPi I don't think there's a great enough need to justify it in the stdlib. Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
Guido van Rossum wrote: Did you consider to just change the words so users can ignore it more easily? Yes, that has also been discussed. Speaking for myself, it would be only slightly better. Speaking for everyone that wants context suppression (using Steven D'Aprano's words): chained exceptions expose details to the caller that are irrelevant implementation details. It seems to me that generating the amount of information needed to track down errors is a balancing act between too much and too little; forcing the print of previous context when switching from exception A to exception B feels like too much: at the very least it's extra noise; at the worst it can be confusing to the actual problem. When the library (or custom class) author is catching A, saying "Yes, expected, now let's raise B instead", A is no longer necessary. Also, the programmer is free to *not* use 'from None', leaving the complete traceback in place. ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
On 1/26/2012 10:47 PM, Glenn Linderman wrote: On 1/26/2012 10:25 PM, Gregory P. Smith wrote: (and on top of all of this I believe we're all settled on having per interpreter hash randomization_as well_ in 3.3; but this AVL tree approach is one nice option for a backport to fix the major vulnerability) If the tree code cures the problem, then randomization just makes debugging harder. I think if it is included in 3.3, it needs to have a switch to turn it on/off (whichever is not default). In case it is not clear, I meant randomization should always be able to be switched off. Another issue occurs to me: when a hash with colliding keys (one that has been attacked, and has trees) has a non-string key added, isn't the flattening process likely to have extremely poor performance? Agreed that the common HTML FORM or JSON attack vectors are unlikely to produce anything except string keys, but if an application grabs those, knows that the user keys are all strings, and adds a few more bits of info to the dict for convenience, using other key types, then ... WHAM? Seems a bit unlikely, but I know I've coded things along that line from time to time... I don't recall doing it in Python Web applications... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
Another issue occurs to me: when a hash with colliding keys (one that has been attacked, and has trees) has a non-string key added, isn't the flattening process likely to have extremely poor performance? Correct. "Don't do that, then" I don't consider it mandatory to fix all issues with hash collision. In fact, none of the strategies fixes all issues with hash collisions; even the hash-randomization solutions only deal with string keys, and don't consider collisions on non-string keys. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
On Fri, Jan 27, 2012 at 9:08 AM, Ethan Furman wrote: > Guido van Rossum wrote: >> >> Did you consider to just change the >> words so users can ignore it more easily? > > > Yes, that has also been discussed. > > Speaking for myself, it would be only slightly better. > > Speaking for everyone that wants context suppression (using Steven > D'Aprano's words): chained exceptions expose details to the caller that are > irrelevant implementation details. > > It seems to me that generating the amount of information needed to track > down errors is a balancing act between too much and too little; forcing the > print of previous context when switching from exception A to exception B > feels like too much: at the very least it's extra noise; at the worst it > can be confusing to the actual problem. When the library (or custom class) > author is catching A, saying "Yes, expected, now let's raise B instead", A > is no longer necessary. > > Also, the programmer is free to *not* use 'from None', leaving the complete > traceback in place. Ok, got it. The developer has to explicitly say "raise from None" and that indicates they have really thought about the issue of suppressing too much information and they are okay with it. I dig that. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Hashing proposal: 64-bit hash
As already mentioned, the vulnerability of 64-bit Python rather theoretical and not practical. The size of the hash makes the attack is extremely unlikely. Perhaps the easiest change, avoid 32-bit Python on the vulnerability, will use 64-bit (or more) hash on all platforms. The performance is comparable to the randomization. Keys order depended code will be braked not stronger than when you change the platform or Python feature version. Maybe all the 64 bits used only for strings, and for other objects -- only the lower 32 bits. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hashing proposal: 64-bit hash
2012/1/27 Serhiy Storchaka : > As already mentioned, the vulnerability of 64-bit Python rather theoretical > and not practical. The size of the hash makes the attack is extremely > unlikely. Perhaps the easiest change, avoid 32-bit Python on the > vulnerability, will use 64-bit (or more) hash on all platforms. The > performance is comparable to the randomization. Keys order depended code will > be braked not stronger than when you change the platform or Python feature > version. Maybe all the 64 bits used only for strings, and for other objects > -- only the lower 32 bits. A tempting idea, but binary incompatible. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Eli Bendersky wrote:
Hello,
Following an earlier discussion on python-ideas [1], we would like to
propose the following PEP for review. Discussion is welcome.
I think you need to emphasize that modules in __preview__ are NOT expected to
have a forward-compatible, stable, API. This is a feature of __preview__, not
a bug, and I believe it is the most important feature.
I see responses to this PEP that assume that APIs will be stable, and that
having a module fail to graduate out of __preview__ should be an extraordinary
event. But if this is the case, then why bother with __preview__? It just adds
complexity to the process -- if __preview__.spam and spam are expected to be
the same, then just spam straight into the std lib and be done with it.
This PEP only makes sense if we assume that __preview__.spam and spam *will*
be different, even if only in minor ways, and that there might not even be a
spam. There should be no expectation that every __preview__ module must
graduate, or that every standard library module must go through __preview__.
If it is stable and uncontroversial, __preview__ adds nothing to the process.
Even when there are candidates for inclusion with relatively stable APIs, like
regex, we should *assume* that there will be API differences between
__preview__.regex and regex, simply because it is less harmful to expect
changes that don't eventuate than to expect stability and be surprised by changes.
This, I believe, rules out Antoine's suggestion that modules remain importable
from __preview__ even after graduation to a full member of the standard
library. We simply can't say have all three of these statements true at the
same time:
1) regular standard library modules are expected to be backward compatible
2) __preview__ modules are not expected to be forward compatible
3) __preview__.spam is an alias to regular standard library spam
At least one of them has to go. Since both 1) and 2) are powerful features,
and 3) is only a convenience, the obvious one to drop is 3). I note that the
PEP, as it is currently written, explicitly states that __preview__.spam will
be dropped when it graduates to spam. This is a good thing and should not be
changed.
Keeping __preview__.spam around after graduation is, I believe, actively
harmful. It adds complexity to the developer's decision-making process
("Should I import spam from __preview__, or just import spam? What's the
difference?"). It gives a dangerous impression that code written for
__preview__.spam will still work for spam.
We should be discouraging simple-minded recipes like
try:
import spam
except ImportError:
from __preview__ import spam
spam.foo(a, b, c)
since they undermine the vital feature of __preview__ that the signature and
even the existence of spam.foo is subject to change.
I would go further and suggest that __preview__ be explicitly called
__unstable__. If that name is scary, and it frightens some users off, good!
The last thing we want is when 3.4 comes around to have dozens of bug reports
along the line of "spam.foo() and __preview__.spam.foo() have different
function signatures and aren't compatible". Of course they do. That's why
__preview__.spam existed in the first place, to allow the API to mature
without the expectation that it was already stable.
Since __preview__.spam (or, as I would prefer, __unstable__.spam) and spam
cannot be treated as drop-in replacements, what is __preview__.spam good for?
Without a stable API, __preview__.spam is not suitable for use in production
applications that expect to run under multiple versions of the standard library.
I think the PEP needs more use-cases on who might use __preview__.spam, and
why. These come to my mind:
* if you don't care about Python 3.x+1, then there is no reason not to
treat Python 3.x's __preview__.spam as stable;
* rapid development proof-of-concept software ("build one to throw away")
can safely use __preview__.spam, since they are expected to be replaced
anyway;
* one-use scripts;
* use at the interactive interpreter;
* any other time where forward-compatibility is not required.
I am reminded of the long, often acrimonious arguments that took place on
Python-Dev a few years back about the API for the ipaddr library. A lot of the
arguments could have been short-circuited if we had said "putting ipaddr into
__preview__ does not constitute acceptance of its API".
(On the other hand, if __preview__ becomes used in the future for library
authors to fob-off criticism for 18 months in the hope it will just be
forgotten, then this will be a bad thing.)
--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Eli Bendersky wrote: try: from __preview__ import thing except ImportError: import thing So no need to target a very specific version of Python. Yep, this is what I had in mind. And it appeared too trivial to place it in the PEP. Trivial and wrong. Since thing and __preview__.thing may have subtle, or major, API differences, how do you use it? try: result = thing.foo(a, b, c) + thing.bar(x) except AttributeError: # Must be the preview version result = thing.foobar(a, c, b, x) -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hashing proposal: 64-bit hash
As already mentioned, the vulnerability of 64-bit Python rather theoretical and not practical. The size of the hash makes the attack is extremely unlikely. Unfortunately this assumption is not correct. It works very good with 64bit-hashing. It's much harder to create (efficiently) 64-bit hash-collisions. But I managed to do so and created strings with a length of 16 (6-bit)-characters (a-z, A-Z, 0-9, _, .). Even 14 characters would have been enough. You need less than twice as many characters for the same effect as in the 32bit-world. Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 27, 2012, at 05:26 PM, Alex wrote: >I'm -1 on this, for a pretty simple reason. Something goes into __preview__, >instead of it's final destination directly because it needs feedback/possibly >changes. However, given the release cycle of the stdlib (~18 months), any >feedback it gets can't be seen by actual users until it's too >late. Essentially you can only get one round of stdlib. I'm -1 on this as well. It just feels like the completely wrong way to stabilize an API, and I think despite the caveats that are explicit in __preview__, Python will just catch tons of grief from users and haters about API instability anyway, because from a practical standpoint, applications written using __preview__ APIs *will* be less stable. It also won't improve the situation for prospective library developers because they're locked into Python's development cycle anyway. I also think the benefit to users is a false one since it will be much harder to write applications that are portable across Python releases. >I think a significantly healthier process (in terms of maximizing feedback >and getting something into it's best shape) is to let a project evolve >naturally on PyPi and in the ecosystem, give feedback to it from an inclusion >perspective, and then include it when it becomes ready on it's own >merits. The counter argument to this is that putting it in the stdlib gets >you signficantly more eyeballs (and hopefully more feedback, therefore), my >only response to this is: if it doesn't get eyeballs on PyPi I don't think >there's a great enough need to justify it in the stdlib. I agree with everything Alex said here. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Fri, 27 Jan 2012 16:10:51 -0500 Barry Warsaw wrote: > > I'm -1 on this as well. It just feels like the completely wrong way to > stabilize an API, and I think despite the caveats that are explicit in > __preview__, Python will just catch tons of grief from users and haters about > API instability anyway, because from a practical standpoint, applications > written using __preview__ APIs *will* be less stable. Well, obviously __preview__ is not for the most conservative users. I think the name clearly conveys the idea that you are trying out something which is not in its definitive state, doesn't it? > >I think a significantly healthier process (in terms of maximizing feedback > >and getting something into it's best shape) is to let a project evolve > >naturally on PyPi and in the ecosystem, give feedback to it from an inclusion > >perspective, and then include it when it becomes ready on it's own > >merits. The counter argument to this is that putting it in the stdlib gets > >you signficantly more eyeballs (and hopefully more feedback, therefore), my > >only response to this is: if it doesn't get eyeballs on PyPi I don't think > >there's a great enough need to justify it in the stdlib. > > I agree with everything Alex said here. The idea that being on PyPI is sufficient is nice but flawed (the IPaddr example). PyPI doesn't guarantee any visibility (how many packages are there?). Furthermore, having users is not a guarantee that the API is appropriate, either; it just means that the API is appropriate for *some* users. On the other hand, __preview__ would clearly signal that something is on the verge of being frozen as an official stdlib API, and would prompt people to actively try it. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27 January 2012 21:48, Antoine Pitrou wrote: > Well, obviously __preview__ is not for the most conservative users. I > think the name clearly conveys the idea that you are trying out > something which is not in its definitive state, doesn't it? Agreed. But that in turn implies to me that __preview__.foo should not be maintained as an alias for foo once it gets "promoted". Firstly, because if you're not comfortable with changing your code to make the simple change to remove the __preview__ prefix in the import, then how could you be comfortable with using a module with no compatibility guarantee anyway? (BTW, I assume that the normal incantation would actually be "from __preview__ import foo", as that limits the module name change to the import statement). > The idea that being on PyPI is sufficient is nice but flawed (the > IPaddr example). PyPI doesn't guarantee any visibility (how many > packages are there?). Furthermore, having users is not a guarantee that > the API is appropriate, either; it just means that the API is > appropriate for *some* users. Agreed entirely. We need a way to signal somehow that a module is being seriously considered for stdlib inclusion. That *would* result in more uptake, and hence more testing and feedback. As an example, I would definitely try out MRAB's regex module if it were in __preview__, but even though I keep meaning to, I've never actually got round to bothering to download from PyPI - I end up just using the stdlib re for my one-off scripts. > On the other hand, __preview__ would clearly signal that something is > on the verge of being frozen as an official stdlib API, and would > prompt people to actively try it. Precisely. It's in effect a "last call for feedback", and people should view it that way, in my opinion. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
On 1/27/2012 2:54 PM, Guido van Rossum wrote: On Fri, Jan 27, 2012 at 9:08 AM, Ethan Furman wrote: Guido van Rossum wrote: Did you consider to just change the words so users can ignore it more easily? Yes, that has also been discussed. Speaking for myself, it would be only slightly better. Speaking for everyone that wants context suppression (using Steven D'Aprano's words): chained exceptions expose details to the caller that are irrelevant implementation details. Especially if the users are non-programmer app users. It seems to me that generating the amount of information needed to track down errors is a balancing act between too much and too little; forcing the print of previous context when switching from exception A to exception B feels like too much: at the very least it's extra noise; at the worst it can be confusing to the actual problem. When the library (or custom class) author is catching A, saying "Yes, expected, now let's raise B instead", A is no longer necessary. I find double tracebacks to be 'jarring'. If there is a double bug, one in both the try and except blocks, it *should* stand out. If there is just one bug and the developer merely wants to rename it and change the message, it should not. Also, the programmer is free to *not* use 'from None', leaving the complete traceback in place. Ok, got it. The developer has to explicitly say "raise from None" and that indicates they have really thought about the issue of suppressing too much information and they are okay with it. I dig that. Now that I have been reminded that 'from x' was already added to raise statements, I am fine with reusing that. I still think it 'sticks out' more than the 'as' version, but when reading code, having (rare) info suppression stick out is not so bad. The PEP does not address the issue of whether the new variation of raise is valid outside of an except block. My memory is that it was not to be and I think it should not be. One advantage of the 'as' form is that it is clear that raising the default as something else is invalid if there is no default. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 27, 2012, at 10:48 PM, Antoine Pitrou wrote: >On Fri, 27 Jan 2012 16:10:51 -0500 >Barry Warsaw wrote: >> >> I'm -1 on this as well. It just feels like the completely wrong way to >> stabilize an API, and I think despite the caveats that are explicit in >> __preview__, Python will just catch tons of grief from users and haters about >> API instability anyway, because from a practical standpoint, applications >> written using __preview__ APIs *will* be less stable. > >Well, obviously __preview__ is not for the most conservative users. I >think the name clearly conveys the idea that you are trying out >something which is not in its definitive state, doesn't it? Maybe. I could quibble about the name, but let's not bikeshed on that right now. The problem as I see it is that __preview__ will be very tempting to use in production. In fact, its use case is almost predicated on that. (We want you to use it so you can tell us if the API is good.) Once people use it, they will probably ship code that relies on it, and then the pressure will be applied to us to continue to support that API even if a newer, better one gets promoted out of __preview__. I worry that over time, for all practical purposes, there won't be much difference between __preview__ and the stdlib. >> >I think a significantly healthier process (in terms of maximizing feedback >> >and getting something into it's best shape) is to let a project evolve >> >naturally on PyPi and in the ecosystem, give feedback to it from an >> >inclusion >> >perspective, and then include it when it becomes ready on it's own >> >merits. The counter argument to this is that putting it in the stdlib gets >> >you signficantly more eyeballs (and hopefully more feedback, therefore), my >> >only response to this is: if it doesn't get eyeballs on PyPi I don't think >> >there's a great enough need to justify it in the stdlib. >> >> I agree with everything Alex said here. > >The idea that being on PyPI is sufficient is nice but flawed (the >IPaddr example). PyPI doesn't guarantee any visibility (how many >packages are there?). Furthermore, having users is not a guarantee that >the API is appropriate, either; it just means that the API is >appropriate for *some* users. I can't argue with that, it's just that I don't think __preview__ solves that problem. And it seems to me that __preview__ introduces a whole 'nother set of problems on top of that. So taking the IPaddr example further. Would having it in the stdlib, relegated to an explicitly unstable API part of the stdlib, increase eyeballs enough to generate the kind of API feedback we're looking for, without imposing an additional maintenance burden on us? If you were writing an app that used something in __preview__, how would you provide feedback on what parts of the API you'd want to change, *and* how would you adapt your application to use those better APIs once they became available 18 months from now? I think we'll just see folks using the unstable APIs and then complaining when we remove them, even though they *know* *upfront* that these APIs will go away. I'm also nervous about it from an OS vender point of view. Should I reject any applications that import from __preview__? Or do I have to make a commitment to support those APIs longer than Python does because the application that uses it is important to me? I think the OS vendor problem is easier with an application that uses some PyPI package, because I can always make that package available to the application by pulling in the version I care about. It's harder if a newer, incompatible version is released upstream and I want to provide both, but I don't think __preview__ addresses that. A robust, standard approach to versioning of modules would though, and I think would better solve what __preview__ is trying to solve. >On the other hand, __preview__ would clearly signal that something is >on the verge of being frozen as an official stdlib API, and would >prompt people to actively try it. I'm not so sure about that. If I were to actively try it, I'm not sure how much motivation I'd have to rewrite key parts of my code when an incompatible version gets promoted to the un__preview__d stdlib. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 27, 2012, at 10:02 PM, Paul Moore wrote: >Agreed entirely. We need a way to signal somehow that a module is >being seriously considered for stdlib inclusion. That *would* result >in more uptake, and hence more testing and feedback. I'm just not convinced that's a message that we can clearly articulate to users of the library. I think most people will see it in the module documentation, just use it, and then complain when it's gone. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Fri, 27 Jan 2012 17:54:14 -0500 Barry Warsaw wrote: > On Jan 27, 2012, at 10:48 PM, Antoine Pitrou wrote: > > >On Fri, 27 Jan 2012 16:10:51 -0500 > >Barry Warsaw wrote: > >> > >> I'm -1 on this as well. It just feels like the completely wrong way to > >> stabilize an API, and I think despite the caveats that are explicit in > >> __preview__, Python will just catch tons of grief from users and haters > >> about > >> API instability anyway, because from a practical standpoint, applications > >> written using __preview__ APIs *will* be less stable. > > > >Well, obviously __preview__ is not for the most conservative users. I > >think the name clearly conveys the idea that you are trying out > >something which is not in its definitive state, doesn't it? > > Maybe. I could quibble about the name, but let's not bikeshed on that > right now. The problem as I see it is that __preview__ will be very tempting > to use in production. In fact, its use case is almost predicated on that. > (We want you to use it so you can tell us if the API is good.) That's my opinion too. But using it in production doesn't mean you lose control on the code and its users. Perhaps you are used to a kind of production where the code gets disseminated all over the GNUniverse :) But for most people "production" means a single server or machine where they have entire control. > If you were writing an app > that used something in __preview__, how would you provide feedback on what > parts of the API you'd want to change, *and* how would you adapt your > application to use those better APIs once they became available 18 months from > now? For the former, the normal channels probably apply (bug tracker or python-dev). For the latter, depending on the API change, catching e.g. AttributeError on module lookup, or TypeError on function call, or explicitly examining the Python version are all plausible choices. Let's take another example: the regex module, where the API is unlikely to change much (since it's meant to be re-compatible), and the main concerns are ease of maintenance, data-wise compatibility with re (rather than API-wise), performance, and the like. > I think we'll just see folks using the unstable APIs and then > complaining when we remove them, even though they *know* *upfront* that these > APIs will go away. Hmm, isn't that a bit pessimistic about our users? > I'm also nervous about it from an OS vender point of view. Should I reject > any applications that import from __preview__? Or do I have to make a > commitment to support those APIs longer than Python does because the > application that uses it is important to me? Well, is the application supported upstream? If yes, then there shouldn't be any additional burden. If no, then you have a complication indeed. > A robust, standard approach to > versioning of modules would though, and I think would better solve what > __preview__ is trying to solve. I don't think versioning can replace API stability. __preview__ is explicitly and visibly special, and that's a protection against us becoming too complacent. > >On the other hand, __preview__ would clearly signal that something is > >on the verge of being frozen as an official stdlib API, and would > >prompt people to actively try it. > > I'm not so sure about that. If I were to actively try it, I'm not sure how > much motivation I'd have to rewrite key parts of my code when an incompatible > version gets promoted to the un__preview__d stdlib. Obviously you would only use a module from __preview__ if the functionality is exciting enough for you (or the cost/benefit ratio is good enough). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
On 1/27/2012 11:39 AM, mar...@v.loewis.de wrote: Another issue occurs to me: when a hash with colliding keys (one that has been attacked, and has trees) has a non-string key added, isn't the flattening process likely to have extremely poor performance? Correct. Thanks for the clarification. "Don't do that, then" I don't consider it mandatory to fix all issues with hash collision. In fact, none of the strategies fixes all issues with hash collisions; even the hash-randomization solutions only deal with string keys, and don't consider collisions on non-string keys. Which is fine, I just wanted the clarification. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
> I don't consider it mandatory to fix all issues with hash collision. > In fact, none of the strategies fixes all issues with hash collisions; > even the hash-randomization solutions only deal with string keys, and > don't consider collisions on non-string keys. How so? None of the patches did, but I think it was said several times that other types (int, tuple, float) could also be converted to use randomized hashes. What's more, there isn't any technical difficulty in doing so. And once you have randomized the hashes for these 4 or 5 built-in types, most third-party types follow since the common case of a __hash__ implementation is to call hash() on one or several constituents. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
Terry Reedy wrote: On 1/27/2012 2:54 PM, Guido van Rossum wrote: On Fri, Jan 27, 2012 at 9:08 AM, Ethan Furman wrote: Guido van Rossum wrote: Did you consider to just change the words so users can ignore it more easily? Yes, that has also been discussed. Speaking for myself, it would be only slightly better. Speaking for everyone that wants context suppression (using Steven D'Aprano's words): chained exceptions expose details to the caller that are irrelevant implementation details. Especially if the users are non-programmer app users. Or beginner programmers, e.g. on the python-list and tutor mailing lists. It is hard enough to get beginners to post the entire traceback without making them bigger. The typical newbie posts just the error message, sometimes not even the exception type. What they will make of chained exceptions, I hate to think. I find double tracebacks to be 'jarring'. If there is a double bug, one in both the try and except blocks, it *should* stand out. If there is just one bug and the developer merely wants to rename it and change the message, it should not. Agreed with all of this. [...] The PEP does not address the issue of whether the new variation of raise is valid outside of an except block. My memory is that it was not to be and I think it should not be. One advantage of the 'as' form is that it is clear that raising the default as something else is invalid if there is no default. I think that raise ... from None should be illegal outside an except block. My reasoning is: 1) It ensures that raise from None only occurs when the developer can see the old exception right there, and not "just in case". 2) I can't think of any use-cases for raise from None outside of an except block. 3) When in doubt, start with something more restrictive, because it is easier to loosen the restriction later if it turns out to be too much, than to change our mind and add the restriction afterwards. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
Terry Reedy wrote: The PEP does not address the issue of whether the new variation of raise is valid outside of an except block. My memory is that it was not to be and I think it should not be. One advantage of the 'as' form is that it is clear that raising the default as something else is invalid if there is no default. Were you speaking of the original (PEP 3134), or this new one (PEP 409)? Because at this point it is possible to do: raise ValueError from NameError outside a try block. I don't see it as incredibly useful, but I don't know that it's worth making it illegal. So the question is: - should 'raise ... from ...' be legal outside a try block? - should 'raise ... from None' be legal outside a try block? ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
How so? None of the patches did, but I think it was said several times that other types (int, tuple, float) could also be converted to use randomized hashes. What's more, there isn't any technical difficulty in doing so. The challenge again is about incompatibility: the more types you apply this to, the higher the risk of breaking third-party code. Plus you still risk that the hash seed might leak out of the application, opening it up again to the original attack. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Fri, Jan 27, 2012 at 11:48 PM, Matt Joiner wrote: > +0. I think the idea is right, and will help to get good quality > modules in at a faster rate. However it is compensating for a lack of > interface and packaging standardization in the 3rd party module world. No, it really isn't. virtualenv and pip already work *beautifully*, so long as you're in an environment where: 1. Due diligence isn't a problem 2. Network connectivity isn't a problem 3. You *already know* about virtual environments and the Python Package Index 4. You either don't need dependencies written in C, or the ones you need are written to compile cleanly under distutils and you aren't on Windows (because Microsoft consider building fully functional binaries from source to be an optional extra people should be charged for rather than a fundamental feature of an operating system) It would probably be worth adding a heading specifically countering this myth, though. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
n Sat, Jan 28, 2012 at 3:26 AM, Alex wrote: > I think a significantly healthier process (in terms of maximizing feedback and > getting something into it's best shape) is to let a project evolve naturally > on > PyPi and in the ecosystem, give feedback to it from an inclusion perspective, > and then include it when it becomes ready on it's own merits. The counter > argument to this is that putting it in the stdlib gets you signficantly more > eyeballs (and hopefully more feedback, therefore), my only response to this > is: > if it doesn't get eyeballs on PyPi I don't think there's a great enough need > to > justify it in the stdlib. And what about a project like regex, which *has* the eyeballs on PyPI, but the core devs aren't confident enough of its maintainability yet to be happy about adding it directly to the stdlib with full backwards compatibility guarantees? The easy answer for us in that context is to just not add it (i.e. the status quo), which isn't a healthy outcome for the overall language ecosystem. Really, regex is the *reason* this PEP exists: we *know* we need to either replace or seriously enhance "re" (since its Unicode handling isn't up to scratch), but we're only *pretty sure* adding "regex" to the stdlib is the right answer. Adding "__preview__.regex" instead gives us a chance to back out if we uncover serious problems (e.g. with the cross-platform support). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue13703] Hash collision security issue
On Sat, 28 Jan 2012 01:53:40 +0100 mar...@v.loewis.de wrote: > > > How so? None of the patches did, but I think it was said several times > > that other types (int, tuple, float) could also be converted to use > > randomized hashes. What's more, there isn't any technical difficulty in > > doing so. > > The challenge again is about incompatibility: the more types you apply this > to, the higher the risk of breaking third-party code. > > Plus you still risk that the hash seed might leak out of the application, > opening it up again to the original attack. Attacks on the hash seed are a different level of difficulty than sending a well-known universal payload to a Web site. Unless the application leaks hash() values directly, you have to guess them from the dict ordering observed in the application's output. IMHO it's ok if our hash function is vulnerable to cryptanalysts rather than script kiddies. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] plugging the hash attack
Hello everyone, In effort to get a fix out before Perl 6 goes mainstream, Barry and I have decided to pronounce on what we want for our stable releases. What we have decided is that 1. Simple hash randomization is the way to go. We think this has the best chance of actually fixing the problem while being fairly straightforward such that we're comfortable putting it in a stable release. 2. It will be off by default in stable releases and enabled by an envar at runtime. This will prevent code breakage from dictionary order changing as well as people depending on the hash stability. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 6:43 AM, Steven D'Aprano wrote:
> This PEP only makes sense if we assume that __preview__.spam and spam *will*
> be different, even if only in minor ways, and that there might not even be a
> spam. There should be no expectation that every __preview__ module must
> graduate, or that every standard library module must go through __preview__.
> If it is stable and uncontroversial, __preview__ adds nothing to the
> process.
Yes, the PEP already points to lzma as an example of a module with a
sufficiently obvious API that it didn't need to go through a preview
round.
> Keeping __preview__.spam around after graduation is, I believe, actively
> harmful. It adds complexity to the developer's decision-making process
> ("Should I import spam from __preview__, or just import spam? What's the
> difference?"). It gives a dangerous impression that code written for
> __preview__.spam will still work for spam.
Yes, this was exactly the reasoning behind removing the names from
__preview__ namespace when the modules graduated. It sets a line in
the sand: "An API compatibility break is not only allowed, it is 100%
guaranteed. If you are not prepared to deal with this, then you are
*not* part of the target audience for the __preview__ namespace. Wait
until the module reaches the main section of the standard library
before you start using it, or else download a third party supported
version with backwards compatibility guarantees from PyPI. The
__preview__ namespace is not designed for anything that requires long
term support spanning multiple Python version - it is intended for use
in single version environments, such as intranet web services and
student classrooms"
> I would go further and suggest that __preview__ be explicitly called
> __unstable__. If that name is scary, and it frightens some users off, good!
Hmm, the problem with "unstable" is that we only mean the *API* is
unstable. The software itself will be as thoroughly tested as
everything else we ship.
> I think the PEP needs more use-cases on who might use __preview__.spam, and
> why. These come to my mind:
>
> * if you don't care about Python 3.x+1, then there is no reason not to
> treat Python 3.x's __preview__.spam as stable;
>
> * rapid development proof-of-concept software ("build one to throw away")
> can safely use __preview__.spam, since they are expected to be replaced
> anyway;
>
> * one-use scripts;
>
> * use at the interactive interpreter;
>
> * any other time where forward-compatibility is not required.
A specific list of use cases is a good idea.
I'd add a couple more:
* in a student classroom where the concept of PyPI and third party
packages has yet to be introduced
* for an intranet web service deployment where due diligence adds
significant overhead to any use of third party packages
> I am reminded of the long, often acrimonious arguments that took place on
> Python-Dev a few years back about the API for the ipaddr library. A lot of
> the arguments could have been short-circuited if we had said "putting ipaddr
> into __preview__ does not constitute acceptance of its API".
Yep, there's a reason 'ipaddr' was high on the list of modules this
could be used for :)
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
Hi, On Tue, Nov 8, 2011 at 10:36, Benjamin Peterson wrote: > 2011/11/8 stefan brunthaler : >> How does that sound? > > I think I can hear real patches and benchmarks most clearly. > I spent the better part of my -20% time on implementing the work as "suggested". Please find the benchmarks attached to this email, I just did them on my system (i7-920, Linux 3.0.0-15, GCC 4.6.1). I branched off the regular 3.3a0 default tip changeset 73977 shortly after your email. I do not have an official patch yet, but am going to create one if wanted. Changes to the existing interpreter are minimal, the biggest chunk is a new interpreter dispatch loop. Merging dispatch loops eliminates some of my optimizations, but my inline caching technique enables inlining some functionality, which results in visible speedups. The code is normalized to the non-threaded-code version of the CPython interpreter (named "vanilla"), so that I can reference it to my preceding results. I anticipate *no* compatibility issues and the interpreter requires less than 100 KiB of extra memory at run-time. Since my interpreter is using 215 of a maximum of 255 instructions, there is room for adding additional derivatives, e.g., for popular Python libraries, too. Let me know what python-dev thinks of this and have a nice weekend, --stefan PS: AFAIR the version without partial stack frame caching also passes all regression tests modulo the ones that test against specific bytecodes. currently processing: bench/binarytrees.py3.py phd-cpy-3a0-thr-cod-pytho arg: 10 | time: 0.161876 | stdev: 0.007780 | var: 0.61 | mem: 6633.60 phd-cpy-3a0-thr-cod-pytho arg: 12 | time: 0.699243 | stdev: 0.019112 | var: 0.000365 | mem: 8142.67 phd-cpy-3a0-thr-cod-pytho arg: 14 | time: 3.388344 | stdev: 0.048042 | var: 0.002308 | mem: 13586.93 phd-cpy-pio-sne-pre-pyt-no-psf arg: 10 | time: 0.153875 | stdev: 0.003828 | var: 0.15 | mem: 6873.73 phd-cpy-pio-sne-pre-pyt-no-psf arg: 12 | time: 0.632572 | stdev: 0.019121 | var: 0.000366 | mem: 8246.27 phd-cpy-pio-sne-pre-pyt-no-psf arg: 14 | time: 3.020988 | stdev: 0.043483 | var: 0.001891 | mem: 13640.27 phd-cpy-pio-sne-pre-pytho arg: 10 | time: 0.150942 | stdev: 0.005157 | var: 0.27 | mem: 6901.87 phd-cpy-pio-sne-pre-pytho arg: 12 | time: 0.660841 | stdev: 0.020538 | var: 0.000422 | mem: 8286.80 phd-cpy-pio-sne-pre-pytho arg: 14 | time: 3.184198 | stdev: 0.051103 | var: 0.002612 | mem: 13680.40 phd-cpy-3a0-van-pytho arg: 10 | time: 0.202812 | stdev: 0.005480 | var: 0.30 | mem: 6633.33 phd-cpy-3a0-van-pytho arg: 12 | time: 0.908456 | stdev: 0.015744 | var: 0.000248 | mem: 8153.07 phd-cpy-3a0-van-pytho arg: 14 | time: 4.364805 | stdev: 0.037522 | var: 0.001408 | mem: 13593.60 ### phd-cpy-3a0-thr-cod-pytho : 1.2887 (avg-sum: 1.416488) ### phd-cpy-pio-sne-pre-pyt-no-psf: 1.4383 (avg-sum: 1.269145) ### phd-cpy-pio-sne-pre-pytho : 1.3704 (avg-sum: 1.331994) ### phd-cpy-3a0-van-pytho : 1. (avg-sum: 1.825358) currently processing: bench/fannkuch.py3.py phd-cpy-3a0-thr-cod-pytho arg: 8 | time: 0.172677 | stdev: 0.006620 | var: 0.44 | mem: 6424.13 phd-cpy-3a0-thr-cod-pytho arg: 9 | time: 1.426755 | stdev: 0.035545 | var: 0.001263 | mem: 6425.20 phd-cpy-pio-sne-pre-pyt-no-psf arg: 8 | time: 0.168010 | stdev: 0.010277 | var: 0.000106 | mem: 6481.07 phd-cpy-pio-sne-pre-pyt-no-psf arg: 9 | time: 1.345817 | stdev: 0.033127 | var: 0.001097 | mem: 6479.60 phd-cpy-pio-sne-pre-pytho arg: 8 | time: 0.165876 | stdev: 0.007136 | var: 0.51 | mem: 6520.00 phd-cpy-pio-sne-pre-pytho arg: 9 | time: 1.351150 | stdev: 0.028822 | var: 0.000831 | mem: 6519.73 phd-cpy-3a0-van-pytho arg: 8 | time: 0.216146 | stdev: 0.012879 | var: 0.000166 | mem: 6419.07 phd-cpy-3a0-van-pytho arg: 9 | time: 1.834247 | stdev: 0.028224 | var: 0.000797 | mem: 6418.67 ### phd-cpy-3a0-thr-cod-pytho : 1.2820 (avg-sum: 0.799716) ### phd-cpy-pio-sne-pre-pyt-no-psf: 1.3544 (avg-sum: 0.756913) ### phd-cpy-pio-sne-pre-pytho : 1.3516 (avg-sum: 0.758513) ### phd-cpy-3a0-van-pytho : 1. (avg-sum: 1.025197) currently processing: bench/fasta.py3.py phd-cpy-3a0-thr-cod-pytho arg: 5 | time: 0.374023 | stdev: 0.010870 | var: 0.000118 | mem: 6495.07 phd-cpy-3a0-thr-cod-pytho arg: 10 | time: 0.714577 | stdev: 0.024713 | var: 0.000611 | mem: 6495.47 phd-cpy-3a0-thr-cod-pytho arg: 15 | time: 1.062866 | stdev: 0.040138 | var: 0.001611 | mem: 6496.27 phd-cpy-pio-sne-pre-pyt-no-psf arg: 5 | time: 0.345621 | stdev: 0.022549 | var: 0.000508 | mem: 6551.87 phd-cpy-pio-sne-pre-pyt-no-psf arg: 10 | time: 0.656174 |
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
2012/1/27 stefan brunthaler : > Hi, > > On Tue, Nov 8, 2011 at 10:36, Benjamin Peterson wrote: >> 2011/11/8 stefan brunthaler : >>> How does that sound? >> >> I think I can hear real patches and benchmarks most clearly. >> > I spent the better part of my -20% time on implementing the work as > "suggested". Please find the benchmarks attached to this email, I just > did them on my system (i7-920, Linux 3.0.0-15, GCC 4.6.1). I branched > off the regular 3.3a0 default tip changeset 73977 shortly after your > email. I do not have an official patch yet, but am going to create one > if wanted. Changes to the existing interpreter are minimal, the > biggest chunk is a new interpreter dispatch loop. > > Merging dispatch loops eliminates some of my optimizations, but my > inline caching technique enables inlining some functionality, which > results in visible speedups. The code is normalized to the > non-threaded-code version of the CPython interpreter (named > "vanilla"), so that I can reference it to my preceding results. I > anticipate *no* compatibility issues and the interpreter requires less > than 100 KiB of extra memory at run-time. Since my interpreter is > using 215 of a maximum of 255 instructions, there is room for adding > additional derivatives, e.g., for popular Python libraries, too. > > > Let me know what python-dev thinks of this and have a nice weekend, Cool. It'd be nice to see a patch. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 8:54 AM, Barry Warsaw wrote: > I think the OS vendor problem is easier with an application that uses some > PyPI package, because I can always make that package available to the > application by pulling in the version I care about. It's harder if a newer, > incompatible version is released upstream and I want to provide both, but I > don't think __preview__ addresses that. A robust, standard approach to > versioning of modules would though, and I think would better solve what > __preview__ is trying to solve. I'd be A-OK with an explicit requirement that *any* module shipped in __preview__ must have a third-party supported multi-version compatible alternative on PyPI. (PEP 2 actually pretty much says that should be the case, but making it mandatory in the __preview__ case would be a good idea). As an OS vendor, you'd then be able to say: "Don't use __preview__, since that will cause problems when we next upgrade the system Python. Use the PyPI version instead." Then the stdlib docs for that module (while it is in __preview__) would say "If you are able to easily use third party packages, package offers this API for multiple Python versions with stronger API stability guarantees. This preview version of the module is for use in environments that specifically target a single Python version and/or where the use of third party packages outside the standard library poses additional complications beyond simply downloading and installing the code." Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 28, 2012, at 11:37 AM, Nick Coghlan wrote: >Then the stdlib docs for that module (while it is in __preview__) >would say "If you are able to easily use third party packages, package > offers this API for multiple Python versions with stronger API >stability guarantees. This preview version of the module is for use in >environments that specifically target a single Python version and/or >where the use of third party packages outside the standard library >poses additional complications beyond simply downloading and >installing the code." Would it be acceptable then for a distro to disable __preview__ or empty it out? The thinking goes like this: if you would normally use an __preview__ module because you can't get approval to download some random package from PyPI, well then your distro probably could or should provide it, so get it from them. In fact, if the number of __preview__ modules is kept low, *and* PyPI equivalents were a requirement, then a distro vendor could just ensure those PyPI versions are available as distro packages outside of the __preview__ stdlib namespace (i.e. in their normal third-party namespace). Then folks developing on that platform could just use the distro package and ignore __preview__. If that's acceptable, then maybe it should be explicitly so in the PEP. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
1. Simple hash randomization is the way to go. We think this has the best chance of actually fixing the problem while being fairly straightforward such that we're comfortable putting it in a stable release. 2. It will be off by default in stable releases and enabled by an envar at runtime. This will prevent code breakage from dictionary order changing as well as people depending on the hash stability. I think this is a good compromise given the widely varying assessments of the issue. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 28, 2012, at 11:27 AM, Nick Coghlan wrote: >* for an intranet web service deployment where due diligence adds >significant overhead to any use of third party packages Which really means that *we* are assuming the responsibility for this due diligence. And of course, we should not add anything to __preview__ without consent (and contributor agreement) of the upstream developers. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 28, 2012, at 11:13 AM, Nick Coghlan wrote: >Really, regex is the *reason* this PEP exists: we *know* we need to >either replace or seriously enhance "re" (since its Unicode handling >isn't up to scratch), but we're only *pretty sure* adding "regex" to >the stdlib is the right answer. Adding "__preview__.regex" instead >gives us a chance to back out if we uncover serious problems (e.g. >with the cross-platform support). I'd also feel much better about this PEP if we had specific ways to measure success. If, for example, regex were added to Python 3.3, but removed from 3.4 because we didn't get enough feedback about it, then I'd consider the approach put forward in this PEP to be a failure. Experiments that fail are *okay* of course, if they are viewed as experiments, there are clear metrics to measure their success, and we have the guts to end the experiment if it doesn't work out. Of course, if it's a resounding success, then that's fantastic too. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 20:43, Steven D'Aprano wrote:
Eli Bendersky wrote:
Hello,
Following an earlier discussion on python-ideas [1], we would like to
propose the following PEP for review. Discussion is welcome.
I think you need to emphasize that modules in __preview__ are NOT
expected to have a forward-compatible, stable, API. This is a feature
of __preview__, not a bug, and I believe it is the most important
feature.
I see responses to this PEP that assume that APIs will be stable,
I didn't see responses like that - the *point* of this pep is to allow
an api we think *should* be in the standard library stabilise and mature
(that's how I see it anyway). There is a difference between "not yet
stable" and "we will make huge gratuitous changes" though. We *might*
make huge gratuitous changes, but only if they're really needed (meaning
they're huge but not gratuitous I guess).
and that having a module fail to graduate out of __preview__ should be
an extraordinary event.
I would say this will probably be the case. Once we add something there
will be resistance to removing it and we shouldn't let things rot in
__preview__ either. I would say failing to graduate would be the
exception, although maybe not extraordinary.
But if this is the case, then why bother with __preview__? It just
adds complexity to the process -- if __preview__.spam and spam are
expected to be the same, then just spam straight into the std lib and
be done with it.
I think you're misunderstanding what was suggested. The suggestion was
that once spam has graduated from __preview__ into stdlib, that
__preview__.spam should remain as an alias - so that code using it from
__preview__ at least has a fighting chance of working.
This PEP only makes sense if we assume that __preview__.spam and spam
*will* be different,
I disagree. Once there is a spam they should remain the same.
__preview__ is for packages that haven't yet made it into the standard
library - not a place for experimenting with apis that are already there.
even if only in minor ways, and that there might not even be a spam.
There should be no expectation that every __preview__ module must
graduate,
Graduate or die however.
or that every standard library module must go through __preview__. If
it is stable and uncontroversial, __preview__ adds nothing to the
process.
Sure. __preview__ is for things that *need* previewing.
All the best,
Michael Foord
Even when there are candidates for inclusion with relatively stable
APIs, like regex, we should *assume* that there will be API
differences between __preview__.regex and regex, simply because it is
less harmful to expect changes that don't eventuate than to expect
stability and be surprised by changes.
This, I believe, rules out Antoine's suggestion that modules remain
importable from __preview__ even after graduation to a full member of
the standard library. We simply can't say have all three of these
statements true at the same time:
1) regular standard library modules are expected to be backward
compatible
2) __preview__ modules are not expected to be forward compatible
3) __preview__.spam is an alias to regular standard library spam
At least one of them has to go. Since both 1) and 2) are powerful
features, and 3) is only a convenience, the obvious one to drop is 3).
I note that the PEP, as it is currently written, explicitly states
that __preview__.spam will be dropped when it graduates to spam. This
is a good thing and should not be changed.
Keeping __preview__.spam around after graduation is, I believe,
actively harmful. It adds complexity to the developer's
decision-making process ("Should I import spam from __preview__, or
just import spam? What's the difference?"). It gives a dangerous
impression that code written for __preview__.spam will still work for
spam.
We should be discouraging simple-minded recipes like
try:
import spam
except ImportError:
from __preview__ import spam
spam.foo(a, b, c)
since they undermine the vital feature of __preview__ that the
signature and even the existence of spam.foo is subject to change.
I would go further and suggest that __preview__ be explicitly called
__unstable__. If that name is scary, and it frightens some users off,
good! The last thing we want is when 3.4 comes around to have dozens
of bug reports along the line of "spam.foo() and
__preview__.spam.foo() have different function signatures and aren't
compatible". Of course they do. That's why __preview__.spam existed in
the first place, to allow the API to mature without the expectation
that it was already stable.
Since __preview__.spam (or, as I would prefer, __unstable__.spam) and
spam cannot be treated as drop-in replacements, what is
__preview__.spam good for? Without a stable API, __preview__.spam is
not suitable for use in production applications that expect to run
under multiple versions of the standard library.
I think the PEP
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 20:48, Steven D'Aprano wrote: Eli Bendersky wrote: try: from __preview__ import thing except ImportError: import thing So no need to target a very specific version of Python. Yep, this is what I had in mind. And it appeared too trivial to place it in the PEP. Trivial and wrong. Since thing and __preview__.thing may have subtle, or major, API differences, how do you use it? No, potentially wrong in cases where the APIs are different. Even with the try...except ImportError dance around StringIO / cStringIO there are some API differences. But for a lot of use cases it works fine (simplejson and json aren't *identical*, but it works for most people). Michael try: result = thing.foo(a, b, c) + thing.bar(x) except AttributeError: # Must be the preview version result = thing.foobar(a, c, b, x) -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
Benjamin Peterson wrote: Hello everyone, In effort to get a fix out before Perl 6 goes mainstream, Barry and I have decided to pronounce on what we want for our stable releases. What we have decided is that 1. Simple hash randomization is the way to go. We think this has the best chance of actually fixing the problem while being fairly straightforward such that we're comfortable putting it in a stable release. 2. It will be off by default in stable releases and enabled by an envar at runtime. This will prevent code breakage from dictionary order changing as well as people depending on the hash stability. Do you have the expectation that it will become on by default in some future release? -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
2012/1/27 Steven D'Aprano : > Benjamin Peterson wrote: >> >> Hello everyone, >> In effort to get a fix out before Perl 6 goes mainstream, Barry and I >> have decided to pronounce on what we want for our stable releases. >> What we have decided is that >> 1. Simple hash randomization is the way to go. We think this has the >> best chance of actually fixing the problem while being fairly >> straightforward such that we're comfortable putting it in a stable >> release. >> 2. It will be off by default in stable releases and enabled by an >> envar at runtime. This will prevent code breakage from dictionary >> order changing as well as people depending on the hash stability. > > > Do you have the expectation that it will become on by default in some future > release? Yes, 3.3. The solution in 3.3 could even be one of the more sophisticated proposals we have today. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
On Fri, Jan 27, 2012 at 5:19 PM, Benjamin Peterson wrote: > Hello everyone, > In effort to get a fix out before Perl 6 goes mainstream, Barry and I > have decided to pronounce on what we want for our stable releases. > What we have decided is that > 1. Simple hash randomization is the way to go. We think this has the > best chance of actually fixing the problem while being fairly > straightforward such that we're comfortable putting it in a stable > release. > 2. It will be off by default in stable releases and enabled by an > envar at runtime. This will prevent code breakage from dictionary > order changing as well as people depending on the hash stability. Okay, good call! -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Michael Foord wrote: On 27/01/2012 20:48, Steven D'Aprano wrote: Eli Bendersky wrote: try: from __preview__ import thing except ImportError: import thing So no need to target a very specific version of Python. Yep, this is what I had in mind. And it appeared too trivial to place it in the PEP. Trivial and wrong. Since thing and __preview__.thing may have subtle, or major, API differences, how do you use it? No, potentially wrong in cases where the APIs are different. Even with the try...except ImportError dance around StringIO / cStringIO there are some API differences. But for a lot of use cases it works fine (simplejson and json aren't *identical*, but it works for most people). Okay, granted, I accept your point. But I think we need to distinguish between these cases. In the case of StringIO and cStringIO, API compatibility is expected, and differences are either bugs or implementation differences that you shouldn't be relying on. In the case of the typical[1] __preview__ module, one of the motivations of adding it to __preview__ is to test the existing API. We should expect changes, even if in practice often there won't be. We might hope for no API changes, but we should plan for the case where there will be. And that rules out the "try import" dance for the typical __preview__ module. There may be modules which graduate and keep the same API. In those cases, people will quickly work out the import dance on their own, it's a very common idiom. But we shouldn't advertise it as the right way to deal with __preview__, since that implies the expectation of API stability, and we want to send the opposite message: __preview__ is the last time the API can change without a big song and dance, so be prepared for it to change. I'm with Nick on this one: if you're not prepared to change "from __preview__ import module" to "import module" in your app, then you certainly won't be prepared to deal with the potential API changes and you aren't the target audience for __preview__. [1] I am fully aware of the folly of referring to a "typical" example of something that doesn't exist yet -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
>> No, potentially wrong in cases where the APIs are different. Even with the >> try...except ImportError dance around StringIO / cStringIO there are some >> API differences. But for a lot of use cases it works fine (simplejson and >> json aren't *identical*, but it works for most people). > > > > Okay, granted, I accept your point. > > But I think we need to distinguish between these cases. > > In the case of StringIO and cStringIO, API compatibility is expected, and > differences are either bugs or implementation differences that you shouldn't > be relying on. > I just recently ran into a compatibility of StringIO and cStringIO. It's a good thing it's documented: "Another difference from the StringIO module is that calling StringIO() with a string parameter creates a read-only object. Unlike an object created without a string parameter, it does not have write methods. These objects are not generally visible. They turn up in tracebacks as StringI and StringO." But it did cause me a couple of minutes of grief until I found this piece in the docs and wrote a work-around. But no, even in the current stable stdlib, the "try import ... except import from elsewhere" trick doesn't "just work" for StringIO/cStringIO. And as far as I can understand this is documented, not a bug or some obscure implementation detail. My point is that if our users accept *this*, in the stable stdlib, I see no reason they won't accept the same happening between __preview__ and a graduated module, when they (hopefully) understand the intention of __preview__. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Michael Foord writes: > >> Assuming the module is then promoted to the the standard library proper in > >> release ``3.X+1``, it will be moved to a permanent location in the > >> library:: > >> > >> import example > >> > >> And importing it from ``__preview__`` will no longer work. > > Why not leave it accessible through __preview__ too? > > +1 Er, doesn't this contradict your point about using try: from __preview__ import spam except ImportError: import spam ? I think it's a bad idea to introduce a feature that's *supposed* to break (in the sense of "make a break", ie, change the normal pattern) with every release and then try to avoid breaking (in the sense of "causing an unexpected failure") code written by people who don't want to follow the discipline of keeping up with changing APIs. If they want that stability, they should wait for the stable release. Modules should become unavailable from __preview__ as soon as they have a stable home. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Executive summary: If the promise to remove the module from __preview__ is credible (ie, strictly kept), then __preview__ will have a specific audience in those who want the stdlib candidate code and are willing to deal with a certain amount of instability in that code. (Whether that audience is big enough to be worth the effort of managing __preview__ is another question.) Barry Warsaw writes: > >> I agree with everything Alex said here. I don't necessarily disagree. But: > I can't argue with that, it's just that I don't think __preview__ > solves [the visibility] problem. I do disagree with that. I frequently refer to the library reference for modules that do what I need, but almost never to PyPI (my own needs are usually not very hard to program, but if there's a stdlib module it's almost surely far more general, robust, and tested than my special-case code would be; PyPI provides far less of a robustness guarantee than a stdlib candidate would). I don't know how big or important a use case this is, though I think that Antoine's point that a similar argument applies to those who develop software for their own internal use (like me, but they have actual standards for QA) is valid. > I think we'll just see folks using the unstable APIs and then > complaining when we remove them, even though they *know* *upfront* > that these APIs will go away. So maybe the Hon. Mr. Broytman would be willing to supply a form letter for those folks, too. "We promised to remove the module from __preview__, and we did. We warned you the API would be likely unstable, and it was. You have no complaint." would be the gist. > A robust, standard approach to versioning of modules would though, > and I think would better solve what __preview__ is trying to solve. I suspect that "robust, standard approach to versioning of modules" is an oxymoron. The semantics of "module version" from the point of view of application developers and users is very complex, and cannot be encapsulated in a linear sequence. The only reliable comparison that can be done on versions is equality (and Python knows that; that's why there is a stdlib bound to the core in the first place!) > I'm not so sure about that. If I were to actively try it, I'm not > sure how much motivation I'd have to rewrite key parts of my code > when an incompatible version gets promoted to the un__preview__d > stdlib. So use the old version of Python. You do that anyway. Or avoid APIs where you are unwilling to deal with more or less frequent changes. You do that anyway. And if you're motivated enough, use __preview__. I don't understand what you think you lose here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 1/27/2012 8:48 PM, Barry Warsaw wrote: > The thinking goes like this: if you would normally use an __preview__ module > because you can't get approval to download some random package from PyPI, well > then your distro probably could or should provide it, so get it from them. That is my thought about the entire __preview__ concept. Anything that would/should go into __preview__ would be better off being packaged for a couple of key distros (e.g., Ubuntu/Fedora/Gentoo) where they would get better visibility than just being on PyPI and would be more flexible in terms of release schedule to allow API changes. If the effort being put into making the __preview__ package was put into packaging those modules for distros, then you would get the same exposure with better flexibility and a better maintenance story. The whole idea of __preview__ seems to be a workaround for the difficult packaging story for Python modules on common distros -- stuffing them into __preview__ is a cheat to get the distro packagers to distribute these interesting modules since we would be bundling them. However, as you have pointed out, it would very desirable to them to not do so. So in the end, these modules may not receive as wide of visibility as the PEP suggests. I could very easily imagine the more stable distributions refusing or patching anything that used __preview__ in order to eliminate difficulties. -- Scott Dial sc...@scottdial.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Eli Bendersky writes: > My point is that if our users accept *this*, in the stable stdlib, I > see no reason they won't accept the same happening between __preview__ > and a graduated module, when they (hopefully) understand the intention > of __preview__. If it doesn't happen with sufficiently high frequency and annoyance factors to make attempting to use both the __preview__ and graduated versions in the same code base unacceptable to most users, then __preview__ is unnecessary, and the PEP should be rejected. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 07:41, Stephen J. Turnbull wrote: > Eli Bendersky writes: > > > My point is that if our users accept *this*, in the stable stdlib, I > > see no reason they won't accept the same happening between __preview__ > > and a graduated module, when they (hopefully) understand the intention > > of __preview__. > > If it doesn't happen with sufficiently high frequency and annoyance > factors to make attempting to use both the __preview__ and graduated > versions in the same code base unacceptable to most users, then > __preview__ is unnecessary, and the PEP should be rejected. API differences such as changing one method to another (perhaps repeated over several methods) is unacceptable for stdlib modules. On the other hand, for a determined user importing from either __preview__ or the graduated version it's only a matter of a few lines in a conditional import. IMHO this is much preferable to having the module either external or in the stdlib, because that imposes another external dependency. But I think that the issue of keeping __preview__ in a later release is just an "implementation detail" of the PEP and shouldn't be seen as its main decision point. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
On Sat, Jan 28, 2012 at 10:33 AM, Ethan Furman wrote: > Because at this point it is possible to do: > > raise ValueError from NameError > > outside a try block. I don't see it as incredibly useful, but I don't know > that it's worth making it illegal. > > So the question is: > > - should 'raise ... from ...' be legal outside a try block? > > - should 'raise ... from None' be legal outside a try block? Given that it would be quite a bit of work to make it illegal, my preference is to leave it alone. I believe that means there's only one open question. Should "raise ex from None" be syntactic sugar for: 1. clearing the current thread's exception state (as I believe Ethan's patch currently does), thus meaning that __context__ and __cause__ both end up being None 2. setting __cause__ to None (so that __context__ still gets set normally, as it is now when __cause__ is set to a specific exception), and having __cause__ default to a *new* sentinel object that indicates "use __context__" I've already stated my own preference in favour of 2 - that approach means developers that think about it can explicitly change exception types such that the context isn't displayed by default, but application and framework developers remain free to insert their own exception handlers that *always* report the full exception stack. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 11:48 AM, Barry Warsaw wrote: > Would it be acceptable then for a distro to disable __preview__ or empty it > out? > > The thinking goes like this: if you would normally use an __preview__ module > because you can't get approval to download some random package from PyPI, well > then your distro probably could or should provide it, so get it from them. In > fact, if the number of __preview__ modules is kept low, *and* PyPI equivalents > were a requirement, then a distro vendor could just ensure those PyPI versions > are available as distro packages outside of the __preview__ stdlib namespace > (i.e. in their normal third-party namespace). Then folks developing on that > platform could just use the distro package and ignore __preview__. > > If that's acceptable, then maybe it should be explicitly so in the PEP. I think that's an excellent idea - in that case, the distro vendor is taking over the due diligence responsibilities, which are the main point of __preview__. Similarly, sumo distributions like ActiveState or Python(x, y) could choose to add the PyPI version. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 3:22 PM, Stephen J. Turnbull wrote: > Executive summary: > > If the promise to remove the module from __preview__ is credible (ie, > strictly kept), then __preview__ will have a specific audience in > those who want the stdlib candidate code and are willing to deal with > a certain amount of instability in that code. People need to remember there's another half to this equation: the core dev side. The reason *regex* specifically isn't in the stdlib already is largely due to (perhaps excessive) concerns about the potential maintenance burden. It's not a small chunk of code and we don't want to deal with another bsddb. That's the main roadblock to inclusion. Not lack of user demand. Not blindness to the problems with re. Just concerns about maintainability. Add to that some niggling concerns about backwards compatibility in obscure corner cases that may not be exercised by current users. And so we have an impasse. Matthew has indicated he's happy to include it and maintain it as part of the core, but it hasn't really gone anywhere because we don't currently have a good way to address those maintainability concerns (aside from saying "you're worrying about it too much", which isn't what I would call persuasive). That's what __preview__ gives us: a way to deal with the *negative* votes that keep positive additions out of the standard library. Most of the PEP's arguments for due diligence etc are actually talking about why we want things in the standard library in the first place, rather than about __preview__ in particular. The core idea behind the __preview__ namespace is to allow *3* possible responses when a module is proposed for stdlib inclusion: 1. Yes, that's a good idea, we'll add it (cf. lzma for 3.3) 2. Maybe, so we'll add it to __preview__ for a release and see if it blows up in our face (hopefully at least regex for 3.3, maybe ipaddr and daemon as well) 3. No, not going to happen. Currently, anything where we would answer "2" ends up becoming a "3" by default, and that's not a good thing for the long-term health of the language. The reason this will be more effective in building core developer confidence than third party distribution via PyPI is due to a few different things: - we all run the test suite, so we get to see that the software builds and tests effectively - we know what our own buildbots cover, so we know it's passing on all those platforms - we'll get to see more of the related discussions in channels we monitor *anyway* (i.e. the bug tracker, python-dev) As far as the criteria for failing to graduate goes, I'd say something that ends up in __preview__ will almost always make it into the main part of the standard library, with the following exceptions: - excessive build process, test suite and buildbot instability. Whether this is due to fragile test cases or fragile code, we don't want to deal with another bsddb. If the test suite can't be stabilised over the course of an entire feature release, then the module would most likely be rejected rather than allowing it to graduate to the standard library. - strongly negative (or just plain confused) user feedback. We deal with feedback on APIs all the time. Sometimes we add new ones, or tweak the existing ones. Occasionally we'll judge them to be irredeemably broken and just plain remove them (cf. CObject, contextlib.nested, Bastion, rexec). This wouldn't change just because a module was in __preview__ - instead, we'd just have another option available to us (i.e. rejecting the module for stdlib inclusion post-preview rather than trying to fix it). Really, the main benefit for end users doesn't lie in __preview__ itself: it lies in the positive effect __preview__ will have on the long term evolution of the standard library, as it aims to turn python-dev's inherent conservatism (which is a good thing!) into a speed bump rather than a road block. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 4:37 PM, Nick Coghlan wrote: > I think that's an excellent idea - in that case, the distro vendor is > taking over the due diligence responsibilities, which are the main > point of __preview__. Heh, contradicted myself in my next email. python-dev handling due diligence is a key benefit for *stdlib inclusion*, not __preview__ per se. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Fri, Jan 27, 2012 at 12:26 PM, Alex wrote: > I think a significantly healthier process (in terms of maximizing feedback and > getting something into it's best shape) is to let a project evolve naturally > on > PyPi and in the ecosystem, give feedback to it from an inclusion perspective, > and then include it when it becomes ready on it's own merits. The counter > argument to this is that putting it in the stdlib gets you signficantly more > eyeballs (and hopefully more feedback, therefore), my only response to this > is: > if it doesn't get eyeballs on PyPi I don't think there's a great enough need > to > justify it in the stdlib. Strongly agree. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
FWIW I'm now -1 for this idea. Stronger integration with PyPI and packaging systems is much preferable. Python core public releases are no place for testing. On Sat, Jan 28, 2012 at 2:42 AM, Matt Joiner wrote: > On Fri, Jan 27, 2012 at 12:26 PM, Alex wrote: >> I think a significantly healthier process (in terms of maximizing feedback >> and >> getting something into it's best shape) is to let a project evolve naturally >> on >> PyPi and in the ecosystem, give feedback to it from an inclusion perspective, >> and then include it when it becomes ready on it's own merits. The counter >> argument to this is that putting it in the stdlib gets you signficantly more >> eyeballs (and hopefully more feedback, therefore), my only response to this >> is: >> if it doesn't get eyeballs on PyPi I don't think there's a great enough need >> to >> justify it in the stdlib. > > Strongly agree. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
On Jan 26, 2012, at 7:19 PM, Ethan Furman wrote: > One of the open issues from PEP 3134 is suppressing context: currently there > is no way to do it. This PEP proposes one. Thanks for proposing fixes to this issue. It is an annoying problem. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
