Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
> On 24 Nov 2015, at 12:53, Christian Heimes wrote: > Right, with Antoine and Alex out of scope and you, Victor and me working > for Red Hat, the air is getting thin. Benjamin is familiar with the ssl > module. Or we can follow Alex's advice and ask somebody from the PyCA > group (Donald, Paul, lvh) or requests (Cory) to get some outside > perspective. I’m not a CPython core developer, so I think that excludes me. For what it’s worth, I think that this is a good PEP, and I’m strongly in favour of it being approved in some form. signature.asc Description: Message signed with OpenPGP using GPGMail ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
There's a lot to process in this thread, but as I see it, the issue breaks down to these questions: * How should PEP 493 be implemented? * What should the default be? * How should PEP 493 be worded to express the right tone to redistributors? Let me take on the implementation details here. On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote: >I would still find having built-in support for the recommendations >in the Python stdlib a better approach As would I. Currently the PEP contains some sample code for reading a configuration file and an environment variable, and assigning the default behavior based on these values. The first improvement to the stdlib I would have made would have been to add a convenience function to ssl to enable and disable the verification. Something like this: def ssl.verify_certificates(enable) Then at the simplest level, an application that wants to enable higher security could do this early in its main entry point: import ssl if hasattr(ssl, 'verify_certificates'): ssl.verify_certificates(True) or ... ssl.verify_certificates(False) and that would work regardless of the platform default, and it would work properly on older versions of Python 2.7 that didn't have the backport. This seems so straightforward, I'm wondering why PEP 493 doesn't consider it. I would like to see the sample code in PEP 493 implemented in the stdlib. It would then be a matter of cherry picking that should a distro decide to backport it. It means one less non-trivial delta from upstream which has to be managed separately. The only difference to the combined approach I'd make is to handle the case where the _cert_verification_config file doesn't exist. (If you included the convenience method above, then the sample code could be simplified down to a default for the `enable` parameter and a call to ssl.verify_certificates().) Is it too late to include both of those suggestions in PEP 493 and Python 2.7.11? Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On 26 November 2015 at 08:57, Barry Warsaw wrote: > There's a lot to process in this thread, but as I see it, the issue breaks > down to these questions: > > * How should PEP 493 be implemented? > > * What should the default be? > > * How should PEP 493 be worded to express the right tone to redistributors? > > Let me take on the implementation details here. > > On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote: > >>I would still find having built-in support for the recommendations >>in the Python stdlib a better approach > > As would I. For what its worth: a PEP telling distributors to patch the standard library is really distasteful to me. We've spent a long time trying to build close relations such that when something doesn't work distributors can share their needs with us and we can make Python out of the box be a good fit. This seems to fly in the exact opposite direction: we're explicitly making it so that Python builds on these vendor's platforms will not be the same as you get by checking out the Python source code. Ugh. -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On Thu, 26 Nov 2015 09:17:02 +1300, Robert Collins wrote: > On 26 November 2015 at 08:57, Barry Warsaw wrote: > > There's a lot to process in this thread, but as I see it, the issue breaks > > down to these questions: > > > > * How should PEP 493 be implemented? > > > > * What should the default be? > > > > * How should PEP 493 be worded to express the right tone to redistributors? > > > > Let me take on the implementation details here. > > > > On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote: > > > >>I would still find having built-in support for the recommendations > >>in the Python stdlib a better approach > > > > As would I. > > For what its worth: a PEP telling distributors to patch the standard > library is really distasteful to me. > > We've spent a long time trying to build close relations such that when > something doesn't work distributors can share their needs with us and > we can make Python out of the box be a good fit. This seems to fly in > the exact opposite direction: we're explicitly making it so that > Python builds on these vendor's platforms will not be the same as you > get by checking out the Python source code. I think we should include the environment variable support in CPython and be done with it (nuke the PEP otherwise). Which is what I've thought from the beginning :) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Help with a book
Hello Python Developers! I'm writing a book about descriptors, and I'm hoping to make it a comprehensive guide, covering just about everything there is to know about descriptors, including many tips to get past certain knowledge barriers and how to avoid certain pitfalls. I'm hoping for it to become the definitive guide for those who want to really understand descriptors. Raymond's Descriptor HowTo Guide is great, but focused. I want to expand on what that guide teaches. Anyway, I am currently reading through my initial rough draft and making corrections for a revised draft. Once I'm done with that, I'd like a few volunteers to read through it. First, to find spelling, grammatical, or consistency mistakes, though those should be few in number, but also to give input as to whether something seems too repetitive, if some more information should be given in a spot, and if I'm even blatantly wrong about anything (hopefully not). Anyone who helps will be able to receive a free ebook of the final copy as well as mention in the Special Thanks. I appreciate any help anyone can give to assist in making this book as great as it can be. Thanks, Jacob Zimmerman Twitter: @jacobz_20 Blog: http://programmingideaswithjake.wordpress.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
In a message of Wed, 25 Nov 2015 15:39:54 -0500, "R. David Murray" writes: >I think we should include the environment variable support in CPython >and be done with it (nuke the PEP otherwise). Which is what I've >thought from the beginning :) > >--David I like this idea too. Laura ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On Nov 24, 2015, at 09:16 AM, Toshio Kuratomi wrote: >It sounds like the implementation sections of the PEP are acceptable Well, see my follow up to MAL's email. From Rob's and RDM's responses, it seems we're not alone. :) I like all your other proposed changes, although I'll note the preference to minimize the unique-to-downstream changes. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On Nov 24, 2015, at 02:12 PM, Nick Coghlan wrote: >On 24 November 2015 at 11:59, Barry Warsaw wrote: >> We'd have to also handle >> migration paths to newer Ubuntu releases, which probably means removing the >> config file on future upgrades. > >You'll need to figure out those migration paths anyway, as the certificate >verification compatibility break will exist between 14.04 and 16.04 >regardless of what happens with PEP 493. Sure, but that's a very different situation. If someone upgraded from 14.04.3 to 14.04.4, they definitely wouldn't expect it to break working code. If it did, it would be a pretty serious regression. Upgrading to LTS maintenance releases should be a no-brainer. Upgrading from one LTS to a newer one, you *know* things are going to change, so you'll review the release notes and do some very careful analysis of your mission critical code. Cheers, -Barry pgpMgKEH4ygoN.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On Nov 25, 2015, at 03:39 PM, R. David Murray wrote: >I think we should include the environment variable support in CPython >and be done with it (nuke the PEP otherwise). Which is what I've >thought from the beginning :) +1. I'd like to see my proposed convenience function added too. I'm -0 on the config file though since while I think it might be useful, I'm not sure we'd actually install that on Ubuntu. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help with a book
As someone who plays around a *lot* with descriptors, I would be interested in reviewing it. I may not have the same knowledge as a core developer, but I can still help. Date: Wed, 25 Nov 2015 14:31:09 -0600 From: sad2proj...@gmail.com To: python-dev@python.org Subject: [Python-Dev] Help with a book Hello Python Developers! I'm writing a book about descriptors, and I'm hoping to make it a comprehensive guide, covering just about everything there is to know about descriptors, including many tips to get past certain knowledge barriers and how to avoid certain pitfalls. I'm hoping for it to become the definitive guide for those who want to really understand descriptors. Raymond's Descriptor HowTo Guide is great, but focused. I want to expand on what that guide teaches. Anyway, I am currently reading through my initial rough draft and making corrections for a revised draft. Once I'm done with that, I'd like a few volunteers to read through it. First, to find spelling, grammatical, or consistency mistakes, though those should be few in number, but also to give input as to whether something seems too repetitive, if some more information should be given in a spot, and if I'm even blatantly wrong about anything (hopefully not). Anyone who helps will be able to receive a free ebook of the final copy as well as mention in the Special Thanks. I appreciate any help anyone can give to assist in making this book as great as it can be. Thanks,Jacob Zimmerman Twitter: @jacobz_20Blog: http://programmingideaswithjake.wordpress.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help with a book
On Thu, Nov 26, 2015 at 7:31 AM, Jacob Zimmerman wrote: > Anyway, I am currently reading through my initial rough draft and making > corrections for a revised draft. Once I'm done with that, I'd like a few > volunteers to read through it. First, to find spelling, grammatical, or > consistency mistakes, though those should be few in number, but also to give > input as to whether something seems too repetitive, if some more information > should be given in a spot, and if I'm even blatantly wrong about anything > (hopefully not). I'd be happy to help out. Wordsmithing is my life, and I've used descriptors some (not as much as Emanuel will have, though!). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On 26 November 2015 at 05:57, Barry Warsaw wrote: > There's a lot to process in this thread, but as I see it, the issue breaks > down to these questions: > > * How should PEP 493 be implemented? > > * What should the default be? > > * How should PEP 493 be worded to express the right tone to redistributors? > > Let me take on the implementation details here. > > On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote: > >>I would still find having built-in support for the recommendations >>in the Python stdlib a better approach > > As would I. > > Currently the PEP contains some sample code for reading a configuration file > and an environment variable, and assigning the default behavior based on these > values. > > The first improvement to the stdlib I would have made would have been to add > a convenience function to ssl to enable and disable the verification. > Something like this: > > def ssl.verify_certificates(enable) > > Then at the simplest level, an application that wants to enable higher > security could do this early in its main entry point: > > import ssl > if hasattr(ssl, 'verify_certificates'): > ssl.verify_certificates(True) > > or > > ... > ssl.verify_certificates(False) > > and that would work regardless of the platform default, and it would work > properly on older versions of Python 2.7 that didn't have the backport. This > seems so straightforward, I'm wondering why PEP 493 doesn't consider it. PEP 476 rejected providing a public indefinitely maintained API for this, so PEP 493 is specifically about helping commercial redistributors offer a smoother transition plan to customers without affecting the public Python level API, and without encouraging a plethora of mutually incompatible transition schemes. > I would like to see the sample code in PEP 493 implemented in the stdlib. It > would then be a matter of cherry picking that should a distro decide to > backport it. It means one less non-trivial delta from upstream which has to > be managed separately. The only difference to the combined approach I'd make > is to handle the case where the _cert_verification_config file doesn't exist. > > (If you included the convenience method above, then the sample code could be > simplified down to a default for the `enable` parameter and a call to > ssl.verify_certificates().) > > Is it too late to include both of those suggestions in PEP 493 and Python > 2.7.11? PEP 493 isn't about attempting to rehash the PEP 476 discussions in search of a different conclusion, so this would need to be a different PEP, preferably one that targets Python 3.6 first and covers more than just HTTPS. Such an API could then be considered as a backport candidate for 2.7.12. (2.7.11rc1 has already been published, so there isn't any scope for updates there). I actually think that's a nicer approach, but also think the recommendations in PEP 493 are much simpler to implement and test, while still meeting the bar of "good enough to solve the problem at hand". If anyone does decide to write that PEP, then the fact that SELinux offers "setenforce 0" and AppArmor can be turned off with a kernel boot option can be used as examples of platform developers letting end users make their own security decisions, even if the platform provider recommends leaving the more opinionated default behaviour enabled. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)
On 26 November 2015 at 06:17, Robert Collins wrote: > On 26 November 2015 at 08:57, Barry Warsaw wrote: >> There's a lot to process in this thread, but as I see it, the issue breaks >> down to these questions: >> >> * How should PEP 493 be implemented? >> >> * What should the default be? >> >> * How should PEP 493 be worded to express the right tone to redistributors? >> >> Let me take on the implementation details here. >> >> On Nov 24, 2015, at 04:04 PM, M.-A. Lemburg wrote: >> >>>I would still find having built-in support for the recommendations >>>in the Python stdlib a better approach >> >> As would I. > > For what its worth: a PEP telling distributors to patch the standard > library is really distasteful to me. I'm not a big fan of it either, but it's the way sustainable commercial open source distribution works in practice: customers want selective upgrades, so customers get selective upgrades; customers want greater platform integration, so customers get greater platform integration; etc, etc. Large Python shops like Google will typically maintain their own custom Python variants with their own patches applied that are suitable and sufficient for their use cases, but may not be suitable for pushing upstream (e.g. they may be Linux specific patches, without support for other platforms). That's why I've focused on providing guidance to redistributors to keep their patches within reasonably consistent bounds for PEPs like bundling pip, backporting the network security enhancements to Python 2.7, and changing the default behaviour for HTTPS verification. Different redistributors have different customer bases, which also differ from the audience for upstream python.org releases, so saying "don't patch Python" is denying the reality of the obligations commercial vendors have to their customers, while "when you patch Python, please ensure you abide by these guidelines" is something redistributors can realistically do. > We've spent a long time trying to build close relations such that when > something doesn't work distributors can share their needs with us and > we can make Python out of the box be a good fit. This seems to fly in > the exact opposite direction: we're explicitly making it so that > Python builds on these vendor's platforms will not be the same as you > get by checking out the Python source code. This has always been true for the RHEL system Python - it gets backported changes, just like the kernel does (albeit nowhere near as many). That's one of the main things long term support customers are paying for - developers continuing to care about a code base after the volunteers working on the project upstream have (entirely reasonably) shifted their attentions to newer versions. We try to keep the patches to a minimum, and avoid having to carry large patches indefinitely by only backporting changes that have already been accepted in later versions, but "never apply patches to upstream projects as a downstream redistributor" isn't a realistic goal. In this particular case, the migration problems were already raised in the PEP 476 discussions, and the decision was made to *not* provide a public API specifically for globally turning off HTTPS certificate verification, since that would either: 1. Need to be added to Python 3 and maintained indefinitely; or 2. Be implemented in Python 2.7 only, and thus create another barrier to 2->3 migration A HTTPS specific flag wasn't considered a suitable long term API design for Python 3, while a multi-protocol configuration design that also handled setting the defaults for other protocols like secure SMTP, IMAP, POP3, SFTP, etc seemed generally desirable (but also doesn't exist anywhere, and poses signficant challenges in designing and implementing a good test matrix). The only concrete things that have changed since PEP 476 was approved is that: * we figured out the monkeypatching based migration recommendations in that PEP didn't actually work for the redistributor case * Robert Kuska came up with a way to do a backwards compatible backport of PEP 476 to RHEL 7.2 that doesn't raise any new barriers to migrating from Python 2 to Python 3 PEP 493 originally only described the latter idea - I added MAL's design for the environment variable based security downgrade based on the original PEP 493 discussions in July. Splitting the latter out to its own PEP that proposes it as a forwards compatible configuration setting akin to "curl --insecure" and "wget --no-check-certificate" for Python 2.7.12, 3.4.4, 3.5.2 and 3.6 upstream would be fine by me, since that would also significantly clarify the fact that the two ideas are entirely orthogonal to each other (except for the need to define a relative priority when you combine the two). Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/py
