[Python-Dev] anomaly

[Mark Rosenblitt-Janssen]

Here's something that might be wrong in Python (tried on v2.7):

>>> class int(str): pass

>>> int(3)
'3'

Mark
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mac popups running make test

[Carol Willing]

On 5/10/15 10:29 AM, Tal Einat wrote:
On Sun, May 10, 2015 at 5:07 PM, Brett Cannon > wrote:




On Sun, May 10, 2015 at 10:04 AM Skip Montanaro
mailto:skip.montan...@gmail.com>> wrote:

I haven't run the test suite in awhile. I am in the midst of
running it on my Mac running Yosemite 10.10.3. Twice now, I've
gotten this popup:


​
I assume this is testing some server listening on localhost.
Is this a new thing, either with the Python test suite or with
Mac OS X? (I'd normally be hidden behind a NAT firewall, but
at the moment I am on a miserable public connection in a
Peet's Coffee, so it takes on slightly more importance...)


It's not new.


Indeed, I've run into this as well.


I've also seen the Crash Reporter pop up many times, but as
far as I could tell, in all cases the test suite output told
me it was expected. Perhaps tests which listen for network
connections should also mention that, at least on Macs?


Wouldn't hurt. Just requires tracking down which test(s) triggers
it (might be more than one and I don't know if answering that
popup applies for the rest of the test execution or once per test
if you use -j).


If anyone starts working on this, let me know if I can help, e.g. 
trying things on my own Mac.


I believe that the message has to do with OS X's sandboxing 
implementation and the setting of the sandbox's entitlement keys. Here's 
an Apple doc: 
https://developer.apple.com/library/ios/documentation/Miscellaneous/Reference/EntitlementKeyReference/Chapters/EnablingAppSandbox.html#//apple_ref/doc/uid/TP40011195-CH4-SW9


I'm unaware of a way to work around this other than using Apple's code 
signing or adjusting target build settings in XCode :( If anyone knows a 
good way to workaround or manually set permission (other than clicking 
the Allow button), I would be interested.


Warmly,
Carol
--
*Carol Willing*
Developer | Willing Consulting
https://willingconsulting.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] anomaly

[Mark Rosenblitt-Janssen]

In case the example given at the start of the thread wasn't
interesting enough, it also works in the other direction:

>>> class str(int):  pass

>>> str('2')
2  #<- an integer!!!

Mark
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 10.05.2015 05:04, Robert Collins wrote:
> On 10 May 2015 at 11:44, Chris Angelico  wrote:
>> On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg  wrote:
>>> By providing a way to intentionally switch off the new default,
>>> we do make people aware of the risks and that's good enough,
>>> while still maintaining the contract people rightly expect of
>>> patch level releases of Python.
>>
>> Just as long as it's the sysadmin, and NOT some random attacker over
>> the internet, who has the power to downgrade security. Environment
>> variables can be attacked in various ways.
> 
> They can, and the bash fun was very good evidence of that.
> 
> OTOH if someones environment is at risk, PATH and PYTHONPATH are
> already very effective attack vectors.

If an attacker has access to the process environment, you're doomed
anyway, so that's not really an argument for or against using
environment variables :-)

You'd just need to create a file os.py and point PYTHONPATH at it.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode literals in Python 2.7

[Nick Coghlan]

On 10 May 2015 at 23:28, Adam Bartoš  wrote:
> Glenn Linderman wrote:
>> Is this going to get released in 3.5, I hope?  Python 3 is pretty
>> limited without some solution for Unicode on the console... probably the
>> biggest deficiency I have found in Python 3, since its introduction. It
>> has great Unicode support for files and processing, which convinced me
>> to switch from Perl, and I like so much else about it, that I can hardly
>> code in Perl any more (I still support a few Perl programs, but have
>> ported most of them to Python).
>
> I'd love to see it included in 3.5, but I doubt that will happen. For one
> thing, it's only two weeks till beta 1, which is feature freeze. And mainly,
> my package is mostly hacking into existing Python environment. A proper
> implementation would need some changes in Python someone would have to do.
> See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm
> not competent to write a patch myself and I have also no feedback to the
> proposed idea. On the other hand, using the package is good enough for me so
> I didn't further bring attention to the proposal.

Right, and while I'm interested in seeing this improved, I'm not
especially familiar with the internal details of our terminal
interaction implementation, and even less so when it comes to the
Windows terminal. Steve Dower's also had his hands full working on the
Windows installer changes, and several of our other Windows folks
aren't C programmers.

PEP 432 (the interpreter startup sequence improvements) will be back
on the agenda for Python 3.6, so the 3.6 time frame seems more
plausible at this point.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] anomaly

[Paul Moore]

On 10 May 2015 at 17:34, Mark Rosenblitt-Janssen
 wrote:
> Here's something that might be wrong in Python (tried on v2.7):
>
 class int(str): pass
>
 int(3)
> '3'

It's not wrong as such. It is allowed to define your own class that
subclasses a builtin class, and it's allowed to shadow builtin names.
So while this is (obviously) bad practice, it's not wrong.

For a simpler example:

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> str

>>> str = "Hello"
>>> str
'Hello'

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Nick Coghlan]

On 11 May 2015 at 18:04, M.-A. Lemburg  wrote:
> On 10.05.2015 05:04, Robert Collins wrote:
>> On 10 May 2015 at 11:44, Chris Angelico  wrote:
>>> On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg  wrote:
 By providing a way to intentionally switch off the new default,
 we do make people aware of the risks and that's good enough,
 while still maintaining the contract people rightly expect of
 patch level releases of Python.
>>>
>>> Just as long as it's the sysadmin, and NOT some random attacker over
>>> the internet, who has the power to downgrade security. Environment
>>> variables can be attacked in various ways.
>>
>> They can, and the bash fun was very good evidence of that.
>>
>> OTOH if someones environment is at risk, PATH and PYTHONPATH are
>> already very effective attack vectors.
>
> If an attacker has access to the process environment, you're doomed
> anyway, so that's not really an argument for or against using
> environment variables :-)

The core issue lies in managing the "user" vs "administrator"
permissions split. Even for self-administered systems, it's
recommended practice to run *without* administrative permissions
normally, and only elevate when you need them. One of the things
you're watching out for in such cases is that it shouldn't be possible
for an attacker to make a change to the user environment, and have
that propagate to have an effect on a process running with
administrative access. One of the recommended hardening measures
against that kind of attack vector is to *turn off* Python's
environment variable processing when launching Python processes with
administrative access.

We didn't care about that in the hash randomisation case, as the
compatibility concern there applied on a per application basis, and
caring about hash order was technically a bug in its own right. By
contrast, in the situation we're worried about for certificate
verification compatibility, the issue is environmental: certificate
management in many private intranets isn't yet to the same standard as
that on the public internet, so administrators may have a valid reason
for defaulting Python back to the old behaviour, and redistributors
may feel obliged to provide an opt-in period prior to switching the
default behaviour to opt-out. Having the new setting be ignored in
Python processes run under a hardened configuration means that an
environment variable based solution won't have the desired effect in
providing that smoother migration path to the more hardened
configuration.

I've now written a draft "recommendations to redistributors" PEP for
Robert's configuration file based design:
https://www.python.org/dev/peps/pep-0493/ (exact file names & config
setting names TBD)

I wouldn't be opposed to seeing that as an upstream Python 2.7.x
feature, but agreement amongst redistributors on using a file-based
approach is the main outcome I'm looking for.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 11:13, Nick Coghlan wrote:
> On 11 May 2015 at 18:04, M.-A. Lemburg  wrote:
>> On 10.05.2015 05:04, Robert Collins wrote:
>>> On 10 May 2015 at 11:44, Chris Angelico  wrote:
 On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg  wrote:
> By providing a way to intentionally switch off the new default,
> we do make people aware of the risks and that's good enough,
> while still maintaining the contract people rightly expect of
> patch level releases of Python.

 Just as long as it's the sysadmin, and NOT some random attacker over
 the internet, who has the power to downgrade security. Environment
 variables can be attacked in various ways.
>>>
>>> They can, and the bash fun was very good evidence of that.
>>>
>>> OTOH if someones environment is at risk, PATH and PYTHONPATH are
>>> already very effective attack vectors.
>>
>> If an attacker has access to the process environment, you're doomed
>> anyway, so that's not really an argument for or against using
>> environment variables :-)
> 
> The core issue lies in managing the "user" vs "administrator"
> permissions split. Even for self-administered systems, it's
> recommended practice to run *without* administrative permissions
> normally, and only elevate when you need them. One of the things
> you're watching out for in such cases is that it shouldn't be possible
> for an attacker to make a change to the user environment, and have
> that propagate to have an effect on a process running with
> administrative access. One of the recommended hardening measures
> against that kind of attack vector is to *turn off* Python's
> environment variable processing when launching Python processes with
> administrative access.

The env var would not be read at Python startup time, only
when loading the ssl module, so the -E switch would not have
the effect of disabling it - unlike the hash seed logic, which
is run (and has to be run) at Python startup time.

> We didn't care about that in the hash randomisation case, as the
> compatibility concern there applied on a per application basis, and
> caring about hash order was technically a bug in its own right. By
> contrast, in the situation we're worried about for certificate
> verification compatibility, the issue is environmental: certificate
> management in many private intranets isn't yet to the same standard as
> that on the public internet, so administrators may have a valid reason
> for defaulting Python back to the old behaviour, and redistributors
> may feel obliged to provide an opt-in period prior to switching the
> default behaviour to opt-out. Having the new setting be ignored in
> Python processes run under a hardened configuration means that an
> environment variable based solution won't have the desired effect in
> providing that smoother migration path to the more hardened
> configuration.
> 
> I've now written a draft "recommendations to redistributors" PEP for
> Robert's configuration file based design:
> https://www.python.org/dev/peps/pep-0493/ (exact file names & config
> setting names TBD)

The Fastly cache seems to be having problems again. I only get:
503 Backend is unhealthy - Details: cache-fra1225-FRA 1431335851 2631441948

> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
> feature, but agreement amongst redistributors on using a file-based
> approach is the main outcome I'm looking for.

Can't we have both ?

I don't think that we can wait for a whole PEP process to
run through to fix this regression in 2.7.9.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Nick Coghlan]

On 11 May 2015 at 19:22, M.-A. Lemburg  wrote:
> On 11.05.2015 11:13, Nick Coghlan wrote:
>> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
>> feature, but agreement amongst redistributors on using a file-based
>> approach is the main outcome I'm looking for.
>
> Can't we have both ?

I'd strongly advise against it, as we're deliberately increasing the
attack surface area here by providing a potential path to carry out a
downgrade attack on the HTTPS certificate verification by forcing it
back to the old behaviour.

The main existing environment variable based attack vector would be to
manage to run a process with administrative privileges and
SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to
a user or world-writable directory by the attacker. Providing a "don't
verify HTTPS" flag at the interpreter level would let an attacker skip
the first step of writing the certificate file to disk somewhere,
making a system compromise harder to detect. (An especially paranoid
SSL implementation would disallow reading certs from locations that
allow write access to non-administrative users, but I don't believe
OpenSSL is that paranoid)

By contrast, the configuration file shouldn't provide a new attack
vector (or simplify any existing attack vector), as if you have the
permissions needed to modify the config file, you likely also have the
permissions needed to modify the system certificate store, and the
latter is a *far* more interesting attack vector than a downgrade
attack solely against Python.

Thus the environment variable based off switch is neither necessary
(as an administrator controlled configuration file can do the job),
nor sufficient (it can't handle the -E switch), *and* it represents an
increase in the attack surface area relative to a Python
implementation without the capability.

> I don't think that we can wait for a whole PEP process to
> run through

Matrix multiplication made it through the PEP process in 8 days. If we
do this as a redistributor recommendation rather than attempting to
get it into upstream Python 2.7, we could potentially propose you take
on the role of BDFL-Delegate and mark it as Accepted as soon as the
two of us agree on a common approach.

The reason I think that's a reasonable way forward is because we
already know there are folks opposed to making the change upstream. If
the PEP just provides recommendations for redistributors that *do*
decide to provide a "global off switch" to revert to the old
behaviour, then the perspective of the folks opposed to the feature is
respected by the fact that this is a feature some redistributors *may*
choose to add to provide a smoother migration path to more secure
default HTTPS handling, rather than something upstream provides by
default.

I assume the Debian, Ubuntu and Suse folks won't care, as they have
all already decided against backporting the change to their long term
support releases where the compatibility break would pose a problem
(and I can certainly sympathise with that perspective given the
dependency on backporting the PEP 466 SSL changes first, and the work
involved in seeking consensus on a smoother migration path from the
old default to the new one).

It would be nice to hear from ActiveState, Enthought & Continuum
Analytics as well, but if they don't chime in here, I don't see any
particular need to go chasing them explicitly.

>to fix this regression in 2.7.9.

We made the decision when PEP 476 was accepted that this change turned
a silent security failure into a noisy one, rather than being a
regression in its own right. PEP 493 isn't about disagreeing with that
decision, it's about providing a smoother upgrade path in contexts
where letting the security failure remain silent is deemed to be
preferred in the near term.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Donald Stufft]

> On May 11, 2015, at 6:15 AM, Nick Coghlan  wrote:
> 
> On 11 May 2015 at 19:22, M.-A. Lemburg  wrote:
>> On 11.05.2015 11:13, Nick Coghlan wrote:
>>> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
>>> feature, but agreement amongst redistributors on using a file-based
>>> approach is the main outcome I'm looking for.
>> 
>> Can't we have both ?
> 
> I'd strongly advise against it, as we're deliberately increasing the
> attack surface area here by providing a potential path to carry out a
> downgrade attack on the HTTPS certificate verification by forcing it
> back to the old behaviour.
> 
> The main existing environment variable based attack vector would be to
> manage to run a process with administrative privileges and
> SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to
> a user or world-writable directory by the attacker. Providing a "don't
> verify HTTPS" flag at the interpreter level would let an attacker skip
> the first step of writing the certificate file to disk somewhere,
> making a system compromise harder to detect. (An especially paranoid
> SSL implementation would disallow reading certs from locations that
> allow write access to non-administrative users, but I don't believe
> OpenSSL is that paranoid)
> 
> By contrast, the configuration file shouldn't provide a new attack
> vector (or simplify any existing attack vector), as if you have the
> permissions needed to modify the config file, you likely also have the
> permissions needed to modify the system certificate store, and the
> latter is a *far* more interesting attack vector than a downgrade
> attack solely against Python.
> 
> Thus the environment variable based off switch is neither necessary
> (as an administrator controlled configuration file can do the job),
> nor sufficient (it can't handle the -E switch), *and* it represents an
> increase in the attack surface area relative to a Python
> implementation without the capability.
> 
>> I don't think that we can wait for a whole PEP process to
>> run through
> 
> Matrix multiplication made it through the PEP process in 8 days. If we
> do this as a redistributor recommendation rather than attempting to
> get it into upstream Python 2.7, we could potentially propose you take
> on the role of BDFL-Delegate and mark it as Accepted as soon as the
> two of us agree on a common approach.
> 
> The reason I think that's a reasonable way forward is because we
> already know there are folks opposed to making the change upstream. If
> the PEP just provides recommendations for redistributors that *do*
> decide to provide a "global off switch" to revert to the old
> behaviour, then the perspective of the folks opposed to the feature is
> respected by the fact that this is a feature some redistributors *may*
> choose to add to provide a smoother migration path to more secure
> default HTTPS handling, rather than something upstream provides by
> default.
> 
> I assume the Debian, Ubuntu and Suse folks won't care, as they have
> all already decided against backporting the change to their long term
> support releases where the compatibility break would pose a problem
> (and I can certainly sympathise with that perspective given the
> dependency on backporting the PEP 466 SSL changes first, and the work
> involved in seeking consensus on a smoother migration path from the
> old default to the new one).
> 
> It would be nice to hear from ActiveState, Enthought & Continuum
> Analytics as well, but if they don't chime in here, I don't see any
> particular need to go chasing them explicitly.
> 
>> to fix this regression in 2.7.9.
> 
> We made the decision when PEP 476 was accepted that this change turned
> a silent security failure into a noisy one, rather than being a
> regression in its own right. PEP 493 isn't about disagreeing with that
> decision, it's about providing a smoother upgrade path in contexts
> where letting the security failure remain silent is deemed to be
> preferred in the near term.
> 


I don't really agree that the decision to disable TLS is an environment one,
it's really a per application decision. This is why I was against having some
sort of global off switch for all of Python because just because one
application needs it turned off doesn't mean you want it turned off for another
Python application. You might have some script that is interacting with a
custom internal server which doesn’t have a valid TLS certificate but then you
also have pip* installed which is reaching out to PyPI and downloading code
from the internet. You might want to disable TLS verification for the first but
you almost certainly don't want it to disable TLS verification for the second
one.

In this regard I think that environment variables are somewhat better because
they are far easier to set per application instead of in a way that affects
every python program. Per application is the *right* scope for this setting,
especially in a system where people may or may no

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Antoine Pitrou]

I'm in entire agreement with Donald below.

Regards

Antoine.


On Mon, 11 May 2015 06:23:11 -0400
Donald Stufft  wrote:
> 
> I don't really agree that the decision to disable TLS is an environment one,
> it's really a per application decision. This is why I was against having some
> sort of global off switch for all of Python because just because one
> application needs it turned off doesn't mean you want it turned off for 
> another
> Python application. You might have some script that is interacting with a
> custom internal server which doesn’t have a valid TLS certificate but then you
> also have pip* installed which is reaching out to PyPI and downloading code
> from the internet. You might want to disable TLS verification for the first 
> but
> you almost certainly don't want it to disable TLS verification for the second
> one.
> 
> In this regard I think that environment variables are somewhat better because
> they are far easier to set per application instead of in a way that affects
> every python program. Per application is the *right* scope for this setting,
> especially in a system where people may or may not realize what is written in
> Python and what isn't. I think it's absolutely wrong to give people a footgun
> in the terms of a switch that turns off all of Python's TLS verification when
> for many applications the fact they use Python is simply an implementation
> detail.
> 
> That being said, since it's not being included in Python core and it's only
> some patch that some downstream's are going to apply I also don't really care
> that much because it's not going to effect me and if it turns out to be a bad
> idea and a footgun like I think it is, then the blame can rest on those
> downstreams and not us :)
> 
> I'm also not a fan of the environment variable either really for a lot of the
> reasons you've outlined here.
> 
> * Ignoring the fact that pip has (via requests/urllib3) worked around this
>   deficiency in Python and isn't going to be affected by this configuration
>   switch at all.
> 
> 
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Nick Coghlan]

On 11 May 2015 at 20:23, Donald Stufft  wrote:
> On May 11, 2015, at 6:15 AM, Nick Coghlan  wrote:
>> We made the decision when PEP 476 was accepted that this change turned
>> a silent security failure into a noisy one, rather than being a
>> regression in its own right. PEP 493 isn't about disagreeing with that
>> decision, it's about providing a smoother upgrade path in contexts
>> where letting the security failure remain silent is deemed to be
>> preferred in the near term.
>
> I don't really agree that the decision to disable TLS is an environment one,
> it's really a per application decision. This is why I was against having some
> sort of global off switch for all of Python because just because one
> application needs it turned off doesn't mean you want it turned off for 
> another
> Python application.

The scenario I'm interested in is the one where it *was* off globally
(i.e. you were already running Python 2.7.8 or earlier) and you want
to manage a global rollout of a new Python version that supports being
configured to verify HTTPS certificates by default, while making the
decision on whether or not to enable HTTPS certificate verification on
a server-by-server basis, rather than having that decision be coupled
directly to the rollout of the updated version of Python.

I agree that the desired end state is where Python 3 is, and where
upstream Python 2.7.9+ is, this is solely about how to facilitate
folks getting from point A to point B without an intervening window of
"I broke the world and now my boss is yelling at me about it" :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 12:47, Nick Coghlan wrote:
> On 11 May 2015 at 20:23, Donald Stufft  wrote:
>> On May 11, 2015, at 6:15 AM, Nick Coghlan  wrote:
>>> We made the decision when PEP 476 was accepted that this change turned
>>> a silent security failure into a noisy one, rather than being a
>>> regression in its own right. PEP 493 isn't about disagreeing with that
>>> decision, it's about providing a smoother upgrade path in contexts
>>> where letting the security failure remain silent is deemed to be
>>> preferred in the near term.
>>
>> I don't really agree that the decision to disable TLS is an environment one,
>> it's really a per application decision. This is why I was against having some
>> sort of global off switch for all of Python because just because one
>> application needs it turned off doesn't mean you want it turned off for 
>> another
>> Python application.
> 
> The scenario I'm interested in is the one where it *was* off globally
> (i.e. you were already running Python 2.7.8 or earlier) and you want
> to manage a global rollout of a new Python version that supports being
> configured to verify HTTPS certificates by default, while making the
> decision on whether or not to enable HTTPS certificate verification on
> a server-by-server basis, rather than having that decision be coupled
> directly to the rollout of the updated version of Python.

I guess we're approaching this from different perspectives :-)

I'm mostly interested in having a switch that can be set on
a per application basis, not globally.

I think the global default is fine and I'm just looking for a way to have
admins disable it on a case-by-case basis for those applications which
have problems with the new default. Hence the env var approach
- the admin would simply edit the application's startup shell
script, add the env var and that's it.

For pip et al. which don't use the ssl module, the admins can simply
continue using older versions for those applications - ones which
don't implement the extra verification. In many cases, this is not
necessary, since production environments typically don't use PyPI
at all: they use a local directory with the needed distribution
files, which is both more secure and reliable.

> I agree that the desired end state is where Python 3 is, and where
> upstream Python 2.7.9+ is, this is solely about how to facilitate
> folks getting from point A to point B without an intervening window of
> "I broke the world and now my boss is yelling at me about it" :)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Donald Stufft]

> On May 11, 2015, at 6:47 AM, Nick Coghlan  wrote:
> 
> On 11 May 2015 at 20:23, Donald Stufft  wrote:
>> On May 11, 2015, at 6:15 AM, Nick Coghlan  wrote:
>>> We made the decision when PEP 476 was accepted that this change turned
>>> a silent security failure into a noisy one, rather than being a
>>> regression in its own right. PEP 493 isn't about disagreeing with that
>>> decision, it's about providing a smoother upgrade path in contexts
>>> where letting the security failure remain silent is deemed to be
>>> preferred in the near term.
>> 
>> I don't really agree that the decision to disable TLS is an environment one,
>> it's really a per application decision. This is why I was against having some
>> sort of global off switch for all of Python because just because one
>> application needs it turned off doesn't mean you want it turned off for 
>> another
>> Python application.
> 
> The scenario I'm interested in is the one where it *was* off globally
> (i.e. you were already running Python 2.7.8 or earlier) and you want
> to manage a global rollout of a new Python version that supports being
> configured to verify HTTPS certificates by default, while making the
> decision on whether or not to enable HTTPS certificate verification on
> a server-by-server basis, rather than having that decision be coupled
> directly to the rollout of the updated version of Python.
> 
> I agree that the desired end state is where Python 3 is, and where
> upstream Python 2.7.9+ is, this is solely about how to facilitate
> folks getting from point A to point B without an intervening window of
> "I broke the world and now my boss is yelling at me about it" :)
> 

Oh, another issue that I forgot to mention--

A fair number of people had no idea that Python wasn't validating TLS before
2.7.9/3.4.3 however as part of the processing of changing that in 2.7.9 a lot
of people became aware that Python's before 2.7.9 didn't validate but that
Python 2.7.9+ does. I worry that if Redhat (or anyone) ships a Python 2.7.9
that doesn't verify by default then they are going to be shipping something
which defies the expectations of those users who were relying on the fact that
Python 2.7.9+ was supposed to be secure by default now. You're (understandibly)
focusing on "I already have my thing running on Python 2.7.8 and I want to
yum update and get 2.7.9 and have things not visibly break", however there is
the other use case of "I'm setting up a new environment, and I installed RHEL
and got 2.7.9, I remembered reading in LWN that 2.7.9 verifies now so I must
be safe". If you *do* provide such a switch, defaulting it to verify and having
people where that breaks go in and turn it off is probably a safer mechanism
since the cases where 2.7.9 verification breaks things for people is a visible
change where the case that someone expects 2.7.9 to verify and it doesn't isn't
a visible change and is easily missed unless they go out of their way to try
and test it against a server with an invalid certificate.

Either way, if there is some sort of global off switch, having that off switch
set to off should raise some kind of warning (like urllib3 does if you use
the unverified HTTPS methods). To be clear, I don't mean that using the built
in ssl module APIs to disable verification should raise a warning, I mean the
hypothetical "make my Python insecurely access HTTPS" configuration file (or
environment variable) that is being proposed.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Nick Coghlan]

On 11 May 2015 10:16 pm, "Robert Kuska"  wrote:

> > >
> >
> > Oh, another issue that I forgot to mention--
> >
> > A fair number of people had no idea that Python wasn't validating TLS
before
> > 2.7.9/3.4.3 however as part of the processing of changing that in 2.7.9
a lot
> > of people became aware that Python's before 2.7.9 didn't validate but
that
> > Python 2.7.9+ does. I worry that if Redhat (or anyone) ships a Python
2.7.9
> > that doesn't verify by default then they are going to be shipping
something
> > which defies the expectations of those users who were relying on the
fact
> > that
> > Python 2.7.9+ was supposed to be secure by default now. You're
> > (understandibly)
> > focusing on "I already have my thing running on Python 2.7.8 and I want
to
> > yum update and get 2.7.9 and have things not visibly break",

As Robert noted, it would be a matter of updating to a 2.7.5 with more
patches backported, rather than rebasing to a newer upstream version.

I can make the "do not change the default behaviour relative to the
corresponding upstream version" guidance explicit in the PEP, though.

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [RELEASE] Python 2.7.10 release candidate 1

[Benjamin Peterson]

It is my privilege to announce the first release candidate of 2.7.10,
the next bugfix release in the 2.7 series.

Downloads are at

   https://www.python.org/downloads/release/python-2710rc1/

The full changelog is at

   https://hg.python.org/cpython/raw-file/80ccce248ba2/Misc/NEWS

Please consider testing 2.7.10rc1 with your application and reporting
bugs to

   https://bugs.python.org

Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 12:15, Nick Coghlan wrote:
> On 11 May 2015 at 19:22, M.-A. Lemburg  wrote:
>> On 11.05.2015 11:13, Nick Coghlan wrote:
>>> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
>>> feature, but agreement amongst redistributors on using a file-based
>>> approach is the main outcome I'm looking for.
>>
>> Can't we have both ?
> 
> I'd strongly advise against it, as we're deliberately increasing the
> attack surface area here by providing a potential path to carry out a
> downgrade attack on the HTTPS certificate verification by forcing it
> back to the old behaviour.
> 
> The main existing environment variable based attack vector would be to
> manage to run a process with administrative privileges and
> SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to
> a user or world-writable directory by the attacker. Providing a "don't
> verify HTTPS" flag at the interpreter level would let an attacker skip
> the first step of writing the certificate file to disk somewhere,
> making a system compromise harder to detect. (An especially paranoid
> SSL implementation would disallow reading certs from locations that
> allow write access to non-administrative users, but I don't believe
> OpenSSL is that paranoid)

Correct. OpenSSL will happily read the cert files from anywhere
you point it to.

> By contrast, the configuration file shouldn't provide a new attack
> vector (or simplify any existing attack vector), as if you have the
> permissions needed to modify the config file, you likely also have the
> permissions needed to modify the system certificate store, and the
> latter is a *far* more interesting attack vector than a downgrade
> attack solely against Python.
> 
> Thus the environment variable based off switch is neither necessary
> (as an administrator controlled configuration file can do the job),
> nor sufficient (it can't handle the -E switch), *and* it represents an
> increase in the attack surface area relative to a Python
> implementation without the capability.

Whether or not -E will have an effect on the env var depends
on the implementation. At the moment, -E only has an effect
on the C runtime, while the stdlib happily reads from os.environ
without taking the flag into account.

As proposed, the PYTHONHTTPSVERIFY would only affect the ssl
module and only be checked when loading this module, i.e. not
at Python startup time.

>> I don't think that we can wait for a whole PEP process to
>> run through
> 
> Matrix multiplication made it through the PEP process in 8 days. If we
> do this as a redistributor recommendation rather than attempting to
> get it into upstream Python 2.7, we could potentially propose you take
> on the role of BDFL-Delegate and mark it as Accepted as soon as the
> two of us agree on a common approach.
> 
> The reason I think that's a reasonable way forward is because we
> already know there are folks opposed to making the change upstream. If
> the PEP just provides recommendations for redistributors that *do*
> decide to provide a "global off switch" to revert to the old
> behaviour, then the perspective of the folks opposed to the feature is
> respected by the fact that this is a feature some redistributors *may*
> choose to add to provide a smoother migration path to more secure
> default HTTPS handling, rather than something upstream provides by
> default.
> 
> I assume the Debian, Ubuntu and Suse folks won't care, as they have
> all already decided against backporting the change to their long term
> support releases where the compatibility break would pose a problem
> (and I can certainly sympathise with that perspective given the
> dependency on backporting the PEP 466 SSL changes first, and the work
> involved in seeking consensus on a smoother migration path from the
> old default to the new one).
> 
> It would be nice to hear from ActiveState, Enthought & Continuum
> Analytics as well, but if they don't chime in here, I don't see any
> particular need to go chasing them explicitly.

I think the approach to only consider a subset of redistributors
as viable targets for such a switch is a bit too narrow.

You are leaving out all the parties which use custom
Python installations to run their applications, e.g.
the Plone and Zope community, the ZenOSS community,
the many Windows applications built on Python, etc.

>> to fix this regression in 2.7.9.
> 
> We made the decision when PEP 476 was accepted that this change turned
> a silent security failure into a noisy one, rather than being a
> regression in its own right. PEP 493 isn't about disagreeing with that
> decision, it's about providing a smoother upgrade path in contexts
> where letting the security failure remain silent is deemed to be
> preferred in the near term.

The change wasn't regression. The missing downgrade path
is a regression.

Some other comments on PEP 493:

* I don't think we really want to add the overhead of
  having to parse an INI file every time Python star

Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: async() function is deprecated in favour of ensure_future().

[Paul Moore]

On 11 May 2015 at 19:50, yury.selivanov  wrote:
> https://hg.python.org/cpython/rev/b78127eafad7
> changeset:   95956:b78127eafad7
> branch:  3.4
> parent:  95953:a983d63e3321
> user:Yury Selivanov 
> date:Mon May 11 14:48:38 2015 -0400
> summary:
>   asyncio: async() function is deprecated in favour of ensure_future().
>
> files:
>   Lib/asyncio/base_events.py   |   2 +-
>   Lib/asyncio/tasks.py |  27 -
>   Lib/asyncio/windows_events.py|   2 +-
>   Lib/test/test_asyncio/test_base_events.py|   6 +-
>   Lib/test/test_asyncio/test_tasks.py  |  48 +
>   Lib/test/test_asyncio/test_windows_events.py |   2 +-
>   Misc/NEWS|   4 +-
>   7 files changed, 57 insertions(+), 34 deletions(-)

Surely this should include a doc change?
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: async() function is deprecated in favour of ensure_future().

[Yury Selivanov]

Yes, I'm in the process of writing it ;)  (as well as for new 
set_task_factory())


Thanks,
Yury

On 2015-05-11 3:42 PM, Paul Moore wrote:

On 11 May 2015 at 19:50, yury.selivanov  wrote:

https://hg.python.org/cpython/rev/b78127eafad7
changeset:   95956:b78127eafad7
branch:  3.4
parent:  95953:a983d63e3321
user:Yury Selivanov 
date:Mon May 11 14:48:38 2015 -0400
summary:
   asyncio: async() function is deprecated in favour of ensure_future().

files:
   Lib/asyncio/base_events.py   |   2 +-
   Lib/asyncio/tasks.py |  27 -
   Lib/asyncio/windows_events.py|   2 +-
   Lib/test/test_asyncio/test_base_events.py|   6 +-
   Lib/test/test_asyncio/test_tasks.py  |  48 +
   Lib/test/test_asyncio/test_windows_events.py |   2 +-
   Misc/NEWS|   4 +-
   7 files changed, 57 insertions(+), 34 deletions(-)

Surely this should include a doc change?
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: async() function is deprecated in favour of ensure_future().

[Paul Moore]

On 11 May 2015 at 20:47, Yury Selivanov  wrote:
> Yes, I'm in the process of writing it ;)  (as well as for new
> set_task_factory())

Cool - sorry for being a nag :-)
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Free lists

[Stefan Behnel]

Serhiy Storchaka schrieb am 09.05.2015 um 21:01:
> Here is a statistic for most called PyObject_INIT or PyObject_INIT_VAR for
> types (collected during running Python tests on 32-bit Linux).

I'm aware that this includes lots of tests for the Python code in the
stdlib, so these numbers are most likely not too far from what real-world
code would give, but wouldn't it be even better to collect statistics from
a (quick) benchmark suite run? Test suites tend to be very regular, flat
and broad, unlike most production code.

Stefan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Minimal async event loop and async utilities (Was: PEP 492: async/await in Python; version 4)

[Paul Moore]

On 6 May 2015 at 16:46, Guido van Rossum  wrote:
> This is actually a great idea, and I encourage you to go forward with it.
> The biggest piece missing from your inventory is probably Task, which is
> needed to wrap a Future around a coroutine.

OK, I've been doing some work on this. You're right, the asyncio
framework makes Future a key component.

But I'm not 100% sure why Future (and Task) have to be so fundamental.
Ignoring cancellation (see below!) I can build pretty much all of a
basic event loop, plus equivalents of the asyncio locks and queues
modules, without needing the concept of a Future at all. The
create_task function becomes simply a function to add a coroutine to
the ready queue, in this context. I can't return a Task (because I
haven't implemented the Task or Future classes) but I don't actually
know what significant functionality is lost as a result - is there a
reasonably accessible example of where using the return value from
create_task is important anywhere?

A slightly more complicated issue is with the run_until_complete
function, which takes a Future, and hence is fundamentally tied to the
Future API. However, it seems to me that a "minimal" implementation
could work by having a run_until_complete() that just took an
awaitable (i.e., anything that you can yield from). Again, is there a
specific reason that you ended up going with run_until_complete taking
a Future rather than just a coroutine? I think (but haven't confirmed
yet by implementing it) that it should be possible to create a
coroutine that acts like a Future, in the sense that you can tell it
from outside (via send()) that it's completed and set its return
value. But this is all theory, and if you have any practical
experience that shows I'm going down a dead end, I'd be glad to know.

I'm not sure how useful this line of attack will be - if the API isn't
compatible with asyncio.BaseEventLoop, it's not very useful in
practice. On the other hand, if I can build a loop without Future or
Task classes, it may indicate that those classes aren't quite as
fundamental as asyncio makes them (which may allow some
simplifications or generalisations).

> I expect you'll also want to build cancellation into your "base async
> framework"; and the primitives to wait for multiple awaitables. The next
> step would be some mechanism to implement call_later()/call_at() (but this
> needs to be pluggable since for a "real" event loop it needs to be
> implemented by the basic I/O selector).

These are where I suspect I'll have the most trouble if I haven't got
a solid understanding of the role of the Future and Task classes (or
alternatively, how to avoid them :-)) So I'm holding off on worrying
about them for now. But certainly they need to be covered. In
particular, call_later/call_at are the only "generic" example of any
form of wait that actually *waits*, rather than returning immediately.
So as you say, implementing them will show how the basic mechanism can
be extended with a "real" selector (whether for I/O, or GUI events, or
whatever).

> If you can get this working it would be great to include this in the stdlib
> as a separate "asynclib" library. The original asyncio library would then be
> a specific implementation (using a subclass of asynclib.EventLoop) that adds
> I/O, subprocesses, and integrates with the selectors module (or with IOCP,
> on Windows).

One thing I've not really considered in the above, is how a
refactoring like this would work. Ignoring the "let's try to remove
the Future class" approach above, my "basic event loop" is mostly just
an alternative implementation of an event loop (or maybe an
alternative policy - I'm not sure I understand the need for policies
yet). So it may simply be a case of ripping coroutines.py, futures.py,
locks.py, log.py, queues.py, and tasks.py out of asyncio and adding a
new equivalent of events.py with my "minimal" loop in it. (So far,
when I've tried to do that I get hit with some form of circular import
problem - I've not worked out why yet, or how asyncio avoids the same
problem).

That in itself would probably be a useful refactoring, splitting out
the IO aspects of asyncio from the event loop / async aspects.

> I don't see any particular hurry to get this in before 3.5; the refactoring
> of asyncio can be done later, in a backward compatible way. It would be a
> good way to test the architecture of asyncio!

Agreed. It's also not at all clear to me how the new async/await
syntax would fit in with this, so that probably needs some time to
settle down. For example, in Python 3.5 would run_until_complete take
an awaitable rather than a Future?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode literals in Python 2.7

[Glenn Linderman]

On 5/11/2015 1:09 AM, Nick Coghlan wrote:

On 10 May 2015 at 23:28, Adam Bartoš  wrote:

Glenn Linderman wrote:

Is this going to get released in 3.5, I hope?  Python 3 is pretty
limited without some solution for Unicode on the console... probably the
biggest deficiency I have found in Python 3, since its introduction. It
has great Unicode support for files and processing, which convinced me
to switch from Perl, and I like so much else about it, that I can hardly
code in Perl any more (I still support a few Perl programs, but have
ported most of them to Python).

I'd love to see it included in 3.5, but I doubt that will happen. For one
thing, it's only two weeks till beta 1, which is feature freeze. And mainly,
my package is mostly hacking into existing Python environment. A proper
implementation would need some changes in Python someone would have to do.
See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm
not competent to write a patch myself and I have also no feedback to the
proposed idea. On the other hand, using the package is good enough for me so
I didn't further bring attention to the proposal.

Right, and while I'm interested in seeing this improved, I'm not
especially familiar with the internal details of our terminal
interaction implementation, and even less so when it comes to the
Windows terminal. Steve Dower's also had his hands full working on the
Windows installer changes, and several of our other Windows folks
aren't C programmers.

PEP 432 (the interpreter startup sequence improvements) will be back
on the agenda for Python 3.6, so the 3.6 time frame seems more
plausible at this point.

Cheers,
Nick.


Wow!  Another bug that'll reach a decade in age before being fixed...
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Minimal async event loop and async utilities (Was: PEP 492: async/await in Python; version 4)

[Guido van Rossum]

On Mon, May 11, 2015 at 1:37 PM, Paul Moore  wrote:

> On 6 May 2015 at 16:46, Guido van Rossum  wrote:
> > This is actually a great idea, and I encourage you to go forward with it.
> > The biggest piece missing from your inventory is probably Task, which is
> > needed to wrap a Future around a coroutine.
>
> OK, I've been doing some work on this. You're right, the asyncio
> framework makes Future a key component.
>
> But I'm not 100% sure why Future (and Task) have to be so fundamental.
> Ignoring cancellation (see below!) I can build pretty much all of a
> basic event loop, plus equivalents of the asyncio locks and queues
> modules, without needing the concept of a Future at all. The
> create_task function becomes simply a function to add a coroutine to
> the ready queue, in this context. I can't return a Task (because I
> haven't implemented the Task or Future classes) but I don't actually
> know what significant functionality is lost as a result - is there a
> reasonably accessible example of where using the return value from
> create_task is important anywhere?
>

In asyncio the Task object is used to wait for the result. Of course if all
you need is to wait for the result you don't need to call create_task() --
so in your situation it's uninteresting. But Task is needed for
cancellation and Future is needed so I/O completion can be implemented
using callback functions.


> A slightly more complicated issue is with the run_until_complete
> function, which takes a Future, and hence is fundamentally tied to the
> Future API. However, it seems to me that a "minimal" implementation
> could work by having a run_until_complete() that just took an
> awaitable (i.e., anything that you can yield from). Again, is there a
> specific reason that you ended up going with run_until_complete taking
> a Future rather than just a coroutine?


Actually it takes a Future *or* a coroutine. (The docs or the arg name may
be confusing.) In asyncio, pretty much everything that takes one takes the
other.


> I think (but haven't confirmed
> yet by implementing it) that it should be possible to create a
> coroutine that acts like a Future, in the sense that you can tell it
> from outside (via send()) that it's completed and set its return
> value. But this is all theory, and if you have any practical
> experience that shows I'm going down a dead end, I'd be glad to know.
>

I don't know -- I never explored that.


> I'm not sure how useful this line of attack will be - if the API isn't
> compatible with asyncio.BaseEventLoop, it's not very useful in
> practice. On the other hand, if I can build a loop without Future or
> Task classes, it may indicate that those classes aren't quite as
> fundamental as asyncio makes them (which may allow some
> simplifications or generalisations).
>

Have you tried to implement waiting for I/O yet?

OTOH you may look at micropython's uasyncio -- IIRC it doesn't have Futures
and it definitely has I/O waiting.


> > I expect you'll also want to build cancellation into your "base async
> > framework"; and the primitives to wait for multiple awaitables. The next
> > step would be some mechanism to implement call_later()/call_at() (but
> this
> > needs to be pluggable since for a "real" event loop it needs to be
> > implemented by the basic I/O selector).
>
> These are where I suspect I'll have the most trouble if I haven't got
> a solid understanding of the role of the Future and Task classes (or
> alternatively, how to avoid them :-)) So I'm holding off on worrying
> about them for now. But certainly they need to be covered. In
> particular, call_later/call_at are the only "generic" example of any
> form of wait that actually *waits*, rather than returning immediately.
> So as you say, implementing them will show how the basic mechanism can
> be extended with a "real" selector (whether for I/O, or GUI events, or
> whatever).
>

Right.


> > If you can get this working it would be great to include this in the
> stdlib
> > as a separate "asynclib" library. The original asyncio library would
> then be
> > a specific implementation (using a subclass of asynclib.EventLoop) that
> adds
> > I/O, subprocesses, and integrates with the selectors module (or with
> IOCP,
> > on Windows).
>
> One thing I've not really considered in the above, is how a
> refactoring like this would work. Ignoring the "let's try to remove
> the Future class" approach above, my "basic event loop" is mostly just
> an alternative implementation of an event loop (or maybe an
> alternative policy - I'm not sure I understand the need for policies
> yet).


A policy is mostly a wrapper around an event loop factory plus state that
records the current event loop.


> So it may simply be a case of ripping coroutines.py, futures.py,
> locks.py, log.py, queues.py, and tasks.py out of asyncio and adding a
> new equivalent of events.py with my "minimal" loop in it. (So far,
> when I've tried to do that I get hit with some form of circular impo

Re: [Python-Dev] Mac popups running make test

[Ned Deily]

In article 
,
 Tal Einat  wrote:
> On Sun, May 10, 2015 at 9:07 PM, Carol Willing <
> willi...@willingconsulting.com> wrote:
> > On 5/10/15 10:29 AM, Tal Einat wrote:
> >  On Sun, May 10, 2015 at 5:07 PM, Brett Cannon  wrote:
> >> On Sun, May 10, 2015 at 10:04 AM Skip Montanaro 
> >> wrote:
> >>> I haven't run the test suite in awhile. I am in the midst of running it
> >>> on my Mac running Yosemite 10.10.3. Twice now, I've gotten this popup:
> >>>  I assume this is testing some server listening on localhost. Is this a
> >>> new thing, either with the Python test suite or with Mac OS X? (I'd
> >>> normally be hidden behind a NAT firewall, but at the moment I am on a
> >>> miserable public connection in a Peet's Coffee, so it takes on slightly
> >>> more importance...)
> >>  It's not new.
> >  Indeed, I've run into this as well.
> >>>  I've also seen the Crash Reporter pop up many times, but as far as I
> >>> could tell, in all cases the test suite output told me it was expected.
> >>> Perhaps tests which listen for network connections should also mention
> >>> that, at least on Macs?
> >>  Wouldn't hurt. Just requires tracking down which test(s) triggers it
> >> (might be more than one and I don't know if answering that popup applies
> >> for the rest of the test execution or once per test if you use -j).
> >  If anyone starts working on this, let me know if I can help, e.g. trying
> > things on my own Mac.
\> >I believe that the message has to do with OS X's sandboxing
> > implementation and the setting of the sandbox's entitlement keys. Here's an
> > Apple doc:
> > https://developer.apple.com/library/ios/documentation/Miscellaneous/Referenc
> > e/EntitlementKeyReference/Chapters/EnablingAppSandbox.html#//apple_ref/doc/u
> > id/TP40011195-CH4-SW9
> > I'm unaware of a way to work around this other than using Apple's code
> > signing or adjusting target build settings in XCode :( If anyone knows a
> > good way to workaround or manually set permission (other than clicking the
> > Allow button), I would be interested.
> I was reading about this a few weeks ago an recall finding a way to ad-hoc
> sign the built python executable. Here's a link below. I haven't tried
> this, though, and don't know if it would work with a python executable
> rather than a proper OSX app. If it does work, it would be useful to add
> this as a tool and/or mention it in the developer docs.
> 
> http://apple.stackexchange.com/a/121010

I believe the issue has to do with the OS X application firewall and not 
sandboxing, as vanilla Python on OS X is not sandboxed.  See:

https://support.apple.com/en-us/HT201642

As described there, codesigned applications are automatically authorized 
to accept inbound connections; that's the workaround proposed in the 
apple.stackexchange cite.  But arbitrarily signing development binaries 
after every compile is probably not a good idea.  Another option is to 
configure the firewall but that probably only can be made to work with a 
framework build of Python which launches Python within an app bundle.  
In any case, please open an issue on the bug tracker so we can follow up 
on this.

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mac popups running make test

[Ezio Melotti]

On Sun, May 10, 2015 at 5:04 PM, Skip Montanaro
 wrote:
>
> ...
> I've also seen the Crash Reporter pop up many times,

I don't know how to get rid of the popup you mentioned, but Windows
had problems with the crash popups for a long time.
Eventually it got fixed:
  https://hg.python.org/cpython/file/default/Lib/test/support/__init__.py#l2202
  http://bugs.python.org/issue11732
  http://bugs.python.org/issue18948
  http://bugs.python.org/issue23314

Perhaps Mac OS has something similar too?

Best Regards,
Ezio Melotti

>
> but as far as I could tell, in all cases the test suite output told me it was 
> expected. Perhaps tests which listen for network connections should also 
> mention that, at least on Macs?
>
> Thx,
>
> Skip
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[Nick Coghlan]

On 12 May 2015 at 04:49, M.-A. Lemburg  wrote:
> On 11.05.2015 12:15, Nick Coghlan wrote:
>> By contrast, the configuration file shouldn't provide a new attack
>> vector (or simplify any existing attack vector), as if you have the
>> permissions needed to modify the config file, you likely also have the
>> permissions needed to modify the system certificate store, and the
>> latter is a *far* more interesting attack vector than a downgrade
>> attack solely against Python.
>>
>> Thus the environment variable based off switch is neither necessary
>> (as an administrator controlled configuration file can do the job),
>> nor sufficient (it can't handle the -E switch), *and* it represents an
>> increase in the attack surface area relative to a Python
>> implementation without the capability.
>
> Whether or not -E will have an effect on the env var depends
> on the implementation. At the moment, -E only has an effect
> on the C runtime, while the stdlib happily reads from os.environ
> without taking the flag into account.

I had an off-list discussion with Christian Heimes about that in
relation to the OpenSSL flags, and he pointed out the reason -E
specifically needs to be a command line switch is because that is the
only way to affect how environment variables are processed during
interpreter startup. Once an application is up and running, further
environment variable sanitisation can be handled at an application
level by whitelisting entries in os.environ and deleting everything
else.

> As proposed, the PYTHONHTTPSVERIFY would only affect the ssl
> module and only be checked when loading this module, i.e. not
> at Python startup time.

Right, the same is true for the configuration file proposal.

>> It would be nice to hear from ActiveState, Enthought & Continuum
>> Analytics as well, but if they don't chime in here, I don't see any
>> particular need to go chasing them explicitly.
>
> I think the approach to only consider a subset of redistributors
> as viable targets for such a switch is a bit too narrow.
>
> You are leaving out all the parties which use custom
> Python installations to run their applications, e.g.
> the Plone and Zope community, the ZenOSS community,
> the many Windows applications built on Python, etc.

No, they already have a solution: monkeypatch (or just plain patch)
the SSL module. That's an upstream supported technique, which is why
it's documented in the PEP.

The problem we (as in Red Hat) ran into was that that technique
*doesn't work* for the case of backporting PEP 476 to Python 2.7.5 as
an opt-in feature.

>>> to fix this regression in 2.7.9.
>>
>> We made the decision when PEP 476 was accepted that this change turned
>> a silent security failure into a noisy one, rather than being a
>> regression in its own right. PEP 493 isn't about disagreeing with that
>> decision, it's about providing a smoother upgrade path in contexts
>> where letting the security failure remain silent is deemed to be
>> preferred in the near term.
>
> The change wasn't regression. The missing downgrade path
> is a regression.

It's a shame we don't have "-X" options in Python 2, as that would be
a nice hard-to-attack option (although it wouldn't play well with
subprocesses)

> Some other comments on PEP 493:
>
> * I don't think we really want to add the overhead of
>   having to parse an INI file every time Python starts up.
>   Please remember that we just parsing of the sysconfig
>   data not long ago because we wanted to avoid this startup
>   time.

Compared to the overhead of reading from the system cert database,
reading a config file at ssl module import time should be trivial.

> * I don't see why the attack surface of using an INI file
>   somewhere in the system should be smaller than e.g. using
>   sitecustomize.py

You can put sitecustomize.py in a user directory, and if there's no
system wide sitecustomize, Python will read it automatically (unless
user site directories are turned off).

> * If done right, we'd also need a switch to ignore this
>   global config file and recommend using it to reduce the
>   attack surface (for the same reason you explain in the
>   PEP)

No, the recommendation there would be to upgrade to a newer version of
Python that doesn't offer this downgrade capability. It's a proposal
for a transition smoothing technique, not a permanent capability (I
did suggest the latter at one point, but the discussion on the issue
tracker persuaded me that was a bad idea, with the increased attack
surface being a key part of that change of heart).

> * I don't think a global switch is the right way forward.
>   Many applications on properly configured systems will
>   work fine with the new default. The downgrade option is
>   only needed for those cases, where they don't and you
>   don't have a good way to fix the application.

And techniques like chroots and containers let you do that selectively.

The key thing I'm after is an agreed technique for backporting to
earlier 2.7

Re: [Python-Dev] Unicode literals in Python 2.7

[Nick Coghlan]

On 12 May 2015 at 06:38, Glenn Linderman  wrote:
> On 5/11/2015 1:09 AM, Nick Coghlan wrote:
> On 10 May 2015 at 23:28, Adam Bartoš  wrote:
> I'd love to see it included in 3.5, but I doubt that will happen. For one
> thing, it's only two weeks till beta 1, which is feature freeze. And mainly,
> my package is mostly hacking into existing Python environment. A proper
> implementation would need some changes in Python someone would have to do.
> See for example my proposal http://bugs.python.org/issue17620#msg234439. I'm
> not competent to write a patch myself and I have also no feedback to the
> proposed idea. On the other hand, using the package is good enough for me so
> I didn't further bring attention to the proposal.
>
> Right, and while I'm interested in seeing this improved, I'm not
> especially familiar with the internal details of our terminal
> interaction implementation, and even less so when it comes to the
> Windows terminal. Steve Dower's also had his hands full working on the
> Windows installer changes, and several of our other Windows folks
> aren't C programmers.
>
> PEP 432 (the interpreter startup sequence improvements) will be back
> on the agenda for Python 3.6, so the 3.6 time frame seems more
> plausible at this point.
>
> Cheers,
> Nick.
>
> Wow!  Another bug that'll reach a decade in age before being fixed...

Yep, that tends to happen with complex cross-platform bugs & RFEs that
require domain expertise in multiple areas to resolve. It's one of the
areas that operating system vendors are typically best equipped to
handle, but we haven't historically had that kind of major
institutional backing for CPython core development (that *is*
changing, but it's a relatively recent phenomenon).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com