Re: [Python-Dev] Python 2.7 patch levels turning two digit

[M.-A. Lemburg]

On 23.06.2014 18:09, Donald Stufft wrote:
> 
> On Jun 23, 2014, at 2:09 AM, Martin v. Löwis  wrote:
> 
>>>
>>> * Should we make use of the potential breakage with 2.7.10
>>>   to introduce a new Windows compiler version for Python 2.7 ?
>>
>> Assuming it is a good idea to continue producing Windows binaries
>> for 2.7, I think it would be a bad idea to switch compilers. It will
>> cause severe breakage of 2.7 installations, much more problematic
>> than switching to two-digit version numbers.
> 
> I agree with this, we’ve just finally started getting things to the point 
> where
> it makes a lot of sense for binary distributions for Windows. Breaking all
> of them on 2.7 would be very bad.

Not sure what you mean. We've had binary wininst distributions
for Windows for more than a decade, and egg and msi distributions
for 8 years :-)

But without access to the VS 2008 compiler that is needed to
compile those extensions, it will become increasingly difficult
for package authors to provide such binary packages, so we have to
ask ourselves:

What's worse: breaking old Windows binaries for Python 2.7
or not having updated and new Windows binaries for Python 2.7
at all in a few years ?

Switching to a newer compiler will make things easier for everyone
and we'd see more binary packages for Windows again.

Given that Python 2.7 support was extended for another 5 years at the
recent Python Language Summit to 2020, we have to face this
breakage sooner or later anyway. Extended support for VS 2008
will end in 2018 (but then: Python developers usually don't have
extended support contracts with MS). Service pack support has already
ended in 2009.

Depending on how you see it, using such an old compiler also
poses security risks. The last security update for VS 2008 dates
back to 2011 (http://support.microsoft.com/kb/2538243).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 23 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-06-17: Released eGenix PyRun 2.0.0 ...   http://egenix.com/go58
2014-07-02: Python Meeting Duesseldorf ...  9 days to go
2014-07-21: EuroPython 2014, Berlin, Germany ...   28 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 patch levels turning two digit

[M.-A. Lemburg]

On 23.06.2014 22:20, Donald Stufft wrote:
> 
> On Jun 23, 2014, at 3:27 PM, M.-A. Lemburg  wrote:
> 
>> On 23.06.2014 18:09, Donald Stufft wrote:
>>>
>>> On Jun 23, 2014, at 2:09 AM, Martin v. Löwis  wrote:
>>>
>>>>>
>>>>> * Should we make use of the potential breakage with 2.7.10
>>>>>  to introduce a new Windows compiler version for Python 2.7 ?
>>>>
>>>> Assuming it is a good idea to continue producing Windows binaries
>>>> for 2.7, I think it would be a bad idea to switch compilers. It will
>>>> cause severe breakage of 2.7 installations, much more problematic
>>>> than switching to two-digit version numbers.
>>>
>>> I agree with this, we’ve just finally started getting things to the point 
>>> where
>>> it makes a lot of sense for binary distributions for Windows. Breaking all
>>> of them on 2.7 would be very bad.
> 
> Err, sorry that “We” was with my pip hat on.
> 
>>
>> Not sure what you mean. We've had binary wininst distributions
>> for Windows for more than a decade, and egg and msi distributions
>> for 8 years :-)
> 
> Nonetheless, changing the compiler will not only break pip, but every
> automated installer tool (easy_install, buildout) that i’m aware of. The
> blow back for binary installation is going to be huge I think.
>
>> But without access to the VS 2008 compiler that is needed to
>> compile those extensions, it will become increasingly difficult
>> for package authors to provide such binary packages, so we have to
>> ask ourselves:
>>
>> What's worse: breaking old Windows binaries for Python 2.7
>> or not having updated and new Windows binaries for Python 2.7
>> at all in a few years ?
> 
> At the risk of getting Guido to post his slide again, I still think the
> solution to the old compiler is to just roll a 2.8 with minimal changes.
> It could even be a good place to move to the ssl backport changes
> too since they were the riskier set of changes in PEP466.
> 
> But either way, if a compiler does change in a 2.7 release we’ll need
> to update a lot of tooling to cope with that, so any plan to do that should
> include that and a timeline for adoption of that.

Sure, and we'd need to hash out possible solutions to minimize
breakage, but first we'll have to see whether we want to consider
this step or not.


BTW: It's strange that I'm arguing for breaking things. I'm usually
on the other side of such arguments :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] surrogatepass - she's a witch, burn 'er! [was: Cleaning up ...]

[M.-A. Lemburg]

On 29.08.2014 02:41, Stephen J. Turnbull wrote:
> In the process of booking up for my other post in this thread, I
> noticed the 'surrogatepass' handler.
> 
> Is there a real use case for the 'surrogatepass' error handler?  It
> seems like a horrible break in the abstraction.  IMHO, if there's a
> need, the application should handle this.  Python shouldn't provide
> it on encoding as the resulting streams are not Unicode conformant,
> nor on decoding UTF-16, as conversion of surrogate pairs is a
> requirement of all Unicode versions since about 1995.

This error handler allows applications to reactivate the Python 2
style behavior of the UTF codecs in Python 3, which allow reading
lone surrogates on input.

Since Python allows working with lone surrogates in Unicode (they
are valid code points) and we're using UTF-8 for marshal, we needed
a way to make sure that Python 3 also optionally supports working
with lone surrogates in such UTF-8 streams (nowadays called CESU-8:
http://en.wikipedia.org/wiki/CESU-8).

See

http://bugs.python.org/issue3672
http://bugs.python.org/issue12892

for discussions.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 29 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...21 days to go
2014-09-27: PyDDF Sprint 2014 ...  29 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] surrogatepass - she's a witch, burn 'er! [was: Cleaning up ...]

[M.-A. Lemburg]

On 29.08.2014 13:22, Isaac Morland wrote:
> On Fri, 29 Aug 2014, M.-A. Lemburg wrote:
> 
>> On 29.08.2014 02:41, Stephen J. Turnbull wrote:
>> Since Python allows working with lone surrogates in Unicode (they
>> are valid code points) and we're using UTF-8 for marshal, we needed
>> a way to make sure that Python 3 also optionally supports working
>> with lone surrogates in such UTF-8 streams (nowadays called CESU-8:
>> http://en.wikipedia.org/wiki/CESU-8).
> 
> If I want that wouldn't I specify "cesu-8" as the encoding?
> 
> i.e., instead of .decode ('utf-8') I would use .decode ('cesu-8').  Right 
> now, trying this I get
> that cesu-8 is an unknown encoding but that could be changed without 
> affecting the behaviour of the
> utf-8 codec.

Why write a new codec that's almost identical to the utf-8 codec,
if you can get the same functionality by explicitly using a
special error handler ?

>From a maintenance POV that does not sound like a good approach.

> It seems to me that .decode ('utf-8') should decode exactly and only valid 
> utf-8, including the
> non-use of surrogate pairs as an intermediate encoding step.

It does in Python 3.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 29 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...21 days to go
2014-09-27: PyDDF Sprint 2014 ...  29 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 29.08.2014 21:47, Alex Gaynor wrote:
> Hi all,
> 
> I've just submitted PEP 476, on enabling certificate validation by default for
> HTTPS clients in Python. Please have a look and let me know what you think.
> 
> PEP text follows.

Thanks for the PEP. I think this is generally a good idea,
but some important parts are missing from the PEP:

 * transition plan:

   I think starting with warnings in Python 3.5 and going
   for exceptions in 3.6 would make a good transition

   Going straight for exceptions in 3.5 is not in line with
   our normal procedures for backwards incompatible changes.

 * configuration:

   It would be good to be able to switch this on or off
   without having to change the code, e.g. via a command
   line switch and environment variable; perhaps even
   controlling whether or not to raise an exception or
   warning.

 * choice of trusted certificate:

   Instead of hard wiring using the system CA roots into
   Python it would be good to just make this default and
   permit the user to point Python to a different set of
   CA roots.

   This would enable using self signed certs more easily.
   Since these are often used for tests, demos and education,
   I think it's important to allow having more control of
   the trusted certs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 29 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...21 days to go
2014-09-27: PyDDF Sprint 2014 ...  29 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 29.08.2014 23:11, Donald Stufft wrote:
> 
> Sorry I was on my phone and didn’t get to fully reply to this.
> 
>> On Aug 29, 2014, at 4:00 PM, M.-A. Lemburg  wrote:
>>
>> On 29.08.2014 21:47, Alex Gaynor wrote:
>>> Hi all,
>>>
>>> I've just submitted PEP 476, on enabling certificate validation by default 
>>> for
>>> HTTPS clients in Python. Please have a look and let me know what you think.
>>>
>>> PEP text follows.
>>
>> Thanks for the PEP. I think this is generally a good idea,
>> but some important parts are missing from the PEP:
>>
>> * transition plan:
>>
>>   I think starting with warnings in Python 3.5 and going
>>   for exceptions in 3.6 would make a good transition
>>
>>   Going straight for exceptions in 3.5 is not in line with
>>   our normal procedures for backwards incompatible changes.
> 
> As far as a transition plan, I think that this is an important
> enough thing to have an accelerated process. If we need
> to provide a warning than let’s add it to the next 3.4 otherwise
> it’s going to be 2.5+ years until we stop being unsafe by
> default.

Fine with me; we're still early in the Python 3.4
patch level releases.

> Another problem with this is that I don’t think it’s actually
> possible to do. Python itself isn’t validating the TLS certificates,
> OpenSSL is doing that. To my knowledge OpenSSL doesn’t
> have a way to say “please validate these certificates and if
> they don’t validate go ahead and keep going and just let me
> get a warning from it”. It’s a 3 way switch, no validation, validation
> if a certificate is provided, and validation always.
> 
> Now that’s strictly for the “verify the certificate chain” portion,
> the hostname verification is done entirely on our end and we
> could do something there… but I’m not sure it makes sense
> to do so if we can’t do it for invalid certificates too.

OpenSSL provides a callback for certificate validation,
so it is possible to issue a warning and continue with
accepting the certificate.

>> * configuration:
>>
>>   It would be good to be able to switch this on or off
>>   without having to change the code, e.g. via a command
>>   line switch and environment variable; perhaps even
>>   controlling whether or not to raise an exception or
>>   warning.
> 
> I’m on the fence about this, if someone provides a certificate
> that we can validate against (which can be done without
> touching the code) then the only thing that really can’t be
> “fixed” without touching the code is if someone has a certificate
> that is otherwise invalid (expired, not yet valid, wrong hostname,
> etc). I’d say if I was voting on this particular thing I’d be -0, I’d
> rather it didn’t exist but I wouldn’t cry too much if it did.

If you're testing code or trying out some new stuff, you
don't want to get a valid cert first, but instead go ahead
with a self signed one. That's the use case.

>> * choice of trusted certificate:
>>
>>   Instead of hard wiring using the system CA roots into
>>   Python it would be good to just make this default and
>>   permit the user to point Python to a different set of
>>   CA roots.
>>
>>   This would enable using self signed certs more easily.
>>   Since these are often used for tests, demos and education,
>>   I think it's important to allow having more control of
>>   the trusted certs.
> 
> 
> Like my other email said, the Python API has everything needed
> to easily specify your own CA roots and/or disable the validations.
> The OpenSSL library also allows you to specify either a directory
> or a file to change the root certificates without code changes. The
> only real problems with the APIs are that the default is bad and
> an unrelated thing where you can’t pass in an in memory certificate.

Are you sure that's possible ? Python doesn't load the
openssl.cnf file and the SSL_CERT_FILE, SSL_CERT_DIR env
vars only work for the openssl command line binary, AFAIK.

In any case, Python will have to tap into the OS CA root
provider using special code and this code could easily be
made to check other dirs/files as well.

The point is that it should be possible to change this default
at the Python level, without needing application code changes.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 29 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
_

Re: [Python-Dev] surrogatepass - she's a witch, burn 'er! [was: Cleaning up ...]

[M.-A. Lemburg]

On 30.08.2014 01:37, Greg Ewing wrote:
> M.-A. Lemburg wrote:
>> we needed
>> a way to make sure that Python 3 also optionally supports working
>> with lone surrogates in such UTF-8 streams (nowadays called CESU-8:
>> http://en.wikipedia.org/wiki/CESU-8).
> 
> I don't think CESU-8 is the same thing. According to the wiki
> page, CESU-8 *requires* all code points above 0x to be split
> into surrogate pairs before encoding. It also doesn't say that
> lone surrogates are valid -- it doesn't mention lone surrogates
> at all, only pairs. Neither does the linked technical report.
> 
> The technical report also says that CESU-8 forbids any UTF-8
> sequences of more than three bytes, so it's definitely not
> "UTF-8 plus lone surrogates".

You're right, it's not the same as UTF-8 plus lone surrogates.

CESU-8 does encode surrogates as individual code points using
the UTF-8 encoding, which is what probably caused it to be
mentioned in discussions when talking about having UTF-8 streams
do the same for lone surrogates.

So let's call the encoding UTF-8-py so that everyone knows
what we're talking about :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 30 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...20 days to go
2014-09-27: PyDDF Sprint 2014 ...  28 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 30.08.2014 04:44, Alex Gaynor wrote:
> Thanks for the rapid feedback everyone!
> 
> I want to summarize the action items and discussion points that have come up 
> so
> far:
> 
> To add to the PEP:
> 
> * Emit a warning in 3.4.next for cases that would raise a Exception in 3.5
> * Clearly state that the existing OpenSSL environment variables will be
>   respected for setting the trust root

I'd also suggest to compile Python with OPENSSL_LOAD_CONF, since that
causes OpenSSL to read the global openssl.cnf file for additional
configuration.

> Discussion points:
> 
> * Disabling verification entirely externally to the program, through a CLI 
> flag
>   or environment variable. I'm pretty down on this idea, the problem you hit 
> is
>   that it's a pretty blunt instrument to swing, and it's almost impossible to
>   imagine it not hitting things it shouldn't; it's far too likely to be used 
> in
>   applications that make two sets of outbound connections: 1) to some internal
>   service which you want to disable verification on, and 2) some external
>   service which needs strong validation. A global flag causes the latter to
>   fail silently when subjected to a MITM attack, and that's exactly what we're
>   trying to avoid. It also makes things much harder for library authors: I
>   write an API client for some API, and make TLS connections to it. I want
>   those to be verified by default. I can't even rely on the httplib defaults,
>   because someone might disable them from the outside.

The reasoning here is the same as for hash randomization. There
are cases where you want to test your application using self-signed
certificates which don't validate against the system CA root list.

In those cases, you do know what you're doing. The test would fail
otherwise and the reason is not a bug in your code, it's just
the fact that the environment you're running it in is a test
environment.

Ideally, all applications should give you this choice, but this is
unlikely to happen, so it's good to be able to change the Python
default, since with the proposed change, most applications will
probably continue to use the Python defaults as they do now.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 30 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...20 days to go
2014-09-27: PyDDF Sprint 2014 ...  28 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 30.08.2014 12:40, Antoine Pitrou wrote:
> On Sat, 30 Aug 2014 12:19:11 +0200
> "M.-A. Lemburg"  wrote:
>>> To add to the PEP:
>>>
>>> * Emit a warning in 3.4.next for cases that would raise a Exception in 3.5
>>> * Clearly state that the existing OpenSSL environment variables will be
>>>   respected for setting the trust root
>>
>> I'd also suggest to compile Python with OPENSSL_LOAD_CONF, since that
>> causes OpenSSL to read the global openssl.cnf file for additional
>> configuration.
> 
> Python links against OpenSSL as a shared library, not statically. It's
> unlikely that setting a compile constant inside Python would affect
> OpenSSL at all.

The change is to the OpenSSL API, not the OpenSSL lib. By setting
the variable you enable a few special calls to the config loader
functions in OpenSSL when calling the initializer it:

https://www.openssl.org/docs/crypto/OPENSSL_config.html

>>> Discussion points:
>>>
>>> * Disabling verification entirely externally to the program, through a CLI 
>>> flag
>>>   or environment variable. I'm pretty down on this idea, the problem you 
>>> hit is
>>>   that it's a pretty blunt instrument to swing, and it's almost impossible 
>>> to
>>>   imagine it not hitting things it shouldn't; it's far too likely to be 
>>> used in
>>>   applications that make two sets of outbound connections: 1) to some 
>>> internal
>>>   service which you want to disable verification on, and 2) some external
>>>   service which needs strong validation. A global flag causes the latter to
>>>   fail silently when subjected to a MITM attack, and that's exactly what 
>>> we're
>>>   trying to avoid. It also makes things much harder for library authors: I
>>>   write an API client for some API, and make TLS connections to it. I want
>>>   those to be verified by default. I can't even rely on the httplib 
>>> defaults,
>>>   because someone might disable them from the outside.
>>
>> The reasoning here is the same as for hash randomization. There
>> are cases where you want to test your application using self-signed
>> certificates which don't validate against the system CA root list.
> 
> That use case should be served with the SSL_CERT_DIR and SSL_CERT_FILE
> env vars (or, better, by specific settings *inside* the application).
> 
> I'm against multiplying environment variables, as it makes it more
> difficult to assess the actual security of a setting. The danger of an
> ill-secure setting is much more severe than with hash randomization.

You have a point there. So how about just a python run-time switch
and no env var ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 30 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...20 days to go
2014-09-27: PyDDF Sprint 2014 ...  28 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 30.08.2014 12:55, Antoine Pitrou wrote:
> On Sat, 30 Aug 2014 12:46:47 +0200
> "M.-A. Lemburg"  wrote:
>> The change is to the OpenSSL API, not the OpenSSL lib. By setting
>> the variable you enable a few special calls to the config loader
>> functions in OpenSSL when calling the initializer it:
>>
>> https://www.openssl.org/docs/crypto/OPENSSL_config.html
> 
> Ah, ok. Do you have experience with openssl.cnf? Apparently, it is
> meant for offline tools such as certificate generation, I am not sure
> how it could impact certification validation.

I'm still exploring this: the OpenSSL documentation is, well,
less than complete on these things, so searching mailing lists
and reading source code appears to be the only reasonable way
to figure out what is possible and what not.

The openssl.cnf config file is indeed mostly used by the various
openssl subcommands (e.g. req and ca), but it can also be used to
configuring engines and my hope is that configuration of
e.g. default certificate stores also becomes possible.

One of the engines can tap into the Windows certificate store,
for example.

>>> That use case should be served with the SSL_CERT_DIR and SSL_CERT_FILE
>>> env vars (or, better, by specific settings *inside* the application).
>>>
>>> I'm against multiplying environment variables, as it makes it more
>>> difficult to assess the actual security of a setting. The danger of an
>>> ill-secure setting is much more severe than with hash randomization.
>>
>> You have a point there. So how about just a python run-time switch
>> and no env var ?
> 
> Well, why not, but does it have a value over letting the code properly
> configure their SSLContext?

Yes, because when Python changes the default to be validating
and more secure, application developers will do the same as
they do now: simply use the defaults ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 30 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...20 days to go
2014-09-27: PyDDF Sprint 2014 ...  28 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 30.08.2014 15:32, R. David Murray wrote:
> On Sat, 30 Aug 2014 14:03:57 +0200, "M.-A. Lemburg"  wrote:
>> On 30.08.2014 12:55, Antoine Pitrou wrote:
>>> On Sat, 30 Aug 2014 12:46:47 +0200
>>> "M.-A. Lemburg"  wrote:
>>>>> That use case should be served with the SSL_CERT_DIR and SSL_CERT_FILE
>>>>> env vars (or, better, by specific settings *inside* the application).
>>>>>
>>>>> I'm against multiplying environment variables, as it makes it more
>>>>> difficult to assess the actual security of a setting. The danger of an
>>>>> ill-secure setting is much more severe than with hash randomization.
>>>>
>>>> You have a point there. So how about just a python run-time switch
>>>> and no env var ?
>>>
>>> Well, why not, but does it have a value over letting the code properly
>>> configure their SSLContext?
>>
>> Yes, because when Python changes the default to be validating
>> and more secure, application developers will do the same as
>> they do now: simply use the defaults ;-)
> 
> But neither of those addresses the articulated use case: someone *using*
> a program implemented in python that does not itself provide a way to
> disable the new default security (because it is *new*).  Only an
> environment variable will do that.
> 
> Since the environment variable is opt-in, I think the "consenting
> adults" argument applies to Alex's demure about "multiple connections".
> It could still emit the warnings.

That would be a possibility as well, yes.

I'd just like to see a way to say: I know what I'm doing
and I'm not in the mood to configure my own CA list, so
please go ahead and accept whatever certs you find --
much like what --no-check-certificate does for wget.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 30 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...20 days to go
2014-09-27: PyDDF Sprint 2014 ...  28 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[M.-A. Lemburg]

On 01.09.2014 10:09, Nick Coghlan wrote:
> On 1 September 2014 17:13, Christian Heimes  wrote:
>> On 01.09.2014 08:44, Nick Coghlan wrote:
>>> Yes, it would have exactly the same security failure modes as
>>> sitecustomize, except it would only fire if the application
>>> imported the ssl module.
>>>
>>> The "-S" and "-I" switches would need to disable the implied
>>> "sslcustomize", just as they disable "import site".
>>
>> A malicious package can already play havoc with your installation with
>> a custom ssl module. If somebody is able to sneak in a ssl.py then you
>> are screwed anyway. sslcustomize is not going to make the situation worse.
> 
> That's not quite true - we're fairly careful about putting the
> standard library before userspace directories, so aside from the
> "current directory" problem, shadowing "ssl" itself can be tricky to
> arrange.

It's really easy to modify sys.modules to override any module
that has already been loaded or add new ones bypassing sys.path
entirely, so the sys.path layout doesn't provide any protection
against such hacks.

If you gain access to one of the dirs on sys.path, you can play
such tricks in sitecustomize.py. Any 3rd party package can do
the same.

We'd have to add digital API signatures to the ssl module to
prevent such stunts :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 08 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-08-27: Released eGenix PyRun 2.0.1 ...   http://egenix.com/go62
2014-09-19: PyCon UK 2014, Coventry, UK ...11 days to go
2014-09-27: PyDDF Sprint 2014 ...  19 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[M.-A. Lemburg]

On 24.09.2014 03:48, Nick Coghlan wrote:
> On 24 September 2014 03:05, Steve Dower  wrote:
>> Larry Hastings wrote:
>>>
>>> On 09/19/2014 03:31 PM, Barry Warsaw wrote:
>>> I think we need a Python 3.5 Release Schedule PEP.
>>>
>>> Just checked it in as PEP 478.  It should show up here in a few minutes:
>>> http://legacy.python.org/dev/peps/pep-0478/
>>>
>>> Key facts:
>>> . Beta 1 is May 24th 2015, about a month after the end of the PyCon US 2015 
>>> sprints.
>>> . Final release is September 13, 2015, just over a year from now.
>>>
>>> Comments?
>>
>> Martin is no longer producing the Windows installers - that task has been 
>> handed to me. I'm planning to have a rewritten installer (also in the same 
>> repo) that should be easier to modify and maintain, as well as being able to 
>> produce alternative packages (such as a Python 3.5 or stdlib merge module, 
>> for example), though that doesn't necessarily need to go into the PEP.
>>
>> I'm also considering/experimenting with installing into "Program Files" by 
>> default, but I suspect that isn't going to work out yet.
>>
>> I'd like to move the Windows versions onto the next release of VC (currently 
>> "VC 14" until the branding team figures out what to call it). There isn't a 
>> promised RTM date for VC 14 yet, so it looks like the best available 
>> compiler by Beta 1 will be a "Go Live" RC. (The "Go Live" marking basically 
>> means "we think this is ready for use, but expect a round of minor 
>> updates/fixes soon - the compiler is least likely to be updated, my guess is 
>> that it'll be Visual Studio UI mostly).
>>
>> I personally don't have any qualms about using the RC compiler for Beta 1, 
>> and Beta 2 will almost certainly use VC 14 RTM, but I know when I proposed 
>> this topic that some people were concerned about having the final version 
>> available for Python 3.5 Beta.
>>
>> So far I've been building regularly with internal versions of VC and haven't 
>> been hitting any major issues with Python (OpenSSL has some issues, but I've 
>> been filing bugs on both sides so they should be worked out soon enough). My 
>> work is at http://hg.python.org/sandbox/steve.dower (branch: VC14) for 
>> anyone interested.
>>
>> For the alphas, I'm contemplating producing two builds (VC 10 and VC 14), 
>> but I obviously want to settle on one or the other by Beta. Last time we 
>> discussed it, there was strong support for changing compiler, but I have a 
>> better idea of the timeline now and it's tighter than I thought...
>>
>> Thoughts, anyone?
> 
> It's ultimately up to Larry as RM, but I'd personally favour targeting
> the newer compiler and runtime, even with the slight risk of
> potentially needing to slip our schedule. There's also a fair amount
> of wiggle room between the first beta and the first release candidate.

I'd rather be conservative here and wait for another Python release
before switching VC versions. There are a few important questions
that need answers before we can consider a new VC version:

 * Will there be free versions available ?

 * Will those free editions include the 64-bit compilers ?

 * Will those free editions include the optimizing compilers ?

 * Is there a roadmap for how long these free versions will remain
   officially available ?

 * Are there issues compiling 3rd party libraries with it ?

   E.g. the numeric and science stacks, the web stacks,
   the deployment stacks, etc.

 * What license terms will the new version have ?

   E.g. GPL compatibility issues, weird exceptions,

 * What will the pricing structure look like ?

   While core devs will get free MSDN licenses, most other 3rd party
   providers will have to buy licenses for the compiler, unless
   they can use the free versions.

An alternative would be targeting VC13 instead of VC14, in case it
has good answers for the above questions. It's been around for a
year now, so there should be more experience available with this
version.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 24 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-09-27: PyDDF Sprint 2014 ...   3 days to go
2014-09-30: Python Meeting Duesseldorf ...  6 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
ht

Re: [Python-Dev] 3.5 release schedule PEP

[M.-A. Lemburg]

Thanks for the insights, Steve.

More below...

On 24.09.2014 18:52, Steve Dower wrote:
> M.-A. Lemburg wrote:
>>
>> I'd rather be conservative here and wait for another Python release before
>> switching VC versions. There are a few important questions that need answers
>> before we can consider a new VC version:
>>
>> * Will there be free versions available ?
>>
>> * Will those free editions include the 64-bit compilers ?
>>
>> * Will those free editions include the optimizing compilers ?
>>
>> * Is there a roadmap for how long these free versions will remain
>> officially available ?
>>
>> * Are there issues compiling 3rd party libraries with it ?
>>
>> E.g. the numeric and science stacks, the web stacks,
>> the deployment stacks, etc.
>>
>> * What license terms will the new version have ?
>>
>> E.g. GPL compatibility issues, weird exceptions,
>>
>> * What will the pricing structure look like ?
>>
>> While core devs will get free MSDN licenses, most other 3rd party
>> providers will have to buy licenses for the compiler, unless
>> they can use the free versions.
>>
>> An alternative would be targeting VC13 instead of VC14, in case it has good
>> answers for the above questions. It's been around for a year now, so there
>> should be more experience available with this version.
> 
> (Nit - it's actually VC12 a.k.a. "Visual Studio 2013" - VC13 was skipped. 
> This is what happens when you have separate engineering and marketing teams 
> :) )

Ah, ok :-)

> I don't have good answers to all of these yet, but none of them are going to 
> be any worse than for VC12. I've forwarded these questions to the people on 
> the VC team who do get to choose the answers, and while I'm not expecting to 
> hear specifics back from them, they are at least aware of the concerns and 
> how important their product is to our community.
> 
> There will be free versions available, but I don't know what format they'll 
> be in. Those free editions should include identical compilers to the paid 
> ones - the cases where that hasn't been true have been bugs or due to 
> assumptions that were proven to be incorrect.
> 
> The main improvement in this version is that all versions from VC14 should be 
> binary compatible, and so there will always be a free compiler, but it may be 
> VC15/16/etc. and not VC14.

That's good news.

> There are certainly issues with 3rd party libraries, largely because all 
> projects have a tendency to take dependencies on compiler/library internals. 
> OpenSSL, for example, redefines the stdout/in/err macros based on the VC 
> version, but the new definitions are no longer valid with VC14, and so they 
> are fixing that. Python itself has a few issues that I have already fixed in 
> my branch. There will certainly be other issues, but an advantage of starting 
> early is that bugs in the compiler itself can be fixed in the compiler.
> 
> The license should not change significantly from previous versions. GPL 
> incompatibilities are because the GPL wants to be incompatible with licenses 
> based on different ideologies - AFAIK there's never been anything in the VC 
> licenses preventing whatever redistribution license you like.

As example: there once was a special clause which explicitly
disallowed "Excluded License[s]" to be used together with
VC redistibutable files. I think this is no longer the case, but
there may be new things in the EULAs.

> Part of my improvements to /PCBuild will help avoid the need for Visual 
> Studio entirely, but the free versions should always be sufficient for 
> building and debugging. I have no insight or control over the pricing 
> structure.

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 24 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-09-27: PyDDF Sprint 2014 ...   3 days to go
2014-09-30: Python Meeting Duesseldorf ...  6 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of C compilers for Python on Windows

[M.-A. Lemburg]

On 10.10.2014 11:26, Larry Hastings wrote:
> 
> On 10/10/2014 08:07 AM, Paul Moore wrote:
>> On 10 October 2014 01:29, Victor Stinner  wrote:
>>> What about the Python stable ABI? Would it be broken if we use a
>>> different compiler?
>>>
>>> What about third party Python extensions?
>>>
>>> What about external dependencies like gzip, bz2, Tk, Tcl, OpenSSL, etc.?
>> The key point for me is that any supported build on Windows supports
>> the exact same ABI.
> 
> Just to make something clear that may not be clear to non-Windows developers: 
> the C library is
> implicitly part of the ABI.  So unless these other compilers bend over 
> backwards to generate code /
> have a C library that behaves *exactly* like MSVC, their ABI will be 
> different, and therefore shared
> libraries compiled with one compiler won't work with the next.  Heck, even 
> shared libraries compiled
> with one version of MSVC won't work with any other version!  (This is 
> something apparently being
> fixed by MSVC 15; apparently they are designing the ABI for forwards 
> compatibility.  Huzzah!)
> 
> So if CPython officially said "we support MSVC and Compiler X", I worry that 
> we'd have third-party
> modules compiled with either one or the other, leaving users unable to mix 
> and match third-party
> extensions as they do today.  ("I want to use library X, which is only 
> available compiled by MSVC. 
> I also want to use library Y, which is only available compiled by Compiler X. 
>  What should I do?"
> "... install Linux?")
> 
> 
> Here's my perspective.  Having your code compilable by more compilers is 
> good.  But a maze of
> #ifdefs is bad.  We still have #ifdef's for Borland C--I'd be very surprised 
> if anyone was compiling
> Python 3 with Borland C.  IMO the benefit from supporting other compilers on 
> Windows is negligible,
> but the costs in maintaining these other compilers is tangible.  Or, worse, 
> we accept changes to
> support these other compilers, but the support is incomplete, or goes 
> unmaintained and breaks (and
> nobody notices).
> 
> So as a practical matter I think I'd prefer if we continued to only support 
> MSVC.  In fact I'd
> prefer it if we removed support for other Windows compilers, instead asking 
> those maintainers to
> publish their own patches / repos, in the way that Stackless does.

I don't think this is special to the Windows platform. We already
do support quite a few compilers in CPython and for multiple
platforms, so keeping support for e.g. MinGW or adding Intel
C support wouldn't really make much difference in the overall
#ifdef picture ;-)

That said, We do need maintainers for this support, so if there
are no people willing to support these compilers in CPython,
we should use the external port hosting approach for these, IMO.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] XP buildbot problem cloning from hg.python.org

[M.-A. Lemburg]

On 26.10.2014 00:14, Ned Deily wrote:
> In article ,
>  David Bolen  wrote:
> 
>> David Bolen  writes:
>>
>>> which appears to die mid-stream while receiving the manifests.
>>>
>>> So I'm sort of hoping there might be some record server-side as to why
>>> things are falling apart mid-way.
>>
>> Just to follow-up to myself, I get the same same error trying to do a
>> clone from my own personal XP machine rather than the buildbot (which
>> is a VM).  I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2.
>>
>> However, the same clones completely successfully under OSX and Linux.
>>
>> So that's sort of strange.
> 
> Very interesting!  I had been doing some housekeeping on some of my 
> older OS X build systems over the past few days and I've run into the 
> same problem.  In particular, I am seeing this failure on an OS X 10.5.8 
> system (running in a Fusion VM) which I've used for years and from which 
> I have regularly cloned repos from hg.python.org.  I spent some time 
> yesterday trying to isolate it.  I came to the conclusion that it was 
> independent of the version of OpenSSL (identical failures occurred with 
> the system's ancient Apple 0.9.7 as well as a newly-build 1.0.1j) and 
> independent of the version of hg (at least with two data points, current 
> and a year-old version) and seemingly independent of the network 
> connection.  I was not able to reproduce the failure on the host OS X 
> system (10.10) and I didn't have problems a few days earlier with 
> various other OS X releases (10.6.x through 10.9.x) also running in VMs 
> on the same host.  I stumbled across a workaround for the problem as I 
> was experiencing it:  adding --uncompressed to hg clone eliminated 
> failures.  You can get more info on the hg failures by adding 
> --traceback and --debugger to the clone command.  After spending way too 
> much time on the issue, I was not in the mood to spend more time 
> isolating the problem after finding a workaround but if others are also 
> seeing it, it might be worth doing.  Sigh.
> 
>   $ hg --version
>   Mercurial Distributed SCM (version 3.1.2)
>   $ hg clone -U http://hg.python.org/cpython cpython
>   real URL is https://hg.python.org/cpython
>   requesting all changes
>   adding changesets
>   adding manifests
>   transaction abort!
>   rollback completed
>   abort: connection ended unexpectedly
>   $ hg clone --uncompressed -U https://hg.python.org/cpython cpython
>   streaming all changes
>   10404 files to transfer, 248 MB of data
>   transferred 248 MB in 44.4 seconds (5.58 MB/sec)

If compression is causing the problem, perhaps there's an incompatibility
with the use zlib version between the host and your client system.

hg.python.org was recently updated to a new Ubuntu version.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 26 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-10-24: Released eGenix pyOpenSSL 0.13.5 ...  http://egenix.com/go63

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (2.7): #22650: test suite: load Unicode test data files from www.pythontest.net

[M.-A. Lemburg]

On 06.11.2014 15:39, Brett Cannon wrote:
> What is pythontest.net? Is it something we control, and if so how do we add
> things to it for tests? Did I miss an email on python-dev or
> python-committers about this?

pythontest.net is a domain owned by the PSF and run by Donald
Stufft and Benjamin (I believe).

https://bugs.python.org/issue22650 has the details.

> On Thu Nov 06 2014 at 8:57:22 AM georg.brandl 
> wrote:
> 
>> https://hg.python.org/cpython/rev/0af36ea1d010
>> changeset:   93417:0af36ea1d010
>> branch:  2.7
>> parent:  93401:3e8d3c4bc17e
>> user:Georg Brandl 
>> date:Thu Nov 06 14:37:49 2014 +0100
>> summary:
>>   #22650: test suite: load Unicode test data files from www.pythontest.net

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 06 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-10-24: Released eGenix pyOpenSSL 0.13.5 ...  http://egenix.com/go63

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] hg.python.org cloning troubles after Sep 13 changes

[M.-A. Lemburg]

Just FYI (and for the archives), to perhaps save someone a few minutes:

I've been hitting a problem with hg pull and hg clone on a box recently
and after staring at it for a while, finally found the cause.

Here's what hg printed:

HG-Python/cpython> hg pull -u -b 2.7
abort: no suitable response from remote hg!

Not much help.

Using --debug:

orig/HG-Python> hg --debug clone ssh://h...@hg.python.org/cpython cpython-2.7
running ssh -C -q -l marc-andre.lemburg -i /home/lemburg/.ssh/id_rsa 
h...@hg.python.org "hg -R cpython
serve --stdio"
sending hello command
sending between command
abort: no suitable response from remote hg!

Still no help.

Then I remembered that the box was newly setup on Sept 13 and checked
Benjamins email. The server host key had also changed. After removing
the offending key and accepting the new one, things started working
again.

The hg on the box is 1.7.5, so newer version may include a better
error message.

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 07 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Redirection of ar.pycon.org

[M.-A. Lemburg]

Hi Facunda,

you should either write to webmas...@pycon.org, the
conference ML or me directly, since I'm managing these the
pycon.org subdomains.

> On Thu, Dec 18, 2014, at 10:59, Facundo Batista wrote:
>> Hi!
>>
>> Don't remember where to ask for changing the redirection of that
>> domain name. Somebody knows?
>>
>> I need for the redirection to be to pycon.python.org.ar (and we'll
>> take care of proper year-by-year redirection from there).

-- 
Marc-Andre Lemburg
Director
Python Software Foundation
http://www.python.org/psf/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compile Python on Windows (OpenSSL)

[M.-A. Lemburg]

On 13.01.2015 23:04, Victor Stinner wrote:
> Hi,
> 
> To compile Python on Windows, there are a few information in the
> Developer Guide:
> https://docs.python.org/devguide/setup.html#windows-compiling
> 
> Python 3.5 now requires Visual Studio 2010 *SP1*, or newer Visual Studio:
> http://bugs.python.org/issue22919#msg233637
> 
> I found PCbuild\readme.txt which is not mentionned in the devguide :-/
> https://hg.python.org/cpython/file/56f717235c45/PCbuild/readme.txt
> (at least not on the Windows section of the setup page)
> 
> I found some clues to build OpenSSL to be able to build the Python ssl
> module, but I still have issues.
> 
> Is there a more complete documentation?
> 
> I found how to install svn.exe, perl.exe and nasm.exe, but not how to
> install nmake.exe. I don't know the command to build OpenSSL.
> 
> I don't care of building OpenSSL, my goal is only to build the Python
> ssl module. Is there a way to install a development version of OpenSSL
> (.lib files if I remember correctly) from an installer/binary?

If you just need the compiled OpenSSL DLLs and .libs, you
can use the ones we ship with the source package of egenix-pyopenssl:

http://www.egenix.com/products/python/pyOpenSSL/#Download

The files are in openssl-win32 / openssl-win64.

Those dirs also have batch files which we use to build the
OpenSSL libs on Windows.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
2015-01-20: Python Meeting Duesseldorf ...http://egenix.com/go69

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compile Python on Windows (OpenSSL)

[M.-A. Lemburg]

On 13.01.2015 23:42, Brian Curtin wrote:
> On Tue, Jan 13, 2015 at 4:36 PM, Victor Stinner
>  wrote:
>> 2015-01-13 23:18 GMT+01:00 Steve Dower :
>>> Technically, Python 3.5 requires Visual Studio 2015
>>
>> For me, it's *very* difficult to find how to install Visual Studio.
>> There are many different websites and web pages which mention Visual
>> Studio with a lot of versions and "flavors" (Express, Community,
>> Ultimate, etc.).
>>
>> Visual Studio 2015 was not released yet :-/
>>
>> My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
>> 2008 Express and VS 2010 Express installed. I understood that
>> "Ultimate" includes a *lot* of things, not only a C compiler.
>>
>> I found a "free" Visual Studio which is in fact Visual Studio 2013
>> Community and I read that it's not free.
>>
>> I sent an email to Brian Curtin to ask to renew my MSDN account. He
>> didn't reply yet.
> 
> I saw that and will send it on, but it's still going to take some time
> to process - usually a week or so.
> 
> In the meantime, the first result searching for Visual Studio 2015
> came up with 
> http://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx,
> which seems to give you VS2015. I haven't tried to run it since I'm
> not on Windows at the moment, but it looks correct.

Just a note of caution: for older preview releases of VS the
only way to get back to a clean system was to reinstall
Windows.

I don't know whether this will be different with VS 2015,
but if you care for your VM, you should probably create
a snapshot before installing VS 2015 preview to make it
easy to revert back, e.g. to install the final VS 2015
version.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
2015-01-20: Python Meeting Duesseldorf ...http://egenix.com/go69

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compile Python on Windows (OpenSSL)

[M.-A. Lemburg]

On 13.01.2015 23:50, Victor Stinner wrote:
> 2015-01-13 23:46 GMT+01:00 M.-A. Lemburg :
>> Just a note of caution: for older preview releases of VS the
>> only way to get back to a clean system was to reinstall
>> Windows.
> 
> Does it mean that it's not possible to have VS 2008 and VS 2015
> installed at the same time?
> 
> VS 2008 is required to build Python 2.7.

Steve already answered this basically. It is well possible to
have multiple VS versions installed at the same time. Before
compiling you just need to run the right environment setup batch file
to prepare everything.

I was just referring to *preview* versions of VS. If you use
one of those, chances are that you'll have a hard time upgrading
to the final version of VS. My recommendation is not to use preview
versions on systems that you cannot easily revert to a state
before the preview install.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
2015-01-20: Python Meeting Duesseldorf ...http://egenix.com/go69

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can Python Be Built Without Distutils

[M.-A. Lemburg]

On 23.01.2015 19:48, Matthias Klose wrote:
> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>> Related to my earlier question regarding building Python on Android
>> and an undefined reference to dlopen error...I have the following
>> question:  Is it possible to build and install Python without having
>> to build and install...or use...distutils?
>>
>> Some background:
>> I can build the python interpreter on my device, and I can build a
>> bunch of modules.  The problem appears when make reaches the part
>> where setup.py is used to build and import modules...specifically when
>> setup.py attempts to import distutils.core.
> 
> you can do this using Setup.local. This works for me building additional
> extensions as builtins.  It might require some tweaking to build everything.

You may want to have a look at the Setup files we're using
in eGenix PyRun, which uses them to force static builds of the
various built-in extensions.

Look for these files:

PyRun/Runtime/Setup.PyRun-2.7
PyRun/Runtime/Setup.PyRun-3.4

in the source archives:

http://www.egenix.com/products/python/PyRun/

> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).

Why ? It's great for finding stuff on your system and configuring
everything without user intervention (well, most of the time :-)).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 23 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can Python Be Built Without Distutils

[M.-A. Lemburg]

On 23.01.2015 21:56, Cyd Haselton wrote:
> On Fri, Jan 23, 2015 at 1:19 PM, M.-A. Lemburg  wrote:
>> On 23.01.2015 19:48, Matthias Klose wrote:
>>> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>>> Related to my earlier question regarding building Python on Android
>>>> and an undefined reference to dlopen error...I have the following
>>>> question:  Is it possible to build and install Python without having
>>>> to build and install...or use...distutils?
>>>>
>>>> Some background:
>>>> I can build the python interpreter on my device, and I can build a
>>>> bunch of modules.  The problem appears when make reaches the part
>>>> where setup.py is used to build and import modules...specifically when
>>>> setup.py attempts to import distutils.core.
>>>
>>> you can do this using Setup.local. This works for me building additional
>>> extensions as builtins.  It might require some tweaking to build everything.
>>
>> You may want to have a look at the Setup files we're using
>> in eGenix PyRun, which uses them to force static builds of the
>> various built-in extensions.
>>
>> Look for these files:
>>
>> PyRun/Runtime/Setup.PyRun-2.7
>> PyRun/Runtime/Setup.PyRun-3.4
>>
>> in the source archives:
>>
>> http://www.egenix.com/products/python/PyRun/
> 
> Do you have a browseable git repo?

No, but we're thinking of mirroring our open-source packages on
Github or Bitbucket.

>>> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>>
>> Why ? It's great for finding stuff on your system and configuring
>> everything without user intervention (well, most of the time :-)).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 23 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can Python Be Built Without Distutils

[M.-A. Lemburg]

On 24.01.2015 21:23, Zachary Ware wrote:
> On Saturday, January 24, 2015, Brett Cannon  wrote:
> 
>> On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith > > wrote:
>>
>>> On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg >> > wrote:
>>>
>>>> On 23.01.2015 19:48, Matthias Klose wrote:
>>>>> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>>>>> Related to my earlier question regarding building Python on Android
>>>>>> and an undefined reference to dlopen error...I have the following
>>>>>> question:  Is it possible to build and install Python without having
>>>>>> to build and install...or use...distutils?
>>>>>>
>>>>>> Some background:
>>>>>> I can build the python interpreter on my device, and I can build a
>>>>>> bunch of modules.  The problem appears when make reaches the part
>>>>>> where setup.py is used to build and import modules...specifically when
>>>>>> setup.py attempts to import distutils.core.
>>>>>
>>>>> you can do this using Setup.local. This works for me building
>>>> additional
>>>>> extensions as builtins.  It might require some tweaking to build
>>>> everything.
>>>>
>>>> You may want to have a look at the Setup files we're using
>>>> in eGenix PyRun, which uses them to force static builds of the
>>>> various built-in extensions.
>>>>
>>>> Look for these files:
>>>>
>>>> PyRun/Runtime/Setup.PyRun-2.7
>>>> PyRun/Runtime/Setup.PyRun-3.4
>>>>
>>>> in the source archives:
>>>>
>>>> http://www.egenix.com/products/python/PyRun/
>>>>
>>>>> Otoh, I would like to get rid off the setup.py altogether (/me ducks
>>>> ...).
>>>>
>>>> Why ? It's great for finding stuff on your system and configuring
>>>> everything without user intervention (well, most of the time :-)).
>>>>
>>>
>>> Because our setup.py is a nightmare of arbitrary code run in a linear
>>> fashion with ad-hoc checks for things that are unlike how any other project
>>> on the planet determines what is available on your system.  It may have
>>> seemed "great" when it was created in 2001.  It really shows its age now.
>>>
>>> It defeats build parallelism and dependency declaration.
>>> It also prevents cross compilation.
>>>
>>> Building an interpreter with a limited standard library on your build
>>> host so that you can run said interpreter to have it drive the remainder of
>>> your build is way more self hosting that we rightfully need to be for
>>> CPython.
>>>
>>
>> So are you suggesting to add the build rules to configure and the Makefile
>> -- and Windows build file -- in order to drop setup.py?
>>
> 
> Windows already doesn't use setup.py. There are a lot more modules built-in
> on Windows, and others have their own project files; distutils isn't used
> at all.

The Windows installers comes with all dependencies included.

On other systems, this is not the case and so determining the
various settings is done using Python, which is a lot easier to
do than e.g. relying on autoconf finding everything and setting up
the correct compile options for each built-in.

All that said: if you can come up with a better system that's
both easy to maintain and provides the same level of user
friendliness, I'm sure this would be considered.

BTW: Parallel execution, cross compilation can be added
to setup.py. I don't think parallel execution is all
that important, but cross compilation would certainly be an
interesting feature to have (this currently has to be done
using the Setup.in approach) and an option to build the
modules statically linked to the interpreter would also be
nice.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 24 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Encoding of PyFrameObject members

[M.-A. Lemburg]

On 06.02.2015 00:27, Francis Giraldeau wrote:
> I need to access frame members from within a signal handler for tracing
> purpose. My first attempt to access co_filename was like this (omitting
>  error checking):
> 
> PyFrameObject *frame = PyEval_GetFrame();
> PyObject *ob = PyUnicode_AsUTF8String(frame->f_code->co_filename)
> char *str = PyBytes_AsString(ob)
> 
> However, the function PyUnicode_AsUTF8String() calls PyObject_Malloc(),
> which is not reentrant. If the signal handler nest over PyObject_Malloc(),
> it causes a segfault, and it could also deadlock.
> 
> Instead, I access members directly:
> char *str = PyUnicode_DATA(frame->f_code->co_filename);
> size_t len = PyUnicode_GET_DATA_SIZE(frame->f_code->co_filename);
> 
> Is it safe to assume that unicode objects co_filename and co_name are
> always UTF-8 data for loaded code? I looked at the PyTokenizer_FromString()
> and it seems to convert everything to UTF-8 upfront, and I would like to
> make sure this assumption is valid.

The macros won't work in all cases, as they don't pay attention
to the different kinds used in the Unicode implementation.

I don't think there's any API you can use to extract the
underlying data without going through PyObject_Malloc()
at some point (you may be lucky if there already is a
UTF-8 version available, but it's not guaranteed).

I guess your best bet is to write your own UTF-8
codec which then copies the data to a buffer that
you can control. Have a look at Objects/stringlib/codecs.h:
utf8_encode.

Alternatively, you can copy the data to a Py_UCS4 buffer
which you allocate using code such as this (untested,
adapted from the UTF-8 encoder):

Py_UCS4 *p;
enum PyUnicode_Kind repkind;
void *repdata;
Py_ssize_t repsize, k;

if (PyUnicode_READY(rep) < 0)
goto error;
repkind = PyUnicode_KIND(rep);
repdata = PyUnicode_DATA(rep);
repsize = PyUnicode_GET_LENGTH(rep);

p = malloc((repsize + 1) * sizeof(Py_UCS4));
for(k=0; k>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 03.04.2015 11:56, Larry Hastings wrote:
> My Windows development days are firmly behind me.  So I don't really have an 
> opinion here.  So I put
> it to you, Windows Python developers: do you care about GnuPG signatures on 
> Windows-specific files? 
> Or do you not care?

Regardless of target platform, I firmly believe we should (continue to)
GPG sign all distribution files as well as provide hash files/values
for them.

This is very useful to detect corrupted downloads or files which
were not created by the original packagers.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 03.04.2015 19:35, Steve Dower wrote:
>> My Windows development days are firmly behind me. So I don't really have an
>> opinion here. So I put it to you, Windows Python developers: do you care 
>> about
>> GnuPG signatures on Windows-specific files? Or do you not care?
> 
> The later replies seem to suggest that they are general goodness that nobody 
> on Windows will use. If someone convinces me (or steamrolls me, that's fine 
> too) that the goodness of GPG is better than a hash then I'll look into 
> adding it into the process. Otherwise I'll happily add hash generation into 
> the upload process (which I'm going to do anyway for the ones displayed on 
> the download page).

FWIW: I regularly check the GPG sigs on all important downloaded
files, regardless of which platform they target, including the
Windows installers for Python or any other Windows installers
I use which provide such sigs.

The reason is simple:
The signature is a proof of authenticity which is not bound to
a particular file format or platform and before running .exes
it's good to know that they were built by the right people and
not manipulated by trojans, viruses or malicious proxies.

Is that a good enough reason to continue providing the GPG
sigs or do you need more proof of goodness ? ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 04.04.2015 00:14, Steve Dower wrote:
> The thing is, that's exactly the same goodness as Authenticode gives, except 
> everyone gets that for free and meanwhile you're the only one who has 
> admitted to using GPG on Windows :)
> 
> Basically, what I want to hear is that GPG sigs provide significantly better 
> protection than hashes (and I can provide better than MD5 for all files if 
> it's useful), taking into consideration that (I assume) I'd have to obtain a 
> signing key for GPG and unless there's a CA involved like there is for 
> Authenticode, there's no existing trust in that key.

Hashes only provide checks against file corruption (and then
only if you can trust the hash values). GPG provides all the
benefits of public key encryption on arbitrary files (not just
code).

The main benefit in case of downloadable installers is to
be able to make sure that the files are authentic, meaning that
they were created and signed by the people listed as packagers.

There is no CA infrastructure involved as for SSL certificates
or Authenticode, but it's easy to get the keys from key servers
given the key signatures available from python.org's download
pages.

If you want to sign a package file using GPG, you will need
to create your own key, upload it to the key servers and then
place the signature up on the download page.

Relying only on Authenticode for Windows installers would
result in a break in technology w/r to the downloads we
make available for Python, since all other files are (usually)
GPG signed:

https://www.python.org/ftp/python/3.4.3/

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/


> Cheers,
> Steve
> 
> Top-posted from my Windows Phone
> 
> From: M.-A. Lemburg<mailto:m...@egenix.com>
> Sent: ‎4/‎3/‎2015 10:55
> To: Steve Dower<mailto:steve.do...@microsoft.com>; Larry 
> Hastings<mailto:la...@hastings.org>; Python 
> Dev<mailto:python-dev@python.org>; 
> python-committers<mailto:python-committ...@python.org>
> Subject: Re: [python-committers] [Python-Dev] Do we need to sign Windows 
> files with GnuPG?
> 
> On 03.04.2015 19:35, Steve Dower wrote:
>>> My Windows development days are firmly behind me. So I don't really have an
>>> opinion here. So I put it to you, Windows Python developers: do you care 
>>> about
>>> GnuPG signatures on Windows-specific files? Or do you not care?
>>
>> The later replies seem to suggest that they are general goodness that nobody 
>> on Windows will use. If someone convinces me (or steamrolls me, that's fine 
>> too) that the goodness of GPG is better than a hash then I'll look into 
>> adding it into the process. Otherwise I'll happily add hash generation into 
>> the upload process (which I'm going to do anyway for the ones displayed on 
>> the download page).
> 
> FWIW: I regularly check the GPG sigs on all important downloaded
> files, regardless of which platform they target, including the
> Windows installers for Python or any other Windows installers
> I use which provide such sigs.
> 
> The reason is simple:
> The signature is a proof of authenticity which is not bound to
> a particular file format or platform and before running .exes
> it's good to know that they were built by the right people and
> not manipulated by trojans, viruses or malicious proxies.
> 
> Is that a good enough reason to continue providing the GPG
> sigs or do you need more proof of goodness ? ;-)
> 
> --
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source
>>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
> 
> 
> ::: Try our new mxODBC.Connect Python Database Interface for free ! 
> 
> 
>eGenix.com Software, Ski

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 04.04.2015 02:49, Donald Stufft wrote:
> 
>> On Apr 3, 2015, at 6:38 PM, M.-A. Lemburg  wrote:
>>
>> On 04.04.2015 00:14, Steve Dower wrote:
>>> The thing is, that's exactly the same goodness as Authenticode gives, 
>>> except everyone gets that for free and meanwhile you're the only one who 
>>> has admitted to using GPG on Windows :)
>>>
>>> Basically, what I want to hear is that GPG sigs provide significantly 
>>> better protection than hashes (and I can provide better than MD5 for all 
>>> files if it's useful), taking into consideration that (I assume) I'd have 
>>> to obtain a signing key for GPG and unless there's a CA involved like there 
>>> is for Authenticode, there's no existing trust in that key.
>>
>> Hashes only provide checks against file corruption (and then
>> only if you can trust the hash values). GPG provides all the
>> benefits of public key encryption on arbitrary files (not just
>> code).
>>
>> The main benefit in case of downloadable installers is to
>> be able to make sure that the files are authentic, meaning that
>> they were created and signed by the people listed as packagers.
>>
>> There is no CA infrastructure involved as for SSL certificates
>> or Authenticode, but it's easy to get the keys from key servers
>> given the key signatures available from python.org's download
>> pages.
> 
> FTR if we’re relying on people to get the GPG keys from the download
> pages then there’s no additional benefit over just using a hash
> published on the same page.

Well, it's still better than just the hashes...

> In order to get additional benefit we’d need to get Steve’s key
> signed by enough people to get him into the strong set.

...but having the key signed by fellow core devs will certainly
add more goodness :-)

>> If you want to sign a package file using GPG, you will need
>> to create your own key, upload it to the key servers and then
>> place the signature up on the download page.
>>
>> Relying only on Authenticode for Windows installers would
>> result in a break in technology w/r to the downloads we
>> make available for Python, since all other files are (usually)
>> GPG signed:
>>
>> https://www.python.org/ftp/python/3.4.3/
>>
>> Cheers,
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source
>>>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
>> 
>>
>> ::: Try our new mxODBC.Connect Python Database Interface for free ! 
>>
>>
>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>   Registered at Amtsgericht Duesseldorf: HRB 46611
>>   http://www.egenix.com/company/contact/
>>
>>
>>> Cheers,
>>> Steve
>>>
>>> Top-posted from my Windows Phone
>>> 
>>> From: M.-A. Lemburg<mailto:m...@egenix.com>
>>> Sent: ‎4/‎3/‎2015 10:55
>>> To: Steve Dower<mailto:steve.do...@microsoft.com>; Larry 
>>> Hastings<mailto:la...@hastings.org>; Python 
>>> Dev<mailto:python-dev@python.org>; 
>>> python-committers<mailto:python-committ...@python.org>
>>> Subject: Re: [python-committers] [Python-Dev] Do we need to sign Windows 
>>> files with GnuPG?
>>>
>>> On 03.04.2015 19:35, Steve Dower wrote:
>>>>> My Windows development days are firmly behind me. So I don't really have 
>>>>> an
>>>>> opinion here. So I put it to you, Windows Python developers: do you care 
>>>>> about
>>>>> GnuPG signatures on Windows-specific files? Or do you not care?
>>>>
>>>> The later replies seem to suggest that they are general goodness that 
>>>> nobody on Windows will use. If someone convinces me (or steamrolls me, 
>>>> that's fine too) that the goodness of GPG is better than a hash then I'll 
>>>> look into adding it into the process. Otherwise I'll happily add hash 
>>>> generation into the upload process (which I'm going to do anyway for the 
>>>> ones displayed on the download p

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 04.04.2015 16:41, Steve Dower wrote:
> "Relying only on Authenticode for Windows installers would result in a break 
> in technology w/r to the downloads we make available for Python, since all 
> other files are (usually) GPG signed"
> 
> This is the point of this discussion. I'm willing to make such a break 
> because I believe Authenticode is so much more convenient for end users that 
> it isn't worth producing GPG signatures. So far, the responses seem to be:
> 
> "I'd use them on Windows" x1
> "I'd consider using them on another OS" x2-3
> "Please don't change" everyone else
> 
> At least that's the impression I'm getting, so I hope that helps clarify why 
> I'm still not convinced it's that critical.

Just to clarify:

I have absolutely nothing against using Authenticode on Windows :-)

I'm only trying to convince you that *additionally* providing GPG
sigs for Windows downloads is a good thing and we should not stop
doing this, since it makes verification of downloaded files
easier. It's not hard to do, can be automated and provides additional
security which can be verified on any platform, not only Windows.

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 04.04.2015 21:02, Kurt B. Kaiser wrote:
> For the record, that is a Symantec/Verisign code signing certificate. We
> paid $1123 for it last April.  It expires April 2017.
> 
> If you don't switch to a different vendor, e.g. startssl, please contact
> me for renewal in 2017.

FWIW: The PSF mostly uses StartSSL nowadays and they also support
code signing certificates. Given that this option is a lot cheaper
than Verisign, I think we should switch, unless there are
significant reasons not to. We should revisit this in 2017.

> KBK
> 
> On Sat, Apr 4, 2015, at 10:35 AM, Steve Dower wrote:
>> Small clarification: there certificates *are* the same format as for SSL,
>> and OpenSSL it's able to validate them in the same way as well as
>> generate them (but not extract embedded ones, AFAICT). But generally SSL
>> certificates are not marked as suitable for code signing so you need to
>> buy a separate one.
>>
>> Both Martin and I have the PSF's code signing cert private key, which is
>> how we can sign with the "Python Software Foundation" name. The public
>> key is embedded into every signed file, just like an SSL cert is attached
>> to a site or an S/MIME cert is embedded in a signed email.
>>
>> Cheers,
>> Steve
>>
>> Top-posted from my Windows Phone
>> ________
>> From: Steve Dower<mailto:steve.do...@microsoft.com>
>> Sent: ‎4/‎4/‎2015 7:25
>> To: Wes Turner<mailto:wes.tur...@gmail.com>; M. -A.
>> Lemburg<mailto:m...@egenix.com>
>> Cc: python-committers<mailto:python-committ...@python.org>;
>> Python-Dev<mailto:python-dev@python.org>
>> Subject: Re: [python-committers] [Python-Dev] Do we need to sign Windows
>> files with GnuPG?
>>
>> "Authenticode does not have a PKI"
>>
>> If you got that from this discussion, I need everyone to at least skim
>> read this: 
>> https://msdn.microsoft.com/en-us/library/ie/ms537361(v=vs.85).aspx
>>
>> Authenticode uses the same certificate infrastructure as SSL (note: not
>> the same certificates). As I see it, anyone running on Windows has access
>> to verification that is at least as good as GPG, and the only people who
>> would benefit from GPG sigs are those checking Windows files on another
>> OS or those with an existing GPG workflow on Windows (before this thread,
>> I knew nobody who used GPG on Windows for anything, so forgive me for
>> thinking this is very rare).
>>
>> Cheers,
>> Steve
>>
>> Top-posted from my Windows Phone
>> 
>> From: Wes Turner<mailto:wes.tur...@gmail.com>
>> Sent: ‎4/‎4/‎2015 6:42
>> To: M. -A. Lemburg<mailto:m...@egenix.com>
>> Cc: Python-Dev<mailto:python-dev@python.org>;
>> python-committers<mailto:python-committ...@python.org>; Larry
>> Hastings<mailto:la...@hastings.org>; Steve
>> Dower<mailto:steve.do...@microsoft.com>
>> Subject: Re: [Python-Dev] [python-committers] Do we need to sign Windows
>> files with GnuPG?
>>
>>
>> So, AFAIU from this discussion:
>>
>> * Authenticode does not have a PKI
>> * GPG does have PKI
>> * ASC signatures are signed checksums
>>
>> As far as downstream packaging on Windows (people who should/could be
>> subscribed to release ANNs):
>>
>> For Choclatey NuGet:
>>
>> * https://chocolatey.org/packages/python
>> * https://chocolatey.org/packages/python.x86
>> * https://chocolatey.org/packages/python2
>> * https://chocolatey.org/packages/python-x86_32
>> * https://chocolatey.org/packages/python3
>>
>> Python(x,y):
>>
>> * https://code.google.com/p/pythonxy/
>>
>> For Anaconda (the MS Azure chosen python distribution):
>>
>> * http://docs.continuum.io/anaconda/install.html#windows-install
>>
>> ...
>>
>> These should/could/are checking GPG signatures for Windows packages
>> downstream.
>>
>> http://www.scipy.org/install.html
>>
>> On Apr 3, 2015 5:38 PM, "M.-A. Lemburg"
>> mailto:m...@egenix.com>> wrote:
>> On 04.04.2015 00:14, Steve Dower wrote:
>>> The thing is, that's exactly the same goodness as Authenticode gives, 
>>> except everyone gets that for free and meanwhile you're the only one who 
>>> has admitted to using GPG on Windows :)
>>>
>>> Basically, what I want to hear is that GPG sigs provide significantly 
>>> better protection than hashes (and I can provide better than MD5 for all 
>>> files if it's 

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 04.04.2015 21:49, Kurt B. Kaiser wrote:
> 
> 
> On Sat, Apr 4, 2015, at 03:35 PM, M.-A. Lemburg wrote:
>> On 04.04.2015 21:02, Kurt B. Kaiser wrote:
>>> For the record, that is a Symantec/Verisign code signing
>>> certificate. We paid $1123 for it last April.  It expires
>>> April 2017.
>>>
>>> If you don't switch to a different vendor, e.g. startssl, please
>>> contact me for renewal in 2017.
>>
>> FWIW: The PSF mostly uses StartSSL nowadays and they also support code
>> signing certificates. Given that this option is a lot cheaper than
>> Verisign, I think we should switch, unless there are significant
>> reasons not to. We should revisit this in 2017.
> 
> Agree - apparently the starlssl process for getting a signing cert is
> complex/obscure, so we should start early.

Not really. Once you have the org verification it's really easy.

> Let me know if I can help providing PSF organization verification.

I already completed that for the current cycle.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 16.04.2015 21:34, "Martin v. Löwis" wrote:
> Am 04.04.15 um 21:54 schrieb M.-A. Lemburg:
>>>> FWIW: The PSF mostly uses StartSSL nowadays and they also support code
>>>> signing certificates. Given that this option is a lot cheaper than
>>>> Verisign, I think we should switch, unless there are significant
>>>> reasons not to. We should revisit this in 2017.
>>>
>>> Agree - apparently the starlssl process for getting a signing cert is
>>> complex/obscure, so we should start early.
>>
>> Not really. Once you have the org verification it's really easy.
>>
>>> Let me know if I can help providing PSF organization verification.
>>
>> I already completed that for the current cycle.
>>
> 
> I had asked the PSF for a StartSSL certificate when the previous
> certificate expired, and the PSF was not able to provide one. After
> waiting several weeks for the PSF to provide the certificate, Kurt then
> kindly went to Verisign.

When was that ? I never received such a request. The account
I'm using was created in Dec 2014 and the validation received
on 2014-12-17. This is valid for about a year:

https://wiki.python.org/psf/PSF%20SSL%20Certificates

Code signing certificates are valid for two years, so switching
to StartSSL probably doesn't make much sense now, unless perhaps
we want to switch to SHA2 and longer RSA keys (if that's possible for
code signing certs - I'd have to check).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 17 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[M.-A. Lemburg]

On 17.04.2015 19:31, "Martin v. Löwis" wrote:
> Am 17.04.15 um 00:46 schrieb M.-A. Lemburg:
>>> I had asked the PSF for a StartSSL certificate when the previous
>>> certificate expired, and the PSF was not able to provide one. After
>>> waiting several weeks for the PSF to provide the certificate, Kurt then
>>> kindly went to Verisign.
>>
>> When was that ? I never received such a request. 
> 
> I sent the request to Jesse Noller, Noah Kantrowitz and Kurt Kaiser on
> 2014-03-17. On 2014-04-15, Jesse indicated that he had given up.

I guess that explains why nothing happened. Jesse owned the StartSSL
account before I took over in Dec last year.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 17 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] typeshed for 3rd party packages (was: Type hints -- a mediocre programmer's reaction)

[M.-A. Lemburg]

On 21.04.2015 05:37, Guido van Rossum wrote:
> On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich  wrote:
>> * Uploading stubs for other people's code is a terrible idea. Who do I
>> contact when I update the interface to my library? The random Joe who
>> "helped" by uploading annotations three months ago and then quit the
>> internet? I don't even want to think about people maliciously adding stubs
>> to PyPI.
>>
> 
> We're certainly not planning to let arbitrary people upload stubs for
> arbitrary code to PyPI that will automatically be used by the type
> checkers. (We can't stop people from publishing their stubs, just as you
> can't stop people from writing blog posts or stackoverflow answers with
> examples for your library.)
> 
> The actual plan is to have a common repository of stubs (a prototype exists
> at https://github.com/JukkaL/typeshed) but we also plan some kind of
> submission review. I've proposed that when submitting stubs for a package
> you don't own, the typeshed owners ask the package owner what their
> position is, and we expect the answers to fall on the following spectrum:
> 
> - I don't want stubs uploaded for my package
> - I'll write the stubs myself
> - I want to review all stubs that are uploaded for my package before they
> are accepted
> - Please go ahead and add stubs for my package and let me know when they're
> ready
> - Go ahead, I trust you
> 
> This seems a reasonable due diligence policy that avoids the scenarios
> you're worried about. (Of course if you refuse stubs a black market for
> stubs might spring into existence. That sounds kind of exciting... :-)

Hmm, that's the first time I've heard about this. I agree with
Jack that it's a terrible idea to allow this for 3rd party
packages.

If people want to contribute stubs, they should contribute them
to the package in the usual ways, not in a side channel. The important
part missing in the above setup is maintenance and to some extent
an external change of the API definitions.

Both require active participation in the package project,
not the separated setup proposed above, to be effective and
actually work out in the long run.

For the stdlib, typesched looks like a nice idea to spread the
workload.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 21 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] typeshed for 3rd party packages

[M.-A. Lemburg]

On 21.04.2015 18:08, Guido van Rossum wrote:
> On Tue, Apr 21, 2015 at 12:33 AM, M.-A. Lemburg  wrote:
> 
>> On 21.04.2015 05:37, Guido van Rossum wrote:
>>> On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich 
>> wrote:
>>>> * Uploading stubs for other people's code is a terrible idea. Who do I
>>>> contact when I update the interface to my library? The random Joe who
>>>> "helped" by uploading annotations three months ago and then quit the
>>>> internet? I don't even want to think about people maliciously adding
>> stubs
>>>> to PyPI.
>>>>
>>>
>>> We're certainly not planning to let arbitrary people upload stubs for
>>> arbitrary code to PyPI that will automatically be used by the type
>>> checkers. (We can't stop people from publishing their stubs, just as you
>>> can't stop people from writing blog posts or stackoverflow answers with
>>> examples for your library.)
>>>
>>> The actual plan is to have a common repository of stubs (a prototype
>> exists
>>> at https://github.com/JukkaL/typeshed) but we also plan some kind of
>>> submission review. I've proposed that when submitting stubs for a package
>>> you don't own, the typeshed owners ask the package owner what their
>>> position is, and we expect the answers to fall on the following spectrum:
>>>
>>> - I don't want stubs uploaded for my package
>>> - I'll write the stubs myself
>>> - I want to review all stubs that are uploaded for my package before they
>>> are accepted
>>> - Please go ahead and add stubs for my package and let me know when
>> they're
>>> ready
>>> - Go ahead, I trust you
>>>
>>> This seems a reasonable due diligence policy that avoids the scenarios
>>> you're worried about. (Of course if you refuse stubs a black market for
>>> stubs might spring into existence. That sounds kind of exciting... :-)
>>
>> Hmm, that's the first time I've heard about this. I agree with
>> Jack that it's a terrible idea to allow this for 3rd party
>> packages.
>>
>> If people want to contribute stubs, they should contribute them
>> to the package in the usual ways, not in a side channel. The important
>> part missing in the above setup is maintenance and to some extent
>> an external change of the API definitions.
>>
>> Both require active participation in the package project,
>> not the separated setup proposed above, to be effective and
>> actually work out in the long run.
>>
>> For the stdlib, typeshed looks like a nice idea to spread the
>> workload.
>>
> 
> I hesitate to speak for others, but IIUC the reason why typeshed was
> started is that companies like PyCharm and Google (and maybe others) are
> *already* creating their own stubs for 3rd party packages, because they
> have a need to type-check code that *uses* 3rd party packages. Their use
> cases are otherwise quite different (the user code type-checked by PyCharm
> is that of PyCharm users, and the code type-checked by Google is their own
> proprietary code) but they both find themselves needing stubs for commonly
> used 3rd party packages. mypy finds itself in a similar position.
> 
> Think of it this way. Suppose you wrote an app that downloaded some files
> from the web using the popular Requests package. Now suppose you wanted to
> run mypy over your app. You're willing to do the work of adding signatures
> to your own app, and presumably there are stubs for those parts of the
> stdlib that you're using, but without stubs for Requests, mypy won't do a
> very good job type-checking your calls into Requests. It's not rocket
> science to come up with stubs for Requests (there aren't that many classes
> and methods) but the Requests package is in maintenance mode, and while
> they respond quickly to security issues they might take their time to
> release a new version that includes your stub files, and until there are a
> lot of people clamoring for stubs for Requests, they might not care at all.

For projects in maintenance mode, it does make sense indeed.

For active ones, I think you'd quickly run into a situation similar
to translation projects: there are always parts which haven't been
translated yet or where the translation no longer matches the original
intent.

Unlike with translations, where missing or poor ones don't have
much effect on the usefulness of the software, a type checker
would complain loudly and probably show lots of false positives
(if you read a type bug as "

Re: [Python-Dev] Clarification of PEP 476 "opting out" section

[M.-A. Lemburg]

On 30.04.2015 02:33, Nick Coghlan wrote:
> Hi folks,
> 
> This is just a note to highlight the fact that I tweaked the "Opting
> out" section in PEP 476 based on various discussions I've had over the
> past few months: https://hg.python.org/peps/rev/dfd96ee9d6a8
> 
> The notable changes:
> 
> * the example monkeypatching code handles AttributeError when looking
> up "ssl._create_unverified_context", in order to accommodate older
> versions of Python that don't have PEP 476 implemented
> * new paragraph making it clearer that while the intended use case for
> the monkeypatching trick is as a workaround to handle environments
> where you *know* HTTPS certificate verification won't work properly
> (including explicit references to sitecustomize.py and Standard
> Operating Environments for Python), there's also a secondary use case
> in allowing applications to provide a system administrator controlled
> setting to globally disable certificate verification (hence the change
> to the example code)
> * new paragraph making it explicit that even though we've improved
> Python's default behaviour, particularly security sensitive
> applications should still provide their own context rather than
> relying on the defaults

Can we please make the monkeypatch a regular part of Python's
site.py which can enabled via an environment variable, say
export PYTHONHTTPSVERIFY=0.

See http://bugs.python.org/issue23857 for the discussion.

Esp. for Python 2.7.9 the default verification from PEP 476
is causing problems for admins who want to upgrade their
Python installation without breaking applications using
Python. They need an easy and official non-hackish way to
opt-out from the PEP 476 default on a per application basis.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 30 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var (was: Clarification of PEP 476 "opting out" section)

[M.-A. Lemburg]

On 07.05.2015 04:30, Nick Coghlan wrote:
>> Can we please make the monkeypatch a regular part of Python's
>> site.py which can enabled via an environment variable, say
>> export PYTHONHTTPSVERIFY=0.
>>
>> See http://bugs.python.org/issue23857 for the discussion.
> ...
> I actually do think it would be good to have such a feature as a
> native part of Python 2.7 in order to provide a nicer "revert to the
> pre-PEP-476 behaviour" experience for Python 2.7 users (leaving the
> "there's no easy way to turn HTTPS certificate verification off
> globally" state of affairs to Python 3), but I don't currently have
> the time available to push for that against the "end users can't be
> trusted not to turn certificate verification off when they should be
> fixing their certificate management instead" perspective.

We're currently working on a new release of eGenix PyRun and this
will include Python 2.7.9.

We do want to add such an env switch to disable the cert verification,
so would like to know whether we can use PYTHONHTTPSVERIFY for this
or not.

We mainly need this to reenable simple use of self-signed certificates
which 2.7.9 disables.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 08 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 08.05.2015 11:36, Nick Coghlan wrote:
> On 8 May 2015 6:52 pm, "M.-A. Lemburg"  wrote:
>>
>> On 07.05.2015 04:30, Nick Coghlan wrote:
>>>> Can we please make the monkeypatch a regular part of Python's
>>>> site.py which can enabled via an environment variable, say
>>>> export PYTHONHTTPSVERIFY=0.
>>>>
>>>> See http://bugs.python.org/issue23857 for the discussion.
>>> ...
>>> I actually do think it would be good to have such a feature as a
>>> native part of Python 2.7 in order to provide a nicer "revert to the
>>> pre-PEP-476 behaviour" experience for Python 2.7 users (leaving the
>>> "there's no easy way to turn HTTPS certificate verification off
>>> globally" state of affairs to Python 3), but I don't currently have
>>> the time available to push for that against the "end users can't be
>>> trusted not to turn certificate verification off when they should be
>>> fixing their certificate management instead" perspective.
>>
>> We're currently working on a new release of eGenix PyRun and this
>> will include Python 2.7.9.
>>
>> We do want to add such an env switch to disable the cert verification,
>> so would like to know whether we can use PYTHONHTTPSVERIFY for this
>> or not.
> 
> That's a slightly misleading quotation of my post, as I'm opposed to the
> use of an environment variable for this, due to the fact that using the
> "-E" switch will then revert to the upstream default behaviour of verifying
> certificates, rather defeating the point of introducing the legacy
> infrastructure compatibility feature in the first place.

Oh, sorry. I read your email implying that you are fine with
the env var approach.

I don't really see the issue with -E, though. It's well possible
to internally set PYTHONHTTPSVERIFY=0 when -E is used to regain
backwards compatibility per default for Python 2.7.

Regarding the config file approach and letting distributions
set their own defaults:

I don't think it's a good idea to have one distribution
default to verifying HTTPS certs via a global config file
and another distribution do the opposite.

Python itself should define the defaults to be used, not
the distributions.

The Python Linux distribution is too complex already due to the
many different ways Python is installed on the systems.

Not having to deal with this complexity was the main motivation
for us to create eGenix PyRun, since it works the same on
all Linux platforms and doesn't use any of the system
wide installed Python interpreters, settings or packages
(unless you tell it to).

> A new informational PEP akin to PEP 394 that defines a config file location
> & contents for downstream redistributors that need a smoother transition
> plan for PEP 476 will let us handle this in a consistent way across
> redistributors that's also compatible with runtime use of the -E switch.

Regardless of whether a global config file is a good idea or not,
I don't think we can wait with 2.7.10 until a whole new PEP process
has run through.

> Cheers,
> Nick.
> 
>>
>> We mainly need this to reenable simple use of self-signed certificates
>> which 2.7.9 disables.
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, May 08 2015)
>>>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
>> 
>>
>> : Try our mxODBC.Connect Python Database Interface for free ! ::
>>
>>eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>Registered at Amtsgericht Duesseldorf: HRB 46611
>>http://www.egenix.com/company/contact/
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 08 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 09.05.2015 02:29, Nick Coghlan wrote:
> On 8 May 2015 8:14 pm, "M.-A. Lemburg"  wrote:
>>
>> On 08.05.2015 11:36, Nick Coghlan wrote:
>>> On 8 May 2015 6:52 pm, "M.-A. Lemburg"  wrote:
>>>>
>>>> On 07.05.2015 04:30, Nick Coghlan wrote:
>>>>>> Can we please make the monkeypatch a regular part of Python's
>>>>>> site.py which can enabled via an environment variable, say
>>>>>> export PYTHONHTTPSVERIFY=0.
>>>>>>
>>>>>> See http://bugs.python.org/issue23857 for the discussion.
>>>>> ...
>>>>> I actually do think it would be good to have such a feature as a
>>>>> native part of Python 2.7 in order to provide a nicer "revert to the
>>>>> pre-PEP-476 behaviour" experience for Python 2.7 users (leaving the
>>>>> "there's no easy way to turn HTTPS certificate verification off
>>>>> globally" state of affairs to Python 3), but I don't currently have
>>>>> the time available to push for that against the "end users can't be
>>>>> trusted not to turn certificate verification off when they should be
>>>>> fixing their certificate management instead" perspective.
>>>>
>>>> We're currently working on a new release of eGenix PyRun and this
>>>> will include Python 2.7.9.
>>>>
>>>> We do want to add such an env switch to disable the cert verification,
>>>> so would like to know whether we can use PYTHONHTTPSVERIFY for this
>>>> or not.
>>>
>>> That's a slightly misleading quotation of my post, as I'm opposed to the
>>> use of an environment variable for this, due to the fact that using the
>>> "-E" switch will then revert to the upstream default behaviour of
> verifying
>>> certificates, rather defeating the point of introducing the legacy
>>> infrastructure compatibility feature in the first place.
>>
>> Oh, sorry. I read your email implying that you are fine with
>> the env var approach.
>>
>> I don't really see the issue with -E, though. It's well possible
>> to internally set PYTHONHTTPSVERIFY=0 when -E is used to regain
>> backwards compatibility per default for Python 2.7.
>>
>> Regarding the config file approach and letting distributions
>> set their own defaults:
>>
>> I don't think it's a good idea to have one distribution
>> default to verifying HTTPS certs via a global config file
>> and another distribution do the opposite.
>>
>> Python itself should define the defaults to be used, not
>> the distributions.
> 
> As a result of the discussions around both PEP 466 and 476, I'm now firmly
> of the view that it's correct for the upstream Python community to assume
> the public internet as its standard operating environment when it comes to
> network security settings, and for those of us working for commercial
> redistributors to subsequently bear the cost of adapting from that upstream
> assumption to the different assumptions that may apply in the context of
> organisational intranets.
> 
> That's also why I've come around to the view that an informational PEP with
> recommendations for redistributors, rather than an actual change to Python
> 2.7, is the right answer (at least initially) when it comes to providing a
> smoother transition plan for PEP 476 - the folks saying "it's a bad idea to
> make this easy to turn off" are *right* from the perspective of operating
> over the public internet, or with well designed internal SSL/TLS
> certificate management, it's just also a *necessary* idea (in my view) to
> help CIOs and other infrastructure leads responsibly and effectively manage
> the wide range of risks associated with internal infrastructure upgrades.

I don't agree. We've broken the contract that people had with Python 2.7
by introducing a major breakage in a patch level release very far
down the line in 2.7.9, without providing an easy and official
way to opt-out that does not involve hacking your installation.

IMO, we should not fall for the somewhat arrogant view that
we know better than all the Python users out there when it
comes to running secure systems.

By providing a way to intentionally switch off the new default,
we do make people aware of the risks and that's good enough,
while still maintaining the contract people rightly expect of
patch level releases of Python.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Sou

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 10.05.2015 05:04, Robert Collins wrote:
> On 10 May 2015 at 11:44, Chris Angelico  wrote:
>> On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg  wrote:
>>> By providing a way to intentionally switch off the new default,
>>> we do make people aware of the risks and that's good enough,
>>> while still maintaining the contract people rightly expect of
>>> patch level releases of Python.
>>
>> Just as long as it's the sysadmin, and NOT some random attacker over
>> the internet, who has the power to downgrade security. Environment
>> variables can be attacked in various ways.
> 
> They can, and the bash fun was very good evidence of that.
> 
> OTOH if someones environment is at risk, PATH and PYTHONPATH are
> already very effective attack vectors.

If an attacker has access to the process environment, you're doomed
anyway, so that's not really an argument for or against using
environment variables :-)

You'd just need to create a file os.py and point PYTHONPATH at it.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 11:13, Nick Coghlan wrote:
> On 11 May 2015 at 18:04, M.-A. Lemburg  wrote:
>> On 10.05.2015 05:04, Robert Collins wrote:
>>> On 10 May 2015 at 11:44, Chris Angelico  wrote:
>>>> On Sun, May 10, 2015 at 4:13 AM, M.-A. Lemburg  wrote:
>>>>> By providing a way to intentionally switch off the new default,
>>>>> we do make people aware of the risks and that's good enough,
>>>>> while still maintaining the contract people rightly expect of
>>>>> patch level releases of Python.
>>>>
>>>> Just as long as it's the sysadmin, and NOT some random attacker over
>>>> the internet, who has the power to downgrade security. Environment
>>>> variables can be attacked in various ways.
>>>
>>> They can, and the bash fun was very good evidence of that.
>>>
>>> OTOH if someones environment is at risk, PATH and PYTHONPATH are
>>> already very effective attack vectors.
>>
>> If an attacker has access to the process environment, you're doomed
>> anyway, so that's not really an argument for or against using
>> environment variables :-)
> 
> The core issue lies in managing the "user" vs "administrator"
> permissions split. Even for self-administered systems, it's
> recommended practice to run *without* administrative permissions
> normally, and only elevate when you need them. One of the things
> you're watching out for in such cases is that it shouldn't be possible
> for an attacker to make a change to the user environment, and have
> that propagate to have an effect on a process running with
> administrative access. One of the recommended hardening measures
> against that kind of attack vector is to *turn off* Python's
> environment variable processing when launching Python processes with
> administrative access.

The env var would not be read at Python startup time, only
when loading the ssl module, so the -E switch would not have
the effect of disabling it - unlike the hash seed logic, which
is run (and has to be run) at Python startup time.

> We didn't care about that in the hash randomisation case, as the
> compatibility concern there applied on a per application basis, and
> caring about hash order was technically a bug in its own right. By
> contrast, in the situation we're worried about for certificate
> verification compatibility, the issue is environmental: certificate
> management in many private intranets isn't yet to the same standard as
> that on the public internet, so administrators may have a valid reason
> for defaulting Python back to the old behaviour, and redistributors
> may feel obliged to provide an opt-in period prior to switching the
> default behaviour to opt-out. Having the new setting be ignored in
> Python processes run under a hardened configuration means that an
> environment variable based solution won't have the desired effect in
> providing that smoother migration path to the more hardened
> configuration.
> 
> I've now written a draft "recommendations to redistributors" PEP for
> Robert's configuration file based design:
> https://www.python.org/dev/peps/pep-0493/ (exact file names & config
> setting names TBD)

The Fastly cache seems to be having problems again. I only get:
503 Backend is unhealthy - Details: cache-fra1225-FRA 1431335851 2631441948

> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
> feature, but agreement amongst redistributors on using a file-based
> approach is the main outcome I'm looking for.

Can't we have both ?

I don't think that we can wait for a whole PEP process to
run through to fix this regression in 2.7.9.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 12:47, Nick Coghlan wrote:
> On 11 May 2015 at 20:23, Donald Stufft  wrote:
>> On May 11, 2015, at 6:15 AM, Nick Coghlan  wrote:
>>> We made the decision when PEP 476 was accepted that this change turned
>>> a silent security failure into a noisy one, rather than being a
>>> regression in its own right. PEP 493 isn't about disagreeing with that
>>> decision, it's about providing a smoother upgrade path in contexts
>>> where letting the security failure remain silent is deemed to be
>>> preferred in the near term.
>>
>> I don't really agree that the decision to disable TLS is an environment one,
>> it's really a per application decision. This is why I was against having some
>> sort of global off switch for all of Python because just because one
>> application needs it turned off doesn't mean you want it turned off for 
>> another
>> Python application.
> 
> The scenario I'm interested in is the one where it *was* off globally
> (i.e. you were already running Python 2.7.8 or earlier) and you want
> to manage a global rollout of a new Python version that supports being
> configured to verify HTTPS certificates by default, while making the
> decision on whether or not to enable HTTPS certificate verification on
> a server-by-server basis, rather than having that decision be coupled
> directly to the rollout of the updated version of Python.

I guess we're approaching this from different perspectives :-)

I'm mostly interested in having a switch that can be set on
a per application basis, not globally.

I think the global default is fine and I'm just looking for a way to have
admins disable it on a case-by-case basis for those applications which
have problems with the new default. Hence the env var approach
- the admin would simply edit the application's startup shell
script, add the env var and that's it.

For pip et al. which don't use the ssl module, the admins can simply
continue using older versions for those applications - ones which
don't implement the extra verification. In many cases, this is not
necessary, since production environments typically don't use PyPI
at all: they use a local directory with the needed distribution
files, which is both more secure and reliable.

> I agree that the desired end state is where Python 3 is, and where
> upstream Python 2.7.9+ is, this is solely about how to facilitate
> folks getting from point A to point B without an intervening window of
> "I broke the world and now my boss is yelling at me about it" :)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 11.05.2015 12:15, Nick Coghlan wrote:
> On 11 May 2015 at 19:22, M.-A. Lemburg  wrote:
>> On 11.05.2015 11:13, Nick Coghlan wrote:
>>> I wouldn't be opposed to seeing that as an upstream Python 2.7.x
>>> feature, but agreement amongst redistributors on using a file-based
>>> approach is the main outcome I'm looking for.
>>
>> Can't we have both ?
> 
> I'd strongly advise against it, as we're deliberately increasing the
> attack surface area here by providing a potential path to carry out a
> downgrade attack on the HTTPS certificate verification by forcing it
> back to the old behaviour.
> 
> The main existing environment variable based attack vector would be to
> manage to run a process with administrative privileges and
> SSL_CERT_DIR and/or SSL_CERT_FILE pointing to a certificate written to
> a user or world-writable directory by the attacker. Providing a "don't
> verify HTTPS" flag at the interpreter level would let an attacker skip
> the first step of writing the certificate file to disk somewhere,
> making a system compromise harder to detect. (An especially paranoid
> SSL implementation would disallow reading certs from locations that
> allow write access to non-administrative users, but I don't believe
> OpenSSL is that paranoid)

Correct. OpenSSL will happily read the cert files from anywhere
you point it to.

> By contrast, the configuration file shouldn't provide a new attack
> vector (or simplify any existing attack vector), as if you have the
> permissions needed to modify the config file, you likely also have the
> permissions needed to modify the system certificate store, and the
> latter is a *far* more interesting attack vector than a downgrade
> attack solely against Python.
> 
> Thus the environment variable based off switch is neither necessary
> (as an administrator controlled configuration file can do the job),
> nor sufficient (it can't handle the -E switch), *and* it represents an
> increase in the attack surface area relative to a Python
> implementation without the capability.

Whether or not -E will have an effect on the env var depends
on the implementation. At the moment, -E only has an effect
on the C runtime, while the stdlib happily reads from os.environ
without taking the flag into account.

As proposed, the PYTHONHTTPSVERIFY would only affect the ssl
module and only be checked when loading this module, i.e. not
at Python startup time.

>> I don't think that we can wait for a whole PEP process to
>> run through
> 
> Matrix multiplication made it through the PEP process in 8 days. If we
> do this as a redistributor recommendation rather than attempting to
> get it into upstream Python 2.7, we could potentially propose you take
> on the role of BDFL-Delegate and mark it as Accepted as soon as the
> two of us agree on a common approach.
> 
> The reason I think that's a reasonable way forward is because we
> already know there are folks opposed to making the change upstream. If
> the PEP just provides recommendations for redistributors that *do*
> decide to provide a "global off switch" to revert to the old
> behaviour, then the perspective of the folks opposed to the feature is
> respected by the fact that this is a feature some redistributors *may*
> choose to add to provide a smoother migration path to more secure
> default HTTPS handling, rather than something upstream provides by
> default.
> 
> I assume the Debian, Ubuntu and Suse folks won't care, as they have
> all already decided against backporting the change to their long term
> support releases where the compatibility break would pose a problem
> (and I can certainly sympathise with that perspective given the
> dependency on backporting the PEP 466 SSL changes first, and the work
> involved in seeking consensus on a smoother migration path from the
> old default to the new one).
> 
> It would be nice to hear from ActiveState, Enthought & Continuum
> Analytics as well, but if they don't chime in here, I don't see any
> particular need to go chasing them explicitly.

I think the approach to only consider a subset of redistributors
as viable targets for such a switch is a bit too narrow.

You are leaving out all the parties which use custom
Python installations to run their applications, e.g.
the Plone and Zope community, the ZenOSS community,
the many Windows applications built on Python, etc.

>> to fix this regression in 2.7.9.
> 
> We made the decision when PEP 476 was accepted that this change turned
> a silent security failure into a noisy one, rather than being a
> regression in its own right. PEP 493 isn't about disagreeing with that
> decision, it's about 

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 12.05.2015 05:03, Nick Coghlan wrote:
> On 12 May 2015 at 04:49, M.-A. Lemburg  wrote:
>> On 11.05.2015 12:15, Nick Coghlan wrote:
>>> By contrast, the configuration file shouldn't provide a new attack
>>> vector (or simplify any existing attack vector), as if you have the
>>> permissions needed to modify the config file, you likely also have the
>>> permissions needed to modify the system certificate store, and the
>>> latter is a *far* more interesting attack vector than a downgrade
>>> attack solely against Python.
>>>
>>> Thus the environment variable based off switch is neither necessary
>>> (as an administrator controlled configuration file can do the job),
>>> nor sufficient (it can't handle the -E switch), *and* it represents an
>>> increase in the attack surface area relative to a Python
>>> implementation without the capability.
>>
>> Whether or not -E will have an effect on the env var depends
>> on the implementation. At the moment, -E only has an effect
>> on the C runtime, while the stdlib happily reads from os.environ
>> without taking the flag into account.
> 
> I had an off-list discussion with Christian Heimes about that in
> relation to the OpenSSL flags, and he pointed out the reason -E
> specifically needs to be a command line switch is because that is the
> only way to affect how environment variables are processed during
> interpreter startup. Once an application is up and running, further
> environment variable sanitisation can be handled at an application
> level by whitelisting entries in os.environ and deleting everything
> else.
> 
>> As proposed, the PYTHONHTTPSVERIFY would only affect the ssl
>> module and only be checked when loading this module, i.e. not
>> at Python startup time.
> 
> Right, the same is true for the configuration file proposal.
> 
>>> It would be nice to hear from ActiveState, Enthought & Continuum
>>> Analytics as well, but if they don't chime in here, I don't see any
>>> particular need to go chasing them explicitly.
>>
>> I think the approach to only consider a subset of redistributors
>> as viable targets for such a switch is a bit too narrow.
>>
>> You are leaving out all the parties which use custom
>> Python installations to run their applications, e.g.
>> the Plone and Zope community, the ZenOSS community,
>> the many Windows applications built on Python, etc.
> 
> No, they already have a solution: monkeypatch (or just plain patch)
> the SSL module. That's an upstream supported technique, which is why
> it's documented in the PEP.

Again, this is not a proper solution for your friendly
sys admin. They won't go in and patch Python, but rather
consider it broken, if it doesn't provide a configuration
option and simply stay with 2.7.8.

We've had that discussion already, so I won't go into details
again.

> The problem we (as in Red Hat) ran into was that that technique
> *doesn't work* for the case of backporting PEP 476 to Python 2.7.5 as
> an opt-in feature.

And neither does it work for people deploying Windows apps
built on Python (where the code is usually hidden away
in a ZIP archive or even compiled into a DLL), or people
deploying Plone or ZenOSS.

It's not only Red Hat's customers that are affected :-)

>>>> to fix this regression in 2.7.9.
>>>
>>> We made the decision when PEP 476 was accepted that this change turned
>>> a silent security failure into a noisy one, rather than being a
>>> regression in its own right. PEP 493 isn't about disagreeing with that
>>> decision, it's about providing a smoother upgrade path in contexts
>>> where letting the security failure remain silent is deemed to be
>>> preferred in the near term.
>>
>> The change wasn't regression. The missing downgrade path
>> is a regression.
> 
> It's a shame we don't have "-X" options in Python 2, as that would be
> a nice hard-to-attack option (although it wouldn't play well with
> subprocesses)
> 
>> Some other comments on PEP 493:
>>
>> * I don't think we really want to add the overhead of
>>   having to parse an INI file every time Python starts up.
>>   Please remember that we just parsing of the sysconfig
>>   data not long ago because we wanted to avoid this startup
>>   time.
> 
> Compared to the overhead of reading from the system cert database,
> reading a config file at ssl module import time should be trivial.

The cert database is only read when importing the ssl
module, not with each Python st

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 12.05.2015 12:04, Donald Stufft wrote:
> 
>> On May 12, 2015, at 3:57 AM, M.-A. Lemburg  wrote:
>>
>> In a user based installation (which most applications shipping
>> their own Python installation are), you can always do this
>> provided you can gain the application user permissions.
> 
> Of course, if the application is shipping it’s own Python then
> it has to actually do something to update to 2.7.9 and it can
> add it’s own option to disable TLS verification. I personally
> think that the application providing that option is the *right* way
> and all these other things are, at best, just temporary shims until
> the applications do that.

I still believe that requiring to monkeypatch Python is a very poor
approach in terms of quality software design. We can do better and
we should.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 12 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 12.05.2015 13:19, Nick Coghlan wrote:
> On 12 May 2015 at 21:17, Nick Coghlan  wrote:
>> Both of those make sense to me as cases where the environment variable
>> based security downgrade approach is the "least bad" answer available,
>> which is why I eventually agreed it should be one of the
>> recommendations in the PEP.
> 
> It occurs to me that the subtitle of PEP 493 could be "All software is
> terrible, but it's often a system administrator's job to make it run
> anyway" :)

+1 :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 12 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PYTHONHTTPSVERIFY env var

[M.-A. Lemburg]

On 12.05.2015 13:21, Donald Stufft wrote:
> 
>> On May 12, 2015, at 7:17 AM, Nick Coghlan  wrote:
>>
>> On 12 May 2015 at 21:09, Donald Stufft  wrote:
>>> If you control the app you don't need to do that. All relevant api accept 
>>> the context parameter. The shims are only useful when you don't control the 
>>> app. So an app shipping their own python doesn't fall under that.
>>
>> I think the "bundled Python" scenario MAL is interested in is this one:
>>
>> 1. An application with a bundled CPython runtime is using the
>> verification defaults
>> 2. Upgraded the bundled Python to 2.7.9
>> 3. Didn't provide new configuration settings to disable certificate 
>> verification
>> 4. Is being upgraded in an environment where verifying certificates
>> makes the app unusable for environmental reasons related to
>> certificate management
>>
>> The PyRun single-file Python interpreter has a similar need, where
>> some apps than ran fine under 2.7.8 will need a way to disable cert
>> verification in 2.7.9+ on a per-application basis, *without* modifying
>> the applications.
>>
>> Both of those make sense to me as cases where the environment variable
>> based security downgrade approach is the "least bad" answer available,
>> which is why I eventually agreed it should be one of the
>> recommendations in the PEP.
>>
> 
> Why is without modifying the app a reasonable goal? If Python is bundled
> with the app then you have direct control over when that upgrade happens,
> so you can delay the upgrade to 2.7.9 until your application which is
> bundling Python has the relevant switches. This is distinctly different
> from a situation like downstream distributors where the version of Python
> being provided is being provided by a group different than the person
> providing the application.

Take a Plone Intranet as example:

The unified installer downloads and installs Python 2.7 for you.
As of Plone 4.3.3 the version is Python 2.7.6.

Now say you are a sys admin and your Intranet users are affected by
some bug in 2.7.6 which is fixed in 2.7.9. The natural approach would
be to upgrade the bundled Python to 2.7.9.

Because it's an Intranet and Plone is used to aggregate
information from other systems which use self-signed
certificates, you don't want to risk breaking your Plone
installation and need a way to disable the cert checks.

The best way to do this is by configuring the bundled
Python to disable the checks, since you would not want
to mess with the Plone application itself.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 12 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

[M.-A. Lemburg]

On 18.11.2017 01:01, Victor Stinner wrote:
> Hi,
> 
> The CPython internals evolved during Python 3.7 cycle. I would like to
> know if we broke the C API or not.
> 
> Nick Coghlan and Eric Snow are working on cleaning up the Python
> initialization with the "on going" PEP 432:
> https://www.python.org/dev/peps/pep-0432/
> 
> Many global variables used by the "Python runtime" were move to a new
> single "_PyRuntime" variable (big structure made of sub-structures).
> See Include/internal/pystate.h.
> 
> A side effect of moving variables from random files into header files
> is that it's not more possible to fully initialize _PyRuntime at
> "compilation time". For example, previously, it was possible to refer
> to local C function (functions declared with "static", so only visible
> in the current file). Now a new "initialization function" is required
> to must be called.
> 
> In short, it means that using the "Python runtime" before it's
> initialized by _PyRuntime_Initialize() is now likely to crash. For
> example, calling PyMem_RawMalloc(), before calling
> _PyRuntime_Initialize(), now calls the function NULL: dereference a
> NULL pointer, and so immediately crash with a segmentation fault.

To prevent a complete crash, would it be possible to initialize
the struct entries to a generic function (or set of such functions
with the right signatures), which then issue a message to stderr
hinting to the missing call to _PyRuntime_Initialize()
before terminating ?

> I'm writing this email to ask if this change is an issue or not to
> embedded Python and the Python C API. Is it still possible to call
> "all" functions of the C API before calling Py_Initialize()?
> 
> I was bitten by the bug while reworking the Py_Main() function to
> split it into subfunctions and cleanup the code to handle the command
> line arguments and environment variables. I fixed the issue in main()
> by calling _PyRuntime_Initialize() as soon as possible: it's now the
> first instruction of main() :-) (See Programs/python.c)
> 
> To give a more concrete example: Py_DecodeLocale() is the recommanded
> function to decode bytes from the operating system, but this function
> calls PyMem_RawMalloc() which does crash before
> _PyRuntime_Initialize() is called. Is Py_DecodeLocale() used to
> initialize Python?
> 
> For example, "void Py_SetProgramName(wchar_t *);" expects a text
> string, whereas main() gives argv as bytes. Calling
> Py_SetProgramName() from argv requires to decode bytes... So use
> Py_DecodeLocale()...
> 
> Should we do something in Py_DecodeLocale()? Maybe crash if
> _PyRuntime_Initialize() wasn't called yet?
> 
> Maybe, the minimum change is to expose _PyRuntime_Initialize() in the
> public C API?
> 
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 23 2017)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python startup time

[M.-A. Lemburg]

On 14.05.2018 18:26, Chris Barker via Python-Dev wrote:
> 
> 
> On Fri, May 11, 2018 at 11:05 AM, Ryan Gonzalez  > wrote:
> 
>  https://refi64.com/uprocd/ 
> 
> 
> very cool -- but *nix only, of course :-(
> 
> But it seems that there is a demand for this sort of thing, and a few
> major projects are rolling their own. So maybe it makes sense to put
> something into the standard library that everyone could contribute to
> and use.
> 
> With regard to forking -- is there another way? I don't have the
> expertise to have any idea if this is possible, but:
> 
> start up python
> 
> capture the entire runtime image as a single binary blob.
> 
> could that blob be simply loaded into memory and run?
> 
> (hmm -- probably not -- memory addresses would be hard-coded then, yes?)
> or is memory virtualized enough these days?

You might want to look into combining this with PyRun:

https://www.egenix.com/products/python/PyRun/

which takes care of mmap'ing the byte code of the stdlib into
memory.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Microsoft to acquire GitHub for $7.5 billion

[M.-A. Lemburg]

Something that may change is the way they treat Github
accounts, after all, MS is very much a sales driven company.

But then there's always the possibility to move to Gitlab
as alternative (hosted or run on PSF VMs), so I would
worry too much.

Do note, however, that the value in Github is not so much with
the products they have, but with the data. Their databases
know more about IT developer than anyone else and given
that Github is put under the AI umbrella in MS should tell
us something :-)


On 04.06.2018 19:02, Antoine Pitrou wrote:
> 
> That's true, but Microsoft has a lot of stakes in the ecosystem.
> For example, since it has its own CI service that it tries to promote
> (VSTS), is it in Microsoft's best interest to polish and improve
> integrations with other CI services?
> 
> Regards
> 
> Antoine.
> 
> 
> On Mon, 4 Jun 2018 09:06:28 -0700
> Guido van Rossum  wrote:
>> On Mon, Jun 4, 2018 at 8:40 AM, Antoine Pitrou  wrote:
>>
>>>
>>> On Mon, 4 Jun 2018 17:03:27 +0200
>>> Victor Stinner  wrote:  

 At this point, I have no opinion about the event :-) I just guess that
 it should make GitHub more sustainable since Microsoft is a big
 company with money and interest in GitHub. I'm also confident that
 nothing will change soon. IMHO there is no need to worry about
 anything.  
>>>
>>> It does spell uncertainty on the long term.  While there is no need to
>>> worry for now, I think it gives a different colour to the debate about
>>> moving issues to Github.
>>>  
>>
>> I don't see how this *increases* the uncertainty. Surely if GitHub had
>> remained independent there would have been be similar concerns about how it
>> would make enough money to stay in business.
>>
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 05 2018)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Computed Goto dispatch for Python 2

[M.-A. Lemburg]

On 28.05.2015 02:17, Parasa, Srinivas Vamsi wrote:
> Hi All,
> 
> This is Vamsi from Server Scripting Languages Optimization team at Intel 
> Corporation.
> 
> Would like to submit a request to enable the computed goto based dispatch in 
> Python 2.x (which happens to be enabled by default in Python 3 given its 
> performance benefits on a wide range of workloads). We talked about this 
> patch with Guido and he encouraged us to submit a request on Python-dev 
> (email conversation with Guido shown at the bottom of this email).

+1.

It's been successful for Python 3, doesn't change semantics and
increases performance.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 28 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Single-file Python executables (was: Computed Goto dispatch for Python 2)

[M.-A. Lemburg]

You might want to have a look at eGenix PyRun, which gives you
an almost complete Python runtime in 4-13MB (depending on what
startup performance needs you have):

http://www.egenix.com/products/python/PyRun/

On 28.05.2015 17:58, Barry Warsaw wrote:
> On May 28, 2015, at 11:39 AM, Donald Stufft wrote:
> 
>> You don’t need a "fully functioning Python" for a single file binary, you
>> only need enough to actually run your application. For example, if you're
>> making an application that can download files over HTTP, you don't need to
>> include parts of the stdlib like xmlrpc, pickle, shelve, marshall, sqlite,
>> csv, email, mailcap, mailbox, imaplib, nntplib, etc.
> 
> There are actually two related but different use cases to "single file
> executables".
> 
> The first is nicely solved by tools like pex, where you don't need to include
> a fully functional Python at the head of the zip file because the environment
> you're deploying it into will have enough Python to make the zip work.  This
> can certainly result in smaller zip files.  This is the approach I took with
> Snappy Ubuntu Core support for Python 3, based on the current situation that
> the atomic upgrade client is written in Python 3.  If that changes and Python
> 3 is removed from the image, then this approach won't work.
> 
> pex (and others) does a great job at this, so unless there are things better
> refactored into upstream Python, I don't think we need to do much here.
> 
> The second use case is as you describe: put a complete functional Python
> environment at the head of the zip file so you don't need anything in the
> target deployment environment.  "Complete" can easily mean the entire stdlib,
> and although that would usually be more bloated than you normally need, hey,
> it's just some extra unused bits so who cares? .  I think this would be
> an excellent starting point which can be optimized to trim unnecessary bits
> later, maybe by third party tools.

See above. This is what eGenix PyRun provides.

Our main motivation is to have a binary which works on all
Unix platforms, without having to rely on the way too many
system dependent Python distribution (with all their quirks
and whistles ;-)).

On Windows, we use py2exe at the moment, but a port of PyRun
to Windows would be possible as well. You'd still need the
separate Python DLL, though in order to stay compatible to
C extensions which link against this.

As for application packaging: we don't have a need to put
everything into a single ZIP file or even concatenate such
a ZIP file to PyRun (which is possible: just add sys.executable to
sys.path to import from the executable).

We have plans to create a tool to make such packaging possible,
though, since it's handy to have for building small executable
apps, e.g. to drive installations or larger applications.

>> Of course deciding which pieces you include in the zip file you're appending
>> to the end of Python is up to whatever tool builds this executable which
>> doesn't need to be part of Python itself. If Python itself gained the ability
>> to operate in that manner than third party tools could handle trying to do
>> the optimizations where it only includes the things it actually needs in the
>> stdlib and excludes things it doesn't. The key thing here is that since
>> you're doing a single file binary, you don't need to have a Python which is
>> suitable to execute random Python code, you only need one that is suitable to
>> execute this particular code so you can specialize what that includes.
> 
> I'd love to see Python itself gain such a tool, but if it had the critical
> pieces to execute in this way, that would enable a common approach to
> supporting this in third party tools, on a variety of platforms.
> 
> I do think single-file executables are an important piece to Python's
> long-term competitiveness.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 28 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.

[M.-A. Lemburg]

On 01.06.2015 12:44, Armin Rigo wrote:
> Hi Larry,
> 
> On 31 May 2015 at 01:20, Larry Hastings  wrote:
>> p.s. Supporting this patch also helps cut into PyPy's reported performance
>> lead--that is, if they ever upgrade speed.pypy.org from comparing against
>> Python *2.7.2*.
> 
> Right, we should do this upgrade when 2.7.11 is out.
> 
> There is some irony in your comment which seems to imply "PyPy is
> cheating by comparing with an old Python 2.7.2": it is inside a thread
> which started because "we didn't backport performance improvements to
> 2.7.x so far".
> 
> Just to convince myself, I just ran a performance comparison.  I ran
> the same benchmark suite as speed.pypy.org, with 2.7.2 against 2.7.10,
> both freshly compiled with no "configure" options at all.  The
> differences are usually in the noise, but range from +5% to... -60%.
> If anything, this seems to show that CPython should take more care
> about performance regressions.  If someone is interested:
> 
> * "raytrace-simple" is 1.19 times slower
> * "bm_mako" is 1.29 times slower
> * "spitfire_cstringio" is 1.60 times slower
> * a number of other benchmarks are around 1.08.
> 
> The "7.0x faster" number on speed.pypy.org would be significantly
> *higher* if we upgraded the baseline to 2.7.10 now.

If someone were to volunteer to set up and run speed.python.org,
I think we could add some additional focus on performance
regressions. Right now, we don't have any way of reliably
and reproducibly testing Python performance.

Hint: The PSF would most likely fund such adventures :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 01 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.

[M.-A. Lemburg]

On 02.06.2015 21:07, Maciej Fijalkowski wrote:
> Hi
> 
> There was a PSF-sponsored effort to improve the situation with the
> https://bitbucket.org/pypy/codespeed2/src being written (thank you
> PSF). It's not better enough than codespeed that I would like, but
> gives some opportunities.
> 
> That said, we have a benchmark machine for benchmarking cpython and I
> never deployed nightly benchmarks of cpython for a variety of reasons.
> 
> * would be cool to get a small VM to set up the web front
> 
> * people told me that py3k is only interesting, so I did not set it up
> for py3k because benchmarks are mostly missing
> 
> I'm willing to set up a nightly speed.python.org using nightly build
> on python 2 and possible python 3 if there is an interest. I need
> support from someone maintaining python buildbot to setup builds and a
> VM to set up stuff, otherwise I'm good to go
> 
> DISCLAIMER: I did facilitate in codespeed rewrite that was not as
> successful as I would have hoped. I did not receive any money from the
> PSF on that though.

I think we should look into getting speed.python.org up and
running for both Python 2 and 3 branches:

 https://speed.python.org/

What would it take to make that happen ?

> Cheers,
> fijal
> 
> 
> On Mon, Jun 1, 2015 at 1:14 PM, M.-A. Lemburg  wrote:
>> On 01.06.2015 12:44, Armin Rigo wrote:
>>> Hi Larry,
>>>
>>> On 31 May 2015 at 01:20, Larry Hastings  wrote:
>>>> p.s. Supporting this patch also helps cut into PyPy's reported performance
>>>> lead--that is, if they ever upgrade speed.pypy.org from comparing against
>>>> Python *2.7.2*.
>>>
>>> Right, we should do this upgrade when 2.7.11 is out.
>>>
>>> There is some irony in your comment which seems to imply "PyPy is
>>> cheating by comparing with an old Python 2.7.2": it is inside a thread
>>> which started because "we didn't backport performance improvements to
>>> 2.7.x so far".
>>>
>>> Just to convince myself, I just ran a performance comparison.  I ran
>>> the same benchmark suite as speed.pypy.org, with 2.7.2 against 2.7.10,
>>> both freshly compiled with no "configure" options at all.  The
>>> differences are usually in the noise, but range from +5% to... -60%.
>>> If anything, this seems to show that CPython should take more care
>>> about performance regressions.  If someone is interested:
>>>
>>> * "raytrace-simple" is 1.19 times slower
>>> * "bm_mako" is 1.29 times slower
>>> * "spitfire_cstringio" is 1.60 times slower
>>> * a number of other benchmarks are around 1.08.
>>>
>>> The "7.0x faster" number on speed.pypy.org would be significantly
>>> *higher* if we upgraded the baseline to 2.7.10 now.
>>
>> If someone were to volunteer to set up and run speed.python.org,
>> I think we could add some additional focus on performance
>> regressions. Right now, we don't have any way of reliably
>> and reproducibly testing Python performance.
>>
>> Hint: The PSF would most likely fund such adventures :-)
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Jun 01 2015)
>>>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
>> 
>>
>> : Try our mxODBC.Connect Python Database Interface for free ! ::
>>
>>eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>Registered at Amtsgericht Duesseldorf: HRB 46611
>>http://www.egenix.com/company/contact/
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 03 201

Re: [Python-Dev] speed.python.org (was: 2.7 is here until 2020, please don't call it a waste.)

[M.-A. Lemburg]

On 04.06.2015 04:08, Tetsuya Morimoto wrote:
>> If someone were to volunteer to set up and run speed.python.org, I think
> we could add some additional focus on performance regressions. Right now,
> we don't have any way of reliably and reproducibly testing Python
> performance.
> 
> I'm very interested in speed.python.org and feel regret that the project is
> standing still. I have a mind to contribute something ...

On 03.06.2015 18:59, Maciej Fijalkowski wrote:> On Wed, Jun 3, 2015 at 3:49 PM, 
R. David Murray
 I think we should look into getting speed.python.org up and
 running for both Python 2 and 3 branches:

  https://speed.python.org/

 What would it take to make that happen ?
>>>
>>> I guess ideal would be some cooperation from some of the cpython devs,
>>> so say someone can setup cpython buildbot
>>
>> What does "set up cpython buildbot" mean in this context?
>
> The way it works is dual - there is a program running the benchmarks
> (the runner) which is in the pypy case run by the pypy buildbot and
> the web side that reports stuff. So someone who has access to cpython
> buildbot would be useful.

Ok, so there's interest and we have at least a few people who are
willing to help.

Now we need someone to take the lead on this and form a small
project group to get everything implemented. Who would be up
to such a task ?

The speed project already has a mailing list, so you could use
that for organizing the details.

We could also create a PSF work group and assign a budget to it,
if that helps.

If you need help with all this, let me know.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 04 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 490: Chain exceptions at C level

[M.-A. Lemburg]

On 20.06.2015 09:30, Victor Stinner wrote:
> Hi,
> 
> I didn't get much feedback on this PEP. Since the Python 3.6 branch is
> open (default), it's probably better to push such change in the
> beginning of the 3.6 cycle, to catch issues earlier.
> 
> Are you ok to chain exceptions at C level by default?

I think it's a good idea to make C APIs available to
simplify chaining exceptions at the C level, but don't
believe that always doing this by default is a good idea.
It should really be a case-by-case decision, IMO.

Note that Python exceptions are cheap to raise in C
(very much unlike in Python), so making this more
expensive by default would introduce a significant
overhead - without much proven benefit.

More below...

> PEP: 490
> Title: Chain exceptions at C level
> Version: $Revision$
> Last-Modified: $Date$
> Author: Victor Stinner 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 25-March-2015
> Python-Version: 3.6
> 
> 
> Abstract
> 
> 
> Chain exceptions at C level, as already done at Python level.
> 
> 
> Rationale
> =
> 
> Python 3 introduced a new killer feature: exceptions are chained by
> default, PEP 3134.
> 
> Example::
> 
> try:
> raise TypeError("err1")
> except TypeError:
> raise ValueError("err2")
> 
> Output::
> 
> Traceback (most recent call last):
>   File "test.py", line 2, in 
> raise TypeError("err1")
> TypeError: err1
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "test.py", line 4, in 
> raise ValueError("err2")
> ValueError: err2
> 
> Exceptions are chained by default in Python code, but not in
> extensions written in C.
> 
> A new private ``_PyErr_ChainExceptions()`` function was introduced in
> Python 3.4.3 and 3.5 to chain exceptions. Currently, it must be called
> explicitly to chain exceptions and its usage is not trivial.
> 
> Example of ``_PyErr_ChainExceptions()`` usage from the ``zipimport``
> module to chain the previous ``OSError`` to a new ``ZipImportError``
> exception::
> 
> PyObject *exc, *val, *tb;
> PyErr_Fetch(&exc, &val, &tb);
> PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
> _PyErr_ChainExceptions(exc, val, tb);
> 
> This PEP proposes to also chain exceptions automatically at C level to
> stay consistent and give more information on failures to help
> debugging. The previous example becomes simply::
> 
> PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
> 
> 
> Proposal
> 
> 
> Modify PyErr_*() functions to chain exceptions
> --
> 
> Modify C functions raising exceptions of the Python C API to
> automatically chain exceptions: modify ``PyErr_SetString()``,
> ``PyErr_Format()``, ``PyErr_SetNone()``, etc.
> 
> 
> Modify functions to not chain exceptions
> 
> 
> Keeping the previous exception is not always interesting when the new
> exception contains information of the previous exception or even more
> information, especially when the two exceptions have the same type.
> 
> Example of an useless exception chain with ``int(str)``::
> 
> TypeError: a bytes-like object is required, not 'type'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: int() argument must be a string, a bytes-like object or
> a number, not 'type'
> 
> The new ``TypeError`` exception contains more information than the
> previous exception. The previous exception should be hidden.
> 
> The ``PyErr_Clear()`` function can be called to clear the current
> exception before raising a new exception, to not chain the current
> exception with a new exception.
> 
> 
> Modify functions to chain exceptions
> 
> 
> Some functions save and then restore the current exception. If a new
> exception is raised, the exception is currently displayed into
> sys.stderr or ignored depending on the function.  Some of these
> functions should be modified to chain exceptions instead.
> 
> Examples of function ignoring the new exception(s):
> 
> * ``ptrace_enter_call()``: ignore exception
> * ``subprocess_fork_exec()``: ignore exception raised by enable_gc()
> * ``t_bootstrap()`` of the ``_thread`` module: ignore exception raised
>   by trying to display the bootstrap function to ``sys.stderr``
> * ``PyDict_GetItem()``, ``_PyDict_GetItem_KnownHash()``: ignore
>   exception raised by looking for a key in the dictionary
> * ``_PyErr_TrySetFromCause()``: ignore exception
> * ``PyFrame_LocalsToFast()``: ignore exception raised by
>   ``dict_to_map()``
> * ``_PyObject_Dump()``: ignore exception. ``_PyObject_Dump()`` is used
>   to debug, to inspect a running process, it should not modify the
>   Python state.
> * ``Py_ReprLeave()``: ignore

Re: [Python-Dev] speed.python.org

[M.-A. Lemburg]

On 23.06.2015 03:58, Zachary Ware wrote:
> On Thu, Jun 4, 2015 at 10:51 AM, Maciej Fijalkowski  wrote:
>> On Thu, Jun 4, 2015 at 4:32 PM, R. David Murray  
>> wrote:
>>> OK, so what you are saying is that speed.python.org will run a buildbot
>>> slave so that when a change is committed to cPython, a speed run will be
>>> triggered?  Is "the runner" a normal buildbot slave, or something
>>> custom?  In the normal case the master controls what the slave
>>> runs...but regardless, you'll need to let us know how the slave
>>> invocation needs to be configured on the master.
>>
>> Ideally nightly (benchmarks take a while). The setup for pypy looks like 
>> this:
>>
>>
>> https://bitbucket.org/pypy/buildbot/src/5fa1f1a4990f842dfbee416c4c2e2f6f75d451c4/bot2/pypybuildbot/builds.py?at=default#cl-734
>>
>> so fairly easy. This already generates a json file that you can plot.
>> We can setup an upload automatically too.
> 
> I've been looking at what it will take to set up the buildmaster for
> this, and it looks like it's just a matter of checking out the
> benchmarks, building Python, testing it, and running the benchmarks.
> There is the question of which benchmark repo to use:
> https://bitbucket.org/pypy/benchmarks or
> https://hg.python.org/benchmarks; ideally, we should use
> hg.python.org/benchmarks for CPython benchmarks, but it looks like
> pypy/benchmarks has the necessary runner, so I suppose we'll be using
> it for now.  Is there interest from both sides to merge those
> repositories?
> 
> The big question for the buildmaster is what options to pass to the
> benchmark runner.  I suppose most of them should match the
> CPythonBenchmark BuildFactory from the PyPy buildbot master
> configuration, but otherwise I'm not sure.
> 
> The other big question is where the benchmarks will be run.  The
> speed.python.org page makes it sound like there's a box intended for
> that purpose (the one running the speed.python.org page?); can anyone
> with access to it contact me to get the build slave set up?

Yes, I believe the machine is currently only running that page.

I've pinged the PSF Infra Team to get you access to it.

Thank you for looking into this !

Cheers,
-- 
Marc-Andre Lemburg
Director
Python Software Foundation
http://www.python.org/psf/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backporting the 3.5+ Windows build project files to 2.7

[M.-A. Lemburg]

On 22.06.2015 19:03, Zachary Ware wrote:
> Hi,
> 
> As you may know, Steve Dower put significant effort into rewriting the
> project files used by the Windows build as part of moving to VC14 as
> the official compiler for Python 3.5.  Compared to the project files
> for 3.4 (and older), the new project files are smaller, cleaner,
> simpler, more easily extensible, and in some cases quite a bit more
> correct.
> 
> I'd like to backport those new project files to 2.7, and Intel is
> willing to fund that work as part of making Python ICC compilable on
> all platforms they support since it makes building Python 2.7 with ICC
> much easier.  I have no intention of changing the version of MSVC used
> for official 2.7 builds, it *will* remain at MSVC 9.0 (VS 2008) as
> determined the last time we had a thread about it.  VS 2010 and newer
> can access older compilers (back to 2008) as a 'PlatformToolset' if
> they're installed, so all we have to do is set the PlatformToolset in
> the projects at 'v90'.  Backporting the projects would make it easier
> to build 2.7 with a newer compiler, but that's already possible if
> somebody wants to put enough work into it, the default will be the old
> compiler, and we can emit big scary warnings if somebody does use a
> compiler other than v90.
> 
> With the stipulation that the officially supported compiler won't
> change, I want to make sure there's no major opposition to replacing
> the old project files in PCbuild.  The old files would move to
> PC\VS9.0, so they'll still be available and usable if necessary.
> 
> Using the backported project files to build 2.7 would require two
> versions of Visual Studio to be installed; VS2010 (or newer) would be
> required in addition to VS2008.  All Windows core developers should
> already have VS2010 for Python 3.4 (and/or VS2015 for 3.5) and I
> expect that anyone else who cares enough to still have VS2008 probably
> has (or can easily get) one of the free editions of VS 2010 or newer,
> so I don't consider this to be a major issue.

To understand this correctly:

In order to build Python 2.7.11 (or a later version), we'd
now need VS 2008 *and* VS 2010 installed ?

This doesn't sound like it would make things easier for
people who need to build their own Python on Windows, e.g.
because they are freezing their apps for distribution, or
using their own internal special builds.

For VS 2008 we now have a long-term solution thanks to MS.
The same is not true for VS 2010 (download links are not official
anymore), so we'd complicate things again by requiring the
mixed compiler setup.

> The backported files could be added alongside the old files in
> PCbuild, in a better-named 'NewPCbuild' directory, or in a
> subdirectory of PC. I would rather replace the old project files in
> PCbuild, though; I'd like for the backported files to be the
> recommended way to build, complete with support from PCbuild/build.bat
> which would make the new project files the default for the buildbots.

If you keep the old VS 2008 build files around, I'd be
fine with having an optional upgrade path to newer
compilers :-)

It's rather unlikely that the project files will change much
in coming years, so there shouldn't be much maintenance
effort.

> I have a work-in-progress branch with the backported files in PCbuild,
> which you can find at
> https://hg.python.org/sandbox/zware/log/project_files_backport.  There
> are still a couple bugs to work out (and a couple unrelated changes to
> PC/pyconfig.h), but most everything works as expected.
> 
> Looking forward to hearing others' opinions,

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 25 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2015-06-25: Released mxODBC 3.3.3 ... http://egenix.com/go79
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78
2015-07-20: EuroPython 2015, Bilbao, Spain ... 25 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backporting the 3.5+ Windows build project files to 2.7

[M.-A. Lemburg]

On 25.06.2015 17:12, Zachary Ware wrote:
> On Thu, Jun 25, 2015 at 8:54 AM, M.-A. Lemburg  wrote:
>> On 22.06.2015 19:03, Zachary Ware wrote:
>>> Using the backported project files to build 2.7 would require two
>>> versions of Visual Studio to be installed; VS2010 (or newer) would be
>>> required in addition to VS2008.  All Windows core developers should
>>> already have VS2010 for Python 3.4 (and/or VS2015 for 3.5) and I
>>> expect that anyone else who cares enough to still have VS2008 probably
>>> has (or can easily get) one of the free editions of VS 2010 or newer,
>>> so I don't consider this to be a major issue.
>>
>> To understand this correctly:
>>
>> In order to build Python 2.7.11 (or a later version), we'd
>> now need VS 2008 *and* VS 2010 installed ?
> 
> Using the backported project files, yes, two versions of VS would be
> required.  However, the second version doesn't have to be VS 2010, it
> could be 2010, 2012, 2013, 2015, or any from the future (until
> Microsoft changes the format of the project files again).
> 
>> This doesn't sound like it would make things easier for
>> people who need to build their own Python on Windows, e.g.
>> because they are freezing their apps for distribution, or
>> using their own internal special builds.
>>
>> For VS 2008 we now have a long-term solution thanks to MS.
>> The same is not true for VS 2010 (download links are not official
>> anymore), so we'd complicate things again by requiring the
>> mixed compiler setup.
> 
> For anyone building 2.7 and any other (current) version of Python on
> the same machine, all the necessary pieces are already in place.
> 
> The main motivation for backporting is to make it easier to build
> Python with ICC.  Pre-backport, Intel gave me a list of 9 steps they
> were doing to build 2.7.10 with ICC.  Post-backport, it's just
> `PCbuild\build.bat -e [-p x64] [-d] "/p:PlatformToolset=Intel C++
> Compiler XE 15.0" "/p:BasePlatformToolset=v90"` (the bracketed options
> being optional, for 64-bit and debug builds, respectively).

Sounds good.

BTW: I remember there was some discussion a while ago to
get ICC licenses to core devs. Has there been any progress
on this ?

> There are some benefits for MSVC builds too, though: it's easier to do
> a parallel build, and OpenSSL and Tcl/Tk/Tix are built by the project
> files, so the whole build can be done much quicker.  The new project
> files for Tcl and Tk also take care of copying their DLLs into PCbuild
> so that _tkinter can find them without having to mess around with
> PATH.  My patch also fixes a couple of long-standing bugs with finding
> the right libraries for 64-bit Tcl/Tk and the test suite undoing the
> work of FixTk.py after any test that imports Tkinter, both of which
> were exacerbated by the rest of the patch.
> 
>> If you keep the old VS 2008 build files around, I'd be
>> fine with having an optional upgrade path to newer
>> compilers :-)
> 
> The old files are moved to PC/VS9.0, and they work as expected as far
> as I've tested them. 

So it's still possible to build with "just" VS 2008 installed
or will the VS 2010 (or later) be required for those old
files as well ?

> Btw, the upgrade path to newer compilers is just
> a side-effect which I'm kind of torn about.  On one hand, making it
> easier to build 2.7 with the "wrong" compiler is bound to lead to
> issues somewhere somehow.  On the other hand, people are already doing
> so anyway, with their own upgraded project files.

I guess will have the discussion about switching compilers
for Python 2.7 again at some point. People writing extensions
will sooner or later want to use features from more recent
compilers for Python 2.7 as well :-)

>> It's rather unlikely that the project files will change much
>> in coming years, so there shouldn't be much maintenance
>> effort.
> 
> Hopefully not :) 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 25 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2015-06-25: Released mxODBC 3.3.3 ... http://egenix.com/go79
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78
2015-07-20: EuroPython 2015, Bilbao, Spain ... 25 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] How are we merging forward from the Bitbucket 3.5 repo?

[M.-A. Lemburg]

On 16.08.2015 16:08, Guido van Rossum wrote:
> I presume the issue here is that Hg is so complicated that everyone knows a
> different subset of the commands and semantics.
> 
> I personally don't know what the commands for cherry-picking a revision
> would be.
> 
> I also don't know exactly what happens when you merge a PR using bitbucket.
> (I'm only familiar with the GitHub PR flow, and I don't like its behavior,
> which seems to always create an extra merge revision for what I consider as
> logically a single commit.)
> 
> BTW When I go to https://bitbucket.org/larry/cpython350 the first thing I
> see (in a very big bold font) is "This is Python version 3.6.0 alpha 1".
> What's going on here? It doesn't inspire confidence.

You are probably looking at the default branch within that repo fork.

This is the 3.5 branch:

https://bitbucket.org/larry/cpython350/branch/3.5

> On Sun, Aug 16, 2015 at 10:13 AM, Larry Hastings  wrote:
> 
>>
>>
>> So far I've accepted two pull requests into bitbucket.com/larry/cpython350
>> in the 3.5 branch, what will become 3.5.0rc2.  As usual, it's the
>> contributor's responsibility to merge forward; if their checkin goes in to
>> 3.5, it's their responsibility to also merge it into the
>> hg.python.org/cpython 3.5 branch (which will be 3.5.1) and default branch
>> (which right now is 3.6).
>>
>> But... what's the plan here?  I didn't outline anything specific, I just
>> assumed we'd do the normal thing, pulling from 3.5.0 into 3.5.1 and
>> merging.  But of the two pull requests so far accepted, one was merged this
>> way, though it cherry-picked the revision (skipping the pull request merge
>> revision Bitbucket created), and one was checked in to 3.5.1 directly (no
>> merging).
>>
>> I suppose we can do whatever we like.  But it'd be helpful if we were
>> consistent.  I can suggest three approaches:
>>
>>1. After your push request is merged, you cherry-pick your revision
>>from bitbucket.com/larry/cpython350 into hg.python.org/cpython and
>>merge.  After 3.5.0 final is released I do a big null merge from
>>bitbucket.com/larry/cpython350 into hg.python.org/cpython.
>>2. After your push request is merged, you manually check in a new
>>equivalent revision into hg.python.org/cpython in the 3.5 branch.  No
>>merging necessary because from Mercurial's perspective it's unrelated to
>>the revision I accepted.  After 3.5.0 final is released I do a big null
>>merge from bitbucket.com/larry/cpython350 into hg.python.org/cpython.
>>3. After your push request is merged, you pull from
>>bitbucket.com/larry/cpython350 into hg.python.org/cpython and merge
>>into 3.5.  In this version I don't have to do a final null merge!
>>
>> I'd prefer 3; that's what we normally do, and that's what I was
>> expecting.  So far people have done 1 and 2.
>>
>> Can we pick one approach and stick with it?  Pretty-please?
>>
>>
>> */arry*
>>
>> ___
>> python-committers mailing list
>> python-committ...@python.org
>> https://mail.python.org/mailman/listinfo/python-committers
>>
>>
> 
> 
> 
> 
> ___
> python-committers mailing list
> python-committ...@python.org
> https://mail.python.org/mailman/listinfo/python-committers
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 16 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2015-08-12: Released mxODBC 3.3.4 ... http://egenix.com/go80
2015-08-22: FrOSCon 2015 ...6 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0484 - the Numeric Tower

[M.-A. Lemburg]

On 14.10.2015 01:37, Raymond Hettinger wrote:
> 
>> On Oct 13, 2015, at 9:16 AM, Random832  wrote:
>>
>>> ## 
>>> ## Decimal has all of the methods specified by the Real abc, but it should
>>> ## not be registered as a Real because decimals do not interoperate with
>>> ## binary floats (i.e.  Decimal('3.14') + 2.71828 is undefined).  But,
>>> ## abstract reals are expected to interoperate (i.e. R1 + R2 should be
>>> ## expected to work if R1 and R2 are both Reals).
>>
>> Why?
> 
> Q.  Why is Python the way it is?
> A.   Because Guido said so ;-)
> 
> IIRC, the answer is that we were being conservative with possibly unintended 
> operations between types with differing precision and with differing notions 
> of what numbers could be exactly representable.
> 
> We could have (and still could) make the choice to always coerce to decimal 
> (every float is exactly representable in decimal).  Further, any decimal 
> float or binary float could be losslessly coerced to a Fraction, but that 
> probably isn't what you really want most of the time.  I think people who 
> work in decimal usually want to stay there and people who work with binary 
> floating point want to stay there as well (invisible coercions being more 
> likely to cause pain than relieve pain).

I can only underline this. Conversion to decimals or fractions should
not be implicit. People needing these types will know when they need
them and apply the required explicit conversions to fit their use case.

E.g. in accounting you'll likely use decimal, in finance and science
you stick to floats.

>From a theoretical point of view, it would make sense to add coercion
to these types, but not from a practical point of view.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Translate Python language

[M.-A. Lemburg]

On 11.11.2015 17:20, Donald Stufft wrote:
> On November 11, 2015 at 11:19:07 AM, Paul Moore (p.f.mo...@gmail.com) wrote:
>> On 11 November 2015 at 15:13, Christophe Bal wrote:
>>> Hello.
>>>
>>> I'm a french teacher and I would like to use Python with young child but
>>> I've a big problem. All the keyword are in english. So I would like to know
>>> if there is a way to hack the AST tools such as to use french keywords and
>>> then translate a list of french keywords to their english regular version.
>>>
>>> Where should I start ? My idea is not to build a new language but simply
>>> translate words using an hacked version of the AST tools.
>>>
>>> Hoping to be clear and I do not pollute the message in this list.
>>
>> Do you really just mean keywords? Most of the English words used in
>> Python (for example "open") aren't actually keywords but are names of
>> functions defined in the builtins and/or standard library. Those are
>> not exposed as distinct objects in the AST, but are simply name
>> lookups.
>>
> 
> You might be able to use an encoding to do this.

Or use Andrew Dalke's fun project LOLPython as basis:

http://www.dalkescientific.com/writings/diary/archive/2007/06/01/lolpython.html

It uses a parser which then translates the source code
into regular Python source code, which it then runs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2015-10-23: Released mxODBC Connect 2.1.5 ... http://egenix.com/go85

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Support of UTF-16 and UTF-32 source encodings

[M.-A. Lemburg]

On 14.11.2015 23:56, Victor Stinner wrote:
> These encodings are rarely used. I don't think that any text editor use
> them. Editors use ascii, latin1, utf8 and... all locale encoding. But I
> don't know any OS using UTF-16 as a locale encoding. UTF-32 wastes disk
> space.

UTF-16 is used a lot for Windows text files, e.g. Unicode
CSV files (the save as "Unicode text file" option writes
UTF-16).

However, nowadays, all text editors also support UTF-8 and
many of these recognize the UTF-8 BOM as identifier to detect
Unicode text files.

> Ok, even if it exists, Python already accepts a very wide range of
> encoding. It is not worth to make the parser much more complex just to
> support encodings which are also never used (for .py files).

Agreed. In Python 2 we decided to only allow ASCII super-sets
for Python source files, which out ruled multi-byte encodings
such as UTF-16 and -32. I don't think we need to make the parser
more complex just to support them. UTF-8 works fine as Python
source code encoding.

> Victor
> Le 14 nov. 2015 20:20, "Serhiy Storchaka"  a écrit :
> 
>> For now UTF-16 and UTF-32 source encodings are not supported. There is a
>> comment in Parser/tokenizer.c:
>>
>> /* Disable support for UTF-16 BOMs until a decision
>>is made whether this needs to be supported.  */
>>
>> Can we make a decision whether this support will be added in foreseeable
>> future (say in near 10 years), or no?
>>
>> Removing commented out and related code will help to refactor the
>> tokenizer, and that can help to fix some existing bugs (e.g. issue14811,
>> issue18961, issue20115 and may be others). Current tokenizing code is too
>> tangled.
>>
>> If the support of UTF-16 and UTF-32 is planned, I'll take this to
>> attention during refactoring. But in many places besides the tokenizer the
>> ASCII compatible encoding of source files is expected.
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>>
> 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 15 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2015-10-23: Released mxODBC Connect 2.1.5 ... http://egenix.com/go85

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reading Python source file

[M.-A. Lemburg]

On 17.11.2015 02:53, Serhiy Storchaka wrote:
> I'm working on rewriting Python tokenizer (in particular the part that reads 
> and decodes Python
> source file). The code is complicated. For now there are such cases:
> 
> * Reading from the string in memory.
> * Interactive reading from the file.
> * Reading from the file:
>   - Raw reading ignoring encoding in parser generator.
>   - Raw reading UTF-8 encoded file.
>   - Reading and recoding to UTF-8.
> 
> The file is read by the line. It makes hard to check correctness of the first 
> line if the encoding
> is specified in the second line. And it makes very hard problems with null 
> bytes and with
> desynchronizing buffered C and Python files. All this problems can be easily 
> solved if read all
> Python source file in memory and then parse it as string. This would allow to 
> drop a large complex
> and buggy part of code.
> 
> Are there disadvantages in this solution? As for memory consumption, the 
> source text itself will
> consume only small part of the memory consumed by AST tree and other 
> structures. As for performance,
> reading and decoding all file can be faster then by the line.

A problem with this approach is that you can no
longer fail early and detect indentation errors et al. while
parsing the data (which may well come from a pipe).

Another related problem is that you have to wait for the full
input data before you can start compiling the code.

I don't think these situations are all that common, though,
so reading in the full source code before compiling it
sounds like a reasonable approach.

We use the same simplification in eGenix PyRun's emulation of
the Python command line interface and it has so far not
caused any problems.

> [1] http://bugs.python.org/issue25643

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 17 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2015-10-23: Released mxODBC Connect 2.1.5 ... http://egenix.com/go85

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reading Python source file

[M.-A. Lemburg]

On 17.11.2015 16:22, Guido van Rossum wrote:
> On Tue, Nov 17, 2015 at 1:59 AM, M.-A. Lemburg  wrote:
>>> [moving from read source line by line to reading all in one go]
>> We use the same simplification in eGenix PyRun's emulation of
>> the Python command line interface and it has so far not
>> caused any problems.
> 
> Curious how you do it? I'd actually be quite disappointed if the
> amount of parsing done by the standard REPL went down.

Oh, that's easy:

elif sys.argv[0] == '-' and not (pyrun_as_string or pyrun_as_module):
# Read the script from stdin
pyrun_as_string = True
pyrun_script = sys.stdin.read()

and then, later on:

# Run the script
try:
pyrun_execute_script(pyrun_script, mode)
except Exception as reason:
if pyrun_interactive:
import traceback
traceback.print_exc()
pyrun_prompt(banner='')
else:
raise
else:
# Enter interactive mode, in case wanted
if pyrun_interactive:
pyrun_prompt()

The REPL is not affected by this, since we use the standard
code.interact() for the prompt. This reads the entry line
by line, joins the lines and tries to compile the entry every
time it receives a new line until it succeeds or fails.

Serhiy's proposed change should not affect this mode of
operation.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 19 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2015-10-23: Released mxODBC Connect 2.1.5 ... http://egenix.com/go85

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python stdlib ssl.SSLContext is missing mode setting ability

[M.-A. Lemburg]

On 19.11.2015 09:14, Cory Benfield wrote:
> 
>> On 19 Nov 2015, at 03:53, Ben Bangert  wrote:
>>
>> In Python 2 and 3, the ssl module's SSLContext object has a way to set
>> SSL options, but not to set SSL modes.
>>
>> The set_mode command and some of the available modes:
>> https://www.openssl.org/docs/manmaster/ssl/SSL_CTX_set_mode.html
>>
>> The most critical mode is SSL_MODE_RELEASE_BUFFERS, which can drop the
>> SSL overhead *per connection* from around 25kb to ~7kb. The pyopenssl
>> library allows the setting of SSLContext modes, it seems very odd that
>> the Python 2/3 ssl modules do not. Though I could understand that
>> perhaps not all SSL libraries Python might build against would have
>> this mode thing available.
>>
> 
> Ben,
> 
> Do we need the ability to set arbitrary modes? Most of the modes mentioned in 
> the OpenSSL documentation are things we actively don’t want the user to set 
> because stuff will randomly break. With that in mind, and with the fact that 
> SSL_MODE_RELEASE_BUFFERS is so obviously better than the standard, should we 
> just instead have the ssl module automatically set SSL_MODE_RELEASE_BUFFERS 
> unconditionally?
> 
> If so, I’m happy to submit a bug/patch to get that to happen.

The mode should only be enabled for OpenSSL versions which are
not affected by this vulnerability:

https://www.rapid7.com/db/vulnerabilities/http-openssl-cve-2014-0198

Other than that it seems like a good way forward. Plenty other
projects have had this enabled per default for years:

http://www.dovecot.org/list/dovecot/2011-October/131381.html
https://svn.boost.org/trac/boost/changeset/71706
https://community.openvpn.net/openvpn/ticket/157

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 19 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2015-10-23: Released mxODBC Connect 2.1.5 ... http://egenix.com/go85

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[M.-A. Lemburg]

I think the PEP is a good step forward to compromise between
the crypto purists (use whatever technologies makes us more
secure even if it breaks things) and those who cannot upgrade
their Python 2.7 because of the PEP 476 change, since it causes their
applications to fail (e.g. because the embedded devices they want to
interface to only support self signed certs).

I would still find having built-in support for the recommendations
in the Python stdlib a better approach, but PEP 493 is good enough
in at least solving real problems people are having.

PS: Would be great to have a PyPI package which implements these
recommendations so that you can simply add it as dependency - and
then please for Python 3 as well, since people with embedded
devices will want to be able to use Python 3 as well ;-)


On 24.11.2015 15:27, Laura Creighton wrote:
> In a message of Tue, 24 Nov 2015 14:05:53 +, Paul Moore writes:
>> Simply adding "people who have no control over their broken
>> infrastructure" with a note that this PEP helps them, would be
>> sufficient here (and actually helps the case for the PEP, so why not?
>> ;-))
> 
> But does it help them?  Or does it increase the power of those who
> hand out certificates and who are intensely security conscious over
> those who would like to get some work done this afternoon?
> 
> Laura
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 24 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deleting with setting C API functions

[M.-A. Lemburg]

On 02.12.2015 13:29, Serhiy Storchaka wrote:
> On 02.12.15 12:06, Victor Stinner wrote:
>> 2015-12-02 9:42 GMT+01:00 Serhiy Storchaka :
>>> You have enough time to update your projects, and you can update them
>>> uniformly for all versions. And may be you will found few weird bugs related
>>> to misuse of Set* API.
>>
>> Did you check popular projects using C extensions to check if they
>> call Set*() functions to delete attributes/items?
> 
> I have checked following projects.
> 
> regex, simplejson, Pillow, PyQt4, LibreOffice, PyGTK, PyICU, pyOpenSSL, 
> libxml2, Boost, psutil,
> mercurial don't use PyObject_SetAttr at all.
> 
> NumPy, pgobject don't use PyObject_SetAttr for deleting.
> 
> PyYAML and lxml use PyObject_SetAttr only in code generated by Cython and 
> never use it for deleting.

While I don't think deleting attributes is a very common thing
to do in any Python code base (unless you need to break circular
references or explicitly want to free resources), the
fact that PyObject_DelAttr() itself is implemented as macro
using the NULL attribute value clearly creates an API incompatibility
when removing this functionality or generating warnings, since
all code using the correct PyObject_DelAttr() at the moment,
would then trigger the warning as well.

As a result, the deprecation would have to be extended across
more releases than the usual cycle.

A first step would be to replace the macro with a proper function
to avoid false positive warnings, even when using the correct API.

Then we could add a warning to the PyObject_SetAttr() function and
hope that not too many projects use the stable ABI as basis to
have C extensions work across several releases.

Overall, I'm not sure whether it's worth the trouble. Documenting
the feature and adding a deprecation notice to just the documentation
would likely be better. We could then remove the functionality
in Python 4.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 02 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Reference has no mention of list comprehensions

[M.-A. Lemburg]

On 03.12.2015 14:37, Paul Moore wrote:
> On 3 December 2015 at 12:51, Laura Creighton  wrote:
>> Intentional or Oversight?
> 
> Hard to find :-)
> 
> https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries
> 
> I went via "Atoms" in the expression section, then followed the links
> in the actual grammar spec.

Strange that the doc search facility doesn't find this:

https://docs.python.org/3/search.html?q=comprehension

The human readable documentation is in the tutorial:

https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

(this is found by the doc search)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 03 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Reference has no mention of list comÃprehensions

[M.-A. Lemburg]

On 03.12.2015 17:09, Ryan Gonzalez wrote:
> 
> 
> On December 3, 2015 8:26:23 AM CST, Laura Creighton  wrote:
>> In a message of Thu, 03 Dec 2015 13:37:17 +, Paul Moore writes:
>>> On 3 December 2015 at 12:51, Laura Creighton  wrote:
 Intentional or Oversight?
>>>
>>> Hard to find :-)
>>>
>>> https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries
>>>
>>> I went via "Atoms" in the expression section, then followed the links
>>> in the actual grammar spec.
>>>
>>> Paul
>>
>> I think the whole use of the language displays as in
>>  
>>  6.2.4. Displays for lists, sets and dictionaries
>>
>>  For constructing a list, a set or a dictionary Python provides 
>>  special syntax called “displays”, each of them in two flavors:
>>
>>either the container contents are listed explicitly, or
>>they are computed via a set of looping and filtering instructions, 
>>called a comprehension.
>>
>> is very odd.  I don't know anybody who talks of 'displays'.  They
>> talk of 'two ways to construct a'.  
>>
>> Who came up with the word 'display' and what does it have going for
>> it that I have missed?  Right now I think its chief virtue is that
>> it is a meaningless noun.  (But not meaningless enough, as I
>> associate displays with output, not construction).
>>
>> I think that 
>>
>>6.2.4 Constructing lists, sets and dictionaries
>>
>> would be a much more useful title, and
>>
>> 6.2.4 Constructing lists, sets and dictionaries -- explicitly or
>> through the use of comprehensions
>>
> 
> What about:
> 
> 6.2.4 Constricting lists, sets, and dictionaries (including comprehensions)
> 
> or something to that effect?
> 
>> an even better one.
>>
>> Am I missing something important about the 'display' language?

I don't think changing a single header is useful in this case.

The grammar uses the token term "display" to mean "representation
of an object". While you normally only think of output when talking
of the representation of an object, it can also refer to the visual
definition of an object when passed to the parser.

A list comprehension is an example of such a visual definition of
an object, hence the token name.

If we were to change the term, we'd have to change it throughout
the reference, grammar and parser implementation.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 03 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Reference has no mention of list comÃprehensions

[M.-A. Lemburg]

On 03.12.2015 18:30, Laura Creighton wrote:
> What I would like is if it were a lot easier for a person who just
> saw a list comprehension for the very first time, and was told what it
> is, to have a much, much easier time finding it in the Reference Manual.

Such a person should more likely be directed to the tutorial
rather than the very technical language spec :-)

> Would a section on comprehensions in general, defining what a comprehension
> is be appropriate?

We already have this in the tutorial:

https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 03 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Reference has no mention of list comÃprehensions

[M.-A. Lemburg]

On 03.12.2015 19:27, Laura Creighton wrote:
> So how do we get search to work so that people in the Language
> Reference who type in 'List Comprehension' get a hit?

It seems that the search index is broken for at least a few
documentation file releases:

ok: 
https://docs.python.org/2.6/search.html?q=comprehension&check_keywords=yes&area=default
not ok: 
https://docs.python.org/2.7/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/3.2/search.html?q=comprehension&check_keywords=yes&area=default
not ok: 
https://docs.python.org/3.3/search.html?q=comprehension&check_keywords=yes&area=default
not ok: 
https://docs.python.org/3.4/search.html?q=comprehension&check_keywords=yes&area=default
not ok: 
https://docs.python.org/3.5/search.html?q=comprehension&check_keywords=yes&area=default

(ok = "/reference/expressions.html is found")

Interestingly, these URLs give different results, e.g.

ok: 
https://docs.python.org/release/2.7.1/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.2/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.3/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.4/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.5/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.6/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.7/search.html?q=comprehension&check_keywords=yes&area=default
ok: 
https://docs.python.org/release/2.7.8/search.html?q=comprehension&check_keywords=yes&area=default
not ok:
https://docs.python.org/release/2.7.9/search.html?q=comprehension&check_keywords=yes&area=default
not ok:
https://docs.python.org/release/2.7.10/search.html?q=comprehension&check_keywords=yes&area=default

Looks like something changed between the 2.7.8 and 2.7.9 release.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 03 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do windows 10 users, like windows 7 users need to install a SP before installing Python will work?

[M.-A. Lemburg]

On 07.12.2015 21:50, Laura Creighton wrote:
> As webmaster, I am dealing with 3 unhappy would-be python users who have
> windows 10.
> 
> Right now their first problem is that when they click on the big
> yellow button here: https://www.python.org/downloads/
> 
> instead of getting a download of 3.5.1 they get a redirect to
> https://www.python.org/downloads/windows/
> 
> I've tried them on both the 
> Download Windows x86 web-based installer
> 
> and
>  
> Download Windows x86-64 web-based installer
> 
> but still no go, they get the  Modify/Repair/Uninstall screen
> like: http://www2.openend.se/~lac/5796.2.png
> 
> I do not know how to help them now.

Have they already tried the regular installers (as opposed to
the web installers) ?

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 07 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Branches in which to fix the SSL tests

[M.-A. Lemburg]

On 07.01.2016 04:06, Martin Panter wrote:
> Currently some SSL tests in the test suite are broken by a recent
> certificate change at https://svn.python.org/; see
>  for the bug report. The tests are
> broken when the test suite is run with the “-unetwork” option enabled,
> and most of the buildbots appear to be affected. (In 3.6 the tests
> have temporarily been disabled as a workaround.) I have a simple patch
> that subsitutes the old root certificate for the new which I would
> like to commit, but I’m not sure which branches to apply it to, or
> even which branches are open to normal maintainence and bug fixes.

As mentioned on the issue tracker I'm not convinced that your
patch is a good solution going forward. Rather than basing
the test on svn.python.org, which can change again in the
future, we should use the domain we have specifically
reserved for stdlib tests, namely pythontest.net and get this
set up in a way so that we can run SSL tests again.

I can help with getting SSL certificates for pythontest.net,
since I'm managing the PSF SSL certificates (well, trying to
keep those managed anyway ;-) - I was not involved in the
choice of new CA for svn.python.org):

https://wiki.python.org/psf/PSF%20SSL%20Certificates

Regarding which branches to apply the final fix, others have
already chimed in. Essentially, any branch which we'll need
to run tests on in the foreseeable future will need to be fixed,
since the change in svn.python.org's SSL setup (the expired
certificate which caused the tests to fail was replaced with
a new one from a different CA, again causing tests to fail)
affects all released test suites.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 07 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update PEP 7 to require curly braces in C

[M.-A. Lemburg]

On 18.01.2016 08:00, Victor Stinner wrote:
> I like if without braces when the body is only one line, especially when
> there is no else block.

Same here.

Compilers warn about these things today, so I don't think
we need to go paranoid ;-)

> Victor
> 
> 
> Le dimanche 17 janvier 2016, Brett Cannon  a écrit :
> 
>> While doing a review of http://bugs.python.org/review/26129/ I asked to
>> have curly braces put around all `if` statement bodies. Serhiy pointed out
>> that PEP 7 says curly braces are optional:
>> https://www.python.org/dev/peps/pep-0007/#id5. I would like to change
>> that.
>>
>> My argument is to require them to prevent bugs like the one Apple made
>> with OpenSSL about two years ago:
>> https://www.imperialviolet.org/2014/02/22/applebug.html. Skipping the
>> curly braces is purely an aesthetic thing while leaving them out can lead
>> to actual bugs.
>>
>> Anyone object if I update PEP 7 to remove the optionality of curly braces
>> in PEP 7?


-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 18 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Update PEP 7 to require curly braces in C

[M.-A. Lemburg]

On 19.01.2016 00:20, Brett Cannon wrote:
> On Sun, 17 Jan 2016 at 11:10 Brett Cannon  wrote:
> 
>> While doing a review of http://bugs.python.org/review/26129/ I asked to
>> have curly braces put around all `if` statement bodies. Serhiy pointed out
>> that PEP 7 says curly braces are optional:
>> https://www.python.org/dev/peps/pep-0007/#id5. I would like to change
>> that.
>>
>> My argument is to require them to prevent bugs like the one Apple made
>> with OpenSSL about two years ago:
>> https://www.imperialviolet.org/2014/02/22/applebug.html. Skipping the
>> curly braces is purely an aesthetic thing while leaving them out can lead
>> to actual bugs.
>>
>> Anyone object if I update PEP 7 to remove the optionality of curly braces
>> in PEP 7?
>>
> 
> Currently this thread stands at:

Make that:

> +1
>   Brett
>   Ethan
>   Robert
>   Georg
>   Nick
>   Maciej Szulik
> +0
>   Guido
> -0
>   Serhiy
> -1
MAL
>   Victor (maybe; didn't specifically vote)
>   Larry
>   Stefan

There are plenty other cases where typos can ruin the flow
of your code in C; the discussed case is not a very common one.

I find the whole discussion a bit strange: In Python we're
advocating not having to use curly braces, because whitespace
already provides the needed semantics for us, yet you are
now advocating that without adding extra curly braces
we'd be in danger of writing wrong code.

The Apple bug can happen in Python just as well:

if a:
run_if_true()
else:
run_if_false()

can turn into (say by hitting a wrong key in the editor):

if a:
run_if_true()
run_if_false()

(run_if_false is now run when a is true, and nothing is
done in case a is false)

So what's the correct way to address this ?

It's having a test for both branches (a is true, a is false),
not starting to write e.g.

if a:
run_if_true()
if not a:
run_if false()

to feel more "secure".

Also note that the extra braces won't necessarily help you.
If you remove "else" from:

if (a) {
run_if_true();
}
else {
run_if_false();
}

you get exactly the same Apply bug as before, only with more
curly braces.

This all sounds a bit like security theater to me ;-)

I'd say: people who feel better using the extra braces can use
them, while others who don't need them can go ahead as always
... and both groups should write more tests :-)

BTW: There are other things in PEP 7 which should probably be updated
instead, e.g. it still says we should use C89, but we're having more
and more C99 code (mostly new library functions) in CPython.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 19 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] More optimisation ideas

[M.-A. Lemburg]

On 30.01.2016 20:15, Steve Dower wrote:
> Brett tried freezing the entire stdlib at one point (as we do for parts of 
> importlib) and reported no significant improvement. Since that rules out code 
> compilation as well as the OS calls, it'd seem the priority is to execute 
> less code on startup.
> 
> Details of that work were posted to python-dev about twelve months ago, IIRC. 
> Maybe a little longer.

Freezing the entire stdlib does improve the startup time,
simply because it removes stat calls, which dominate the startup
time at least on Unix.

It also allows sharing the stdlib byte code in memory, since it gets
stored in static C structs which the OS will happily mmap into
multiple processes for you without any additional effort.

Our eGenix PyRun does exactly that. Even though the original
motivation is a different one, the gained improvement in
startup time is a nice side effect:

http://www.egenix.com/products/python/PyRun/

Aside: The encodings don't really make much difference here. The
dictionaries aren't all that big, so generating them on the fly doesn't
really create much overhead. The trade off in terms of maintainability/speed
definitely leans toward maintainability. For the larger encoding
tables we already have C implementations with appropriate data
structures to make lookup speed vs. storage needs efficient.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 31 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/


> Top-posted from my Windows Phone
> 
> -Original Message-
> From: "Serhiy Storchaka" 
> Sent: ‎1/‎30/‎2016 10:22
> To: "python-dev@python.org" 
> Subject: Re: [Python-Dev] More optimisation ideas
> 
> On 30.01.16 18:31, Steve Dower wrote:
>> On 30Jan2016 0645, Serhiy Storchaka wrote:
>>> $ ./python -m timeit -s "import codecs; from encodings.cp437 import
>>> decoding_table" -- "codecs.charmap_build(decoding_table)"
>>> 10 loops, best of 3: 4.36 usec per loop
>>>
>>> Getting rid from charmap_build() would save you at most 4.4 microseconds
>>> per encoding. 0.0005 seconds if you have imported *all* standard
>>> encodings!
>>
>> Just as happy to be proven wrong. Perhaps I misinterpreted my original
>> profiling and then, embarrassingly, ran with the result for a long time
>> without retesting.
> 
> AFAIK the most time is spent in system calls like stat or open. 
> Archiving the stdlib into the ZIP file and using zipimport can decrease 
> Python startup time (perhaps there is an open issue about this).
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org
> 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

[M.-A. Lemburg]

On 03.02.2016 22:03, Victor Stinner wrote:
> Hi,
> 
> There is an old discussion about the performance of PyMem_Malloc()
> memory allocator. CPython is stressing a lot memory allocators. Last
> time I made statistics, it was for the PEP 454:
> "For example, the Python test suites calls malloc() , realloc() or
> free() 270,000 times per second in average."
> https://www.python.org/dev/peps/pep-0454/#log-calls-to-the-memory-allocator
> 
> I proposed a simple change: modify PyMem_Malloc() to use the pymalloc
> allocator which is faster for allocation smaller than 512 bytes, or
> fallback to malloc() (which is the current internal allocator of
> PyMem_Malloc()).
> 
> This tiny change makes Python up to 6% faster on some specific (macro)
> benchmarks, and it doesn't seem to make Python slower on any
> benchmark:
> http://bugs.python.org/issue26249#msg259445
> 
> Do you see any drawback of using pymalloc for PyMem_Malloc()?

Yes: You cannot free memory allocated using pymalloc with the
standard C lib free().

It would be better to go through the list of PyMem_*() calls
in Python and replace them with PyObject_*() calls, where
possible.

> Does anyone recall the rationale to have two families to memory allocators?

The PyMem_*() APIs were needed to have a cross-platform malloc()
implementation which returns standard C lib free()able memory,
but also behaves well when passing 0 as size.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Feb 04 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

[M.-A. Lemburg]

On 04.02.2016 13:29, Victor Stinner wrote:
> Hi,
> 
> 2016-02-04 11:17 GMT+01:00 M.-A. Lemburg :
>>> Do you see any drawback of using pymalloc for PyMem_Malloc()?
>>
>> Yes: You cannot free memory allocated using pymalloc with the
>> standard C lib free().
> 
> That's not completly new.
> 
> If Python is compiled in debug mode, you get a fatal error with a huge
> error message if you free the memory allocated by PyMem_Malloc() using
> PyObject_Free() or PyMem_RawFree().
> 
> But yes, technically it's possible to use free() when Python is *not*
> compiled in debug mode.

Debug mode is a completely different beast ;-)

>> It would be better to go through the list of PyMem_*() calls
>> in Python and replace them with PyObject_*() calls, where
>> possible.
> 
> There are 536 calls to the functions PyMem_Malloc(), PyMem_Realloc()
> and PyMem_Free().
> 
> I would prefer to modify a single place having to replace 536 calls :-/

You have a point there, but I don't think it'll work out
that easily, since we are using such calls to e.g. pass
dynamically allocated buffers to code in extensions (which then
have to free the buffers again).

>>> Does anyone recall the rationale to have two families to memory allocators?
>>
>> The PyMem_*() APIs were needed to have a cross-platform malloc()
>> implementation which returns standard C lib free()able memory,
>> but also behaves well when passing 0 as size.
> 
> Yeah, PyMem_Malloc() & PyMem_Free() help to have a portable behaviour.
> But, why not PyObject_Malloc() & PObject_Free() were not used in the
> first place?

Good question. I guess developers simply thought of PyObject_Malloc()
being for PyObjects, not arbitrary memory buffers, most likely
because pymalloc was advertised as allocator for Python objects,
not random chunks of memory.

Also: PyObject_*() APIs were first introduced with pymalloc, and
no one really was interested in going through all the calls to
PyMem_*() APIs and convert those to use the new pymalloc at the
time.

All this happened between Python 1.5.2 and 2.0.

One of the reasons probably also was that pymalloc originally
did not return memory back to the system malloc(). This was
changed only some years ago.

> An explanation can be that PyMem_Malloc() can be called without the
> GIL held. But it wasn't true before Python 3.4, since PyMem_Malloc()
> called (indirectly) PyObject_Malloc() when Python was compiled in
> debug mode, and PyObject_Malloc() requires the GIL to be held.
> 
> When I wrote the PEP 445, there was a discussion about the GIL. It was
> proposed to allow to call PyMem_xxx() without the GIL:
> https://www.python.org/dev/peps/pep-0445/#gil-free-pymem-malloc
> 
> This option was rejected.

AFAIR, the GIL was not really part of the consideration at the time.
We used pymalloc for PyObject allocation, that's all.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Feb 04 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

[M.-A. Lemburg]

On 04.02.2016 14:25, Victor Stinner wrote:
> Thanks for your feedback, you are asking good questions :-)
> 
> 2016-02-04 13:54 GMT+01:00 M.-A. Lemburg :
>>> There are 536 calls to the functions PyMem_Malloc(), PyMem_Realloc()
>>> and PyMem_Free().
>>>
>>> I would prefer to modify a single place having to replace 536 calls :-/
>>
>> You have a point there, but I don't think it'll work out
>> that easily, since we are using such calls to e.g. pass
>> dynamically allocated buffers to code in extensions (which then
>> have to free the buffers again).
> 
> Ah, interesting. But I'm not sure that we delegate the responsability
> of freeing the memory to external libraries. Usually, it's more the
> opposite: a library gives us an allocated memory block, and we have to
> free it. No?

Sometimes, yes, but we also do allocations for e.g.
parsing values in Python argument tuples (e.g. using
"es" or "et"):

https://docs.python.org/3.6/c-api/arg.html

We do document to use PyMem_Free() on those; not sure whether
everyone does this though.

> I checked if we call directly malloc() to pass the buffer to a
> library, but I failed to find such case.
>
> Again, in debug mode, calling free() on a memory block allocated by
> PyMem_Malloc() will likely crash. Since we run the Python test suite
> with a Python compiled in debug mode, we would already have detected
> such bug, no?

The Python test suite doesn't test Python C extensions,
so it's not surprising that it passes :-)

> See also my old issue http://bugs.python.org/issue18203 which replaced
> almost all direct calls to malloc() with PyMem_Malloc() or
> PyMem_RawMalloc().
> 
>> Good question. I guess developers simply thought of PyObject_Malloc()
>> being for PyObjects,
> 
> Yeah, I also understood that, but in practice, it looks like
> PyMem_Malloc() is slower than so using it makes the code less
> efficient than it can be.
> 
> Instead of teaching developers that well, in fact, PyObject_Malloc()
> is unrelated to object programming, I think that it's simpler to
> modify PyMem_Malloc() to reuse pymalloc ;-)

Perhaps if you add some guards somewhere :-)

Seriously, this may work if C extensions use the APIs
consistently, but in order to tell, we'd need to check
few. I know that I switched over all mx Extensions to
use PyObject_*() instead of PyMem_*() or native malloc()
several years ago and have not run into any issues.

I guess the main question then is whether pymalloc is good enough
for general memory allocation needs; and the answer may well be
"yes".

BTW: Tuning pymalloc for commonly used object sizes is
another area where Python could gain better performance,
i.e. reserve more / pre-allocate space for often used block
sizes. pymalloc will also only work well for small blocks
(up to 512 bytes). Everything else is routed to the
system malloc().

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Feb 04 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

[M.-A. Lemburg]

On 12.02.2016 12:18, Victor Stinner wrote:
> ping?

Sorry, your email must gotten lost in my inbox.

> 2016-02-08 15:18 GMT+01:00 Victor Stinner :
>> 2016-02-04 15:05 GMT+01:00 M.-A. Lemburg :
>>> Sometimes, yes, but we also do allocations for e.g.
>>> parsing values in Python argument tuples (e.g. using
>>> "es" or "et"):
>>>
>>> https://docs.python.org/3.6/c-api/arg.html
>>>
>>> We do document to use PyMem_Free() on those; not sure whether
>>> everyone does this though.
>>
>> It's well documented. If programs start to crash, they must be fixed.
>>
>> I don't propose to "break the API" for free, but to get a speedup on
>> the overall Python.
>>
>> And I don't think that we can say that it's an API change, since we
>> already stated that PyMem_Free() must be used.
>>
>> If your program has bugs, you can use a debug build of Python 3.5 to
>> detect misusage of the API.

Yes, but people don't necessarily do this, e.g. I have
for a very long time ignored debug builds completely
and when I started to try them, I found that some of the
things I had been doing with e.g. free list implementations
did not work in debug builds.

>>> The Python test suite doesn't test Python C extensions,
>>> so it's not surprising that it passes :-)
>>
>> What do you mean by "C extensions"? Which modules?
>>
>> Many modules in the stdlib have "C accelerators" and the PEP 399 now
>> *require* to test the C and Python implementations.

Yes, but those are part of the stdlib. You'd need to check
a few C extensions which are not tested as part of the stdlib,
e.g. numpy, scipy, lxml, pillow, etc. (esp. ones which implement custom
types in C since these will often need the memory management
APIs).

It may also be a good idea to check wrapper generators such
as cython, swig, cffi, etc.

>>>> Instead of teaching developers that well, in fact, PyObject_Malloc()
>>>> is unrelated to object programming, I think that it's simpler to
>>>> modify PyMem_Malloc() to reuse pymalloc ;-)
>>>
>>> Perhaps if you add some guards somewhere :-)
>>
>> We have runtime checks but only implemented in debug mode for efficiency.
>>
>> By the way, I proposed once to add an environment variable to allow to
>> enable these checks without having to recompile Python.  Since the PEP
>> 445, it became easy to implement this. What do you think?
>> https://www.python.org/dev/peps/pep-0445/#add-a-new-pydebugmalloc-environment-variable
>>
>> "This alternative was rejected because a new environment variable
>> would make Python initialization even more complex. PEP 432 tries to
>> simplify the CPython startup sequence."
>>
>> The PEP 432 looks stuck, so I don't think that we should block
>> enhancements because of this PEP. Anyway, my idea should be easy to
>> implement.

I suppose such a flag would create a noticeable runtime
performance hit, since the compiler would no longer be
able to inline the PyMem_*() APIs if you redirect those
APIs to other sets at runtime.

I also don't see much point in carrying around such
baggage in production builds of Python, since you'd most
likely only want to use the tools to debug C extensions during
their development.

>>> Seriously, this may work if C extensions use the APIs
>>> consistently, but in order to tell, we'd need to check
>>> few.
>>
>> Can you suggest me names of projects that must be tested?

See above for a list of starters :-)

It would be good to add a few more that work on text or
larger chunks of memory, since those will most likely utilize
the memory allocators more than other extensions which mostly
wrap (sets of) C variables.

Some of them may also have benchmarks, so in addition to
checking whether they work with the change, you could also
test performance.

>>> I guess the main question then is whether pymalloc is good enough
>>> for general memory allocation needs; and the answer may well be
>>> "yes".
>>
>> What do you mean by "good enough"? For the runtime performance,
>> pymalloc looks to be faster than malloc(). What are your other
>> criterias? Memory fragmentation?

Runtime performance, difference in memory consumption (arenas
cannot be freed if there are still small chunks allocated),
memory locality. I'm no expert in this, so can't really
comment much.

I suspect that lib C and OS provided allocators will have
advantages as well, but since pymalloc redirects to them for
all larger memory chunks, it's probably an overall 

Re: [Python-Dev] PEP 493: HTTPS verification migration tools for Python 2.7

[M.-A. Lemburg]

On 24.02.2016 12:28, Cory Benfield wrote:
> 
>> On 24 Feb 2016, at 10:32, Nick Coghlan  wrote:
>>
>> Security Considerations
>> ---
>>
>> Relative to the behaviour in Python 3.4.3+ and Python 2.7.9->2.7.11, this
>> approach does introduce a new downgrade attack against the default security
>> settings that potentially allows a sufficiently determined attacker to revert
>> Python to the default behaviour used in CPython 2.7.8 and earlier releases.
>> However, such an attack requires the ability to modify the execution
>> environment of a Python process prior to the import of the ``ssl`` module,
>> and any attacker with such access would already be able to modify the
>> behaviour of the underlying OpenSSL implementation.
>>
> 
> I’m not entirely sure this is accurate. Specifically, an attacker that is 
> able to set environment variables but nothing else (no filesystem access) 
> would be able to disable hostname validation. To my knowledge this is the 
> only environment variable that could be set that would do that.

An attacker with access to the OS environment of a process would
be able to do lots of things. I think disabling certificate checks
is not one of the highest ranked attack vectors you'd use, given
such capabilities :-)

Think of LD_PRELOAD attacks, LD_LIBRARY_PATH manipulations, shell PATH
manipulations (think spawned processes), compiler flag manipulations
(think "pip install sourcepkg"), OpenSSL reconfiguration, etc.

Probably much easier than an active attack would be to simply extract
sensitive information from the environ and use this for more direct
attacks, e.g. accessing databases, payment services, etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Feb 24 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 493: HTTPS verification migration tools for Python 2.7

[M.-A. Lemburg]

On 24.02.2016 21:39, Cory Benfield wrote:
> 
>> On 24 Feb 2016, at 12:19, M.-A. Lemburg  wrote:
>>
>> On 24.02.2016 12:28, Cory Benfield wrote:
>>>
>>>> On 24 Feb 2016, at 10:32, Nick Coghlan  wrote:
>>>>
>>>> Security Considerations
>>>> ---
>>>>
>>>> Relative to the behaviour in Python 3.4.3+ and Python 2.7.9->2.7.11, this
>>>> approach does introduce a new downgrade attack against the default security
>>>> settings that potentially allows a sufficiently determined attacker to 
>>>> revert
>>>> Python to the default behaviour used in CPython 2.7.8 and earlier releases.
>>>> However, such an attack requires the ability to modify the execution
>>>> environment of a Python process prior to the import of the ``ssl`` module,
>>>> and any attacker with such access would already be able to modify the
>>>> behaviour of the underlying OpenSSL implementation.
>>>>
>>>
>>> I’m not entirely sure this is accurate. Specifically, an attacker that is 
>>> able to set environment variables but nothing else (no filesystem access) 
>>> would be able to disable hostname validation. To my knowledge this is the 
>>> only environment variable that could be set that would do that.
>>
>> An attacker with access to the OS environment of a process would
>> be able to do lots of things. I think disabling certificate checks
>> is not one of the highest ranked attack vectors you'd use, given
>> such capabilities :-)
>>
>> Think of LD_PRELOAD attacks, LD_LIBRARY_PATH manipulations, shell PATH
>> manipulations (think spawned processes), compiler flag manipulations
>> (think "pip install sourcepkg"), OpenSSL reconfiguration, etc.
>>
>> Probably much easier than an active attack would be to simply extract
>> sensitive information from the environ and use this for more direct
>> attacks, e.g. accessing databases, payment services, etc.
> 
> To be clear, I’m not suggesting that this represents a reason not to do any 
> of this, just that we should not suggest that there is no risk here: there 
> is, and it is a new attack vector.

Fair enough :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Feb 24 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Very old git mirror under github user "python-git"

[M.-A. Lemburg]

On 28.02.2016 18:46, Georg Brandl wrote:
> On 02/27/2016 11:45 PM, Matthias Bussonnier wrote:
>> Hi all, 
>>
>>
>>> On Feb 27, 2016, at 14:21, Alexander Walters >> > wrote:
>>>
>>> Can we even ask github to pull it down and reasonably expect them to comply?
>>>  Their entire model is built on everyone forking everyone else.
>>
>> Why the model is everyone forking, some of the help page of GitHub actually 
>> tell
>> you to contact GitHub support, like if you desire to "detach" a fork. 
>>
>> Every reasonable requests I made to GitHub and the few interactions I had 
>> with
>> the support always went well. 
>> This did include asking GitHub to contact user as their pages were confusing,
>> and might be misleading others.
>>
>> So I would suggest 
>>
>> 1) asking GitHub to contact author, potentially forwarding him/her a message
>> from this list asking him/her to bring that down or transfer the control to 
>> you.
>> That should be easy to do as it will not force GitHub to provide anyone with 
>> the
>> emails of the the owner of python-git.
>>
>> 2) in the case of no response from author ask politely GitHub that the repo 
>> is
>> confusing for user, and ask what they can do about that.
> 
> These are both fine.  Although I don't see much confusion; there's bound to be
> hundreds of forks of CPython, if not already, then definitely once we move to
> GitHub.
> 
>> 3) If still nothing can be done make a DMCA request. You can likely argue 
>> that
>> the logo/name are used without PSF content. 
>> https://help.github.com/articles/dmca-takedown-policy/ 
> 
> Please no.  There is absolutely no call using such a blunt instrument, just 
> for
> a case of minor inconvenience.  It could also be blown up into a PR disaster,
> probably rightly so.

I frankly don't understand what all the fuzz is about. The repo
in question hasn't been touched in 7 years. It refers to Python 2.7
alpha 0.

It also clearly reads "Unofficial Python SVN auto-updating mirror",
so there's no confusion either.

The talk about DMCA requests really doesn't apply. Python is
open-source. Anyone can fork it, at any version they like, as
long as the license is respected.

The trademark use is also perfectly in line with our TM policy.
The logo is a bit blurred, but that's really the only nit I could
find.

Asking the owner to take the repo down is still a good thought,
but there's definitely nothing wrong with it per se.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Feb 28 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What does a double coding cookie mean?

[M.-A. Lemburg]

On 16.03.2016 01:28, Guido van Rossum wrote:
> I agree that the spirit of the PEP is to stop at the first coding
> cookie found. Would it be okay if I updated the PEP to clarify this?
> I'll definitely also update the docs.

+1

The only reason to read up to two lines was to address the use of
the shebang on Unix, not to be able to define two competing
source code encodings :-)

> On Tue, Mar 15, 2016 at 2:04 PM, Brett Cannon  wrote:
>>
>>
>> On Tue, 15 Mar 2016 at 13:31 Guido van Rossum  wrote:
>>>
>>> I came across a file that had two different coding cookies -- one on
>>> the first line and one on the second. CPython uses the first, but mypy
>>> happens to use the second. I couldn't find anything in the spec or
>>> docs ruling out the second interpretation. Does anyone have a
>>> suggestion (apart from following CPython)?
>>>
>>> Reference: https://github.com/python/mypy/issues/1281
>>
>>
>> I think the spirit of PEP 263 is for the first specified encoding to win as
>> the support of two lines is to support shebangs and not multiple encodings
>> :) . I also think the fact that tokenize.detect_encoding() doesn't
>> automatically read two lines from its input also suggests the intent is
>> "first encoding wins" (and that is the semantics of the function).
> 
> 
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Mar 16 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-03-07: Released eGenix pyOpenSSL 0.13.14 ... http://egenix.com/go89
2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What does a double coding cookie mean?

[M.-A. Lemburg]

On 17.03.2016 15:55, Guido van Rossum wrote:
> On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka  wrote:
>>> Should we recommend that everyone use tokenize.detect_encoding()?
>>
>> Likely. However the interface of tokenize.detect_encoding() is not very
>> simple.
> 
> I just found that out yesterday. You have to give it a readline()
> function, which is cumbersome if all you have is a (byte) string and
> you don't want to split it on lines just yet. And the readline()
> function raises SyntaxError when the encoding isn't right. I wish
> there were a lower-level helper that just took a line and told you
> what the encoding in it was, if any. Then the rest of the logic can be
> handled by the caller (including the logic of trying up to two lines).

I've uploaded the code I posted yesterday, modified to address
some of the issues it had to github:

https://github.com/malemburg/python-snippets/blob/master/detect_source_encoding.py

I'm pretty sure the two-lines read can be optimized away and
put straight into the regular expression used for matching.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Mar 18 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-03-07: Released eGenix pyOpenSSL 0.13.14 ... http://egenix.com/go89
2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

From openstack-dev-bounces+archive=mail-archive@lists.openstack.org Sat Mar 
19 08:12:23 2016
Return-path: 

Envelope-to: arch...@mail-archive.com
Delivery-date: Sat, 19 Mar 2016 08:12:23 -0700
Received: from bolt10a.mxthunder.net ([209.105.224.168])
by mail-archive.com with esmtp (Exim 4.76)
(envelope-from 
)
id 1ahIYA-0002fr-Cy
for arch...@mail-archive.com; Sat, 19 Mar 2016 08:12:22 -0700
Received: by bolt10a.mxthunder.net (Postfix, from userid 12345)
id 3qRVGK3w2Rz19ktG; Fri, 18 Mar 2016 08:56:36 -0700 (PDT)
Received: from lists.openstack.org (lists.openstack.org [50.56.173.222])
(using TLSv1 with cipher AES256-SHA (256/256 bits))
(No client certificate requested)
by bolt10a.mxthunder.net (Postfix) with ESMTPS id 3qRVFs4wjPz19kcC
for ; Fri, 18 Mar 2016 08:56:33 -0700 (PDT)
Received: from localhost ([127.0.0.1] helo=lists.openstack.org)
by lists.openstack.org with esmtp (Exim 4.76)
(envelope-from )
id 1agwhl-0005a9-59; Fri, 18 Mar 2016 15:52:49 +
Received: from g4t3426.houston.hp.com ([15.201.208.54])
 by lists.openstack.org with esmtp (Exim 4.76)
 (envelope-from ) id 1agwhj-0005YM-O5
 for openstack-...@lists.openstack.org; Fri, 18 Mar 2016 15:52:47 +
Received: from G4W9121.americas.hpqcorp.net (g4w9121.houston.hp.com
 [16.210.21.16]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits))
 (No client certificate requested)
 by g4t3426.houston.hp.com (Postfix) with ESMTPS id 605A664
 for ; Fri, 18 Mar 2016 15:52:47 + (UTC)
Received: from G4W9121.americas.hpqcorp.net (16.210.21.16) by
 G4W9121.americas.hpqcorp.net (16.210.21.16) with Microsoft SMTP Server (TLS)
 id 15.0.1076.9; Fri, 18 Mar 2016 15:52:38 +
Received: from G4W6304.americas.hpqcorp.net (16.210.26.229) by
 G4W9121.americas.hpqcorp.net (16.210.21.16) with Microsoft SMTP Server (TLS)
 id 15.0.1076.9 via Frontend Transport; Fri, 18 Mar 2016 15:52:38 +
Received: from G9W0750.americas.hpqcorp.net ([169.254.9.246]) by
 G4W6304.americas.hpqcorp.net ([16.210.26.229]) with mapi id 14.03.0169.001;
 Fri, 18 Mar 2016 15:52:38 +
From: "Hayes, Graham" 
To: "OpenStack Development Mailing List (not for usage questions)"
 
Thread-Topic: [openstack-dev] [all][infra][ptls] tagging reviews, making
 tags searchable
Thread-Index: AQHRgSbdQRQMpxFOeUKGrubFuR+3RA==
Date: Fri, 18 Mar 2016 15:52:37 +
Message-ID: 
<325f898546fbbf4487d24d6d606a277e1b17a...@g9w0750.americas.hpqcorp.net>
References: 

 <325f898546fbbf4487d24d6d606a277e1b179...@g9w0750.americas.hpqcorp.net>
 

Accept-Language: en-IE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [89.101.195.206]
MIME-Version: 1.0
X-Topics: Infrastructure
Cross-project co

Re: [Python-Dev] What does a double coding cookie mean?

[M.-A. Lemburg]

On 17.03.2016 01:29, Guido van Rossum wrote:
> I've updated the PEP. Please review. I decided not to update the
> Unicode howto (the thing is too obscure). Serhiy, you're probably in a
> better position to fix the code looking for cookies to pick the first
> one if there are two on the same line (or do whatever you think should
> be done there).

Thanks, will do.

> Should we recommend that everyone use tokenize.detect_encoding()?

I'd prefer a separate utility for this somewhere, since
tokenize.detect_encoding() is not available in Python 2.

I've attached an example implementation with tests, which works
in Python 2.7 and 3.

> On Wed, Mar 16, 2016 at 5:05 PM, Guido van Rossum  wrote:
>> On Wed, Mar 16, 2016 at 12:59 AM, M.-A. Lemburg  wrote:
>>> The only reason to read up to two lines was to address the use of
>>> the shebang on Unix, not to be able to define two competing
>>> source code encodings :-)
>>
>> I know. I was just surprised that the PEP was sufficiently vague about
>> it that when I found that mypy picked the second if there were two, I
>> couldn't prove to myself that it was violating the PEP. I'd rather
>> clarify the PEP than rely on the reasoning presented earlier here.

I suppose it's a rather rare case, since it's the first time
that I heard about anyone thinking that a possible second line
could be picked - after 15 years :-)

>> I don't like erroring out when there are two different cookies on two
>> lines; I feel that the spirit of the PEP is to read up to two lines
>> until a cookie is found, whichever comes first.
>>
>> I will update the regex in the PEP too (or change the wording to avoid 
>> "match").
>>
>> I'm not sure what to do if there are two cooking on one line. If
>> CPython currently picks the latter we may want to preserve that
>> behavior.
>>
>> Should we recommend that everyone use tokenize.detect_encoding()?
>>
>> --
>> --Guido van Rossum (python.org/~guido)
> 
> 
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Mar 17 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-03-07: Released eGenix pyOpenSSL 0.13.14 ... http://egenix.com/go89
2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

#!/usr/bin/python
"""
Utility to detect the source code encoding of a Python file.

Marc-Andre Lemburg, 2016.

Supports Python 2.7 and 3.

"""
import sys
import re
import codecs

# Debug output ?
_debug = True

# PEP 263 RE
PEP263 = re.compile(b'^[ \t]*#.*?coding[:=][ \t]*([-.a-zA-Z0-9]+)',
re.MULTILINE)

###

def detect_source_encoding(code, buffer_size=400):

""" Detect and return the source code encoding of the Python code
given in code.

code must be given as bytes.

The function uses a buffer to determine the first two code lines
with a default size of 400 bytes/code points.  This can be adjusted
using the buffer_size parameter.

"""
# Get the first two lines
first_two_lines = b'\n'.join(code[:buffer_size].splitlines()[:2])
# BOMs override any source code encoding comments
if first_two_lines.startswith(codecs.BOM):
return 'utf-8'
# .search() picks the first occurrance
m = PEP263.search(first_two_lines)
if m is None:
return 'ascii'
return m.group(1).decode('ascii')

# Tests

def _test():

l = (
  (b"""\
# No encoding
""", 'ascii'),
  (b"""\
# coding: latin-1
""", 'latin-1'),
  (b"""\
#!/usr/bin/python
# coding: utf-8
""", 'utf-8'),
  (b"""\
coding=123
# The above could be detected as source code encoding
""", 'ascii'),
  (b"""\
# coding: latin-1
# coding: utf-8
""", 'latin-1'),
  (b"""\
# No encoding on first line
# No encoding on second line
# coding: u

Re: [Python-Dev] What does a double coding cookie mean?

[M.-A. Lemburg]

On 17.03.2016 18:53, Serhiy Storchaka wrote:
> On 17.03.16 19:23, M.-A. Lemburg wrote:
>> On 17.03.2016 15:02, Serhiy Storchaka wrote:
>>> On 17.03.16 15:14, M.-A. Lemburg wrote:
>>>> On 17.03.2016 01:29, Guido van Rossum wrote:
>>>>> Should we recommend that everyone use tokenize.detect_encoding()?
>>>>
>>>> I'd prefer a separate utility for this somewhere, since
>>>> tokenize.detect_encoding() is not available in Python 2.
>>>>
>>>> I've attached an example implementation with tests, which works
>>>> in Python 2.7 and 3.
>>>
>>> Sorry, but this code doesn't match the behaviour of Python interpreter,
>>> nor other tools. I suggest to backport tokenize.detect_encoding() (but
>>> be aware that the default encoding in Python 2 is ASCII, not UTF-8).
>>
>> Yes, I got the default for Python 3 wrong. I'll fix that. Thanks
>> for the note.
>>
>> What other aspects are different than what Python implements ?
> 
> 1. If there is a BOM and coding cookie, the source encoding is "utf-8-sig".

Ok, that makes sense (even though it's not mandated by the PEP;
the utf-8-sig codec didn't exist yet).

> 2. If there is a BOM and coding cookie is not 'utf-8', this is an error.

It's an error for Python, but why should a detection function
always raise an error for this case ? It would probably be a good
idea to have an errors parameter to leave this to the use to decide.

Same for unknown encodings.

> 3. If the first line is not blank or comment line, the coding cookie is
> not searched in the second line.

Hmm, the PEP does allow having the coding cookie in the
second line, even if the first line is not a comment. Perhaps
that's not really needed.

> 4. Encoding name should be canonized. "UTF8", "utf8", "utf_8" and
> "utf-8" is the same encoding (and all are changed to "utf-8-sig" with BOM).

Well, that's cosmetics :-) The codec system will take care of
this when needed.

> 5. There isn't the limit of 400 bytes. Actually there is a bug with
> handling long lines in current code, but even with this bug the limit is
> larger.

I think it's a reasonable limit, since shebang lines may only be
127 long on at least Linux (and probably several other Unix systems
as well).

But just in case, I made this configurable :-)

> 6. I made a mistake in the regular expression, missed the underscore.

I added it.

> tokenize.detect_encoding() is the closest imitation of the behavior of
> Python interpreter.

Probably, but that doesn't us on Python 2, right ?

I'll upload the script to github later today or tomorrow to
continue development.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Mar 17 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-03-07: Released eGenix pyOpenSSL 0.13.14 ... http://egenix.com/go89
2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What does a double coding cookie mean?

[M.-A. Lemburg]

On 17.03.2016 15:02, Serhiy Storchaka wrote:
> On 17.03.16 15:14, M.-A. Lemburg wrote:
>> On 17.03.2016 01:29, Guido van Rossum wrote:
>>> Should we recommend that everyone use tokenize.detect_encoding()?
>>
>> I'd prefer a separate utility for this somewhere, since
>> tokenize.detect_encoding() is not available in Python 2.
>>
>> I've attached an example implementation with tests, which works
>> in Python 2.7 and 3.
> 
> Sorry, but this code doesn't match the behaviour of Python interpreter,
> nor other tools. I suggest to backport tokenize.detect_encoding() (but
> be aware that the default encoding in Python 2 is ASCII, not UTF-8).

Yes, I got the default for Python 3 wrong. I'll fix that. Thanks
for the note.

What other aspects are different than what Python implements ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Mar 17 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/

2016-03-07: Released eGenix pyOpenSSL 0.13.14 ... http://egenix.com/go89
2016-02-19: Released eGenix PyRun 2.1.2 ...   http://egenix.com/go88

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 27.05.2016 06:54, Raymond Hettinger wrote:
> 
>> On May 25, 2016, at 3:29 AM, Christian Heimes  wrote:
>>
>> I have three hashing-related patches for Python 3.6 that are waiting for
>> review. Altogether the three patches add ten new hash algorithms to the
>> hashlib module: SHA3 (224, 256, 384, 512), SHAKE (SHA3 XOF 128, 256),
>> BLAKE2 (blake2b, blake2s) and truncated SHA512 (224, 256).
> 
> Do we really need ten?  I don't think the standard library is the place to 
> offer all variants of hashing.  And we should avoid getting in a cycle of 
> "this was just released by NIST" and "nobody uses that one anymore".  Is any 
> one of them an emergent best practice (i.e. starting to be commonly used in 
> network protocols because it is better, faster, stronger, etc)?
> 
> Your last message on https://bugs.python.org/issue16113 suggests that these 
> aren't essential and that there is room for debate about whether some of them 
> are standard-library worthy (i.e. we will have them around forever).

I can understand your eagerness to get this landed, since it's
been 4 years since work started, but I think we should wait with
the addition until OpenSSL has them:

https://github.com/openssl/openssl/issues/439

The current patch is 1.2MB for SHA-3 - that's pretty heavy for just
a few hash functions, which aren't in any wide spread use yet and
probably won't be for quite a few years ahead.

IMO, relying on OpenSSL is a better strategy than providing
(and maintaining) our own compatibility versions. Until OpenSSL
has them, people can use Björn's package:

https://github.com/bjornedstrom/python-sha3

Perhaps you could join forces with Björn to create a standard
SHA-3 standalone package on PyPI based on your two variants
which we could recommend to people in the docs ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 27 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 27.05.2016 13:03, Donald Stufft wrote:
> 
>> On May 27, 2016, at 6:54 AM, M.-A. Lemburg  wrote:
>>
>> IMO, relying on OpenSSL is a better strategy than providing
>> (and maintaining) our own compatibility versions. Until OpenSSL
>> has them, people can use Björn's package:
> 
> Even now, hashlib doesn’t rely on OpenSSL if I recall, I mean it will
> use it if OpenSSL is available but otherwise it has internal implementations
> too.

I know, but still don't think that's a good idea. It makes
sense in case you don't want to carry around OpenSSL all the
time, but how often does that happen nowadays ?

BTW: If I recall correctly, those hash implementations predate
the deeper support for OpenSSL we now have in Python.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 27 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 27.05.2016 17:44, Chris Barker - NOAA Federal wrote:
>>> , which aren't in any wide spread use yet and
>> probably won't be for quite a few years ahead.
> 
> Anything added to the stdlib now will be in py3.6+, yes?
> 
> Which won't be in widespread use for quite a few years yet, either.
> 
> So if ( and that's a big if) it's possible to anticipate what will be
> in widespread use in a couple years, getting it in now would be a good
> thing.

You cut away the important part of what I said:
"The current patch is 1.2MB for SHA-3 - that's pretty heavy for
just a few hash functions, ..."

If people want to use the hashes earlier, this is already possible
via a separate package, so we're not delaying their use.

It is clear that SHA-3 will get more traction in coming years (*),
but I'm pretty sure that OpenSSL will have good implementations by
the time people will actively start using the new hash algorithm
and then hashlib will automatically make that available (hashlib
uses the OpenSSL EVP abstraction, so will be able to use any
new algorithms added to OpenSSL).

However, if we add the reference implementation now, we'd then be
left with 1.2MB unnecessary code in the stdlib.

The question is not so much: is SHA-3 useful or not, it's
whether we want to maintain this forever going forward or
not.

(*) People are just now starting to move from SHA-1 to SHA-2
and SHA-2 was standardized in 2001. Python received SHA-2 support
in 2006. So there's plenty of time to decide :-)

> -CHB
> 
> 
> 
>>
>> IMO, relying on OpenSSL is a better strategy than providing
>> (and maintaining) our own compatibility versions. Until OpenSSL
>> has them, people can use Björn's package:
>>
>> https://github.com/bjornedstrom/python-sha3
>>
>> Perhaps you could join forces with Björn to create a standard
>> SHA-3 standalone package on PyPI based on your two variants
>> which we could recommend to people in the docs ?!
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Experts (#1, May 27 2016)
> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> Python Database Interfaces ...   http://products.egenix.com/
> Plone/Zope Database Interfaces ...   http://zope.egenix.com/
>> 
>>
>> ::: We implement business ideas - efficiently in both time and costs :::
>>
>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>   Registered at Amtsgericht Duesseldorf: HRB 46611
>>   http://www.egenix.com/company/contact/
>>  http://www.malemburg.com/
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 27 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 27.05.2016 18:41, Chris Barker wrote:
> On Fri, May 27, 2016 at 9:35 AM, M.-A. Lemburg  wrote:
> 
>>> So if ( and that's a big if) it's possible to anticipate what will be
>>> in widespread use in a couple years, getting it in now would be a good
>>> thing.
>>
>> You cut away the important part of what I said:
>> "The current patch is 1.2MB for SHA-3 - that's pretty heavy for
>> just a few hash functions, ..."
>>
>> If people want to use the hashes earlier, this is already possible
>> via a separate package, so we're not delaying their use.
>>
> 
> That's true for ANY addition to the stdlib -- it could always be made
> available in a third party lib. (unless you want to use it in another part
> of the stdlib...)

Well, any addition for which someone already wrote a package,
but yes...

>> It is clear that SHA-3 will get more traction in coming years (*),
>> but I'm pretty sure that OpenSSL will have good implementations by
>> the time people will actively start using the new hash algorithm
>> and then hashlib will automatically make that available (hashlib
>> uses the OpenSSL EVP abstraction, so will be able to use any
>> new algorithms added to OpenSSL).
>>
>> However, if we add the reference implementation now, we'd then be
>> left with 1.2MB unnecessary code in the stdlib.
>>
> 
> I'm probably showing my ignorance here, but couldn't we swap in the OpenSSL
> implementation when that becomes available?

We could, but only if we don't expose separate interfaces
for the hashes and not add them to hashlib.

hashlib.algorithms
hashlib.algorithms_guaranteed

> -CHB
> 
> 
> (*) People are just now starting to move from SHA-1 to SHA-2
>> and SHA-2 was standardized in 2001. Python received SHA-2 support
>> in 2006. So there's plenty of time to decide :-)
> 
> 
> can't deny the history, nor the inertia -- but that doesn't make it a good
> thing...
> 
> 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 27 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 27.05.2016 22:58, Ryan Gonzalez wrote:
> On May 27, 2016 3:04 PM, "Victor Stinner"  wrote:
>>
>> Le vendredi 27 mai 2016, M.-A. Lemburg  a écrit :
>>>
>>> The current patch is 1.2MB for SHA-3 - that's pretty heavy for just
>>> a few hash functions, which aren't in any wide spread use yet and
>>> probably won't be for quite a few years ahead.
>>
>>
>> Oh wow, it's so fat? Why is it so big? Can't we use a lighter version?
>>
> 
> The stark majority of the patch is Lib/test/vectors/sha3_224.txt, which
> seems to be (as the file path implies) just test data. A whopping >1k LOC
> of really long hashes.

Right. There's about 1MB test data in the patch, but even
without that data, the patch adds more than 6400 lines of code.

If we add this now, there should at least be an exit strategy
to remove the code again, when OpenSSL ships with the same
code, IMO.

Aside: BLAKE2 has already landed in OpenSSL 1.1.0:

https://github.com/openssl/openssl/tree/master/crypto/blake2

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 27 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 27.05.2016 23:46, Donald Stufft wrote:
> 
>> On May 27, 2016, at 5:41 PM, M.-A. Lemburg  wrote:
>>
>> If we add this now, there should at least be an exit strategy
>> to remove the code again, when OpenSSL ships with the same
>> code, IMO.
> 
> I think it is a clear win to have the fallback implementations in cases where 
> people either don’t have OpenSSL or don’t have a new enough OpenSSL for those 
> implementations. Not having the fallback just makes it more difficult for 
> people to rely on those hash functions.

This will only be needed once the stdlib itself starts requiring
support for some of these hashes and for that we could add
a pure Python implementation, eg.

https://github.com/coruus/py-keccak

In all other cases, you can simply add the support via a
package such as Björn's or Christian's.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 27 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

[M.-A. Lemburg]

On 28.05.2016 23:13, Christian Heimes wrote:
> On 2016-05-27 14:41, M.-A. Lemburg wrote:
>> On 27.05.2016 22:58, Ryan Gonzalez wrote:
>>> On May 27, 2016 3:04 PM, "Victor Stinner"  wrote:
>>>>
>>>> Le vendredi 27 mai 2016, M.-A. Lemburg  a écrit :
>>>>>
>>>>> The current patch is 1.2MB for SHA-3 - that's pretty heavy for just
>>>>> a few hash functions, which aren't in any wide spread use yet and
>>>>> probably won't be for quite a few years ahead.
>>>>
>>>>
>>>> Oh wow, it's so fat? Why is it so big? Can't we use a lighter version?
>>>>
>>>
>>> The stark majority of the patch is Lib/test/vectors/sha3_224.txt, which
>>> seems to be (as the file path implies) just test data. A whopping >1k LOC
>>> of really long hashes.
>>
>> Right. There's about 1MB test data in the patch, but even
>> without that data, the patch adds more than 6400 lines of code.
> 
> The KeccakCodePackage is rather large. I already removed all unnecessary
> files and modified some files so more code is shared between 32 and
> 64bit optimized variants. Please keep in mind that the KCP contains
> multiple implementations with different optimizations for CPU
> architectures. I already removed the ARM NEON optimization.
> I also don't get your obsession with lines of code. The gzip and expat
> are far bigger than the KeccakCodePackage.

For a small piece of code, it's fine to have a copy in the
stdlib, but for larger chunks such as this one, I think we
ought to consider alternative options, since I don't think
it's good to have to carry around this baggage forever.

OpenSSL will eventually have good enough support for what
most Python users will need from these new hash functions.
That's why I think it's better to have a discussion of whether
we need to full package in the stdlib or better only provide
limited support built into the stdlib and refer people to
PyPI packages for things that you don't need every day.

Regarding the stories for zlib and expat, I only remember
that expat was essentially unmaintained when we added it
and the existing version at the time had known bugs (but
could be wrong).

For zlib, I have no clue as to why we have a copy in the stdlib.
That lib is available on all systems nowadays. Perhaps it wasn't
when we added it; don't remember. If so, it's a good example of
why adding copies to the stdlib is not such a good idea :-)

>> If we add this now, there should at least be an exit strategy
>> to remove the code again, when OpenSSL ships with the same
>> code, IMO.
>>
>> Aside: BLAKE2 has already landed in OpenSSL 1.1.0:
>>
>> https://github.com/openssl/openssl/tree/master/crypto/blake2
> 
> Except BLAKE2 in OpenSSL is severely castrated and tailored towards a
> very limited use case. The implementation does not support any of the
> useful advanced features like keyed hashing (MAC), salt,
> personalization, tree hashing and variable hash length.

I bet that the use cases they put into OpenSSL is what
most people will eventually use, so essentially the same
reasoning we use for putting stuff into the stdlib.

Besides, the code just landed in OpenSSL. It's likely they'll
continue to optimize it and possibly also add the variants
they left out initially.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 29 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Infrastructure] bugs.python.org not reachable in IPv6?

[M.-A. Lemburg]

On 04.11.2013 16:10, Benjamin Peterson wrote:
> 2013/11/4 M.-A. Lemburg :
>> On 04.11.2013 11:01, Victor Stinner wrote:
>>> Hi,
>>>
>>> bugs.python.org is still not responding on IPv6. Can someone please
>>> remove the  record from python.org DNS, or fix the IPv6
>>> configuration?
>>>
>>> It's an issue on my PC because my PC has IPv6 address and so it cannot
>>> reach bugs.python.org. wget is really slow because it tries first
>>> IPv6, but then falls back to IPv4. Firefox tries first IPv4 and so
>>> doesn't have the issue.
>>
>> The infrastructure team cannot do anything about this, since
>> the tracker is hosted by Upfront Systems.
> 
> But the PSF controls the DNS, no?
> 
> ;; AUTHORITY SECTION:
> python.org. 2396IN  NS  ns2.p11.dynect.net.
> python.org. 2396IN  NS  ns3.p11.dynect.net.
> python.org. 2396IN  NS  ns1.p11.dynect.net.
> python.org. 2396IN  NS  ns4.p11.dynect.net.

Yes, but only Upfront Systems knows about the correct IPv6 address :-)

Simply removing it would be short term work-around, though.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 04 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2013-11-19: Python Meeting Duesseldorf ... 15 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Infrastructure] bugs.python.org not reachable in IPv6?

[M.-A. Lemburg]

On 04.11.2013 11:01, Victor Stinner wrote:
> Hi,
> 
> bugs.python.org is still not responding on IPv6. Can someone please
> remove the  record from python.org DNS, or fix the IPv6
> configuration?
> 
> It's an issue on my PC because my PC has IPv6 address and so it cannot
> reach bugs.python.org. wget is really slow because it tries first
> IPv6, but then falls back to IPv4. Firefox tries first IPv4 and so
> doesn't have the issue.

The infrastructure team cannot do anything about this, since
the tracker is hosted by Upfront Systems.

They are hosting the instance at Hetzner in Germany, which does
support IPv6.

Do we have someone with login permission for the server running
RoundUp ?

If not, we should probably find someone at Upfront Systems to check
the configuration.

> Victor
> 
> 2013/10/12 Antoine Pitrou :
>>
>> Opened issue at http://psf.upfronthosting.co.za/roundup/meta/issue528
>>
>> Regards
>>
>> Antoine.
>>
>>
>>
>> Le samedi 12 octobre 2013 à 14:40 +0200, Victor Stinner a écrit :
>>> Hi,
>>>
>>> The DNS server of python.org announce the IP address 2a01:4f8:131:2480::3:
>>>
>>> $ host -t  bugs.python.org
>>> bugs.python.org has IPv6 address 2a01:4f8:131:2480::3
>>>
>>>
>>> The problem is that I cannot connect to this IP address:
>>>
>>> $ ping6 -c 4 2a01:4f8:131:2480::3
>>> PING 2a01:4f8:131:2480::3(2a01:4f8:131:2480::3) 56 data bytes
>>>
>>> --- 2a01:4f8:131:2480::3 ping statistics ---
>>> 4 packets transmitted, 0 received, 100% packet loss, time 2999ms
>>>
>>>
>>> Do you have a the same issue, or is it just me?
>>>
>>> Victor
>>> 
>>> Infrastructure mailing list
>>> infrastruct...@python.org
>>> https://mail.python.org/mailman/listinfo/infrastructure
>>> Unsubscribe: 
>>> https://mail.python.org/mailman/options/infrastructure/solipsis%40pitrou.net
>>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
> 
> Infrastructure mailing list
> infrastruct...@python.org
> https://mail.python.org/mailman/listinfo/infrastructure
> Unsubscribe: 
> https://mail.python.org/mailman/options/infrastructure/mal%40egenix.com
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 04 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2013-11-19: Python Meeting Duesseldorf ... 15 days to go

: Try our mxODBC.Connect Python Database Interface for free ! ::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com