Re: [Python-Dev] Python 2.7 patch levels turning two digit

[Martin v. Löwis]

>  * Is it a good strategy to ship to Python releases for every
>single OpenSSL security release or is there a better way to
>handle these 3rd party issues ?

At least for Windows, a new release certainly needs to be made.
It could be possible to produce MSI patch files, but this would
still be a new release.

>I think we should link to the OpenSSL libs dynamically rather
>than statically in Python 2.7 for Windows so that it's possible
>to provide drop-in updates for such issues.

It is possible to provide drop-in updates regardless of whether the
OpenSSL libs are dynamically linked, as the _ssl module itself is a
dynamic lib.

>  * Should we try to avoid two digit patch level release numbers
>by using some other mechanism such as e.g. a release date
>after 2.7.9 ?

If it was for me, then yes, certainly: the development of 2.7 should
just stop :-)

>  * Should we make use of the potential breakage with 2.7.10
>to introduce a new Windows compiler version for Python 2.7 ?

Assuming it is a good idea to continue producing Windows binaries
for 2.7, I think it would be a bad idea to switch compilers. It will
cause severe breakage of 2.7 installations, much more problematic
than switching to two-digit version numbers.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 patch levels turning two digit

[Martin v. Löwis]

Am 23.06.14 22:04, schrieb Antoine Pitrou:
> Le 23/06/2014 15:27, M.-A. Lemburg a écrit :
>>
>> Not sure what you mean. We've had binary wininst distributions
>> for Windows for more than a decade, and egg and msi distributions
>> for 8 years :-)
>>
>> But without access to the VS 2008 compiler that is needed to
>> compile those extensions,
> 
> It does seem to be available:
> http://www.microsoft.com/en-us/download/details.aspx?id=13276
> 
> What am I missing?

I believe (without testing) that this is just the service pack.
Installing it requires a pre-existing installation of Visual
Studio 2008, or else the installer will refuse to do anything.
Note that it also won't install on top of Visual Studio Express:
you need a licensed copy of Visual Studio to install the service
pack.

Visual Studio 2008 still *is* available to MSDN users. It's
just not available through regular sales channels anymore.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 patch levels turning two digit

[Martin v. Löwis]

Am 23.06.14 21:53, schrieb Ned Deily:
> It does seem like a conundrum.  As I have no deep Windows experience to 
> be able to have an appreciation of all of the technical issues involved, 
> I ask out of ignorance: would it be possible and desirable to provide a 
> transition period of n 2.7.x maintenance releases (where n is between 1 
> and, say, 3) where we would provide 2 sets of Windows installers, one 
> set (32- and 64-bit) with the older compiler and CRT and one with the 
> newer, and campaign to get users and packagers who provide binary 
> extensions to migrate? 

The question is how exactly you implement the transition. I see two
alternatives:
1. "Hijack" the 2.7 name space, in particular the name "python27.dll",
   along with registry keys, the .pyd extension, etc.
   Doing so would permit users to mix binaries from different compilers,
   and doing so would lead to crashes. Users would have to be careful
   to either install packages built for the old compiler or packages
   for the new compiler, and never mix.
2. "Sandbox" the 2.7 name space; come up with new names for everything.
   E.g. "python27vs13.dll", ".pydvs13" (or "_vs13.pyd"),
   "C:\Python27vs13", along with the corresponding changes to PyPI,
   pip, buildout, etc. which would need to learn to look for the right
   variant of a Python 2.7 package.
   This should work, but might take several years to implement: you
   need to find all the places in existing code that refer to the
   "old" names.
   If you do it right, you are done about the time when VS 13 becomes
   unavailable, so you'd then do another such transition to VS 2015,
   which promises forward-binary compatibility to future releases of
   Visual Studio.

> Would that mitigate the pain, assuming that 
> Steve (or someone else) would be willing to build the additional 
> installers for the transition period?  I've done something similar on a 
> smaller scale with the OS X 32-bit installer for 2.7.x but that impact 
> is much less as the audience for that installer is much smaller.

Well, the question really is whether precompiled extension modules
available from PyPI would work on both compilers. I understand that
for OSX, you typically don't have precompiled binaries for the extension
modules, so installation compiles the modules from scratch. This is
easier, as it can use the ABI of the Python which will be installed
to.

If you go the "parallel ABIs" route, extension authors have to provide
two parallel sets of packages as well. Given 32-bit and 64-bit packages,
this will make actually two additional packages - just as if they had
to support another Python version.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 patch levels turning two digit

[Martin v. Löwis]

Am 23.06.14 22:31, schrieb Barry Warsaw:
> On Jun 23, 2014, at 04:20 PM, Donald Stufft wrote:
> 
>> At the risk of getting Guido to post his slide again, I still think the
>> solution to the old compiler is to just roll a 2.8 with minimal changes.
> 
> No.  It's not going to happen, for all the reasons discussed previously.
> Python 2.8 is not a solution to anything.
> 
> If a new, incompatible compiler suite is required, why can't there just be
> multiple Windows downloads on https://www.python.org/download/releases/2.7.7/
> ?  Well, on reason is that you'd have to convince MvL or someone else to take
> over the work that would require, but that's gotta be *much* lighter weight
> than releasing a Python 2.8.

See my other message. It's actually heavier, since it requires changes
to distutils, PyPI, pip, buildout etc., all which know how to deal with
Python minor version numbers, but are unaware of the notion of competing
ABIs on Windows (except that they know how to deal with 32-bit vs. 64-bit).

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7 patch levels turning two digit

[Martin v. Löwis]

Am 23.06.14 22:31, schrieb Barry Warsaw:
> Well, on reason is that you'd have to convince MvL or someone else to take
> over the work that would require, but that's gotta be *much* lighter weight
> than releasing a Python 2.8.

Just to point this out in a separate message: it will have to be
somebody else. I stepped down as the Windows release maintainer for 2.7
when I learned about the extended life of 2.7, much because I feared
that exactly the thing would happen that we see happen now - and I
didn't want to be the one who would have to deal with it. It is a mess,
and it will get bigger the more time passes.

Playing-the-role-of-Cassandra-ly y'rs,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

[Martin v. Löwis]

Am 01.07.14 09:44, schrieb Victor Stinner:
> scandir(fd) must not close the file descriptor, it should be done by
> the caller. Handling the lifetime of the file descriptor is a
> difficult problem, it's better to let the user decide how to handle
> it.

This is an open issue still: when is the file descriptor closed.

I think the generator returned from scandir needs to support a
.close method that guarantees to close the file descriptor.
AFAICT, the pure-Python prototype of scandir already does, but
it should be specified in the PEP.

While we are at it: is it intended that the generator will also
support the other generator methods, in particular .send and
.throw?

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] buildbot.python.org down again?

[Martin v. Löwis]

Am 08.07.14 16:48, schrieb Guido van Rossum:
> May the true owner of buildbot.python.org 
> stand up!

Well, I think that's me (atleast by my definition of "true owner").
I requested that the machine be set up, and I deployed the software
that is running the service (it was also me who originally introduced
buildbot to the Python project).

On the other hand, I'm not at all "in charge" of that infrastructure
piece. I haven't logged into the machine in many months, and it's
Antoine who currently maintains its configuration. So I don't want to
be pinged when the machine is down.

> (But I do think there may well not be anyone who feels they own it. And
> that's a problem for its long term viability.)

I don't think that's actually the case for "ownership". But then, I also
think that ownership is not a very important concept for pydotorg. Most
owners will likely agree that they lose their right to have a say in it
when they stop maintaining the piece that they own.

> Generally speaking, as an organization we should set up a process for
> managing ownership of *all* infrastructure in a uniform way. I don't
> mean to say that we need to manage all infrastructure uniformly, just
> that we need to have a process for identifying and contacting the
> owner(s) for each piece of infrastructure, as well as collecting other
> information that people besides the owners might need to know. You can
> use a wiki page for that list for all I care, but have a process for
> what belongs there, how/when to update it, and even an owner for the
> wiki page!

Unfortunately, that plan keeps failing. Everybody agrees that such a
list would be useful, so everybody makes their own list. I was
maintaining such a list in the Python wiki for some time, until a
board member decided that a publically-visible inventory is not
appropriate, and it must be a password-protected wiki - where I now
keep forgetting where the wiki is, in the first place, let alone
remembering how to log in.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3121, 384 Refactoring Issues

[Martin v. Löwis]

Am 12.07.14 17:19, schrieb Nick Coghlan:
> Using the stable ABI for standard library extensions also serves to
> decouple them further from the internal details of the CPython runtime,
> making it more likely they will be able to run correctly on alternative
> interpreters (since emulating or otherwise supporting the limited API is
> easier than supporting the whole thing).

There are two features to be gained for the standard library from that

A. with proper module shutdown support, it will be possible to release
   objects that are currently held in C global/static variables, as the
   C global variables will go away. This, in turn, is a step forward in
   the desire to allow for proper leak-free interpreter shutdown, and
   in the desire to base interpreter shutdown on GC.

B. with proper use of heap types (instead of the static type objects),
   support for the multiple-interpreter feature will be improved, since
   type objects will be per-interpreter, instead of being global. This,
   in turn, is desirable since otherwise state changes can leak from
   one interpreter to the other.

So I still maintain that the change is desirable even for the standard
library.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bluetooth 4.0 support in "socket" module

[Martin v. Löwis]

Am 14.07.14 15:57, schrieb Tim Tisdall:
> Also, is there a method to test changes against all the different *nix
> variations?  Is Bluez the standard across the different *nix variations?

Perhaps not the answer you expected, but: Python uses autoconf for
feature testing. You can be certain that the API *will* vary across
system vendors. For example, FreeBSD apparently uses ng_hci(4):

http://www.unix.com/man-page/freebsd/4/ng_hci/

If you add features, all you need to make sure that Python continues
to compile when the platform feature is not present. People using the
other systems are then free to contribute support for their platforms.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Surely "nullable" is a reasonable name?

[Martin v. Löwis]

Am 04.08.14 09:12, schrieb Larry Hastings:
> It's my contention that "nullable" is the correct name.  But I've been
> asked to bring up the topic for discussion, to see if a consensus forms
> around this or around some other name.

I have personally no problems with calling a type "nullable" even in
Python, and, as a type *adjective* this seems to be the right choice
(i.e. I wouldn't say "noneable int" or "allow_none int"; the former is
no established or intuitive term, the latter is not an adjective).

As a type *flag*, flexibility in naming is greater. zeroes=True formally
creates a subtype (of string), and it doesn't hurt that it is not an
adjective. "allow_zeroes" might be more descriptive. bitwise=True
doesn't really create a subtype of int. For the feature in question,
I find both "allow_none" and "nullable" acceptable; "noneable" is not.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 4000 to explicitly declare we won't be doing a Py3k style compatibility break again?

[Martin v. Löwis]

Am 18.08.14 08:45, schrieb Nick Coghlan:
> It's certainly the one that has caused the most churn in CPython and
> the standard library - the ripples still haven't entirely settled on
> that front :)

For people porting their libraries and applications, the challenge is
often even bigger: they need to learn a new programming concept. For
many developers, it is a novel idea that character strings are not
just bytes. A similar split is in the number types (integers vs.
floats), but most developers have learned the distinction when they
learned programming. That a text file is not a file that contains text
(but bytes interpreted as text) is surprising. In addition, you also
have to learn a lot of facts (what is the ASCII encoding, what is
the iso-8859-1 encoding, what is UTF-8 (and how does it differ from
Unicode)).

When you have all that understood, you *then* run into the design
choices to be made for your software.

> 
> I think Guido's right that there's also a "death of a thousand cuts"
> aspect for large existing code bases, though, especially those that
> are lacking comprehensive test suites.

I think the second big challenge is "my dependencies are not ported
to Python 3". There is little you can do about it, short of porting
the dependencies yourself (fortunately, Python and most of its libraries
are free software).

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Martin v. Löwis]

Am 19.08.14 19:43, schrieb Ben Hoyt:
 The official policy is that we want them [support for bytes paths in 
 stdlib functions] to go away, but reality so far has not budged. We will 
 continue to hold our breath though. :-)
>>>
>>> Does that mean that new APIs should explicitly not support bytes? I'm
>>> thinking of os.scandir() (PEP 471), which I'm implementing at the
>>> moment. I was originally going to make it support bytes so it was
>>> compatible with listdir, but maybe that's a bad idea. Bytes paths are
>>> essentially broken on Windows.
>>
>> Bytes paths are "essential" on Unix, though, so I don't think we should
>> create new low-level APIs that don't support bytes.
> 
> Fair enough. I don't quite understand, though -- why is the "official
> policy" to kill something that's "essential" on *nix?

I think the people defending the "Unix file names are just bytes" side
often miss an important detail: displaying file names to the user, and
allowing the user to enter file names.

A script that just needs to traverse a directory tree and look at files
by certain criteria can easily do so with not worrying about a text
interpretation of the file names.

When it comes to user interaction, it becomes apparent that, even on
Unix, file names are not just bytes. If you do "ls -l" in your shell,
the "system" (not just the kernel - but ultimately the terminal program,
which might be the console driver, or an X11 application) will interpret
the file name as having an encoding, and render them with a font.

So for Python, the question is: which of the use cases (processing
all files, vs. showing them to the user) should be better supported?
Python 3 took the latter as an answer, under the assumption that this
is the more common case.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https:bugs.python.org -- Untrusted Connection (Firefox)

[Martin v. Löwis]

Am 21.08.14 17:44, schrieb Nick Coghlan:
> I've now raised this issue with the infrastructure team. The current
> hosting arrangements for bugs.python.org were put in place when the
> PSF didn't have any on-call system administrators of its own, but now
> that we do, it may be time to migrate that service to a location where
> we can switch to a more appropriate SSL certificate.

Just to relay Noah's response: it's actually not the hosting that
prevents installation of a proper certificate, it's the limitation
that the certificate we could deploy would include "python.org" as
a server name, which is considered risky regardless of where the
service is hosted. There are solutions to that as well, of course.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Martin v. Löwis]

Am 22.08.14 01:56, schrieb Glenn Linderman:
> 0 and 47 are certainly originally derived from ASCII.  However, there
> could be lots of encodings that are not ASCII compatible (but in
> practice, probably very few, since most encodings _are_ ASCII
> compatible) that could be fit those constraints.
> 
> So while as a technical matter, Cameron is correct that Unix only treats
> 0 & 47 as special, and that is insufficient to declare that encodings
> must be ASCII compatible, as a practical matter, since most encodings
> are ASCII compatible anyway, it would be hard to find very many that
> could be used successfully with Unix file names that are not ASCII
> compatible, that could comply with the 0 & 47 requirements.

More importantly, existing encodings that are distinctively *not*
ASCII compatible (e.g. the EBCDIC ones) do not put the slash into 47
(instead, it is at 91 at EBCDIC, 47 is the BEL control character).

There are boundary cases, of course. VISCII is "mostly ASCII
compatible", putting graphic characters into some of the control
characters, but using those that aren't used in ASCII, anyway.

And then there is the YUSCII family of encodings, which definitely
is not ASCII compatible, as it does not contain Latin characters,
but still puts the / into 47 (and also keeps the ASCII digits and
special characters in their positions). There is also SI 960, which
has the slash, the ASCII uppercase letters, digits and special
characters, but replaces the lower-case characters with Hebrew.

So yes, Unix doesn't mandate ASCII-compatible encodings; but it
still mandates ASCII-inspired encodings. I wonder how you would
run "gcc", though, on an SI 960 system; you'ld have to type
חדד.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path support

[Martin v. Löwis]

Am 24.08.14 03:11, schrieb Greg Ewing:
> Isaac Morland wrote:
>> In HTML 5 it allows non-ASCII-compatible encodings as long as U+FEFF
>> (byte order mark) is used:
>>
>> http://www.w3.org/TR/html-markup/syntax.html#encoding-declaration
>>
>> Not sure about XML.
> 
> According to Appendix F here:
> 
> http://www.w3.org/TR/xml/#sec-guessing
> 
> an XML parser needs to be prepared to try all the encodings it
> supports until it finds one that works well enough to decode
> the XML declaration, then it can find out the exact encoding
> used.

That's not what this section says. Instead, it says that
you need to auto-detect UCS-4, UTF-16, UTF-8 from the BOM,
or guess them or EBCDIC from the encoding of 'https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multilingual programming article on the Red Hat Developer blog

[Martin v. Löwis]

Am 17.09.14 10:56, schrieb Steven D'Aprano:
> On Wed, Sep 17, 2014 at 09:21:56AM +0900, Stephen J. Turnbull wrote:
> 
>> Guido's mantra is something like "Python's str doesn't contain
>> characters or even code points[1], it contains code units."
> 
> But is that true?

It used to be true, and stopped being so with PEP 393. In particular,
Python 3.2 and before would expose UTF-16 in the narrow build, so the
elements of a string would be code units. Since Python 3.3, the
surrogate code points are not longer interpreted as UTF-16 code units.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.5 release schedule PEP

[Martin v. Löwis]

Am 24.09.14 14:34, schrieb Antoine Pitrou:
> On Wed, 24 Sep 2014 17:12:35 +1000
> Nick Coghlan  wrote:
>> On 24 Sep 2014 15:15, "Tim Golden"  wrote:
>>>
>>> On 23/09/2014 18:05, Steve Dower wrote:

 I'm also considering/experimenting with installing into "Program
 Files" by default, but I suspect that isn't going to work out yet.
>>>

>> It might be better to offer that as a supported option in 3.5, and then
>> make it the default in 3.6.
> 
> I would be surprised if this weren't already a supported option. Decent
> Windows installers generally allow you to override the installation
> path (it's been a while I haven't used the Windows Python installer,
> sorry).

I regularly test whether it installs properly into Program Files
(although regularly actually means "for every major release"). Also,
users regularly report if it doesn't work.

As it turns out, 3.4 indeed got a new problem when installing into
Program Files, namely that pip would not install. As discussed elsewhere
in this thread, installation into Program Files requires elevated
privileges, which meant that I had to reschedule the pip installation
procedure within the MSI to run in the system context.

> 
> I think making the move in 3.5 would be a good idea. Experts can
> override the installation path and choose C:\PythonXY. There's no
> actual breakage since the path changes for every major release, anyway
> (i.e. people would have had to change the path from "C:\Python34" to
> "C:\Python35"; instead, they will have to change it to "C:\Program
> Files\Python35").

I expect breakage elsewhere (as also discussed):
- some code will fail because of the space in the path
- some installations will fail because of the restricted access to
  Program Files (which, of course, is also the main reason why people
  want that install location)

It's of course now up to Steve to rule on the default install location;
if he changes it, he will get lots of both praise and blame, if he
doesn't change it, he will get a little blame for continuing the
tradition.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Impact of Windows PowerShell OneGet ?

[Martin v. Löwis]

Am 29.10.14 20:34, schrieb Glenn Linderman:
> New package manager from M$... article here
> .

I've looked at it, but only by reading its code, not trying it out.
Some notes.

First, what is Chocolatey? It's a PowerShell library, and a package
infrastructure. A Chocolatey package is a PowerShell script that
installs a piece of software on the system. The software itself comes
in a different format. There are installer helpers for exe, msi and msu,
Python packages (running setup.py), Ruby packages, etc. There appears to
be a notion of dependencies also.

OneGet is similar; it is also a PowerShell library. It has the notion
of "providers", which are classes that can make a package installed.
There is the ARP provider (Add-Remove-Programs), which can report that
a package is already installed. There is the MSI provider, which can
install an MSI file that is already on disk. And there is the web
provider which can download a file over http. They claim to include
a re-implementation of Chocolatey (meaning that Chocolatey packages
can be installed), however, ISTM that this only supports exe and msi
packages.

I haven't seen anything resembling dependencies in OneGet.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Impact of Windows PowerShell OneGet ?

[Martin v. Löwis]

> Most likely, OneGet won't replace pip/PyPI, any more than apt or yum
> does; but it may be worth having Python itself available that way.
> That might simply mean having someone package up Python and put it on
> an appropriate server, or maybe python.org could end up hosting a
> repo. 

Python is already available in Chocolatey:

https://chocolatey.org/packages/python

Given that OneGet intends to support Chocolatey as a repository,
it might just work already. All python.org would have to guarantee
is stable URLs for the Python msi files.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New Windows installer for Python 3.5

[Martin v. Löwis]

Am 04.01.15 00:34, schrieb Steve Dower:
> so I'm keen to hear whatever feedback people have. 

One issue that I always wanted to address is patch releases. There are
two aspects I dislike about the current implementation

a) an upgrade install first uninstalls the previous installation
(removing all files), and then reinstalls all "new" files. In many patch
releases, a lot of .py files remain unmodified, so it should
speed up the upgrade if they would not need to be replaced.
b) Installer supports patch files (.msp); I always wanted to provide
them in the hope that this would reduce the download size. IIUC, it
would require stable component IDs for components to be upgraded, which
I could not manage to provide.

So, do you have any plans on dealing with a or b?

Other issues:
- what MSI version do you require, and what is the minimum
  Windows version supporting that MSI version?
- Since you are going to install into Program Files by default,
  I think the library should be precompiled by default - there
  is little chance that a regular user can save .pyc files when
  running Python. It might be possible to even include the pyc
  files in the distribution, if we can arrange to somehow support
  relative paths in co_filename.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

[Martin v. Löwis]

Am 10.02.15 um 16:41 schrieb Steve Dower:
> One of my main engineering concerns is lack of shared knowledge a.k.a. the 
> truck factor (not CPython specific, btw, just every project I work on).

Wrt. the installer, I think this is a lost cause. IIUC, you aren't
really getting familiar with msi.py, and you are the one who has the
highest chances - so I was the only real maintainer of it.
Likewise, the Wise installer was only maintained by Tim Peters.

So chances are really high that you will be the only one making
substantial changes to the WiX files, although some people might learn
how to build the installer.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

[Martin v. Löwis]

Am 10.02.15 um 15:43 schrieb Paul Moore:
> On 10 February 2015 at 14:13, Steve Dower  wrote:
>> I was also surprised as I was working on the installer, so +1 on changing
>> it.
> 
> On a procedural note, does this require a change to the PEP (assuming
> it's generally acceptable)? Or is a patch to the code and docs
> sufficient?

There should be a PEP if the change is likely controversial. In any
case, there should be documentation of the rationale for the change, and
the docs itself is the wrong place for it. The commit message could be
the right place, but if the rationale is a bit more verbose, a bug
tracker item would be right. All IMO, of course.

Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows

[Martin v. Löwis]

Am 10.02.15 um 18:45 schrieb Steve Dower:
> So I've enumerated the problems with PATH on Windows before (and my 
> subsequent dislike for modifying it through the installer)

It's quite comforting to hear this - I was arguing against that addition
for years (to the point of refusing to implement it).
So you are certainly not alone here, either.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[Martin v. Löwis]

Am 04.04.15 um 21:54 schrieb M.-A. Lemburg:
>>> FWIW: The PSF mostly uses StartSSL nowadays and they also support code
>>> signing certificates. Given that this option is a lot cheaper than
>>> Verisign, I think we should switch, unless there are significant
>>> reasons not to. We should revisit this in 2017.
>>
>> Agree - apparently the starlssl process for getting a signing cert is
>> complex/obscure, so we should start early.
> 
> Not really. Once you have the org verification it's really easy.
> 
>> Let me know if I can help providing PSF organization verification.
> 
> I already completed that for the current cycle.
> 

I had asked the PSF for a StartSSL certificate when the previous
certificate expired, and the PSF was not able to provide one. After
waiting several weeks for the PSF to provide the certificate, Kurt then
kindly went to Verisign.

Kind regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[Martin v. Löwis]

Am 05.04.15 um 06:43 schrieb Steve Dower:
> Now I just have to find the time to learn how to use it...

I always sign with Kleopatra on Windows. It's really simple: just drag
all files you want to sign onto it, configure "detached" signatures, and
it will place the signature next to the original file.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python3 Stable ABI

[Martin v. Löwis]

Am 13.04.15 um 23:28 schrieb Zachary Ware:
> In issue23903, I've created a script that will produce PC/python3.def
> by scraping the header files in Include.  

See my comment in the issue. Having a script to check is good; having it
generate the def file automatically is bad. It's typically the case that
changes to the header files didn't consider the stable ABI, so each
change needs to be reviewed manually. Typically, further changes to
other source files will be necessary.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?

[Martin v. Löwis]

Am 17.04.15 um 00:46 schrieb M.-A. Lemburg:
>> I had asked the PSF for a StartSSL certificate when the previous
>> certificate expired, and the PSF was not able to provide one. After
>> waiting several weeks for the PSF to provide the certificate, Kurt then
>> kindly went to Verisign.
> 
> When was that ? I never received such a request. 

I sent the request to Jesse Noller, Noah Kantrowitz and Kurt Kaiser on
2014-03-17. On 2014-04-15, Jesse indicated that he had given up.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode literals in Python 2.7

[Martin v. Löwis]

Am 02.05.15 um 21:57 schrieb Adam Bartoš:
> Even if sys.stdin contained a file-like object with proper encoding
> attribute, it wouldn't work since sys.stdin has to be instance of  'file'>. So the question is, whether it is possible to make a file instance
> in Python that is also customizable so it may call my code. For the first
> thing, how to change the value of encoding attribute of a file object.

If, by "in Python", you mean both "in pure Python", and "in Python 2",
then the answer is no. If you can add arbitrary C code, then you might
be able to hack your C library's stdio implementation to delegate fread
calls to your code.

I recommend to use Python 3 instead.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] How shall we conduct the Python 3.5 beta and rc periods? (Please vote!)

[Martin v. Löwis]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Am 13.05.15 um 09:59 schrieb Larry Hastings:
> When you say "branch testing", you mean "running the buildbots
> against it"?  Right now the UI for doing that is pretty clunky.
> Kicking off a build against a server-side clone (iirc) requires
> clicking through a couple web pages, filling out a form, and
> clicking on a teeny-tiny button.  It would help *tremendously* here
> if I could get this automated, so I could run a script locally that
> made everything happen.
> 
> Is there a remote API for starting builds?  Or existing automation
> of any kind?  Who should I talk to about this stuff?

Antoine, or me. For branch builds, it would be better to configure
them into the buildbot configuration, instead of trying to force them
from the outside.

To make this happen, you need to add a repository URL and branch name
into the buildbot configuration, and a post-push hook on the repository
to trigger the build. It's actually possible to configure a bitbucket
POST hook to trigger a buildbot build, but we haven't yet integrated
that into the buildbot master.

Regards,
Martin

-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iEYEARECAAYFAlVfivYACgkQavBT8H2dyNIzCACdG3yHShN/ZEc1sIiOVYj0lcg0
K9IAnjqLCFN+EewBPLfh651wQUq64nun
=0j5m
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 453 (ensurepip) updated

[Martin v. Löwis]

Am 20.10.13 14:27, schrieb Nick Coghlan:
> I have posted the latest version of PEP 453 to python.org. 

This version looks good to me, and I accept it for inclusion in Python 3.4.

I'd like to thank Nick for carefully editing this PEP, and I'd like to
cite it as an archetype for a well-written PEP. It's very precise, and
it elaborates on rejected proposals and the motivation for rejection.

I'd also like to thank Donald for pushing this, and for continued work
on the implementation of the PEP. I see that this all took longer than
expected (also due to my fault in providing timely reviews). I suggest
that some of the stuff that needs to be done still is delegated, so that
Donald doesn't have to do all of it.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384 (stable api) question

[Martin v. Löwis]

Am 07.11.13 13:44, schrieb Thomas Heller:

> I thought that the stable API would keep exactly the same across
> releases - is this expectation wrong or is this a bug?

Oscar is right - this change doesn't affect the ABI, just the API.

That said, please file an issue reporting what change you see in
your compiler warnings. 3.4 is not released yet, perhaps there is
something that can be done.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #19512: Add a new _PyDict_DelItemId() function, similar to

[Martin v. Löwis]

Am 08.11.13 10:25, schrieb Nick Coghlan:
> Likely a mistake - the stable ABI is hard to review properly (since it
> can depend on non local preprocessor checks, so a mistake may not be
> obvious in a diff), we don't currently have a systematic approach to
> handling changes and there's no automated test to catch inadvertent
> additions or (worse) removals :(

Actually, there is, for Windows. The .def file is an explicit list of
symbols; additions can easily be reviewed. Everything that's not added
there is not in the stable ABI (whether or not it is included in the
#ifdef). Removals cause linker failures.

It would be possible to automatically find out whether any functions
declared under the stable API are actually exported also, by parsing
the preprocessor output for Python.h

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode Exception messages in py2.7

[Martin v. Löwis]

Am 15.11.13 00:57, schrieb Chris Barker:
> Maybe so -- but we are either maintaining 2.7 or not -- it WIL be
> around for along time yet...

Procedurally, it's really easy. Ultimately it's up to the release
manager to decide which changes go into a release and which don't, and
Benjamin has already voiced an opinion.

In addition, Guido van Rossum has voiced an opinion a while ago that
he doesn't consider fixing bugs for 2.7 very useful, and would rather
see maintenance focus on ongoing support for new operating systems,
compiler, build environments, etc. The rationale is that people who
have lived with the glitches of 2.x for so long surely have already
made their work-arounds, so they aren't helped with receiving bug fixes.

The same is true in your case: you indicated that you *already* work
around the problem. It may have been tedious when you had to do it,
but now it's done - and you might not even change your code even if
Python 2.7.x gets changed, since you might want to support older 2.7.x
release for some time.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 3154 for 3.4?

[Martin v. Löwis]

Am 19.11.13 20:59, schrieb Antoine Pitrou:
> That's integrated to the built-in buffering. It's not really an
> additional constraint: the frame sizes simply dictate how buffering
> happens in practice. The main point of framing is to *simplify* the
> buffering logic (of course, the old buffering logic is still there for
> protocols <= 3, unfortunately).

I wonder why this needs to be part of the pickle protocol at all,
if it really is "below" the opcodes. Anybody desiring framing could
just implement a framing version of the io.BufferedReader, which
could be used on top of a socket connection (say) to allow fetching
larger blocks from the network stack. This would then be transparent
to the pickle implementation; the framing reader would, of course,
provide the peek() operation to allow the unpickler to continue to use
buffering.

Such a framing BufferedReader might even be included in the standard
library.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 3154 for 3.4?

[Martin v. Löwis]

Am 19.11.13 21:28, schrieb Antoine Pitrou:

> Well, unless you propose a patch before Saturday, I will happily ignore
> your proposal.

See

http://bugs.python.org/file32709/framing.diff

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 3154 for 3.4?

[Martin v. Löwis]

Am 19.11.13 23:50, schrieb Antoine Pitrou:
> Ok, thanks. So now that I look at the patch I see the following problems
> with this idea:
> 
> - "pickle + framing" becomes a different protocol than "pickle" alone,
>   which means we lose the benefit of protocol autodetection. It's as
>   though pickle.load() required you to give the protocol number,
>   instead of inferring it from the pickle bytestream.

Not necessarily. Framing becomes a different protocol, yes. But
autodetection would still be possible (it actually is possible in
my proposed definition).

> - it is less efficient than framing built inside pickle, since it
>   adds separate buffers and memory copies (while the point of framing
>   is to make buffering more efficient).

Correct. However, if the intent is to reduce the number of system
calls, then this is still achieved.

> Your idea is morally similar to saying "we don't need to optimize the
> size of pickles, since you can gzip them anyway". 

Not really. In the case of gzip, it might be that the size reduction
of properly saving bytes in pickle might be even larger. Here,
the wire representation, and the number of system calls is actually
(nearly) identical.

> However, the fact
> that the _pickle module currently goes to lengths to try to optimize
> buffering, implies to me that it's reasonable to also improve the
> pickle protocol so as to optimize buffering.

AFAICT, the real driving force is the desire to not read-ahead
more than the pickle is long. This is what complicates the code.
The easiest (and most space-efficient) solution to that problem
would be to prefix the entire pickle with a data size field
(possibly in a variable-length representation), i.e. to make a
single frame.

If that was done, I would guess that Tim's concerns about brittleness
would go away (as you couldn't have a length field in the middle of
data). IMO, the PEP has nearly the same flaw as the HTTP chunked
transfer, which also puts length fields in the middle of the payload
(except that HTTP makes it worse by making them optional).

Of course, a single length field has other drawbacks, such as having
to pickle everything before sending out the first bytes.

Regards,
Martin




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 3154 for 3.4?

[Martin v. Löwis]

Am 20.11.13 06:18, schrieb Tim Peters:
> BTW, I'm not a web guy:  in what way is HTTP chunked transfer mode
> viewed as being flawed?  Everything I ever read about it seemed to
> think it was A Good Idea.

It just didn't work for some time, see e.g.

http://bugs.python.org/issue1486335
http://bugs.python.org/issue1966
http://bugs.python.org/issue1312980
http://bugs.python.org/issue3761

It's not that the protocol was underspecified - just the implementation
was "brittle" (if I understand that word correctly). And I believe (and
agree with you) that the cause for this "difficult to implement"
property is that the framing is in putting framing "in the middle"
of the stack (i.e. not really *below* pickle itself, but into pickle
but below the opcodes - just like http chunked transfer is "in" http,
but below the content encoding).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 428 - pathlib - ready for approval

[Martin v. Löwis]

Am 20.11.13 17:04, schrieb Eric V. Smith:
> I think the confusion comes from the difference between what NTFS can do
> and what the Win32 (or whatever it's now called) layer allows you to do.
> Rumor has it that the old Posix subsystem allowed NTFS to create 2 files
> in the same directory that differed only by case.

Actually, the Windows API *does* support case-sensitivity in file names,
through the FILE_FLAG_POSIX_SEMANTICS flag to CreateFileW.

Unfortunately, in recent Windows versions, the kernel object manager
(a layer above the file system driver, but below the Windows API)
started setting the kernel variable for case insensitivity of object
names to true, defeating the purpose of this flag:

http://support.microsoft.com/kb/929110

So if you set ObCaseInsensitive to 0, reboot, then use CreateFileW
with FILE_FLAG_POSIX_SEMANTICS, then you should be able to create
two files in the same directory that differ only in case - i.e. the
system would be case-sensitive.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Martin v. Löwis]

Am 22.11.13 01:58, schrieb Steve Dower:

> I'm happy to work on a PEP and changes for what I described above, if
> there's enough interest? I can also update distutils to detect and
> build with any available compiler, though this may be more of a
> feature than we'd want for 2.7 at this point.

I don't think a PEP is necessary - Guido essentially pre-approved
changes needed to make Python 2.7 work with newer tools and operating
systems.

A patch for this would be appreciated - perhaps you would want
to put it into your sandbox on hg.python.org.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Martin v. Löwis]

Am 22.11.13 10:00, schrieb Richard Tew:
> That there are people who would consider using the trademark to force
> us to change the name we've been using without harm for 14 years,
> worries me.  It's one thing to perhaps use it to stop someone scamming
> Python users, and another to suggest using it as a weapon against us
> for this?  Really?

Unfortunately, Christian's original question was unclear. If his plan
had been to release "Python 2.8" (as his original question suggested),
then he would have faced quite some opposition (and, in a sense,
releasing a "python28.dll" is quite close to that).

IMO, calling it "Stackless Python 2.8" is fine, and I also believe that
this complies with the current trademark policy. The objective of this
policy is to give Python companies equal choices in naming their
products, i.e. nobody would be allowed to call something "Python"
without further qualification (except for the traditional CPython
release).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Martin v. Löwis]

Am 22.11.13 17:54, schrieb Christian Tismer:
> The discussion is over, but I cannot let this comment go through without
> citing my original question, again:
> 
>> My question
>> ---
>>
>> I have created a very clean Python 2.7.6+ based CPython with the
>> Stackless
>> additions, that compiles with VS 2010, using the adapted project
>> structure
>> of Python 3.3.X, and I want to publish that on the Stackless website
>> as the
>> official "Stackless Python 2.8". If you consider Stackless as official
>> ;-) .
> 
> Can it be that these sentences have morphed into something different
> when going out to the mailing list? Or maybe there is a filter in the
> brains?
> If one removes the word "Stackless" everywhere, the above text reads
> still almost syntactic correctly, but changes it's meaning a lot.

Hmm. This isn't the original *question*. Instead, the question read

# Can I rely on PEP 404 that the "Python 2.8" namespace never will clash
# with CPython?

So while the word "Stackless" wasn't removed everywhere, it certainly
wasn't in the question.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Martin v. Löwis]

Am 22.11.13 19:10, schrieb Steve Dower:
> I'd really want to update distutils.msvc9compiler to detect later
> versions as well, since that would make 'pip install' work properly
> for a large (majority?) of users for a large (majority?) of packages
> with extension modules. Some may consider this PEP-worthy (or at
> least worth arguing about), though I'm happy to just contribute a
> patch. (Not referring to my existing patch for this - I have a far
> more compatible approach in mind.)

A PEP on 2.7 seems questionable - if this would really need a PEP,
it would be right out (IMO). A PEP would ask for community input,
weighing possibly different design choices.

Instead, I think this needs explicit RM approval, such as any other
borderline bugfix patch. I'd personally support it, including any
distutils change (assuming the changes "look" backwards-compatible) -
but it still would be for Benjamin to rule about it.

> There's probably also value in making the same changes to Python 3.4.

Perhaps. However, Python 3.4 is likely being replaced before VS 2010
ends its life, and people will be more quick to forward-port to 3.5.

> I'm also getting in touch with my colleague who currently owns MSVCRT
> to figure out the full scope of what may happen once we start
> allowing mismatched versions in the same process. 

There used to be an MSDN article about it, but I think it was
incomplete. It mentioned (IIRC) a) locale, b) malloc, c) struct FILE.
Not sure whether it mentioned CRT file handles, and I'm fairly
sure that it didn't mention errno. I also don't think that timezone
issues were mentioned (although there used to be a separate article
about CRT timezones).

So if you can get somebody to compile a complete list, publishing it
as a KB article would certainly be appreciated beyond the Python
project.

>> A patch for this would be appreciated - perhaps you would want to
>> put it into your sandbox on hg.python.org.
> 
> I don't have a sandbox - how can I get one?

You are not a Python committer yet, are you? If you are, go to
hg.python.org/cpython, and invoke the server-side clone. If you
are not - does your company agree if you would become one?

In any case, patches or a clone on bitbucket would work just
as well.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Running the unit test as root/administrator

[Martin v. Löwis]

Am 05.12.13 02:04, schrieb Vajrasky Kok:
> Cool. What about Linux/Unix/BSD with root account? If we have
> something similar, I may plan to write unit test for spwd module.

Can you please phrase your question more explicit? What is it that
you want to be done before writing unit tests for the spwd module?

Anybody could run the test suite, and somebody might run it as root -
so you must be asking for something else.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Running the unit test as root/administrator

[Martin v. Löwis]

Am 05.12.13 16:21, schrieb Vajrasky Kok:
> On Thu, Dec 5, 2013 at 11:06 PM, "Martin v. Löwis"  wrote:
>>
>> Can you please phrase your question more explicit? What is it that
>> you want to be done before writing unit tests for the spwd module?
> 
> I am asking buildbot of Linux/Unix/BSD with root account. Do we have it now?

Ah. I don't think we have one. If somebody would want to donate one, I
suggest to run it in a VM, to reduce the (valid) security concerns that
Guido has voiced. If a snapshot of the VM is made, it would be easy to
restore in case a commit performs serious damage.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Running the unit test as root/administrator

[Martin v. Löwis]

Am 05.12.13 16:31, schrieb Chris Angelico:
>> Ah. I don't think we have one. If somebody would want to donate one, I
>> suggest to run it in a VM, to reduce the (valid) security concerns that
>> Guido has voiced. If a snapshot of the VM is made, it would be easy to
>> restore in case a commit performs serious damage.
> 
> Are we worried about malicious or accidental damage, here?

Accidental damage is a serious risk, IMO. Malicious damage is a risk
for the paranoid: it might be that an SSH key gets stolen, and is then
used to commit a trojan. Of course, that would affect all build slaves
(but less so if they run in a restricted account), and we would disable
the key as soon as we notice, but some damage might have happened.

> I might be able to offer a buildbot VM, though I don't know what the
> requirements are.

The machine should be up "most of the time", including internet
connectivity. A fixed IP address is no requirement (although I'm
unsure how resilient the slaves are when the IP changes). The
admin (i.e. you) needs to set up the box initially, and then care
for it in case of problems; a response time of a few days for
most of the year would be appreciated (although we can also deal
with operators that leave for longer periods of time - we just
disable the slave then, and reenable it when the admin brings
it back up). The commitment should be for a "long" period, i.e.
a few years.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: Python 2.x and 3.x usage survey

[Martin v. Löwis]

Am 31.12.13 07:12, schrieb Tim Peters:
> [Dan Stromberg]
>> I keep hearing naysayers, nay saying about Python 3.x.
>>
>> Here's a 9 question, multiple choice survey I put together about
>> Python 2.x use vs Python 3.x use.
>>
>> I'd be very pleased if you could take 5 or 10 minutes to fill it out.
> 
> If you run Python 3 while filling out the survey, like I did, it takes
> less than 1 minute to complete.  This may be surprising, since people
> usually say Python 3 is slower than Python 2.  Those must people who
> don't fill out surveys ;-)

So for the Python 4 survey, I propose to have just a single question:

* Have you heard of Python 4?

That will prove that Python 4 is even faster than Python 3 :-)
Of course, that is also because it has a JIT compiler, and runs
on 16 cores with no GIL.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: Python 2.x and 3.x usage survey

[Martin v. Löwis]

Am 31.12.13 11:04, schrieb Steven D'Aprano:
> On Tue, Dec 31, 2013 at 08:16:33AM +0100, Lennart Regebro wrote:
>> On Tue, Dec 31, 2013 at 6:31 AM, Dan Stromberg  wrote:
>>> So far the results are looking good for 3.x.
>>
>> Python-dev probably is a bit special.
> 
> Why? Most Python-Dev people have day jobs, and the version of Python 
> that they use in their day job is subject to exactly the same outside 
> pressures as everyone else ("RedHat ships with Python 2.6, so that's the 
> version we're using").

Yes and no. I answered "more 2.x" to that question because my current
projects have dependencies (Twisted in particular); however, I also
answered that I ported code - which actually wasn't my own, and I ported
it only to promote Python 3.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Buildbot - "slave lost"

[Martin v. Löwis]

Am 31.12.13 01:24, schrieb Chris Angelico:
> Does Buildbot retain a constant TCP socket to its server? 

In short: yes. A little bit longer: It uses the Twisted
PerspectiveBroker protocol. That has nearly-transparent
reconnects (but as your case shows, not fully transparent),
and does regular ping messages to keep the connection
alive.

So it should be able to handle a failover from one link to
the other, but it's certainly better to bind it to the more
reliable transport. I believe you can somehow configure the
frequency of ping messages so that you network doesn't believe
the connection goes idle, plus it will attempt a reconnect
if the network did indeed cancel the connection.

If you wanted to study this further, you could look into the
slave's twisted log file.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5

[Martin v. Löwis]

Am 07.01.14 15:08, schrieb Daniel Holth:
> Isn't it true that if you have bytes > 127 or surrogate escapes then
> encoding to latin1 is no longer as fast as memcpy?

You mean "decoding from latin1" (i.e. bytes to string)?

No, the opposite is true. It couldn't use memcpy before, but does now
(see _PyUnicode_FromUCS1).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASED] Python 3.4.0b2

[Martin v. Löwis]

Am 08.01.14 16:03, schrieb Nick Coghlan:
> On 9 January 2014 00:43, Bob Hanson  wrote:
>> When I read this comment of yours, Guido, I immediately started
>> wondering about this. You may well be right -- indeed, I have a
>> very old install (c.2007) which has not been updated (other than
>> one or three new MS "drivers").
>>
>> Perhaps the Python 3.4.0b2 MSI installer uses a new capability,
>> which, as you say, causes the installer to at least attempt to
>> upgrade...?
> 
> I believe the pip bootstrapping involves an MSI feature we haven't
> previously used (MvL would be able to confirm). If so, then MSI may be
> looking for a new version to interpret that new setting.

That's not true. The pip bootstrapping uses a custom action, and
we already have one that is similar (compile to pyc), although that
isn't run by default.

My guess is that it might try verifying signatures, and somehow tries
to obtain the CA certificates (although it's puzzling that it would get
them from akamai - perhaps MS is hosting the CA bundle there).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASED] Python 3.4.0b2

[Martin v. Löwis]

Am 06.01.14 17:26, schrieb Michael Urman:
> Here's some more guesswork. Does it seem possible that msiexec is
> trying to verify the revocation status of the certificate used to sign
> the python .msi file? Per
> http://blogs.technet.com/b/pki/archive/2006/11/30/basic-crl-checking-with-certutil.aspx
> it looks like crl.microsoft.com is the host; this is hosted on akamai:
>crl.microsoft.com is an alias for crl.www.ms.akadns.net.
>crl.www.ms.akadns.net is an alias for a1363.g.akamai.net.

I think that could be close. The MSI file has two signatures in it: the
PSF code signing signature, and a Verisign timestamping signature.

For the PSF certificate, the CRL is at csc3-2010-crl.verisign.com,
which is (here) a CNAME for crl.ws.symantec.com.edgekey.net, which
in turn is a CNAME for e6845.ce.akamaiedge.net.

The timestamping signature has its CRL at ts-crl.ws.symantec.com,
which is a CNAME for crl.ws.symantec.com.edgekey.net again.

So the most plausible reason is indeed that it tries to download
CRLs, though not Microsoft ones, but Verisign/Symantic ones.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python3 "complexity"

[Martin v. Löwis]

> Right.  But even latin-1, or better, cp1252 (on windows) does not solve it 
> because these have undefined
> code points. 

That's not true. latin-1 does not have undefined code points.


Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Common subset of python 2 and python 3

[Martin v. Löwis]

Am 12.01.14 18:39, schrieb Nachshon David Armon:
>>> I propose that this new version of python use the python 3 unicode model.
>>> As the version of python will be fully compatible with both python 2 and
>>> with python 3 but NOT necsesarily with all existing code in either. It is
>>> designed as a porting tool only.

I don't think that it is possible to write an interpreter that is fully
compatible for all it accepts. Would you think that the program

print(repr(2**80).endswith("L"))

is in the subset that should be supported by both Python 2 and Python 3?

Notice that it prints "True" in Python 2 and "False" in Python 3. So if
this common-version interpreter *rejects* the above program, which
operation (**, repr, endswith) would you want to ban from subset?

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Enable Hostname and Certificate Chain Validation

[Martin v. Löwis]

Am 23.01.14 07:45, schrieb Scott Dial:
> Anecdotally, I already know of a system at work that is using HTTPS
> purely for encryption, because the authentication is done in-band. So, a
> self-signed cert was wholly sufficient. The management tools use a
> RESTful interface over HTTPS for control, but you are telling me this
> will be broken by default now. What do I tell our developers (who often
> adopt the latest and greatest versions of things to play with)?

If they play with the newest version before actually using it in
production, all is well. You can then tell them that they have
four options:
- not upgrade to the newest Python release (at least not until
  they are willing to pursue any of the other alternatives)
- update the code to disable cert validation, or explicitly
  add the self-signed cert as a trusted one programmatically.
- update the client system configuration, to add the self-signed
  certificate as trusted (system-wide or per user).
- update the server, to use a cert signed by one of the
  trusted CAs.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Add PyType_GetSlot

[Martin v. Löwis]

I'd like to resolve a long-standing issue of the stable ABI in 3.4:

http://bugs.python.org/issue17162

The issue is that, since PyTypeObject is opaque, module authors cannot
get at tp_free, which they may need to in order to implement tp_dealloc
properly.

Rather than providing the proposed specific wrapper for tp_dealloc, I
propose to add a generic PyType_GetSlot function. From a stability point
of view, exposing slot values is uncritical - it's just that the layout
of the type object is hidden.

Any objection to adding this before RC1?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

[Martin v. Löwis]

I've debugged this a little bit. I couldn't originally see where the
problem is, since I expected that the code dealing with shared
references shouldn't ever trigger - none of the tuples in the example
are actually shared (i.e. they all have a ref-count of 1, except for
the outer list, which is both a parameter and bound in a variable).

Debugging reveals that it is actually the many integer objects which
trigger the sharing code. So a much simplified example of Victor's
benchmarking code can use

data = [0]*1000

The difference between version 2 and version 3 here is that v2 marshals
a lot of "0" integers, whereas version 3 marshals a single one, and then
a lot of references to this integer.

Since "0" is a small integer, and thus a singleton anyway, this doesn't
affect the unmarshal result. If the integers were larger, and actually
shared, the umarshal result under v2 would be "more correct".

If the integers are not shared, v2 and v3 have about the same runtime,
e.g. seen when using

data = [1000*1000 for i in range(1000)]

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add PyType_GetSlot

[Martin v. Löwis]

Am 29.01.14 03:46, schrieb Larry Hastings:

> So this would be a new public ABI function?

Correct.

> Would it be 100% new code, or would you need to refactor code internally
> to achieve it?

See the patch - it's new code.

> Also, just curious: what is typeslots.h used for?  I tried searching for
> a couple of those macros, and their only appearance in trunk was their
> definition.

It's meant for use by extension modules. See xxlimited.c:Xxo_Type_slots
for an application.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docstring hack for signature information has to go

[Martin v. Löwis]

Am 03.02.14 15:43, schrieb Larry Hastings:
> A: We create a PyMethodDefEx structure with an extra field: "const char
> *signature".  We add a new METH_SIGNATURE (maybe just METH_SIG?) flag to
> the flags, indicating that this is an extended structure.  When
> iterating over the PyMethodDefs, we know how far to advance the pointer
> based on this flag.
> 
> B: Same as A, but we add three unused pointers (void *reserved1 etc) to
> PyMethodDefEx to give us some room to grow.
> 
> C: Same as A, but we add two fields to PyMethodDefEx.  The second new
> field identifies the "version" of the structure, telling us its size
> somehow.  Like the lStructSize field of the OPENFILENAME structure in
> Win32.  I suspect YAGNI.

D: Add a new type slot for method signatures. This would be a
tp_signature field, along with a new slot id Py_tp_signature. The
signature field itself could be

struct PyMethodSignature {
   char *method_name;
   char *method_signature;
};

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docstring hack for signature information has to go

[Martin v. Löwis]

Am 05.02.14 17:04, schrieb Georg Brandl:
> Mostly unrelated question while seeing the "char *" here: do we (or do we
> want to) support non-ASCII names for functions implemented in C?

I didn't try, but I think it should work.
methodobject.c:meth_get__name__ uses PyUnicode_FromString, which in turn
decodes from UTF-8.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Windows 'for current user' installation - 32/64-bit registrations overwrite each other

[Martin v. Löwis]

Am 10.03.14 14:03, schrieb Jurko Gospodnetić:
>   Is this as issue or desired behaviour? 

See my response in the tracker. It's desired by Microsoft.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 4: don't remove anything, don't break backward compatibility

[Martin v. Löwis]

Am 10.03.14 18:01, schrieb Tres Seaver:
> On 03/10/2014 12:49 PM, Brett Cannon wrote:
> 
>> I think it got lost in email threading, but Barry pointed out that
>> Guido famously hates double digit version numbers (as do I, probably
>> partially because he does after all these years =).
> 
> "Guido hates them" isn't an argument:  its a ukase. 

Indeed, it is - so obey it then :-) We live in a dictatorship, after all.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 463: Exception-catching expressions

[Martin v. Löwis]

Am 12.03.14 04:58, schrieb Chris Angelico:
> On Wed, Mar 12, 2014 at 2:20 PM, Ethan Furman  wrote:
>> I sure hope this is accepted.  I could have used it at least a half-dozen
>> times in the last week -- which is more often than I would have used the
>> ternary-if!  :)
> 
> Where do we go from here? I've not made any edits for some time - no
> material edits for a good while - how do I request pronouncement?

I think you (or someone) first needs to find a BDFL delegate.
Traditionally, with syntax changes, there is a good chance that Guido
doesn't want to delegate at all, so ask him whether he wants to delegate
or not.

You should also add a section on the implementation status; it's unclear
(to me) whether the proposed implementation actually matches the
specification.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP URLs

[Martin v. Löwis]

Am 14.03.14 15:04, schrieb Barry Warsaw:
> On Mar 14, 2014, at 04:48 PM, Nick Coghlan wrote:
> 
>> I opened https://github.com/python/pythondotorg/issues/297 to ask for
>> an ETA on when we can expect them to be fully integrated.
> 
> Thanks Nick.  On the bug I suggest creating peps.python.org.

And I suggested setting up a reverse proxy.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] default hg.python.org/cpython is now 3.5

[Martin v. Löwis]

Am 17.03.14 16:18, schrieb Benjamin Peterson:
> 
> 
> On Mon, Mar 17, 2014, at 7:10, Brett Cannon wrote:
>> On Mon, Mar 17, 2014 at 4:51 AM, Victor Stinner
>> wrote:
>>
>>> Until when should we fix bugs in the branch 3.3? Branches 3.1 and 3.2 only
>>> accept security fixes, right?
>>>
>>
>> Typically we do one last release before shutting the last bugfix branch
>> down, but that's Georg's call since 3.3.5 was released so recently.
> 
> Given that, I propose 3.3 goes into security fix mode immediately.

+1

Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows

[Martin v. Löwis]

Am 17.03.14 22:10, schrieb Jurko Gospodnetić:
> Fixing
> this required manually cleaning up leftover CPython 3.4.0rc3 windows
> installer registry entries. Note that the issue could not be fixed by
> using the CPython 3.4.0rc3 installer as it failed due to the same problem.

Did you try the 3.4.0rc3 "repair" installation? That should have
undeleted (repaired) the files that you had manually deleted.

>   I compared MSI packaging related CPython source code (Tools/msi
> folder) in 2.6.2 & 2.6.3 releases but I can not see anything suspicious
> there. It could be that the observed beaviour change between those two
> versions is a result of the final release packager changing his used
> Windows Installer version, but I have not rebuilt CPython, and its MSI
> installer to test this theory out.

I believe I switched the entire Windows installation at least once in
this time (from XP to Windows 7), so it's unlikely related to my
personal installation.

>   Should I open a 'Add/Remove Programs' dialog related issue in the
> CPython issue tracker?

Please do. It would also good if somebody volunteered to reproduce
the issue.

> And possibly a separate one for making CPython
> installations not fail without possible recovery if a previous CPython
> installation has already been removed?

Please don't. Or, at least, make it more specific. If you have manually
corrupted your Python installation (by deleting essential files), you
have to accept that the system installation procedures will fail.

It might be possible to recover from the loss of a specific such file;
for that, we would have to identify what the file is. However, the
standard procedure should be to repair the installation before
attempting an upgrade.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 22.03.14 22:17, schrieb Cory Benfield:
> I am 100%, overwhelmingly in favour of this. Without this PEP, Python 2.7
> is a security liability, any it becomes nothing short of irresponsible to
> use Python 2.7 for any form of networking code that hits the open
> internet.

Agreed (although this might be a slight overstatement if taken
literally). However, the right consequence should be to use Python 3.4
instead.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 22.03.14 22:11, schrieb Nick Coghlan:

> Title: Network Security Enhancement Exception for All Branches

I'm -0 on the general idea, and -1 on the inclusion of the 2.7 branch
into the policy.

The PEP trades security concerns against stability and maintainability.
I see a maintenance threat coming out of it: existing installations will
fail to operate after updating Python and some support libraries. Bug
fix releases ought to focus on maintenance much more than moving along
with general security threats. Software that ages needs to be retired,
instead of trying to fix it endlessly (which is a futile exerciese,
anyway). Everybody knows it, and has accepted it, so why should we fight
what seems to be a fundamental law of software evolution?

In the specific case of 2.7, I am hoping that we retire the branch soon.
If so, it would be counter-productive to backport the current features
to 2.7. They are still in progress, and users wouldn't be helped with
getting a new feature in just the final bug fix release of 2.7, with no
chance of then getting updates of the same feature, as there won't be
any further bugfix releases.

Finally, doing this in the 2.7 branch likely involves more effort
than I'm personally willing to provide.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 22.03.14 23:05, schrieb Donald Stufft:
> I think the pep doesn't mandate that someone does. It still requires someone 
> to care enough to actually write the patch. It just allows such a patch to be 
> merged. 

Does it actually? Unfortunately, I never got around to writing the PEP
on security-only branches, so the terminology of PEP 466 is unclear.
It says "maintenance brances", which could mean three things:
- all branches that have ever been made off default in
  hg.python.org/cpython (i.e. back to 2.0). This would be the
  traditional definition of "maintenance branch" in a version control
  system, but is probably not intended by the PEP
- all branches from which still releases are made (whether security-only
  or bug fix)
- all bug fix branches

Clarification of this is just a minor nit, though - although the
maintainers of the old maintenance branches should (IMO) have a say
whether they accept a certain change on a certain branch.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 22.03.14 23:21, schrieb Donald Stufft:
> "Just use Python 3.4" ignores the reality of production software. I wish it 
> were that simple because I love 3.4

But I think so does the PEP (i.e. ignore the reality of production
software). The risk of breaking existing installations (which might
not be affected by the purported security threat being fixed) is just
too high.

Users that feel threatened by the lack of SSL features in Python should
have long started planning on how to get to Python 3. It's not that they
haven't been warned.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 22.03.14 23:33, schrieb Nick Coghlan:
> Hard to maintain legacy software is a fact of life, and way too much
> of it is exposed to the public internet. This PEP is about doing what
> we can to mitigate the damage caused both by other people's mistakes,
> and also the inherent challenges of migrating from the error prone
> POSIX text model to something more reasonable.
> 
> I *don't* think its reasonable to expect us to do this without support
> from the corporate users that caused the problem in the first place
> (by continuing to deploy older versions of Python without investing
> adequately in their upkeep), so I'd encourage everyone employed by a
> commercial user of Python to remind their management chains of the
> risks of failing to invest development time in any upstream
> dependencies that they expect to keep pace with the dynamic nature of
> the internet.

I hope indeed you are successful in activating resources. However,
putting them on this backporting project seems like a waste. They
should rather go into porting stuff to 3.x where people need it.

As responsible maintainers, we should just advise our users that
Python 2.7 is a dead horse, and that they should stop riding it.
More professionally, we should set an official end-of-life date
for 2.7 (alas, we should have done that two years ago).

I hope that the language summit can agree to stopping bug fix
releases for 2.7 in 2014.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 22.03.14 23:39, schrieb Ned Deily:
> Regarding the python.org binary installers, I think past practice has 
> been for us to update third-party libraries as necessary in maintenance 
> releases when there is good cause and with the concurrence of the 
> release manager, so I don't see this as a big issue.  For the OS X 
> binary installer, the issue for OpenSSL has been that we dynamically 
> link to the system-supplied OpenSSL libraries and that, for various 
> reasons, Apple has deprecated (and frozen at non-current OpenSSL 
> releases) the use of those libraries in favor of their own security 
> frameworks.  So, for multiple reasons, including the risk that OpenSSL 
> may be dropped from an upcoming major release of OS X, we need to start 
> supplying our own version with all OS X binary installers.  That's the 
> plan regardless of the outcome of this PEP.

The Windows case is similar but different. I also update OpenSSL
releases between Python bug fix releases, but only to OpenSSL bug fix
releases (likewise for all other third party software). So 2.7 uses
0.9.8 (currently 0.9.8y), 3.3 and 3.4 use 1.0.1 (currently 1.0.1e).

I still wish it was possible to drop OpenSSL altogether on both OSX
and Windows, and use the vendor TLS libraries instead. This would
reduce the maintenance burden, and might even improve performance.
However, it's getting difficult since the ssl module exposes more
and more OpenSSL API as-is (rather than trying to provide an abstraction).

For Windows, this approach was unfortunately hopeless as long as
we had to support XP. Now that Vista becomes the baseline, I hope
I can try to provide much of the ssl module natively using CryptoAPI.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 23.03.14 00:02, schrieb Benjamin Peterson:
> As (I believe) previously discussed and documented in PEP 373, this date
> currently will be May 2015.

Ah! Thanks for reminding me. I think PEP 466 then needs to take a
position on that, as well. By the past schedule, this probably means
two more bug fix releases for 2.7 (which is fine with me). However,
I expect this to be a problem for the PEP, since it is unrealistic
to expect that all aspects of the desired features are complete by
2015 (i.e. with no need for further updates in the coming years).

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Martin v. Löwis]

Am 23.03.14 01:15, schrieb Christian Heimes:
> On 23.03.2014 01:01, Antoine Pitrou wrote:
>> This is a bit limited. There are remotely-exploitable security issues
>> which are still open on the bug tracker; they are not
>> cryptography-related, but that shouldn't make a difference.
>>
>> (for example the dreaded XML issues have never been properly fixed,
>> AFAICT)
> 
> True, you may blame me for the situation. Only a handful of people were
> interested in the XML issues. I ran out of steam and moved to more sapid
> topics, too

I don't think anybody wanted to assign blame (although I can sympathize
with your urge to accept the blame). The fact is that this is a
volunteer project: we do what we can and have fun doing.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466 (round 2): Network security enhancements for Python 2.7

[Martin v. Löwis]

Am 23.03.14 08:07, schrieb Nick Coghlan:
> Several significant changes in this revision:
> 
> - scope narrowed to just Python 2.7 plus permission for commercial
> redistributors to use the same strategy in their long term support
> releases

Thanks; the rationale is now much clearer, and also indicates yet
another alternative path: fork Python.

The PEP indicates that vendors are likely to fork Python anyway
(as they always did, in a small scale). This could become more official
and coordinated. Create an "enhanced TLS" clone of cpython, and start
applying changes to 2.6, 2.7, and any other branches you consider
relevant. Keep it merged with the cpython code base. This model
has worked for many years for Stackless Python.

Then, vendors have the choice of either releasing from the official
CPython repository, or from the enhanced TLS one. All reasoning on
application-level feature testing still applies: applications would
have to feature-test whether they run on python.org release, or
on an enhanced release.

For Windows in particular, it would be up to ActiveState to decide
whether they build binaries from python.org, or from the enhanced
TLS repo. They could actually opt to provide both, leaving the choice
to users.

Even if this notion of forking is not acceptable, I suggest
that you could still start working on the code in a separate clone,
and the decision on the PEP could be deferred until a proposed
implementation is ready. I see it as a risk of the PEP that the
implementation might not be available before May 2015, in which
case the PEP would become irrelevant.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On porting to Python 3 as the answer

[Martin v. Löwis]

Am 23.03.14 17:22, schrieb Guido van Rossum:
> At Dropbox I work with a large group of very capable developers on
> several large code bases that are currently in 2.7. We are constantly
> changing our code to make it more secure (there are several teams
> specifically in charge of that). And yet porting to Python 3 is
> completely out of scope, for a variety of reasons.
> 
> Please stop your wishful thinking.

I can stop expressing it; I don't think I can stop wishing it :-)

If it's really unrealistic that Dropbox will ever port the code
to Python 3, would you then think that Python 3 is a doomed project,
since it won't ever see significant usage?

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.4 buildbots available

[Martin v. Löwis]

Am 22.03.14 22:03, schrieb Benjamin Peterson:
> 
> On Sat, Mar 22, 2014, at 11:10, Antoine Pitrou wrote:
>>
>> Hello,
>>
>> I've created the 3.4 category in the buildbots setup:
>> http://buildbot.python.org/all/waterfall?category=3.4.stable
>>
>> I've also retired 3.2 and 3.3 buildbots. Someone will have to update
>> the text and URLs at https://www.python.org/dev/buildbot/.
> 
> We probably want a http://buildbot.python.org/3.4.stable/ redirect, too,
> then.
> 

Done!

Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements

[Martin v. Löwis]

Am 25.03.14 14:47, schrieb Nick Coghlan:
> The PEP says to sync with Python 3, and that has full cross platform
> support. The Linux focus just comes from the fact that Linux is where
> the problem is most evident.

However, it fails to address a critical detail: the upcoming maintenance
end for 2.7. This makes it less platform-agnostic
than you say: For Linux, many users rely on the system vendor
to provide Python binary, and they may chose to continue improving
the backported APIs even after python-dev's maintenance ends.
OTOH, for Windows, most users rely on the binaries from python.org,
which will stop evolving in May 2015. Of course, there is still
ActiveState, but there isn't the same kind of vendor choice that
you see for Linux.

So I think the PEP should make it clear whether:
a) you expect that the maintenance period for Python 2.7 is to
   be extended beyond 2015,
b) you expect that the backports will evolve in the years to come,
   or whether this will be a one-shot activity
c) and if the answer to b was "will evolve", whether you expect this
   will create forks of 2.7 after maintenance has ended (as the
   then-security-only 2.7 branch won't see any further improvement
   to the backport, according to the PEP)

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C code: %s vs %U

[Martin v. Löwis]

[Assuming you are talking about PyUnicode_FromFormatV]
> %s is a string.

No. %s is a char*; C does not have a "string" type.
The string behind the pointer should be UTF-8 encoded;
other encodings are tolerated through the "replace" error
handler.

> %U is unicode?

No. This is a PyObject* whose Python type is 'str'
(i.e. an object for which PyUnicode_Check succeeds)

> If so, then %s should only be used when it is certain the string in
> question has no unicode in it?

No. If you have a char*, use %s; using %U would crash.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] freeze build slave

[Martin v. Löwis]

I have created a buildbot configuration to test freeze. At the moment,
it has only one builder:

http://buildbot.python.org/all/waterfall?show=AMD64%20Ubuntu%20LTS%20Freeze%203.x

which currently fails as freeze doesn't actually work.

The test itself works by first building Python in release mode,
then installing it, then running freeze on a test program, then
building the test programm (and ultimately running it).

The question then is: how should that integrate with the rest
of the builders? I can see three alternatives:
A. (status quo) run the test on a selected subset of the Unix
   builders
B. run the test on all Unix builders.
C. integrate the test with the Unix regular Unix builder

Evaluating these alternatives:
B: pro: wider testing
   con: each such build takes the slave lock, so slaves
   will have to do one additional full build per commit
   (or two if the fix gets checked into 3.4 as well).
   In addition, each slave will need disk space for one
   additional build tree plus one Python installation,
   per branch.
C: pro: compared to B, build time is reduced (need only
   to build once per branch); disk space is also reduced
   con: it would test a debug build, not a release build

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] libpython added to ABI tracker

[Martin v. Löwis]

Am 01.04.14 13:45, schrieb Nick Coghlan:
> Interesting to see the UCS2 removal there for 3.3. That's a genuine
> removal from the public ABI as part of PEP 393. I guess the reason
> nobody complained is because most 3.2 Linux builds used the UCS4 ABI
> instead, and the stable ABI hadn't seen broad adoption on Windows in
> the 3.2->3.3 time frame.

Not really. The intention was that the stable ABI wouldn't have any
UCS2/UCS4 denotation in the function names, see

http://hg.python.org/cpython/file/9186f4a18584/PC/python3.def

Functions that explicitly referred to Py_UNICODE were banned from
the ABI; functions that were mapped but shouldn't have been mapped
were meant to be unmapped.

However, it seems that this wasn't properly implemented, see

http://bugs.python.org/issue17432

> Regardless, this service already shows we've made some mistakes with
> the stable ABI in previous releases - it is indicating there are new
> symbols in the stable ABI for 3.3 and 3.4 that aren't properly guarded
> with version constraints. That means it is currently possible to set
> Py_LIMITED_API=0x0302 and get something that won't actually run
> properly on 3.2.

Depends on the operating system. On Windows, the import library has
much less additions; anything declared in the header files that
is not in python3.def is a bug in the header files (by default).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.7 Windows installers (Was: death to 2.7; long live 2.7)

[Martin v. Löwis]

Am 10.04.14 03:22, schrieb Benjamin Peterson:
> We'll keep doing what we're currently doing for another year, making
> normal bug fix releases with installers. After that, we _won't_ close
> 2.7 to normal bug fixes as is currently implied by the release schedule.

After thinking about this plan, I believe it is best if I step down as
the Windows installers engineer right away. To my knowledge, no specific
date for the next 2.7 bugfix release has been set, so this should allow
to find a replacement before the next release.

I would have volunteered to do two more releases if these had been the
definite last ones, as it would have been impractical to replace me for
just two releases. If this is going until 2020, anybody up to the task
should start working on it now.

I'll be willing to help anybody taking over with advise, but I'll
consider the 2.7 branch otherwise dead (i.e. not worthwhile
contributing to).

I'll be happy to continue making installers for the 3.x branches,
but if anybody considering to volunteer would do so on the condition
that he/she gets it "all", I'd be happy to hand these other branches
over as well.

Of course, anybody taking over that role would be free to either keep
or replace the current technology (e.g. switching to InstallShield
or WiX or whatever).

The one option I'd rather not support is an "open" committee of
Windows installer engineers. Anybody taking this role will have a
copy of the PSF code signing key, so I'd like to keep this list
rather small, and the people attached to the task.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appeal for reviews

[Martin v. Löwis]

Am 13.04.14 03:07, schrieb Benjamin Peterson:
> On Sat, Apr 12, 2014, at 17:30, Stephen J. Turnbull wrote:
>> I apologize for the tone.  I need to go *right* now, and can't fix
>> that.  Really, I'm sympathetic and my goal is not just to defend
>> python-dev, but to help you get the reviews your work deserves.
>> Please read with that in mind.
> 
> I don't think Nikolaus is wrong to post here. I often tell people that
> sometimes the only way to get your patches in is to constantly poke us
> about it.

I agree. If I had more time than I actually do, I would have responded
with my 5-for-1 deal. Instead, I just looked at some of the patches,
and commented on further improvements.

As for the request "Are you sure that the patch is ready": this is
*very* difficult to answer for the author. We all have experienced
that patches that we considered good were critized out of nowhere,
and I just did the same to Nikolaus. There is just no way to tell
in advance whether people will approve a certain change or not, because
people will apply other criteria than oneself.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appeal for reviews

[Martin v. Löwis]

Am 13.04.14 08:36, schrieb Stephen J. Turnbull:

> I admit the tone was biased toward nagging or "blaming the victim",
> and again I apologize for causing misunderstanding.  Nikolaus isn't
> "wrong" for posting here.  My claim is that in current circumstances,
> core-mentorship would be a more *effective* channel because
> 
>  - core-mentorship is *explicitly* for poking Those Who Can Help
>(among other requests for help);

It would be worth an experiment. I know that I wouldn't have reviewed
Nikolaus' patches if he had posted to core-mentorship - I'm not a core
mentor.

> I also suggested that some tweaks to the weekly issue report might
> help to catch the attention of those who can commit patches, but my
> ideas about that are still basically vaporware.

Well, the code of the weekly reports is free software, contributions
are welcome :-)

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Windows installers and OpenSSL

[Martin v. Löwis]

Am 10.04.14 15:41, schrieb Paul Moore:
> Given the OpenSSL vulnerability and the fact that we bundle OpenSSL
> with the Windows installers (1.0.1e in Python 3.4.0) should we be
> releasing updated installers?

As others have said: certainly, and only for 3.4.0 (i.e. 2.7 in
particular is not affected - I'm glad I didn't update OpenSSL
there past 0.9.8).

My feeling with these things is that it is often better to wait
until the dust settles - people in a hurry of fixing security
bugs tend to introduce new ones in the process.

I'm tempted to experiment with installer patch files for this
(.msp); it's technically just a single DLL that would need to be
replaced. Contributions are welcome.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Windows installers and OpenSSL

[Martin v. Löwis]

Am 13.04.14 21:41, schrieb Steve Dower:
> I'm willing to embark on migrating the entire installer to WiX, which
> doesn't directly fix any particular issue, but could significantly
> reduce the overhead of building and maintaining the Windows installers.

I somewhat doubt that it could reduce the overhead - the actual
overhead for the MSI generation is fairly small. It's more that the
current Python-based approach is not too familiar to people, in
particular to those that are familiar with WiX (which I, in turn, am
not).

> Martin - are you at PyCon today? Can we chat to figure out what how much
> of the work you do can be automated? (If not, email on or off list is
> good too.)

Unfortunately, I'm not, and I can't really do chats ATM due to timezone
differences. It's all fairly automated; what takes time is that there
is manual steps in-between (e.g. getting the sources from Mercurial,
launching Visual Studio, running the CHM generator, running the MSI
generator, signging with PGP, uploading the files, running the
installer for testing, uninstalling the previous installation, and so
on). It could be done in a single batch file mostly, except that I
never got to write this batch file.

If it's automated, it would still take an hour or so, but it would be
possible to do other things while waiting for the build to complete.

> Alternatively, if you don't want me mucking about with this, tell me and
> I'll stop (being willing does not mean being keen ;) )

If it's that, I'd say stop. It's not that I want you to, but that I'm
skeptical that it solves anything. If you do want to start using WiX,
please do run the current build process at least once before.

The basic instruction for that is "run msi.py". It will tell you
if it's missing files, and anybody familiar with the build process
should be able to figure out how to get the missing files (if not,
just ask).

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.7 Windows installers (Was: death to 2.7; long live 2.7)

[Martin v. Löwis]

Am 13.04.14 22:02, schrieb Steve Dower:
> I replied to the other email before I saw this one.

Same here :-)

> Consider this my self-nomination to take over, pending a quick email to
> Microsoft's lawyers to make sure it's okay (it should be, but IANAL and
> they wrote the policy).
> 
> My plan would be to rewrite the installer using WiX but retaining
> compatibility with the existing MSIs. I'd also automate as much as
> possible to make it very easy for someone else with the certificates to
> build/verify the release.
> 
> Happy to answer any other questions or provide more background about
> myself if people ask.

As I just said: to get started, run the current build process. Without
knowing WiX in detail, I'd still claim that msi.py is superiour in
terms of expressiveness (i.e. it can better compute what files go into
the MSI). I'm almost certain that you need a full programming language
to actually perform the entire build; one alternative to msi.py would
be to have a Python script generating the WiX configuration files.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] this is what happens if you freeze all the modules required for startup

[Martin v. Löwis]

Am 14.04.14 23:51, schrieb Brett Cannon:
> It was realized during PyCon that since we are freezing importlib we
> could now consider freezing all the modules to cut out having to stat or
> read them from disk.
[...]
> Thoughts?

They still get read from disk, except that it is the operating system
that does the reading. So what you really save is the access to many
tiny files; something that can also be achieved with the zipfile import.
So I wonder how your all-frozen binary compares to a standard binary
with a python35.zip.

If it is comparable, I'd rather extend on that route, i.e. promote
putting the standard library into a zip file in the default
installation, and also find a way where (say) /usr/bin/hg could
conveniently specify a zip file that will contain the Mercurial
byte code. For example, we could support a -Z option for the interpreter
which would allow to append a zip file to a script that gets put on
sys.path.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] this is what happens if you freeze all the modules required for startup

[Martin v. Löwis]

Am 17.04.14 20:47, schrieb Brett Cannon:
> Because people keep bringing it up, below is the results of hacking up
> the interpreter to include a sys.path entry for ./python35.zip instead
> of hard-coding to /usr/lib/python35.zip and simply zipped up Lib/
> recursively. TL;DR, zipimport performance no longer measures up
> (probably because of stat caching and such that importlib introduced).
> 
> ### normal_startup ###
> 
> Min: 0.510211 -> 2.667958: 5.23x slower
> 

Not sure how to interpret this: what is 5.23x slower than what?

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] this is what happens if you freeze all the modules required for startup

[Martin v. Löwis]

Am 17.04.14 20:47, schrieb Brett Cannon:
> Because people keep bringing it up, below is the results of hacking up
> the interpreter to include a sys.path entry for ./python35.zip instead
> of hard-coding to /usr/lib/python35.zip and simply zipped up Lib/
> recursively. TL;DR, zipimport performance no longer measures up
> (probably because of stat caching and such that importlib introduced).

[I found the answer on what is being compared in replies]

So how did you create the zip file? Any chance that you may have
compressed the pyc files?

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tix version needed to build 2.7 Windows installer?

[Martin v. Löwis]

Am 08.05.14 18:59, schrieb Brian Curtin:
> This is mostly a question for Martin, but perhaps someone else would also 
> know.
> 
> I'm trying to build the 2.7 installers so I can backport the path
> option from 3.3, but I can't seem to figure out which version of Tix
> is necessary to have a complete build. So far any of them on
> http://svn.python.org/projects/external don't appear to be configured
> to work with the combination of tcl and tk that are used on 2.7, per
> the buildbot external scripts. It's another issue that Tix isn't even
> downloaded by the scripts, but I'll do it manually right now just to
> get this going.
> 
> Any tips?

Ah, 2.7. I was using a checkout of tix-8.4.3.x from Nov 13, 2010, one
where python.mak was pointing to Tcl 8.5.2. It may not have been tagged.

In 2.7, it wasn't checked out out automatically because Tix requires the
original Tcl/Tk path names, so building it automatically would have failed.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Where is our official policy of what platforms we do support?

[Martin v. Löwis]

Am 15.05.14 16:20, schrieb Skip Montanaro:
> On Thu, May 15, 2014 at 8:26 AM, Antoine Pitrou  wrote:
>> We already have such buildbots, they are in the "unstable" category.
>> You can browse through existing buildbots here:
>> https://www.python.org/dev/buildbot/
> 
> I can't see how to distinguish "stable" from "unstable" (or to view
> just the "unstable" category. What do those two categories have to do
> with "supported" and "unsupported"?

There are two competing definitions of "stable" vs. "unstable" when it
comes to buildbots.

1. "stable buildbot" (my preferred definition)
   * the buildbot is available most of the time, an operator will deal
 with it when it happens to go down, and the builds always complete.
   * consequentially, an unstable buildbot is one that tends to
 disconnect, and takes a long time for the operator to recover
2. "stable platform"
   * all tests are expected to pass all the time; i.e. there are no
 tests that randomly fail, and no platform-specific failures
   * thus, a failed test is an indication that a new bug has been
 introduced

When Antoine talked about "unofficial" buildbots, he was referring to
the fact that we accept nearly any buildbot offered, but put them into
the unstable category, which they often belong to by either definition
of unstable.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Where is our official policy of what platforms we do support?

[Martin v. Löwis]

Am 14.05.14 16:28, schrieb Barry Warsaw:
> On May 14, 2014, at 02:20 PM, Brett Cannon wrote:
> 
>> Do we want an official policy written down in a PEP (yes, I can write it)?
>> Should I keep closing these patches and saying that we are not adding
>> support for new operating systems and be hand-wavy about it?
> 
> Yes, I think a PEP describing both policy and implementation (i.e. which
> platforms we officially support) is worthwhile.  While I think you could write
> a new PEP, I also think it might just make sense to co-opt PEP 11 and broaden
> its scope, since as you say, removing support is only half the story.

I'd be careful to use the word "support". AFAIK, we do not offer
support for Python on any platform, beyond accepting bug reports in the
bug tracker. Paid support for Python is certainly offered by some
companies, but not by us.

IIUC, Brett is asking the question "What platforms is CPython supposed
to work on?". Thanks to autoconf, it is impossible to give a complete
list of such platforms. It might be that Python builds just fine, with
no changes required, on a system that we've never heard of.

So if you want to give a complete list of "supported" platforms, it
might be the question "What platforms do we care about?", in the sense
that someone would be willing to invest time if it stops working.
Again, as we do not offer paid support, this list might get misleadingly
long. I'd be personally willing to invest time looking into a lot of
problems - just not within the next 12 months (but surely some day :-)

The only reasonable positive platform list I can think of is
"What platforms do we consider release-critical?", in the sense of
not letting a release proceed if it fails to work on one of these
platforms. We've had such a list for a long time, it is:
- Microsoft Windows
- Linux
- Apple Mac OS X

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASED] Python 3.4.1

[Martin v. Löwis]

Am 24.05.14 12:15, schrieb Herbert Griebel:
> Found the issue. To reproduce the problem install Python 3.4.0, then
> rename the install folder (e.g. C:\Python34 to C:\Python34x) and install
> Python 3.4.1.

Please understand that installation of 3.4.1 attempts uninstallation of
3.4 first. Without testing, my guess is that it is the pip
uninstallation which fails, due to python.exe not being there anymore.

If you want to diagnose this further, run the installer with

msiexec /i python-3.4.1.msi /l*v python.log

Either study the log file yourself, or post it to bugs.python.org.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Updating turtle.py

[Martin v. Löwis]

Am 31.05.14 05:32, schrieb Terry Reedy:
> I have two areas of questions about updating turtle.py. First the module
> itself, then a turtle tracker issue versus code cleanup policies.
> 
> A. Unlike most stdlib modules, turtle is copyrighted and licensed by an
> individual.
> '''
> # turtle.py: a Tkinter based turtle graphics module for Python
> # Version 1.1b - 4. 5. 2009
> # Copyright (C) 2006 - 2010  Gregor Lingl
> # email: gli...@aon.at
> '''
> I am not sure what the copyright covers other than the exact text
> contributed, with updates, by Gregor. It certainly does not cover the
> API and whatever code he copied from the previous version (unless that
> was also by him, and I have no idea how much he copied when
> reimplementing). I don't think it should cover additions made by others
> either. Should there be another line to cover these?

He should provide a contributor form, covering his past contributions.
Would you like to contact him about this?

Adding a license up-front (as you propose) is counter-productive; the
author may not agree to your specific licensing terms. If he was
unwilling to agree to the contributor form (which I doubt, knowing
him personally), the only option would be to remove the code from the
distribution.

> _ver = "turtle 1.1b- - for Python 3.1   -  4. 5. 2009"
> '''
> Obsolete; delete or alter (how)?

Delete. It's a private variable, and the true version number is
maintained by Mercurial, and not in the code itself (or else is the
version of Python it ships with).

> A particular example: Gregor added intermediate layers to isolate turtle
> from tkinter. (He had a plan to add other backends, but I believe he
> abandoned that.) If someone wanted to reduce the layering and make the
> code easier to understand and maintain, while speeding execution, would
> that be allowed now?

It would be good if there would be a new maintainer of the module. Maybe
Gregor would be willing to reactivate his contributions when asked,
maybe he would be willing to hand it over to somebody else. A maintainer
would have the ultimate say in architectural changes.

Without a maintainer, I'd rather not make architectural changes.

> Responding today, I cautioned that clean-up only patches, such as she
> apparently would like to start with, are not in favor. 

I would not say that. I recall that I asked Gregor to make a number of
style changes before he submitted the code, and eventually agreed to the
code when I thought it was "good enough". However, continuing on that
path sounds reasonable to me.

It is the mixing of clean-up patches with functional changes that is not
in favor.

> Since she only marked the issue for 3.5, I also cautioned that 3.5-only
> cleanups would make fixing bugs in other issues harder. Is the code
> clean-up policy the same for all branches?

I don't think that we should be taken hostage by merging restrictions
of the DVCS - we switched to the DVCS precisely with the promise that
merging would be easier. Given the number of bug fixes that the turtle
module has seen, I'd suggest that it is less work to restrict cleanup
to 3.5, and then deal with any forward-porting of bug fixing when it
actually happens.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Updating turtle.py

[Martin v. Löwis]

Am 31.05.14 10:09, schrieb Stephen J. Turnbull:
> AFAICT Python policy is that someone should ask Gregor (a precedent is
> the Fredrik Lundh/ElementTree case).  AIUI, there's been a five-year
> span since Gregor's been active, so I would think it's basically a
> matter of courtesy.  Most likely he's not interested in returning as
> maintainer, or he can't be contacted with reasonable effort. 

I would not be so sure about this, see

http://python4kids.net/turtle.html

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Updating turtle.py

[Martin v. Löwis]

Am 01.06.14 00:21, schrieb Terry Reedy:
>>> Responding today, I cautioned that clean-up only patches, such as she
>>> apparently would like to start with, are not in favor.
>>
>> I would not say that. I recall that I asked Gregor to make a number of
>> style changes before he submitted the code, and eventually agreed to the
>> code when I thought it was "good enough". However, continuing on that
>> path sounds reasonable to me.
> 
> I am not sure what you mean by 'that path', to be continued on.

The path of improving the coding style of the turtle module.

>> I'd suggest that it is less work to restrict cleanup
>> to 3.5, and then deal with any forward-porting of bug fixing when it
>> actually happens.
> 
> This would make it non-trivial for any patch hitting a difference.

Indeed. OTOH, it's simpler for anybody doing the code cleanup to do it
only on one branch.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

[Martin v. Löwis]

Am 06.06.14 17:41, schrieb Steve Dower:
> Hi all
> 
> I would like to propose moving Python 3.5 to use Visual C++ 14.0 as
> the main compiler.

This is fine with me, but I'm worried about the precise timing of doing
so. I assume that you would plan to do this moving before VC++ 14 is
actually released. This worries me for three reasons:
1. what is the availability of the compiler during the testing phase,
   and what will it be immediately after the testing ends (where
   traditionally people would have to buy licenses, or wait for VS
   Express to be released)?
2. what is the risk of installing a beta compiler on what might
   otherwise be a "production" developer system? In particular, could
   it interfere with other VS installations, and could it require a
   complete system reinstall when the final release of VC 14 is
   available?
3. what is the chance of the final release being delayed beyond
   the planned release date of Python 3.5? Microsoft has a bad
   track record of meeting release dates (or the tradition of not
   announcing any for that reason); the blog says that it will
   be available "sometime in 2015". Now, Python 3.5 might appear
   November 2015, so what do we do if VS 2015 is not released
   by the time 3.5b1 is planned?

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

[Martin v. Löwis]

Am 06.06.14 19:31, schrieb Brian Curtin:

>> If that's a non-issue, or if we can actually drop XP support, I'm all for it.
> 
> Extended support ended in April of this year, so I think we should put
> XP as unsupported for 3.5 in PEP 11 -
> http://legacy.python.org/dev/peps/pep-0011/
> 
> I seem to remember that we were waiting for this anyway.

We don't actually need to explicitly put XP there, as PEP 11 ties our
support to the Microsoft product life cycle. XP is not supported by
Python anymore.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

[Martin v. Löwis]

Am 06.06.14 20:25, schrieb Brian Curtin:
> We're going to have to change it at some point, otherwise we're going
> to have people in 2018 scrambling to find VS2008, which will be 35
> versions too old by then.

Not sure whether you picked 2018 deliberately: extended support for
VS2008 Professional ends on April 10, 2018.

In any case, the extension problem will occur regardless of what
you do:
- if you switch compilers within 2.7, applications may crash
- if you switch compilers and declare it 2.8, you extensions
  might not be available precompiled for some time (in particular
  if the developers of some package have abandoned 2.7)
- if you don't switch compilers, availability of the tool chain
  will be terrible.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

[Martin v. Löwis]

Am 06.06.14 21:20, schrieb "Martin v. Löwis":
> 2. what is the risk of installing a beta compiler on what might
>otherwise be a "production" developer system? In particular, could
>it interfere with other VS installations, and could it require a
>complete system reinstall when the final release of VC 14 is
>available?

I found an official answer here:

http://www.visualstudio.com/en-us/downloads/visual-studio-14-ctp-vs

"Installing a CTP release will place a computer in an unsupported state.
For that reason, we recommend only installing CTP releases in a virtual
machine, or on a computer that is available for reformatting."

So there is no promise that you will not need to reformat the system
during the evolution of the compiler.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com