Re: [Python-Dev] How far to go with user-friendliness

[Michael Foord]

On Tuesday, 14 July 2015, Christie Wilson  wrote:
>> If people do misspell it, I think they do learn not to in after it
happens a few times.
>
> Unless the line silently executes and they don't notice the mistake for
years :'(

Indeed. This has been a problem with mock, misspelled (usually
misremembered) assert methods silently did nothing.

With this fix in place several failing tests were revealed in code bases!

As for assret, it's the common misspelling people have told me about. It
seems a ridiculous thing for people to get worked up about, but people
enjoy getting worked up.

Michael


> On Tue, Jul 14, 2015 at 9:15 AM, Ron Adam  wrote:
>>
>>
>> On 07/14/2015 09:41 AM, Steven D'Aprano wrote:
>>>
>>> On Tue, Jul 14, 2015 at 02:06:14PM +0200, Dima Tisnek wrote:

 >https://bugs.python.org/issue21238  introduces detection of
 >missing/misspelt mock.assert_xxx() calls on getattr level in Python
 >3.5
 >
 >Michael and Kushal are of the opinion that "assret" is a common typo
 >of "assert" and should be supported in a sense that it also triggers
 >AttributeError and is not silently ignored like a mocked user
 >attribute.
 >
 >I disagree
>>>
>>> I must admit I don't use mock so don't quite understand what is going on
>>> in this bug report. But I don't imagine that anything good will come out
>>> of treating*one*  typo differently from all the other possible typos.
>>> Why should "assret" be treated differently from other easy-to-make typos
>>> like "asert", "assrt", "asset"? Or "assort", which is not only a
>>> standard and common English word, but "e" and "o" are right next to each
>>> other on Dvorak keyboards, making it an easy typo to make.
>>>
>>> Surely this is an obvious case where the Zen should apply. "Special
>>> cases aren't special enough..." -- either all such typos raise
>>> AttributeError, or they are all silent.
>>
>> I agree with Steven that it doesn't seem correct to not raise
AttributeError here.
>>
>> For what it's worth, I have a life long sleep disorder and am a tarrable
(<-- like this)  speller because of it.   I still don't want spell, or
grammar, checkers to not report my mistakes.  And I don't recall ever
making the particular error of using "assret" in place of "assert".  I'd be
more likely to mispell it as "assirt" if I wasn't already so familiar with
"assert".
>>
>> If people do misspell it, I think they do learn not to in after it
happens a few times.
>>
>> Regards,
>>Ron
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/bobcatfish%40gmail.com
>
>
>
> --
> Christie Wilson

-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How far to go with user-friendliness

[Michael Foord]

On Wednesday, 15 July 2015, Robert Collins 
wrote:
> On 15 July 2015 at 12:59, Nick Coghlan  wrote:
>>
>> There is zero urgency here, so nothing needs to change for 3.5.
>> Robert's plan is a fine one to propose for 3.6 (and the PyPI mock
>> backport).
>
> Right - the bad API goes back to the very beginning. I'm not planning


I disagree it's a bad api. It's part of why mock was so easy to use and
part of why it was so successful. With the new check for non-existent
assert methods it's no longer dangerous and so doesn't need fixing.

So -1 from me.

Michael


> on writing the new thing I sketched, though it should be straight
> forward if someone wishes to do so. I'll probably file a ticket in the
> tracker asking for it once this thread quiesces.
>
> -Rob
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>

-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO

[Michael Foord]

On 18/03/14 16:44, Antoine Pitrou wrote:


Hello Xavier,

It is not obvious your message is appropriate for python-dev. It looks 
like mere advertising; if it is not, please explain.


To clarify what this mailing-list is about: "On this list the key 
Python developers discuss the future of the language and its 
implementation. Topics include Python design issues, release 
mechanics, and maintenance of existing releases."


(from https://mail.python.org/mailman/listinfo/python-dev)



Unless you're offering all the core-devs free robots. In which case it's 
fine.


Michael


Regards

Antoine.


Le 17/03/2014 23:05, Xavier Salort a écrit :

Hi,

We are the manufacturer of the humanoid robot NAO :
http://www.youtube.com/watch?v=nNbj2G3GmAo
We are now offering great opportunities for developers to use NAO and
would like to get in touch with you and your members :
http://www.youtube.com/watch?v=_AxErdP0YI8

Let me know if you may be interested.
Thank you!

Xavier
--
Xavier Salort
Area Sales Manager
M + 1 857 247 






___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fwd: Jython Report

[Michael Foord]

Below is the Jython "status update" report on Jython I received from Jim Baker 
and summarised in the Language Summit. It comes with one addendum from Frank:

Jim's list is fantastic - the one bit I'd like to add to the list:

Jython now supports a buffer protocol that parallels CPython's C API buffer 
protocol. This provided the basis for support of buffer() and memoryview(). The 
work was done with Jython3 in mind and will be a huge boost to that eventual 
effort.

Begin forwarded message:

> From: Jim Baker 
> Subject: Re: Jython Report
> Date: 7 April 2014 06:42:51 BST
> To: Michael Foord 
> Cc: Frank Wierzbicki 
> 
> Recent changes to trunk (last 6 months)
> 
> * Recently tagged a soft beta 2!
> * Java 7 JVM is now the minimum version, which gives a larger base of 
> functionality to work with (such as using Java 7's AutoCloseable to imply 
> corresponding context manager support in using Python code)
> * Enable mixing Python and Java types in the bases of a class when using a 
> metaclass
> * Added support for buffer, memoryview, although not complete yet with 
> respect to Java integration
> * Console and encoding support, such as unicodedata/idna updates
> * Many, many small fixes
> 
> About to be in trunk, to support beta 3
> 
> * socket-reboot reimplements socket/select/ssl on top of Netty 4, a popular 
> event loop networking framework for the JVM (used by a large number of 
> performant projects in Java space and originally part of JBoss). There was no 
> ssl support before, but now socket and especially select semantics are much 
> closer to CPython as well (basically close to the Windows socket model). 
> * socket-reboot in turn enables requests and thereby pip. A branch of pip 
> currently works, actually modifying an upstream vendor lib (html5lib) so that 
> it doesn't use isolated UTF-16 surrogates in literals, since this is not 
> actually legal unicode, nor does it work in Jython's UTF-16 based 
> representation. Ironically this usage is to detect such illegal use in input 
> streams.
> * Relative star imports, which seems to impact a number of interesting 
> projects.
> * Performance tuning of sre. Jython has a port of CPython's sre, however our 
> use of UTF-16 requires expansion into an array of codepoints. Currently this 
> is done on demand, which can potentially add another O(n) factor in 
> evaluating regexes. A pull request we will apply memoizes. In the future, we 
> will rewrite the logic in sre so that it does next/prev, much like JRuby 
> currently does for similar encoding issues.
> 
> Related work
> 
> * Other PyPA tooling including virtualenv and wheel needs more diagnosis to 
> see why they currently fail on Jython, but our hope is that this is minor. 
> * New project jythontools by a number of Jython developers (including Frank 
> and Jim). This includes a number of projects that will help evolve Jython, 
> but outside the usual release schedule and the usual problem of being in core 
> (such as eventual deprecation):
>   - Clamp - precise integration with Java, enabling such capabilities as 
> Java directly importing Python modules without explicitly initializing the 
> Jython runtime or using object factories. Future work will enable Java 
> annotation integration, as decorators. Integrates with setuptools; future 
> integration as well with Maven via Aether.
>   - Jiffy - provide a CFFI backend for Jython. Right now it is pure 
> vaporware, but cursory examination of cffi.backend_ctypes suggests that it 
> should be straightforward and of modest effort to provide a similar backend 
> by using JFFI, which Jython and JRuby both use to access native runtime 
> services (such as Posix API) as part of the Java native runtime project.
> * The Patois project has been started to collect examples for 
> cross-implementation support, as seen in surrogate support, but it will be a 
> good question to get that really going, vs just talking about it.
> * JyNI - simply adding this jar to the classpath enables C extension API 
> support. Note that this project has been licensed by its developer (not a 
> Jython committer) under an LGPL license.
> 
> Release schedule
> 
> * Complete beta 2
> * Beta 3 is forthcoming, likely in 2 weeks
> * For beta 4, need to perform a comprehensive bug triage - what will be in, 
> not in for 2.7.0
> * EuroPython sprint to finalize a release candidate for 2.7.0?
> 
> Future
> 
> * Mostly around performance, Java integration, and of course the usual bug 
> fixes
> * Python bytecode compiler remains important, including for support targeting 
> Android and removing restriction on getting too large a method for the JVM
> * More hooks for Java integration, includi

Re: [Python-Dev] git history conundrum

[Michael Foord]

> On 28 Apr 2019, at 22:55, Chris Withers  wrote:
> 
>> On 28/04/2019 22:21, Robert Collins wrote:
>> Thank you!
> 
> Thank me when we get there ;-) Currently in Dec 2018 with a wonderful Py2 
> failure:
> 
> ==
> ERROR: test_autospec_getattr_partial_function 
> (mock.tests.testhelpers.SpecSignatureTest)
> --
> Traceback (most recent call last):
>  File "mock/tests/testhelpers.py", line 973, in 
> test_autospec_getattr_partial_function
>autospec = create_autospec(proxy)
>  File "mock/mock.py", line 2392, in create_autospec
>for entry in dir(spec):
> TypeError: __dir__() must return a list, not str
> 
> Once we're done, I'll need a username/password that can write to 
> https://pypi.org/project/mock/ ...

I can add you as a maintainer. Ping me off-list. 

Michael



> 
>> If I understand correctly this is just the hg style branch backport 
>> consequence, multiple copies of a change. Should be safe to skip those.
> 
> Yep, current script I've been using is here, high level highlighted:
> 
> https://github.com/cjw296/mock/blob/backporting/backport.py#L102-L125
> 
> cheers,
> 
> Chris
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: Preserve keyword argument order in unittest.mock call repr output

[Michael Foord]

Sent from my iPhone

> On 22 Jun 2019, at 08:15, Karthikeyan Singaravelan  
> wrote:
> 
> From Python 3.6 the order of keyword arguments to a function is preserved. In 
> https://bugs.python.org/issue21256 the order of keyword arguments for 
> unittest.mock's repr were sorted to return deterministic output for usage in 
> doctest and other use cases. This currently gives little inconsistent output 
> where the keyword argument order in below mock call is (b=2, a=1) but it's 
> sorted in the error message and mock_calls list to give (a=1, b=2). 
> 
> I have opened https://bugs.python.org/issue37212 to preserve the order. It 
> was recommended in the issue 21256 too at 
> https://bugs.python.org/issue21256#msg216512 . The drawback is that it's 
> backwards incompatible where tests that assert error messages might have used 
> the sorted order. Due to equality implementation call(a=1, b=2) == call(b=2, 
> a=1) is still True so assertions are not affected. There are no test failures 
> in mock test suite except the test in the original issue where sorted output 
> is asserted. Sorting the keyword arguments was also not documented. I propose 
> removing this sorted order in 3.9 and to preserve the original order. The 
> change is straightforward and I can add a PR if this change is accepted.


+1 go ahead. The test that will fail was synthetic for the sorting and it won't 
cause real tests to fail. 

Michael


> 
 from unittest.mock import Mock, call
 m = Mock(name='m')
 m(b=2, a=1)
> 
 call(a=1, b=2) == call(b=2, a=1)
> True
 m.mock_calls
> [call(a=1, b=2)]
 m.assert_called_with(c=1)
> Traceback (most recent call last):
>  File "", line 1, in 
>  File 
> "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", 
> line 870, in assert_called_with
>raise AssertionError(_error_message()) from cause
> AssertionError: expected call not found.
> Expected: m(c=1)
> Actual: m(a=1, b=2)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/6F2NGCERZLZ2CDGYIAR5QOHMGAAE5VHE/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JWPDLKNM5X27FRWJG7UOAFXHNRUGH44S/

Re: [Python-Dev] PLY in stdlib (was cffi in stdlib)

[Michael Foord]

On 27 Feb 2013, at 11:00, David Beazley  wrote:

>> 
>> From: Eli Bendersky 
>> 
>> I'll be the first one to admit that pycparser is almost certainly not
>> generally useful enough to be exposed in the stdlib. So just using it as an
>> implementation detail is absolutely fine. PLY is a more interesting
>> question, however, since PLY is somewhat more generally useful. That said,
>> I see all this as implementation details that shouldn't distract us from
>> the main point of whether cffi should be added.
>> 
> 
> Regarding the inclusion of PLY or some subcomponent of it in the standard 
> library, it's not an entirely crazy idea in my opinion.

+1 PLY is capable and well tried-and-tested. We used it in Resolver One to 
implement a pretty large grammar and it is (in my opinion) best of breed in the 
Python parser generator world. Being stable and widely used, with an "available 
maintainer", makes it an ideal candidate for standard library inclusion.

Michael

>  LALR(1) parsers have been around for a long time, are generally known to 
> anyone who's used yacc/bison, and would be useful outside the context of cffi 
> or pycparser.  PLY has also been around for about 12 years and is what I 
> would call stable.  It gets an update about every year or two, but that's 
> about it.   PLY is also relatively small--just two files and about 4300 lines 
> of code (much of which could probably be scaled down a bit).
> 
> The only downside to including PLY might be the fact that there are very few 
> people walking around who've actually had to *implement* an LALR(1) parser 
> generator.  Some of the code for that is extremely hairy and mathematical.   
> At this time, I don't think there are any bugs in it, but it's not the sort 
> of thing that one wants to wander into casually.Also, there are some 
> horrible hacks in PLY that I'd really like to get rid of, but am currently 
> stuck with due to backwards compatibility issues. 
> 
> Alex Gaynor has been working on a PLY variant (RPLY) geared at RPython and 
> which has a slightly different programming interface.I'd say if we were 
> to go down this route, he and I should work together to put together some 
> kind of more general "parsing.lalr" package (or similar) that  cleans it up 
> and makes it more suitable as a library for building different kinds of 
> parsing tools on top of. 
> 
> Cheers,
> Dave
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

Hello all,

PyCon, and the Python Language Summit, is nearly upon us. We have a good number 
of people confirmed to attend. If you are intending to come to the language 
summit but haven't let me know please do so.

The agenda of topics for discussion so far includes the following:

* A report on pypy status - Maciej and Armin
* Jython and IronPython status reports - Dino / Frank 
* Packaging (Doug Hellmann and Monty Taylor at least)
* Cleaning up interpreter initialisation (both in hopes of finding areas
  to rationalise and hence speed things up, as well as making things
  more embedding friendly). Nick Coghlan
* Adding new async capabilities to the standard library (Guido)
* cffi and the standard library - Maciej
* flufl.enum and the standard library - Barry Warsaw
* The argument clinic - Larry Hastings

If you have other items you'd like to discuss please let me know and I can add 
them to the agenda.

All the best,

Michael Foord

--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 28 Feb 2013, at 07:36, Georg Brandl  wrote:

> Am 27.02.2013 17:51, schrieb Michael Foord:
>> Hello all,
>> 
>> PyCon, and the Python Language Summit, is nearly upon us. We have a good 
>> number of people confirmed to attend. If you are intending to come to the 
>> language summit but haven't let me know please do so.
>> 
>> The agenda of topics for discussion so far includes the following:
>> 
>> * A report on pypy status - Maciej and Armin
>> * Jython and IronPython status reports - Dino / Frank 
>> * Packaging (Doug Hellmann and Monty Taylor at least)
>> * Cleaning up interpreter initialisation (both in hopes of finding areas
>>  to rationalise and hence speed things up, as well as making things
>>  more embedding friendly). Nick Coghlan
>> * Adding new async capabilities to the standard library (Guido)
>> * cffi and the standard library - Maciej
>> * flufl.enum and the standard library - Barry Warsaw
>> * The argument clinic - Larry Hastings
>> 
>> If you have other items you'd like to discuss please let me know and I can 
>> add them to the agenda.
> 
> May I in absentia propose at least a short discussion of the XML fixes
> and accompanying security releases?  FWIW, for 3.2 and 3.3 I have no
> objections to secure-by-default.
> 

Sure. It would be good if someone who *will* be there can champion the 
discussion.

Michael

> Georg
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 27 Feb 2013, at 18:50, Antoine Pitrou  wrote:

> On Wed, 27 Feb 2013 16:51:16 +
> Michael Foord  wrote:
> 
>> Hello all,
>> 
>> PyCon, and the Python Language Summit, is nearly upon us. We have a good 
>> number of people confirmed to attend. If you are intending to come to the 
>> language summit but haven't let me know please do so.
>> 
>> The agenda of topics for discussion so far includes the following:
>> 
>> * A report on pypy status - Maciej and Armin
>> * Jython and IronPython status reports - Dino / Frank 
>> * Packaging (Doug Hellmann and Monty Taylor at least)
>> * Cleaning up interpreter initialisation (both in hopes of finding areas
>>  to rationalise and hence speed things up, as well as making things
>>  more embedding friendly). Nick Coghlan
>> * Adding new async capabilities to the standard library (Guido)
>> * cffi and the standard library - Maciej
>> * flufl.enum and the standard library - Barry Warsaw
>> * The argument clinic - Larry Hastings
>> 
>> If you have other items you'd like to discuss please let me know and I can 
>> add them to the agenda.
> 
> Perhaps someone wants to discuss
> http://www.python.org/dev/peps/pep-0428/, but I won't be there and the
> PEP isn't terribly up-to-date either :-)

If you can find someone familiar with pathlib to champion the discussion it is 
more likely to happen and be productive... Getting the PEP up to date before 
the summit will also help. (I very much like the *idea* of pathlib and the bits 
I've seen / read through - but I haven't used it in anger yet so I don't feel 
qualified to champion it myself.)

Michael

> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 27 Feb 2013, at 19:01, fwierzbi...@gmail.com wrote:

> On Wed, Feb 27, 2013 at 8:51 AM, Michael Foord  
> wrote:
>> If you have other items you'd like to discuss please let me know and I can 
>> add them to the agenda.
> 
> I'd like to discuss merging Jython's standard Lib (the *.py files). We
> have in the past had agreement that this would be a good idea - I just
> want to bring it up since I think this is probably the year that I'm
> actually going to do it.

Added to the agenda.

Michael

> 
> -Frank
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 28 Feb 2013, at 03:42, Barry Warsaw  wrote:

> On Feb 27, 2013, at 04:51 PM, Michael Foord wrote:
> 
>> If you have other items you'd like to discuss please let me know and I can
>> add them to the agenda.
> 
> I'd like to have some discussions around promotion of Python 3, how we can
> accelerate its adoption, availability of supporting packages, what critical
> projects are still missing, etc.
> 

Added to the agenda.

Michael


> -Barry
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 3 Mar 2013, at 01:29, Trent Nelson  wrote:

> On Wed, Feb 27, 2013 at 08:51:16AM -0800, Michael Foord wrote:
>> If you have other items you'd like to discuss please let me know and I
>> can add them to the agenda.
> 
>Hmm, seems like this might be a good forum to introduce the
>parallel/async stuff I've been working on the past few months.
>TL;DR version is I've come up with an alternative approach for
>exploiting multiple cores that doesn't rely on GIL-removal or
>STM (and has a negligible performance overhead when executing
>single-threaded code).  (For those that are curious, it lives
>in the px branch of the sandbox/trent repo on hg.p.o, albeit
>in a very experimental/prototype/proof-of-concept state (i.e.
>it's an unorganized, undocumented, uncommented hackfest); on
>the plus side, it works.  Sort of.)
> 
>Second suggestion: perhaps a little segment on Snakebite?  What
>it is, what's available to committers, feedback/kvetching from
>those who have already used it, etc.
> 

I've added both to the agenda.

>(I forgot the format of these summits -- is there a projector?)
> 

I've asked for a projector, yes.

Michael

>Trent.


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 4 Mar 2013, at 06:26, Robert Collins  wrote:

> On 4 March 2013 18:54, Guido van Rossum  wrote:
>> On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins
>>  wrote:
>>> I'd like to talk about overhauling - not tweaking, overhauling - the
>>> standard library testing facilities.
>> 
>> That seems like too big a topic and too vague a description to discuss
>> usefully. Perhaps you have a specific proposal? Or at least just a use
>> case that's poorly covered?
> 
> I have both - I have a draft implementation for a new test result API
> (and forwards and backwards compat code etc), and use cases that drive
> it. I started a thread here -
> http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html
> , with blog posts
> https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/
> https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/
> https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/
> https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/
> 
> They are focused on subunit, but much of subunit's friction has been
> due to issues encountered from the stdlibrary TestResult API - in
> particular three things:
> - the single-active-test model that the current API (or at least
> implementation) has.
> - the expectation that all test outcomes will originate from the same
> interpreter (or something with a live traceback object)
> - the inability to supply details about errors other than the exception
> 
> All of which start to bite rather deep when working on massively
> parallel test environments.
> 
> It is of course possible for subunit and related tools to run their
> own implementation, but it seems ideal to me to have a common API
> which regular unittest, nose, py.test and others can all agree on and
> use : better reuse for pretty printers, GUI displays and the like
> depend on some common API.
> 
>> TBH, your choice of words is ambiguous -- are you interested in
>> overhauling the facilities for testing *of* the standard library (i.e.
>> the 'test' package), or the testing facilities *provided by* the
>> standard library (i.e. the unittest module)?
> 
> Sorry! Testing facilities provided by the standard library. They
> should naturally facilitate testing of the standard library too.


We can certainly talk about it - although as Guido says, something specific may 
be easier to have a useful discussion about.

Reading through your blog articles it seemed like a whole lot of subunit 
context was required to understand the specific proposal you're making for the 
TestResult. It also *seems* like you're redesigning the TestResult for a single 
use case (distributed testing) with an api that looks quite "odd" for anything 
that isn't that use case. I'd rather see how we can make the TestResult play 
*better* with those requirements. That discussion probably belongs in another 
thread - or at the summit.

Michael

> 
> -Rob
> 
>> --
>> --Guido van Rossum (python.org/~guido)
> 
> 
> 
> -- 
> Robert Collins 
> Distinguished Technologist
> HP Cloud Services
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 4 Mar 2013, at 19:26, Berker Peksağ  wrote:

> On Mon, Mar 4, 2013 at 9:14 PM, Barry Warsaw  wrote:
>> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>> 
 $ python -m unittest discover
 $ python setup.py test
 $ python setup.py nosetests
 $ python -m nose test
 $ nosetests-X.Y
 
 Besides having a multitude of choices, there's almost no way to
 automatically discover (e.g. by metadata inspection or some such) how to
 invoke the tests.  You're often lucky if there's a README.test and it's
 still accurate.
>>> 
>>> I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
>>> Typing "python -m unittest discover" is too cumbersome.
>> 
>> Where is this work being done (e.g. is there a PEP)?
> 
> There is an open issue on the tracker: http://bugs.python.org/issue14266
> 


Indeed, and unittest2 (the backport) which has to work with Python 2.6 (where 
"python -m package_name" doesn't work) has "unit2" as a shortcut. So it has an 
advantage over the standard library version here.

I'd like to see pyunit as a short-cut for "python -m unittest discover", with a 
"pyunit-3.x" variant too. 

Barry objects that Linux distributions won't want to support all of these, 
which is frankly their problem.

Michael

> --Berker
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 28 Feb 2013, at 13:49, Brett Cannon  wrote:

> 
> 
> 
> On Thu, Feb 28, 2013 at 6:34 AM, Michael Foord  
> wrote:
> 
> On 28 Feb 2013, at 07:36, Georg Brandl  wrote:
> 
> > Am 27.02.2013 17:51, schrieb Michael Foord:
> >> Hello all,
> >>
> >> PyCon, and the Python Language Summit, is nearly upon us. We have a good 
> >> number of people confirmed to attend. If you are intending to come to the 
> >> language summit but haven't let me know please do so.
> >>
> >> The agenda of topics for discussion so far includes the following:
> >>
> >> * A report on pypy status - Maciej and Armin
> >> * Jython and IronPython status reports - Dino / Frank
> >> * Packaging (Doug Hellmann and Monty Taylor at least)
> >> * Cleaning up interpreter initialisation (both in hopes of finding areas
> >>  to rationalise and hence speed things up, as well as making things
> >>  more embedding friendly). Nick Coghlan
> >> * Adding new async capabilities to the standard library (Guido)
> >> * cffi and the standard library - Maciej
> >> * flufl.enum and the standard library - Barry Warsaw
> >> * The argument clinic - Larry Hastings
> >>
> >> If you have other items you'd like to discuss please let me know and I can 
> >> add them to the agenda.
> >
> > May I in absentia propose at least a short discussion of the XML fixes
> > and accompanying security releases?  FWIW, for 3.2 and 3.3 I have no
> > objections to secure-by-default.
> >
> 
> Sure. It would be good if someone who *will* be there can champion the 
> discussion.
> 
> While Christian is in the best position to discuss this, I did review his 
> various monkeypatch fixes + expat patches so I can attempt to answer any 
> questions people may have.

I've put you next to the topic in the agenda Brett :-)

Michael

--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 4 Mar 2013, at 20:02, Daniel Holth  wrote:

> On Mon, Mar 4, 2013 at 2:26 PM, Berker Peksağ  wrote:
>> On Mon, Mar 4, 2013 at 9:14 PM, Barry Warsaw  wrote:
>>> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>>> 
> $ python -m unittest discover
> $ python setup.py test
> $ python setup.py nosetests
> $ python -m nose test
> $ nosetests-X.Y
> 
> Besides having a multitude of choices, there's almost no way to
> automatically discover (e.g. by metadata inspection or some such) how to
> invoke the tests.  You're often lucky if there's a README.test and it's
> still accurate.
 
 I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
 Typing "python -m unittest discover" is too cumbersome.
>>> 
>>> Where is this work being done (e.g. is there a PEP)?
>> 
>> There is an open issue on the tracker: http://bugs.python.org/issue14266
>> 
>> --Berker
> 
> setup.py's setup(test_suite="x")... not sure if this is a distutils or
> setuptools feature. PEP 426 has an extension mechanism that could do
> the job.


This is a setuptools extension. There was some discussion for 
packaging/distutils2 of having test support but I have no idea if that has been 
picked up for the new bunch of packaging related work.

Michael

> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 1 Mar 2013, at 18:38, Antoine Pitrou  wrote:

> On Fri, 1 Mar 2013 09:32:23 -0500
> Barry Warsaw  wrote:
>> 
>>> On the other hand in some ways Jython is sort of like Python on a
>>> weird virtual OS that lets the real OS bleed through some. This may
>>> still need to be checked in that way (there's are still checks of >> os.name == 'nt'> right?)
>> 
>> Yeah, but that all old code ;)
> 
> Hmm, what do you mean? `os.name == 'nt'` is still the proper way to
> test that we're running on a Windows system (more accurately, over the
> Windows API).
> 

It has been used incorrectly in a few places in the Python standard library - 
Windows support code that would work correctly on IronPython is skipped because 
os.name is *not* 'nt' on IronPython.  That was the case in the past anyway. 
It's quite some time since I've used IronPython now.

Michael

> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 5 Mar 2013, at 00:23, Robert Collins  wrote:

> On 5 March 2013 13:21, Michael Foord  wrote:
>> 
> 
>> We can certainly talk about it - although as Guido says, something specific 
>> may be easier to have a useful discussion about.
>> 
>> Reading through your blog articles it seemed like a whole lot of subunit 
>> context was required to understand the specific
>> proposal you're making for the TestResult. It also *seems* like you're 
>> redesigning the TestResult for a single use case
>> (distributed testing) with an api that looks quite "odd" for anything that 
>> isn't that use case. I'd rather see how we can
>> make the TestResult play *better* with those requirements. That discussion 
>> probably belongs in another thread - or at
>> the summit.
> 
> Right - all I wanted was to flag that you and I and any other
> interested parties should discuss this at the summit :).

I've added a testing topic to the agenda. At the very least you could outline 
your streaming test result proposal, or kick off a meta discussion. We'll 
probably time limit the discussion so some specific focus will make it more 
productive - or maybe you can get a feel for how open to major changes in this 
area other python devs are.

Michael

> 
> -Rob
> 
> 
> 
> 
> 
> 
> -- 
> Robert Collins 
> Distinguished Technologist
> HP Cloud Services


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 4 Mar 2013, at 22:24, Barry Warsaw  wrote:

> On Mar 04, 2013, at 05:04 PM, Brett Cannon wrote:
> 
>> Sure, but that has nothing to do with programmatic package discovery.
>> That's something you will have to do as a person in making a qualitative
>> decision along the same lines as API design. Flipping a bit in a config
>> file saying "I have tests" doesn't say much beyond you flipped a bit, e.g.
>> no idea on coverage, quality, etc.
> 
> What I'm looking for is something that automated tools can use to easily
> discover how to run a package's tests.  I want it to be dead simple for
> developers of a package to declare how their tests are to be run, and what
> extra dependencies they might need.  It seems like PEP 426 only addresses the
> latter.  Maybe that's fine and a different PEP is needed to describe automated
> test discover, but I still think it's an important use case.
> 
> Imagine:
> 
> * Every time you upload a package to PyPI, snakebite runs your test suite on
>   a variety of Python versions and platforms.  You get a nice link to the
>   Jenkins results so you and your users get a good sense of overall package
>   quality.
> 
> * You have an automated gatekeeper that will prevent commits or uploads if
>   your coverage or test results get worse instead of better.
> 
> * Distro packagers can build tools that auto-discover the tests so that they
>   are run automatically when the package is built, ensuring high quality
>   packages specifically targeted to those distros.
> 
> As a community, we know how important tests are, so I think our tools should
> reflect that and make it easy for those tests to be expressed.  As a selfish
> side-effect, I want to reduce the amount of guesswork I need to perform in
> order to know how to run a package's test when I `$vcs clone` their
> repository. ;)
> 


Distutils2 had a way of specifying this in the metadata. It looks like this 
hasn't made it into the reboot:

 http://alexis.notmyidea.org/distutils2/distutils/newcommands.html

Michael 

> Cheers,
> -Barry
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 4 Mar 2013, at 20:00, Robert Collins  wrote:

> On 5 March 2013 05:34, Brett Cannon  wrote:
>> 
>> 
>> 
>> On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw  wrote:
>>> 
>>> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
>>> 
 It is of course possible for subunit and related tools to run their
 own implementation, but it seems ideal to me to have a common API
 which regular unittest, nose, py.test and others can all agree on and
 use : better reuse for pretty printers, GUI displays and the like
 depend on some common API.
>>> 
>>> And One True Way of invoking and/or discovering how to invoke, a package's
>>> test suite.
>> 
>> 
>> How does unittest's test discovery not solve that?
> 
> Three reasons
> a) There are some bugs (all filed I think) - I intend to hack on
> these in the near future - that prevent discovery working at all for
> some use cases.


The only discovery related issues I'm aware of are:

* Issue 16079 (filed by you) - trivial to fix just needs a test
* Issue 15010 obscure and unlikely to be an issue for standard discovery

I'm not aware of any other discovery related issues. Please let me know (or add 
me as nosy) to them.

> b) discovery requires magic parameters that are project specific
> (e.g. is it 'discover .' or 'discover . lib' to run it). This is
> arguably a setup.py/packaging entrypoint issue.


This was addressed by Barry - and yes discovery has to be done with the right 
parameters. If you layout your project in a particular way then "python -m 
unittest discover" in the project root will just work. This is project specific 
metadata though and not a particular problem of any testing library.

> c) Test suites written for e.g. Twisted, or nose, or other
> non-stdunit-runner-compatible test runners will fail to execute even
> when discovered correctly.
> 
> There are ways to solve this without addressing a/b/c - just defining
> a standard command to run that signals success/failure with it's exit
> code. Packages can export a particular flavour of that in their
> setup.py if they have exceptional needs, and do nothing in the common
> case. That doesn't solve 'how to introspect a package test suite' but
> for distro packagers - and large scale CI integration - that doesn't
> matter.
> 
> For instance testrepository offers a setuptools extension to let it be
> used trivially, I believe nose does something similar.
> 

unittest2 also has setuptools compatible test command.

> Having something that would let *any* test suite spit out folk's
> favourite test protocol de jour would be brilliant of course :).
> [junit-xml, subunit, TAP, ...]
> 

Yes.

Michael

> -Rob
> 
> -- 
> Robert Collins 
> Distinguished Technologist
> HP Cloud Services
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)

[Michael Foord]

On 5 Mar 2013, at 07:19, Antoine Pitrou  wrote:

> On Mon, 4 Mar 2013 15:47:37 -0800
> Eli Bendersky  wrote:
>> On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou  wrote:
>> 
>>> On Mon, 4 Mar 2013 13:26:57 -0800
>>> Eli Bendersky  wrote:
 [Splitting into a separate thread]
 
 Do we really need to overthink something that requires a trivial alias to
 set up for one's own convenience?
 
 Picking a Python version (as Barry mentions) is just one of the problems.
 What's wrong with:
 
 alias rupytests='python3 -m unittest discover"
 alias runpytests2='python2 -m unittest discover"
 
 ?
 
 Don't get me wrong, I love the "discover" option and agree that it should
 be the recommended way to go - but isn't this largely a documentation
>>> issue?
>>> 
>>> I would personally call it a typing issue :-) "python -m unittest
>>> discover" is just too long.
>>> 
>> 
>> Command-line options for advanced capabilities can get long, yes.
> 
> The whole point is that discovery is not "advanced capability", it's
> pretty basic by today's standards. So it should actually be the default
> behaviour (like it is with nose).
> 


For Python 3.3 onwards "python -m unittest" does run test discovery by default. 
However if you want to provide parameters you still need the "discover" 
subcommand to disambiguate from the other command line options. So I agree - a 
shorthand command would be an improvement.

Michael



> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)

[Michael Foord]

On 5 Mar 2013, at 09:02, Glyph  wrote:

> On Mar 4, 2013, at 11:13 PM, Robert Collins  wrote:
> 
>> In principle maybe. Need to talk with the trial developers, nose
>> developers, py.test developers etc - to get consensus on a number of
>> internal API friction points.
> 
> Some of trial's lessons might be also useful for the stdlib going forward, 
> given the hope of doing some event-loop stuff in the core.
> 
> But, I feel like this might be too much to cover at the language summit; 
> there could be a test frameworks summit of its own, of about equivalent time 
> and scope, and we'd still have a lot to discuss.
> 
> Is there a unit testing SIG someone from Twisted ought to be a member of, to 
> represent Trial, and to get consensus on these points going forward?


The testing-on-python mailing list is probably the best place (and if doesn't 
have that status already I'd be keen to elevate it to "official sig for Python 
testing issues" status).

http://lists.idyll.org/listinfo/testing-in-python

Like the "massively distributed testing" use case, I'd be very happy for the 
standard library testing capabilities to better support this use case - but I 
wouldn't like to design their apis around that as the sole use case. :-)

Michael

> 
> -glyph
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

[Michael Foord]

On 5 Mar 2013, at 05:39, Jeff Hardy  wrote:

> On Mon, Mar 4, 2013 at 4:39 PM, Michael Foord  
> wrote:
>> 
>> On 1 Mar 2013, at 18:38, Antoine Pitrou  wrote:
>> 
>>> On Fri, 1 Mar 2013 09:32:23 -0500
>>> Barry Warsaw  wrote:
>>>> 
>>>>> On the other hand in some ways Jython is sort of like Python on a
>>>>> weird virtual OS that lets the real OS bleed through some. This may
>>>>> still need to be checked in that way (there's are still checks of >>>> os.name == 'nt'> right?)
>>>> 
>>>> Yeah, but that all old code ;)
>>> 
>>> Hmm, what do you mean? `os.name == 'nt'` is still the proper way to
>>> test that we're running on a Windows system (more accurately, over the
>>> Windows API).
>>> 
>> 
>> It has been used incorrectly in a few places in the Python standard library 
>> - Windows support code that would work correctly on IronPython is skipped 
>> because os.name is *not* 'nt' on IronPython.  That was the case in the past 
>> anyway. It's quite some time since I've used IronPython now.
> 
> I think you misremembered - there's lots of code that uses
> `sys.platform == 'win32'` to detect Windows, but sys.platform is 'cli'
> for IronPython. I'm pretty sure `os.name has always been 'nt' (when
> running on Windows), and if not, it definitely is now.
> 
> Jython sets os.name to 'java' (IIRC), so there isn't a uniform way to
> detect Windows across all implementations.
> 


Ah, I'm sure you're correct. 

Thanks.

> - Jeff


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to

[Michael Foord]

On 19 Mar 2013, at 17:26, Antoine Pitrou  wrote:

> On Wed, 20 Mar 2013 01:22:58 +0100 (CET)
> michael.foord  wrote:
>> http://hg.python.org/cpython/rev/684b75600fa9
>> changeset:   82811:684b75600fa9
>> user:Michael Foord 
>> date:Tue Mar 19 17:22:51 2013 -0700
>> summary:
>>  Closes issue 17467. Add readline and readlines support to 
>> unittest.mock.mock_open
> 
> Wasn't it possible to re-use an existing implementation (such as
> TextIOBase or StringIO) rather than re-write your own?
> 
> (it's not even obvious your implementation is correct, BTW. How about
> universal newlines?)

mock_open makes it easy to put a StringIO in place if that's what you want. 
It's just a simple helper function for providing some known data *along with 
the Mock api* to make asserts that it was used correctly. It isn't presenting a 
full file-system. My suggestion to the implementor of the patch was that read / 
readline / readlines be disconnected - but the patch provided allows them to be 
interleaved and I saw no reason to undo that.

If users want more complex behaviour (like universal newline support) they can 
use mock_open along with a StringIO.

Michael


> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to

[Michael Foord]

On 20 Mar 2013, at 00:09, Antoine Pitrou  wrote:

> On Tue, 19 Mar 2013 21:44:15 -0700
> Michael Foord  wrote:
>> 
>> mock_open makes it easy to put a StringIO in place if that's what you want. 
>> It's just a simple helper function for providing some known data *along with 
>> the Mock api* to make asserts that it was used correctly. It isn't 
>> presenting a full file-system. My suggestion to the implementor of the patch 
>> was that read / readline / readlines be disconnected - but the patch 
>> provided allows them to be interleaved and I saw no reason to undo that.
>> 
>> If users want more complex behaviour (like universal newline support) they 
>> can use mock_open along with a StringIO.
> 
> This is not about complex behaviour but simply correct behaviour.
> For the record, universal newlines are enabled by default in Python 3:
> 
>>>> with open("foo", "wb") as f: f.write(b"a\r\nb\rc\n")
> ... 
> 7
>>>> with open("foo", "r") as f: print(list(f))
> ... 
> ['a\n', 'b\n', 'c\n']
> 

mock_open is •not• presenting a mock filesystem, but is about providing a mock 
object to avoid •either• reading or writing to the real filesystem. You don't 
•tend• to do both with a single file handle  - I know it's possible but 
mock_open is a convenience function for the common case.

This commit simply adds support for readline and readlines, whereas before only 
read was supported.

If you want to add support for additional functionality feel free to propose a 
patch.

Michael



> 
> Regards
> 
> Antoine.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to

[Michael Foord]

On 20 Mar 2013, at 01:03, Antoine Pitrou  wrote:

> On Wed, 20 Mar 2013 00:50:27 -0700
> Michael Foord  wrote:
>> 
>> If you want to add support for additional functionality feel free to propose 
>> a patch.
> 
> This isn't about additional functionality, this is about
> correctness. You don't want to write multiple, slightly different,
> implementations of readline() and friends (which is what Python 2 did).
> 


This change allows you to set a series of return values for readlines when 
mocking open.

We are not reading data from anywhere the user is supplying it pre-canned.

Do you have a specific problem with it? 

Michael

> Regards
> 
> Antoine.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to

[Michael Foord]

On 19 Mar 2013, at 23:09, "Gregory P. Smith"  wrote:

> 
> On Tue, Mar 19, 2013 at 9:44 PM, Michael Foord  
> wrote:
>> 
>> On 19 Mar 2013, at 17:26, Antoine Pitrou  wrote:
>> 
>> > On Wed, 20 Mar 2013 01:22:58 +0100 (CET)
>> > michael.foord  wrote:
>> >> http://hg.python.org/cpython/rev/684b75600fa9
>> >> changeset:   82811:684b75600fa9
>> >> user:Michael Foord 
>> >> date:Tue Mar 19 17:22:51 2013 -0700
>> >> summary:
>> >>  Closes issue 17467. Add readline and readlines support to 
>> >> unittest.mock.mock_open
>> >
>> > Wasn't it possible to re-use an existing implementation (such as
>> > TextIOBase or StringIO) rather than re-write your own?
>> >
>> > (it's not even obvious your implementation is correct, BTW. How about
>> > universal newlines?)
>> 
>> mock_open makes it easy to put a StringIO in place if that's what you want. 
>> It's just a simple helper function for providing some known data *along with 
>> the Mock api* to make asserts that it was used correctly. It isn't 
>> presenting a full file-system. My suggestion to the implementor of the patch 
>> was that read / readline / readlines be disconnected - but the patch 
>> provided allows them to be interleaved and I saw no reason to undo that.
>> 
>> If users want more complex behaviour (like universal newline support) they 
>> can use mock_open along with a StringIO.
> 
> It'd be good to mention that in the unittest.mock.rst docs.
>  

I'll look at clarifying the intent and limitations of mock_open in the docs - 
plus an example of using it with a StringIO.

It maybe that the support for interleaving of read and readline (etc) is  just 
unnecessary and setting them separately is enough (which simplifies the 
implementation). I know Toshio had a specific use case needing readline support 
so I'll check with him.

Michael



>> 
>> Michael
>> 
>> 
>> >
>> > Regards
>> >
>> > Antoine.
>> >
>> >
>> > ___
>> > Python-Dev mailing list
>> > Python-Dev@python.org
>> > http://mail.python.org/mailman/listinfo/python-dev
>> > Unsubscribe: 
>> > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>> 
>> 
>> --
>> http://www.voidspace.org.uk/
>> 
>> 
>> May you do good and not evil
>> May you find forgiveness for yourself and forgive others
>> May you share freely, never taking more than you give.
>> -- the sqlite blessing
>> http://www.sqlite.org/different.html
>> 
>> 
>> 
>> 
>> 
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
> 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): Process DEFAULT values in mock side_effect that returns iterator.

[Michael Foord]

On 7 April 2013 14:44, andrew.svetlov  wrote:

> http://hg.python.org/cpython/rev/18fd64f1de2d
> changeset:   83179:18fd64f1de2d
> branch:  3.3
> user:Andrew Svetlov 
> date:Sun Apr 07 16:42:24 2013 +0300
> summary:
>   Process DEFAULT values in mock side_effect that returns iterator.
>
> Patch by Michael Ford.
>
> files:
>   Lib/unittest/mock.py   |  2 ++
>   Lib/unittest/test/testmock/testmock.py |  4 
>   2 files changed, 6 insertions(+), 0 deletions(-)
>
>
> diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py
> --- a/Lib/unittest/mock.py
> +++ b/Lib/unittest/mock.py
> @@ -904,6 +904,8 @@
>  result = next(effect)
>  if _is_exception(result):
>  raise result
> +if result is DEFAULT:
> +result = self.return_value
>  return result
>
>  ret_val = effect(*args, **kwargs)
> diff --git a/Lib/unittest/test/testmock/testmock.py
> b/Lib/unittest/test/testmock/testmock.py
> --- a/Lib/unittest/test/testmock/testmock.py
> +++ b/Lib/unittest/test/testmock/testmock.py
> @@ -906,6 +906,10 @@
>  self.assertRaises(StopIteration, mock)
>  self.assertIs(mock.side_effect, this_iter)
>
> +def test_side_effect_iterator_default(self):
> +mock = Mock(return_value=2)
> +mock.side_effect = iter([1, DEFAULT])
> +self.assertEqual([mock(), mock()], [1, 2])
>
>  def test_assert_has_calls_any_order(self):
>  mock = Mock()
>
> --
> Repository URL: http://hg.python.org/cpython
>
>

This was committed without a NEWS entry.

Michael



> ___
> Python-checkins mailing list
> python-check...@python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]

[Michael Foord]

On 20 May 2013, at 18:26, Mark Janssen  wrote:

>>> I'm hoping that core developers don't get caught-up in the "doctests are bad
>>> meme".
>>> 
>>> Instead, we should be clear about their primary purpose which is to test
>>> the examples given in docstrings.
>>> In other words, doctests have a perfectly legitimate use case.
>> 
>> But more than just one ;-)  Another great use has nothing to do with
>> docstrings:  using an entire file as "a doctest".   This encourages
>> writing lots of text explaining what you're doing,. with snippets of
>> code interspersed to illustrate that the code really does behave in
>> the ways you've claimed.
> 
> +1, very true.  I think doctest excel in almost every way above
> UnitTests.  I don't understand the popularity of  UnitTests, except
> perhaps for GUI testing which doctest can't handle.  I think people
> just aren't very *imaginative* about how to create good doctests that
> are *also* good documentation.
> 

Doc tests have lots of problems for unit testing.

* Every line is a test with *all* output part of the test - in unit tests you 
only assert the specific details you're interested in
* Unordered types are a pain with doctest unless you jump through hoops
* Tool support for editing within doctests is *generally* worse
* A failure on one line doesn't halt execution, so you can get many many 
reported errors from a single failure
* Try adding diagnostic prints and then running your doctests!
* Tools support in terms of test discovery and running individual tests is not 
as smooth
* Typing >>> and ... all the time is really annoying
* Doctests practically beg you to write your code first and then copy and paste 
terminal sessions - they're the enemy of TDD
* Failure messages are not over helpful and you lose the benefit of some of the 
new asserts (and their diagnostic output) in unittest
* Tests with non-linear code paths (branches) are more painful to express in 
doctests

and so on...

However doctests absolutely rock for testing documentation / docstring 
examples. So I'm with Raymond on this one.

All the best,

Michael

> That serves two very good purposes in one.  How can you beat that?
> The issues of teardown and setup are fixable and even more beautifully
> solved with doctests -- just use the lexical scoping of the program to
> determine the execution environment for the doctests.
> 
>> picking-your-poison-ly y'rs  - tim
> 
> Cheers,
> 
> Mark
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] lament for the demise of unbound methods

[Michael Foord]

On 4 Jul 2013, at 19:00, Guido van Rossum  wrote:

> Thanks for the code pointers. So it's all about monkeypatching. :-) I have 
> only a little sympathy, as there still seems to be a way to do this, it's 
> just less convenient. Too bad.
> 


I've also lamented the death of bound methods in Python 3 for mock 
"autospeccing". Autospec introspects objects and provides mock objects with the 
same attributes - and with the same method signatures. For methods it needs to 
trim the first argument (because instances are called externally without self 
of course). Not being able to tell the difference between a module level 
function and an unbound method caused some pain then. (I worked round it by 
flowing the information about where the object came from through the code but 
it did add ugliness).

Michael


> --Guido
> 
> On Thu, Jul 4, 2013 at 9:42 AM, Chris Withers  wrote:
> Hi Guido,
> 
> I've bumped into this a couple of times.
> 
> First time was when I wanted to know whether what I had was a classmethod, 
> staticmethod or normal method here:
> 
> https://github.com/Simplistix/testfixtures/blob/master/testfixtures/replace.py#L59
> 
> This resulted in having to trawl through __dict__ here:
> 
> https://github.com/Simplistix/testfixtures/blob/master/testfixtures/resolve.py#L17
> 
> ...rather than just using getattr.
> 
> I bumped into it again, yesterday, trying to add support for classes to this 
> lightweight dependency injection framework I'm developing:
> 
> https://github.com/Simplistix/mush/blob/master/tests/test_runner.py#L189
> 
> Here's my local copy of that test:
> 
> https://gist.github.com/cjw296/db64279c69cdc0c5e112
> 
> The workaround I was playing with this morning is a wrapper so that I know I 
> have a class method, although what I really want to write at this line is:
> 
> https://gist.github.com/cjw296/db64279c69cdc0c5e112#file-gistfile1-txt-L40
> 
> runner = Runner(T0, C1.meth, C2.meth1, C2.meth2)
> 
> ...but if I do that, how can the runner know that what it gets for its second 
> argument is a class method of C1?
> (which is this case means that it should do C1().meth() rather than C1.meth())
> 
> cheers,
> 
> Chris
> 
> 
> On 04/07/2013 17:25, Guido van Rossum wrote:
> Chris, what do you want to do with the knowledge you are seeking?
> 
> --Guido van Rossum (sent from Android phone)
> 
> On Jul 4, 2013 4:28 AM, "Chris Withers"  > wrote:
> 
> Hi All,
> 
> In Python 2, I can figure out whether I have a method or a function,
> and, more importantly, for an unbound method, I can figure out what
> class the method belongs to:
> 
>  >>> class MyClass(object):
> ...   def method(self): pass
> ...
>  >>> MyClass.method
> 
>  >>> MyClass.method.im_class
> 
> 
> There doesn't appear to be any way in Python 3 to do this, which is
> a little surprising and frustrating...
> 
> What am I missing here?
> 
> Chris
> 
> --
> Simplistix - Content Management, Batch Processing & Python Consulting
>  - http://www.simplistix.co.uk
> _
> Python-Dev mailing list
> Python-Dev@python.org 
> http://mail.python.org/__mailman/listinfo/python-dev
> 
> Unsubscribe:
> http://mail.python.org/__mailman/options/python-dev/__guido%40python.org
> 
> 
> 
> __
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> __
> 
> -- 
> Simplistix - Content Management, Batch Processing & Python Consulting
> - http://www.simplistix.co.uk
> 
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido) 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] lament for the demise of unbound methods

[Michael Foord]

On 5 Jul 2013, at 12:07, "Martin v. Löwis"  wrote:

> Am 05.07.13 11:23, schrieb Michael Foord:
>> I've also lamented the death of bound methods in Python 3 for mock
>> "autospeccing". Autospec introspects objects and provides mock
>> objects with the same attributes - and with the same method
>> signatures.
> 
> I wonder why you need to figure out the signatures in advance.
> Can you just wait until the function is actually used, and then
> process the parameters as you get them?
> 

How does that solve the problem? Given a call and a reference to the original 
"function object" I need to know whether or not to trim the first argument from 
the original signature or not (remove self if the corresponding function object 
is actually a method).

Michael

> Regards,
> Martin


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] lament for the demise of unbound methods

[Michael Foord]

On 5 Jul 2013, at 12:26, Łukasz Langa  wrote:

> On 5 lip 2013, at 12:07, Martin v. Löwis  wrote:
> 
>> I wonder why you need to figure out the signatures in advance.
>> Can you just wait until the function is actually used, and then
>> process the parameters as you get them?
>> 
> 
> My guess is that Michael's design lets mock objects be introspected as well, 
> i.e. they don't appear as magical as they really are to the user code.
> 

This is also true. Doing it up front has some conveniences - for example 
dir(...) works correctly.

Michael

> -- 
> Best regards,
> Łukasz Langa
> 
> WWW: http://lukasz.langa.pl/
> Twitter: @llanga
> IRC: ambv on #python-dev
> 


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest.TestSuite holding references to unittest.TestCase instances too long

[Michael Foord]

Sent from my iPhone

On 2 Aug 2013, at 14:51, Matt McClure  wrote:

> It seems unittest.TestSuite holds references to unittest.TestCase instances 
> after the test runs, until the test suite finishes. In a large suite, where 
> the TestCase instances consume memory during execution, that can lead to 
> exhausting all available memory and the OS killing the test process.

Well individual tests not releasing resources could be seen as a bug in those 
tests.

That aside, there's an open bug for this with some discussion and a proposed 
fix:

http://bugs.python.org/issue11798

The agreed on approach just needs doing. 

Michael


> 
> What do you think of a change like this?
> 
> $ hg diff
> diff -r 3bd55ec317a7 Lib/unittest/suite.py
> --- a/Lib/unittest/suite.py   Thu Aug 01 23:57:21 2013 +0200
> +++ b/Lib/unittest/suite.py   Fri Aug 02 07:42:22 2013 -0400
> @@ -90,7 +90,12 @@
>  if getattr(result, '_testRunEntered', False) is False:
>  result._testRunEntered = topLevel = True
>  
> -for test in self:
> +while True:
> +try:
> +test = self._tests.pop(0)
> +except IndexError:
> +break
> +
>  if result.shouldStop:
>  break
>  
> See also the conversation on django-developers[1] that led me here.
> 
> [1]: https://groups.google.com/forum/#!topic/django-developers/XUMetDSGVT0
> 
> -- 
> Matt McClure
> http://matthewlmcclure.com
> http://www.mapmyfitness.com/profile/matthewlmcclure
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest.TestSuite holding references to unittest.TestCase instances too long

[Michael Foord]

Sent from my iPhone

On 2 Aug 2013, at 19:19, Antoine Pitrou  wrote:

> Le Fri, 2 Aug 2013 18:13:13 +0300,
> Michael Foord  a écrit :
>> 
>> On 2 Aug 2013, at 14:51, Matt McClure 
>> wrote:
>> 
>>> It seems unittest.TestSuite holds references to unittest.TestCase
>>> instances after the test runs, until the test suite finishes. In a
>>> large suite, where the TestCase instances consume memory during
>>> execution, that can lead to exhausting all available memory and the
>>> OS killing the test process.
>> 
>> Well individual tests not releasing resources could be seen as a bug
>> in those tests.
>> 
>> That aside, there's an open bug for this with some discussion and a
>> proposed fix:
>> 
>> http://bugs.python.org/issue11798
>> 
>> The agreed on approach just needs doing.
> 
> The patch is basically ready for commit, except for a possible doc
> addition, no?

Looks to be the case, reading the patch it looks fine. I'm currently on holiday 
until Monday. If anyone is motivated to do the docs too and commit that would 
be great. Otherwise I'll get to it on my return.

Michael 



> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest.TestSuite holding references to unittest.TestCase instances too long

[Michael Foord]

Sent from my iPhone

On 3 Aug 2013, at 19:07, "R. David Murray"  wrote:

> On Sat, 03 Aug 2013 10:27:30 -0400, Matt McClure  
> wrote:
>> Michael Foord  voidspace.org.uk> writes:
>>> On 2 Aug 2013, at 19:19, Antoine Pitrou  pitrou.net> wrote:
>>>> The patch is basically ready for commit, except for a possible doc
>>>> addition, no?
>>> 
>>> Looks to be the case, reading the patch it looks fine. I'm currently on
>> holiday until Monday.
>>> If anyone is motivated to do the docs too and commit that would be great.
>> Otherwise I'll
>>> get to it on my return.
>> 
>> It looks like the patch is based on what will become 3.4. Would backporting
>> it to 2.7 be feasible?  What's involved in doing so?
> 
> That depends on how likely Michale thinks it is that it might break
> things.
> 

It smells to me like a new feature rather than a bugfix, and it's a moderately 
big change. I don't think it can be backported to 2.7 other than through 
unittest2.

Michael 


>> I took a crack at the docs.
> 
> Thanks.  Please post your patch to the issue, it will get lost here.
> 
> --David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Use cached builtins.

[Michael Foord]

On 3 Oct 2013, at 12:05, Nick Coghlan  wrote:

> 
> On 3 Oct 2013 06:00, "Victor Stinner"  wrote:
> >
> > I don't remember where, but I remember that I also saw things like
> > "str=str, len=len, ...". So you keep the same name, but you use fast
> > local lookups instead of slow builtin lookups.
> 
> functools uses the local binding trick in lru_cache as a speed hack (pretty 
> sure it uses an underscore prefix, though).
> 


Inside a function you *have* to use an underscore prefix (well, some alternate 
name anyway) - otherwise the assignment makes the name local and the lookup 
would fail with an unbound local error. In order to do the binding at 
definition time rather than call time (so only once) the assignment is often 
done in the function signature. This is *particularly* ugly as it not only 
messes up the using of the builtins but screws your function signature too. So 
It should really only be done where benchmarking proves it makes a difference. 
I've never managed to find such a case...

Michael

> However lru_cache *is* likely to end up being speed critical *and* it's 
> binding local variables , so it's actually shifting a lot more work to 
> compile time than merely trading a builtin lookup for a global lookup does.
> 
> For most code though, introducing that kind of complexity isn't worth the 
> cost in readability.
> 
> Cheers,
> Nick.
> 
> >
> > Victor
> >
> > 2013/10/2 Antoine Pitrou :
> > > On Wed,  2 Oct 2013 18:16:48 +0200 (CEST)
> > > serhiy.storchaka  wrote:
> > >> http://hg.python.org/cpython/rev/d48ac94e365f
> > >> changeset:   85931:d48ac94e365f
> > >> user:Serhiy Storchaka 
> > >> date:Wed Oct 02 19:15:54 2013 +0300
> > >> summary:
> > >>   Use cached builtins.
> > >
> > > What's the point? I don't think it's a good idea to uglify the code if
> > > there isn't a clear benefit.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > > ___
> > > Python-Dev mailing list
> > > Python-Dev@python.org
> > > https://mail.python.org/mailman/listinfo/python-dev
> > > Unsubscribe: 
> > > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (3.3): PythonCAD is now on PyQt, use Wing as a prominent PyGtk example.

[Michael Foord]

On 6 October 2013 11:45, georg.brandl  wrote:

> http://hg.python.org/cpython/rev/942b9420e7e9
> changeset:   86065:942b9420e7e9
> branch:  3.3
> parent:  86062:6ced4fb4f711
> user:Georg Brandl 
> date:Sun Oct 06 12:46:13 2013 +0200
> summary:
>   PythonCAD is now on PyQt, use Wing as a prominent PyGtk example.
>
>

Wing is only a good example of PyGtk until Wing 5 is out of beta. They too
have switched to PyQt...

Michael


> Found by Helge Stenström on docs@.
>
> files:
>   Doc/library/othergui.rst |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
>
>
> diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst
> --- a/Doc/library/othergui.rst
> +++ b/Doc/library/othergui.rst
> @@ -20,7 +20,7 @@
>of the library, GTK+ 2.  It provides an object oriented interface
> that
>is slightly higher level than the C one.  There are also bindings to
>`GNOME `_.  One well known PyGTK application
> is
> -  `PythonCAD `_. An online `tutorial
> +  `WingIDE `_. An online `tutorial
>`_ is available.
>
> `PyQt `_
>
> --
> Repository URL: http://hg.python.org/cpython
>
> ___
> Python-checkins mailing list
> python-check...@python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyCon US 2014

[Michael Foord]

On 10 Oct 2013, at 01:53, Eric Snow  wrote:

> Registration is now open for PyCon US 2014.  Are there any plans yet
> for the language summit?  Just the day (e.g. Thursday April 10) will
> suffice.  Then we can make arrangements appropriately.  Thanks.


Sorry for the late reply. Yes there will be a language summit. I'm pretty sure 
it will be on Thursday as the last few years. I'm getting definite confirmation 
of this.

Michael

> 
> -eric
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: Spammy stuff (was Re: congrats on 3.5! Alas, windows 7 users are having problems installing it)

[Michael Foord]

Sent from my iPhone

> On 15 Sep 2019, at 23:25, Tim Peters  wrote:
> 
> [Tim]
>> While python-dev has several "official" moderators, best I can tell
>> I'm the only one who has reviewed these messages for years.
> 
> I should clarify that!  That's not meant to be a dig at the other
> moderators.  I review everything because I'm retired and am near the
> computer many hours almost every day.  That's been so for at least a
> decade.  So the others rarely get a chance - the queue is very likely
> empty if they ever look.  After a few years of that, they may well
> stop looking ;-)

Personally I resent anything that takes time away from you making posts on 
Facebook. 

> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/ZZXYE2PPVEURQ23CEOFYS5YPBJR5BDDT/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QK43OIYXVV2LE2LTWR2X6VRPN2IYXOJ4/

[Python-Dev] Patch logging module for IronPython compatibility

[Michael Foord]

Hello all,

Issue 5287 is a patch for the logging module for compatibility with
IronPython. IronPython provides sys._getframe but it throws an exception
if you call it with a non-zero depth. This may be fixed in a future
version of IronPython.

http://bugs.python.org/issue5287

It doesn't at all change the behaviour on other platforms (does an
explicit platform check I'm afraid) but fixes a nasty problem with the
logging module not working at all on IronPython. As this is a bugfix for
IronPython at least and IronPython 2.6 is currently being worked on
(tracking Python 2.6) it would be great to get this into 2.6-maint.

All the best,

Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch logging module for IronPython compatibility

[Michael Foord]

Michael Foord wrote:

Hello all,

Issue 5287 is a patch for the logging module for compatibility with
IronPython. IronPython provides sys._getframe but it throws an exception
if you call it with a non-zero depth. This may be fixed in a future
version of IronPython.

http://bugs.python.org/issue5287

It doesn't at all change the behaviour on other platforms (does an
explicit platform check I'm afraid) but fixes a nasty problem with the
logging module not working at all on IronPython. As this is a bugfix for
IronPython at least and IronPython 2.6 is currently being worked on
(tracking Python 2.6) it would be great to get this into 2.6-maint.


I've submitted an alternative patch that catches the error _getframe 
raises on IronPython. As it is possible that _getframe will work on 
IronPython in the future (although it is likely to be enabled by a 
switch as tracking Python frames has a performance cost) this is a 
slightly more future proof solution.


Michael


All the best,

Michael




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2 very interesting projects - Python / Finance

[Michael Foord]

Tim Peters wrote:

[Aahz]
  

...
This is spam, and you have now jeopardized your correct posting to the
Python Job Board.  The other website administrators will be informed and
we will discuss whether spamming python-dev warrants withdrawing it.



To be fair, a python-dev moderator approved the posting, so in their
judgment it wasn't spam.

  


I saw it in the queue and hit reject. I think he may have signed up and 
reposted - either that or I *didn't* hit reject when I intended to.


Michael

It was in my judgment, but someone else approved it before I managed
to hit the "discard" button.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IO implementation: in C and Python?

[Michael Foord]

Antoine Pitrou wrote:

Georg Brandl  gmx.net> writes:
  

I just hope everyone updates both versions when making changes to IO.



My proposal is just organizational, it is neutral in terms of whether or not the
Python version is correctly maintained.

We can hope that the IO lib *semantics* won't change too much in the future
(although there is an IMO legitimate request for a setblocking() method:
http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to
willingly use the Python version if the C version is available.


  


If they're functionally equivalent and single set of tests is run on 
both then -- assuming good tests -- breakage would be noticed...


Michael


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

[Michael Foord]

Brett Cannon wrote:



On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach 
> wrote:


On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon mailto:br...@python.org>> wrote:

Now, from what I can tell, Antoine is suggesting having _pyio
and a _io and then io is simply:

try: from _io import *
except ImportError: from _pyio import *

That works for testing as you can then have test classes have
an attribute for the module to use and then create two
subclasses which set what module to use (kind of like how
test_warnings currently does it). But this only really works
for complete module replacements, not modules like pickle
where only key portions have been rewritten (which happens
more often than the complete rewrite).


A slight change would make it work for modules where only key
functions have been rewritten.  For example, pickle.py could read:

from _pypickle import *
try: from _pickle import *
except ImportError: pass


True, although that still suffers from the problem of overwriting 
things like __name__, __file__, etc.


What do you mean overwriting __name__ and __file__? Doing import * in a 
pure Python file doesn't override these.


Michael


-Brett



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Attention Bazaar mirror users

[Michael Foord]

Steve Holden wrote:

Steven Bethard wrote:
  

On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore  wrote:


PS Just for my own information, am I correct in thinking that it is
*only* Bazaar in the (D)VCS world that has this problem, to any real
extent? I know old Mercurial clients can interact with newer servers
(ie, the wire protocol hasn't changed), I'm fairly sure that older
Subversion clients can talk to newer servers (at least, I've never
cared what client version I'm running).
  

I've definitely had an SVN server tell me that I needed to upgrade my
client to 1.5.



Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my
client to 1.5, which is a little tedious = particularly as the
information comes in the form of a hard-to-decipher error message and a
refusal to work. So much for backward compatibility.
  


Hmm... I've been using 1.5 clients with 1.4 and earlier servers for 
quite a while now. My guess is that you deciphered the error message wrong!


A working copy created by a 1.5 client can only be manipulated by a 1.5 
client (I sometimes have several clients on windows boxen) but work fine 
with earlier servers in my experience.


Michael


Fortunately I still had the SVN 1.4 client on the Ubuntu machine that
was hosting the Samba shares, so I could use ssh and the command line to
maintain things.

regards
 Steve
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

[Michael Foord]

Steven Bethard wrote:

On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon  wrote:
  

But there is another issue with this: the pure Python code will never call
the extension code because the globals will be bound to _pypickle and not
_pickle. So if you have something like::

  # _pypickle
  def A(): return _B()
  def _B(): return -13

  # _pickle
  def _B(): return 42

  # pickle
  from _pypickle import *
  try: from _pickle import *
  except ImportError: pass

If you import pickle and call pickle.A() you will get -13 which is not what
you are after.



Maybe I've missed this and someone else already suggested it, but
couldn't we provide a (probably C-coded) function
``replace_globals(module, globals)`` that would, say, replace the
globals in _pypickle with the globals in pickle? Then you could write
something like::

from _pypickle import *
try:
from _pickle import *
module = __import__('_pickle')
except ImportError:
module = __import__('_pypickle')
replace_globals(module, globals())

Steve
  


Swapping out module globals seems to be a backwards way to do things to 
me. Why not have one set of tests that test the low level APIs 
(identical tests whether they are written in C or Python) - and as they 
live in their own module are easy to test in isolation. And then a 
separate set of tests for the higher level APIs, which can even mock out 
the low level APIs they use. There shouldn't be any need for switching 
out objects in the scope of the lower level APIs - that seems like a 
design smell to me.


Michael

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python wins Linux New Media Award for Best Open Source Programming Language

[Michael Foord]

Hello all,

Not sure if this is the same as the LinuxQuestions award, but it looks 
different:

(German)
http://www.linux-magazin.de/news/cebit_2009_openstreetmap_erntet_zwei_linux_new_media_awards

I particularly like this snippet from the google translation:

The prize was Martin von Löwis of the Python Foundation on behalf of the 
Python community itself.


http://translate.google.co.uk/translate?hl=en&sl=de&u=http://www.linux-magazin.de/news/cebit_2009_openstreetmap_erntet_zwei_linux_new_media_awards&ei=VByxSfSnM9nHjAfb6ojPBQ&sa=X&oi=translate&resnum=1&ct=result&prev=/search%3Fq%3Dcebit%2B2009%2B%2BLinux%2BNew%2BMedia%2BAwards%2Bpython%26hl%3Den%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-GB:official%26hs%3DP9E

All the best,

Michael Foord
--
http://www.ironpythoninaction.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Michael Foord]

Martin v. Löwis wrote:

In light of this, what I'd love to see (but sadly can't really help
with, and am not optimistic about happening) is for:

- python to grow a decent, cross platform, package management system

- the standard library to actually shrink to a point where only
libraries that are not released elsewhere are included

I'd be interested to know how many users of python also felt this way ;-)



I don't like the standard library to shrink. It's good that batteries
are included.
  
I have mixed feelings. It is great that the batteries are included, but 
some batteries are showing their age or not maintained (who maintains 
IDLE? - does the calendar module really warrant being in the standard 
library? - imaplib is really not useful and IMAPClient which isn't in 
the standard library is much better [1]).


If a library is well maintained then there seems to be little point in 
moving it into the standard library as it may actually be harder to 
maintain, and if a library has no active developers then do we really 
want it in the standard library...


On the other hand there are some standard tools that a significant 
portion of the community use (Python Imaging Library and the PyWin32 
extensions for example) that aren't in the standard library.


I think other developers have similar mixed feelings, or at least there 
are enough people on both sides of the fence that it is very hard to 
make changes. Perhaps this is the way it should be.


Michael

[1] http://freshfoo.com/wiki/CodeIndex


Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecated __cmp__ and total ordering

[Michael Foord]

Hello Mart,

This has been discussed before. Guido was against automatically filling 
in these methods based I think on the fact that this may not be what you 
want - worth searching the archives for.


See here for a class decorator that provides all rich comparison methods 
for classes that only implement one (e.g. __lt__):


http://code.activestate.com/recipes/576529/

Michael Foord

Mart Sõmermaa wrote:
__cmp__ used to provide a convenient way to make all ordering 
operators work by defining a single method. For better or worse, it's 
gone in 3.0.


To provide total ordering without __cmp__ one has to implement all of 
__lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all but 
a few cases it suffices only to provide a "real" implementation for 
e.g. __lt__ and define all the other methods in terms of it as follows:


class TotalOrderMixin(object):
def __lt__(self, other):
raise NotImplemented # override this

def __gt__(self, other):
return other < self

def __le__(self, other):
return not (other < self)

def __ge__(self, other):
return not (self < other)

__eq__ and __ne__ are somewhat special, although it is possible to 
define them in terms of __lt__


def __eq__(self, other):
return not (self == other)

def __ne__(self, other):
return self < other or other < self

it may be inefficient.

So, to avoid the situation where all the projects that match 
http://www.google.com/codesearch?q=__cmp__+lang%3Apython have to 
implement their own TotalOrderMixin, perhaps one could be provided in 
the stdlib? Or even better, shouldn't a class grow automagic __gt__, 
__le__, __ge__ if __lt__ is provided, and, in a similar vein, __ne__ 
if __eq__ is provided?



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecated __cmp__ and total ordering

[Michael Foord]

Raymond Hettinger wrote:


[Mart Sõmermaa]

To provide total ordering without __cmp__ one has to implement all of
 __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all
but a few cases it suffices only to provide a "real" implementation for
e.g. __lt__ and define all the other methods in terms of it as follows:


FWIW, I'm working on a solution for the problem using class decorators.
The idea is that it would scan a class and fill-in missing methods based
on the ones already there.  That way, any one of the four ordering
relations can be provided as a starting point and we won't have to
annoint one like __lt__ as the one true underlying method.

When it's ready, I'll bring it to python-dev for discussion.


Is there something you don't like about this one: 
http://code.activestate.com/recipes/576529/



Michael




Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecated __cmp__ and total ordering

[Michael Foord]

Raymond Hettinger wrote:


[Mart Sõmermaa]

To provide total ordering without __cmp__ one has to implement all of
 __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all
but a few cases it suffices only to provide a "real" implementation 
for
e.g. __lt__ and define all the other methods in terms of it as 
follows:


[Raymond Hettinger]

FWIW, I'm working on a solution for the problem using class decorators.
The idea is that it would scan a class and fill-in missing methods 
based

on the ones already there.  That way, any one of the four ordering
relations can be provided as a starting point and we won't have to
annoint one like __lt__ as the one true underlying method.

When it's ready, I'll bring it to python-dev for discussion.


[Michael Foord]
Is there something you don't like about this one: 
http://code.activestate.com/recipes/576529/


Yes, that recipe has the basic idea!



It was originally written after you issued a challenge at PyCon UK last 
year.



I think the implementation can be cleaned-up quite a bit
and it can be made as fast as hand-written code (not
the setup time, but the actual introduced method).


OK

I'll take it back to Christian and Menno and see what we can come up with.

Michael





Raymond 



--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecated __cmp__ and total ordering

[Michael Foord]

Mart Sõmermaa wrote:



On Tue, Mar 10, 2009 at 3:57 PM, Michael Foord 
mailto:fuzzy...@voidspace.org.uk>> wrote:


Is there something you don't like about this one:
http://code.activestate.com/recipes/576529/


Yes -- it is not in the standard library. As I said, eventually all 
the 15,000 matches on Google Code need to update their code and copy 
that snippet to their util/, write tests for it etc.


That question was in reply to Raymond who said he was working on his own 
version.


Michael



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body

[Michael Foord]

Brett Cannon wrote:
Without knowing what StatementSkipped is (just some singleton? If so 
why not just used SkipStatement instance that was raised?) and 
wondering if we are just going to continue to adding control flow 
exceptions that directly inherit from BaseException or some 
ControlFlowException base class, the basic idea seems fine by me.




Note that using exceptions for control flow can  be bad for other 
implementations of Python. For example exceptions on the .NET framework 
are very expensive. (Although there are workarounds such as not really 
raising the exception - but they're ugly).


Isn't it better practise for exceptions to be used for exceptional 
circumstances rather than for control flow?


Michael

On Sun, Mar 15, 2009 at 05:56, Nick Coghlan > wrote:


PEP 377 is a proposal to allow context manager __enter__() methods to
skip the body of the with statement by raising a specific (new) flow
control exception.

Since there is a working reference implementation now, I thought
it was
time to open it up for broader discussion.

Full PEP attached, or you can find it in the usual place at
http://www.python.org/dev/peps/pep-0377

Cheers,
Nick.

P.S. I expect a rationale for the StatementSkipped value binding is
probably going to be pretty high on the list of questions that aren't
currently covered by the PEP. I hope to write more on that some time
this week.

--
Nick Coghlan   |   ncogh...@gmail.com 
  |   Brisbane, Australia
---

PEP: 377
Title: Allow __enter__() methods to skip the statement body
Version: $Revision: 70384 $
Last-Modified: $Date: 2009-03-15 22:48:49 +1000 (Sun, 15 Mar 2009) $
Author: Nick Coghlan mailto:ncogh...@gmail.com>>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 8-Mar-2009
Python-Version: 2.7, 3.1
Post-History: 8-Mar-2009


Abstract


This PEP proposes a backwards compatible mechanism that allows
``__enter__()``
methods to skip the body of the associated ``with`` statement. The
lack of
this ability currently means the ``contextlib.contextmanager``
decorator
is unable to fulfil its specification of being able to turn arbitrary
code into a context manager by moving it into a generator function
with a yield in the appropriate location. One symptom of this is that
``contextlib.nested`` will currently raise ``RuntimeError`` in
situations where writing out the corresponding nested ``with``
statements would not [1].

The proposed change is to introduce a new flow control exception
``SkipStatement``, and skip the execution of the ``with``
statement body if ``__enter__()`` raises this exception.


Proposed Change
===

The semantics of the ``with`` statement will be changed to include a
new ``try``/``except``/``else`` block around the call to
``__enter__()``.
If ``SkipStatement`` is raised by the ``__enter__()`` method, then
the main section of the ``with`` statement (now located in the
``else``
clause) will not be executed. To avoid leaving the names in any ``as``
clause unbound in this case, a new ``StatementSkipped`` singleton
(similar to the existing ``NotImplemented`` singleton) will be
assigned to all names that appear in the ``as`` clause.

The components of the ``with`` statement remain as described in
PEP 343 [2]::

   with EXPR as VAR:
   BLOCK

After the modification, the ``with`` statement semantics would
be as follows::

   mgr = (EXPR)
   exit = mgr.__exit__  # Not calling it yet
   try:
   value = mgr.__enter__()
   except SkipStatement:
   VAR = StatementSkipped
   # Only if "as VAR" is present and
   # VAR is a single name
   # If VAR is a tuple of names, then StatementSkipped
   # will be assigned to each name in the tuple
   else:
   exc = True
   try:
   try:
   VAR = value  # Only if "as VAR" is present
   BLOCK
   except:
   # The exceptional case is handled here
   exc = False
   if not exit(*sys.exc_info()):
   raise
   # The exception is swallowed if exit() returns true
   finally:
   # The normal and non-local-goto cases are handled here
   if exc:
   exit(None, None, None)

With the above change in place for the ``with`` statement semantics,
``contextlib.contextmanager()`` will then be modified to raise
``SkipStatement`` instead of ``RuntimeError`` when the underlying
generator doesn't yield.


Rationale for Change


Currentl

Re: [Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body

[Michael Foord]

Martin v. Löwis wrote:

Note that using exceptions for control flow can  be bad for other
implementations of Python. For example exceptions on the .NET framework
are very expensive.



Why do you say that? What specific implementation of .NET are you
referring to? What do you mean by "very"?
  
I'm talking about IronPython on the Microsoft .NET framework - although 
it is likely that the same is true of IronPython on Mono.


On the .NET framework the setup for exception handling is virtually free 
until an exception is raised. Once an exception is raised it takes a lot 
longer (expensive in time). This means that in IronPython exception 
handling code (try... except and try... finally blocks) are much faster 
than CPython if no exception is raised - but much more expensive if an 
exception is raised.


You can see this in a comparison of IronPython 2 and Python 2.5 running 
PyBench:


http://ironpython.codeplex.com/Wiki/View.aspx?title=IP201VsCPy25Perf

TryExcept:  26ms  888ms  -97.1%  63ms  890ms  -92.9%
TryRaiseExcept: 58234ms 1286ms +4428.6% 58474ms 
1298ms +4404.6%



Isn't it better practise for exceptions to be used for exceptional
circumstances rather than for control flow?



This is an ongoing debate (in Python, and outside). I'm in the camp
that says that exceptions are a control flow mechanism just like
loops, conditionals, and recursion. With exceptions, you get essentially
multiple alternative outcomes of a function call, rather than just a
single result. In principle, it would be possible to eliminate the
return statement altogether, but it is useful syntactic sugar.
  


Using exceptions for control flow is akin to goto. Sometimes useful but 
a dubious practise. :-)


Michael



Regards,
Martin
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body

[Michael Foord]

Nick Coghlan wrote:

Note that using exceptions for control flow can  be bad for other
implementations of Python. For example exceptions on the .NET framework
are very expensive. (Although there are workarounds such as not really
raising the exception - but they're ugly).



Is it that exceptions are expensive, or setting up a try/except block is
expensive? The reason the SkipStatement idea is tenable at all (even in
CPython) is that try/except is fairly cheap when no exception is raised.
  


It is the raising of the exception that is expensive.

Michael

(In this specific case, my initial patch does slow things down a bit,
since one side effect of the extra try/except block is to disallow a
couple of stack based optimisations that are used in the current CPython
implementation of the with statement)

  

Isn't it better practise for exceptions to be used for exceptional
circumstances rather than for control flow?



This *is* an exceptional circumstance: a typical __enter__ method will
just return or raise some other exception. I suppose you could use some
kind of dedicated thread-local state instead of an exception to indicate
that the underlying generator didn't yield, but a new control flow
exception seemed like the most straightforward option.

I'm somewhat intrigued by Glyph's idea though - if I can figure out a
way to make it practical, it does offer very some interesting
possibilities (and would, in effect, bring reusable embedded code blocks
to Python...).

Cheers,
Nick.

  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body

[Michael Foord]

Martin v. Löwis wrote:

I'm talking about IronPython on the Microsoft .NET framework - although
it is likely that the same is true of IronPython on Mono.



I see. It would be interesting to find out why this is so much slower -
I cannot believe that it is inherent in the commercial .NET framework,
but rather expect that it is some issue in IronPython (*). Also, the
test case measured doesn't entirely reflect what is proposed, since it
catches the exception in the same function - for a realistic comparison,
the raise should occur in a function call (so to measure the overhead
of stack unwinding also).

Regards,
Martin

(*) My wild guess is that IronPython feels obliged to provide traceback
objects, and that this a costly operation - I just can't believe that
exceptions are themselves costly on .NET, in the Microsoft
implementation. In the specific case, it would be possible to suppress
traceback generation.
  
I have discussed this issue with the IronPython team several times. They 
say that it is a deliberate design decision in .NET - to minimize the 
cost of exception handling code in the absence of exceptions at the 
expense of slower performance when exceptions are raised.


Googling for ".NET exceptions performance" would seem to confirm that.

Apparently this article on the managed exception model was written by 
one of the core developers:


http://blogs.msdn.com/cbrumme/archive/2003/10/01/51524.aspx

"This is light years away from returning a -1 from your function call. 
Exceptions are inherently non-local, and if there’s an obvious and 
enduring trend for today’s architectures, it’s that you must remain 
local for good performance.


Relative to straight-line local execution, exception performance will 
keep getting worse. Sure, we might dig into our current behavior and 
speed it up a little. But the trend will relentlessly make exceptions 
perform worse.


How do I reconcile the trend to worse performance with our 
recommendation that managed code should use exceptions to communicate 
errors? By ensuring that error cases are exceedingly rare. We used to 
say that exceptions should be used for exceptional cases, but folks 
pushed back on that as tautological."


Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body

[Michael Foord]

Aahz wrote:

On Sun, Mar 15, 2009, Michael Foord wrote:
  
Note that using exceptions for control flow can  be bad for other  
implementations of Python. For example exceptions on the .NET framework  
are very expensive. (Although there are workarounds such as not really  
raising the exception - but they're ugly).


Isn't it better practise for exceptions to be used for exceptional  
circumstances rather than for control flow?



It seems to me that we as a development community already made a decision
when we switched to StopIteration as the primary mechanism for halting
``for`` loops.  (Not that it was really a new decision because parts of
the Python community have always advocated using exceptions for control
flow, but the ``for`` loop enshrines it.)  I doubt that using exceptions
for control flow in ``with`` blocks will cause anywhere near so much a
performance degradation.
  
Well, StopIteration is still an implementation detail that only 
occasionally bleeds through to actual programming. It says nothing about 
whether using exceptions for non-exceptional circumstances (control 
flow) is good practise. Personally I think it makes the intent of code 
less easy to understand - in effect the exceptions *are* being used as a 
goto.


Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body

[Michael Foord]

Aahz wrote:

On Sun, Mar 15, 2009, Michael Foord wrote:
  

Aahz wrote:


On Sun, Mar 15, 2009, Michael Foord wrote:
  
  
Note that using exceptions for control flow can  be bad for other   
implementations of Python. For example exceptions on the .NET 
framework  are very expensive. (Although there are workarounds such 
as not really  raising the exception - but they're ugly).


Isn't it better practise for exceptions to be used for exceptional   
circumstances rather than for control flow?


It seems to me that we as a development community already made a decision
when we switched to StopIteration as the primary mechanism for halting
``for`` loops.  (Not that it was really a new decision because parts of
the Python community have always advocated using exceptions for control
flow, but the ``for`` loop enshrines it.)  I doubt that using exceptions
for control flow in ``with`` blocks will cause anywhere near so much a
performance degradation.
  
  
Well, StopIteration is still an implementation detail that only

occasionally bleeds through to actual programming. It says nothing
about whether using exceptions for non-exceptional circumstances
(control flow) is good practise. Personally I think it makes the
intent of code less easy to understand - in effect the exceptions
*are* being used as a goto.



Let me know how you'd rewrite this more clearly without a control-flow
exception:

try:
for field in curr_fields:
for item in record[field]:
item = item.lower()
for filter in excludes:
if match(item, filter):
raise Excluded
except Excluded: 
continue 


This is pretty much the canonical example showing why control-flow
exceptions are a Good Thing.  They're a *structured* goto.
  


You didn't include all the code - so impossible to match the exact 
semantics. Breaking out of multiple loops with a return is a cleaner way 
to handle it IMHO.



def find_excludes():
   for field in curr_fields:
   for item in record[field]:
   item = item.lower()
   for filter in excludes:
   if match(item, filter):
   return
  
while something:

   find_excludes()

Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What level of detail wanted for import and the language reference?

[Michael Foord]

Brett Cannon wrote:
At this point importlib is done for its public API for Python 3.1. 
That means it's time to turn my attention to making sure the semantics 
of import are well documented. But where to put all of the details? 
The language reference for import 
(http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement) 
explains the basics, but is lacking all of the details of PEP 302 and 
other stuff like __path__ that have existed for ages.


My question is if I should flesh out the details in the language 
reference or do it in importlib's intro docs. The main reason I could 
see not doing it in the langauge reference (or at least duplicating 
it) is it would be somewhat easier to reference specific objects in 
importlib but I am not sure if the language reference should try to 
stay away from stdlib references.


Having the Python import semantics well documented will be an 
*excellent* side effect of importlib. Thank you for your astonishing 
efforts on this project.


Personally I would rather see the import semantics themselves in the 
language reference.


Michael


-Brett


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] an unimportant question, ...

[Michael Foord]

Christian Tismer wrote:

... but I'm curious.

Hi Guido,

while working on Psyco, I stumbled over a log entry in modsupport.h:



   19-Aug-2002  GvR1012Changes to string object struct for
   interning changes, saving 3 bytes.



The change to stringobject was this  (rev. 28308):

Before:

typedef struct {
PyObject_VAR_HEAD
long ob_shash;
PyObject *ob_sinterned;
char ob_sval[1];
} PyStringObject;


After:

typedef struct {
PyObject_VAR_HEAD
long ob_shash;
int ob_sstate;
char ob_sval[1];
} PyStringObject;


Now, the internals are very clear to me. What I don't understand
is where the three saved bytes should be.

Thinking of the time where this change was made, I cannot imagine
that this comment was about the size diff between pointer and int,
and if this was meant, I still don't get how this could save three
bytes?

With unaligned ob_sval, structure packing and ob_sstate being
unsigned char one could save 3 bytes, but we don't do that.

Well, as said, this is no important question. I am just asking
myself what I don't see here, or if the comment is just sub-optimal :-)



At Resolver we've found it useful to short-circuit any doubt and just 
refer to comments in code as 'lies'. :-)


Michael


all the best -- chris


p.s.: won't make it to PyCon this time, see you soon at the piggies



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.6 64-bit Mac Release

[Michael Foord]

Ronald Oussoren wrote:


On 1 Apr, 2009, at 8:45, Peck, Jon wrote:

Apparently the Mac Python 2.6.1 Installer image does not include 
64-bit binaries.  Is this going to change?  Is there some technical 
reason why these are not included?  We are hoping to support that in 
our next 64-bit release.


The 2.6 installer image does not include 64-bit binaries.  As of this 
week the script that creates the installer can build an installer that 
does support 64-bit code as well, but that only works on Leopard systems.


I'm thinking about how to distribute binaries that support 64-bit code 
without unduly complicating the world. The easiest option for me would 
be to have two installers: one 32-bit only that supports OSX 10.3.9 
and later and a 4-way universal one that supports OSX Leopard and 
later.  It might be possible to have a single installer that supports 
64-bit code on Leopard but is usable on 10.3.9 as well, but I haven't 
checked yet how much that would complicate the build.




Two installers sounds OK to me, particularly if it simplifies the build 
process but allows us to still support 64bit.


Michael


Ronald



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Wing IDE and python.wpr

[Michael Foord]

Hello all,

How many are using the Wing IDE to work on core Python?

It would be nice to have a 'python.wpr' checked in to trunk, as I have 
to recreate the project file every time I do a new checkout. Would this 
be useful for anyone else? Where is a good place for it to live? 
Littering the top level directory seems like a bad idea but I can't see 
anywhere else immediately *obvious* (no reason it has to live at the top 
level).


Wing can be configured to use two files for the project - one file for 
the basic configuration (which would be checked in) and one for your 
personal settings (which files you have open, how many windows you are 
using etc) and would be svn-ignored.


Michael Foord

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Wing IDE and python.wpr

[Michael Foord]

Michael Foord wrote:

Hello all,

How many are using the Wing IDE to work on core Python?

It would be nice to have a 'python.wpr' checked in to trunk, as I have 
to recreate the project file every time I do a new checkout. Would 
this be useful for anyone else? Where is a good place for it to live? 
Littering the top level directory seems like a bad idea but I can't 
see anywhere else immediately *obvious* (no reason it has to live at 
the top level).


Wing can be configured to use two files for the project - one file for 
the basic configuration (which would be checked in) and one for your 
personal settings (which files you have open, how many windows you are 
using etc) and would be svn-ignored.

The Wing project file is now checked in. It is Misc/python-wing.wpr

The project is configured with SVN integration enabled, with two file 
configuration and the wpu file SVN ignored plus the main project 
directory added.


The wpr file is text so changes are diff friendly.

There is an issue with the way the project is displayed - the Misc 
directory is the top-level with '..' showing as another directory in the 
project. This issue will be resolved in the next version of Wing.


There are various other feature-requests now with Wing to better support 
using it for developing Python. Currently the debugger doesn't work with 
a newly built version of Python and the executable name / location is 
platform dependent and so setting a custom executable would only work on 
one platform.


It would be easy to add custom tools to (for example) integrate regrtest 
or do the configure / make dance on a fresh checkout.


All the best,

Michael



Michael Foord




--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] unittest package

[Michael Foord]

Hello all,

The unittest module is around 1500 lines of code now, and the tests are 
3000 lines.


It would be much easier to maintain as a package rather than a module. 
Shall I work on a suggested structure or are there objections in principle?


Obviously all the functionality would still be available from the 
top-level unittest namespace (for backwards compatibility).


Michael

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Michael Foord]

Chris Withers wrote:

Olemis Lang wrote:
On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers 
 wrote:

Tarek Ziadé wrote:


- PyPI mirroring (PEP 381)
I don't see why PyPI isn't just ported to GAE with an S3 data 
storage bit
and be done with it... Offline mirrors for people behind firewalls 
already

have solutions out there...


-1 ... IMHO ...


For what reason?


GAE does suffer from blackouts - which is the problem we are attempting 
to solve with mirroring.


I don't see why we should tie vital Python infrastructure to the 
proprietary APIs of a single vendor and outsource delivery entirely to 
them. If we have the manpower to do this ourselves it seems better to do 
it and retain control.


Added to which GAE is a commercial service and beyond a certain level 
bandwidth / cycles needs paying for. This may not be an issue in itself 
(either Google may waive charges or the PSF may be willing to pay).


Michael



Chris




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Michael Foord]

Collin Winter wrote:

On Fri, Apr 3, 2009 at 2:27 AM, Antoine Pitrou  wrote:
  

Thomas Wouters  python.org> writes:


Pystone is pretty much a useless benchmark. If it measures anything, it's the
  

speed of the bytecode dispatcher (and it doesn't measure it particularly well.)
PyBench isn't any better, in my experience.

I don't think pybench is useless. It gives a lot of performance data about
crucial internal operations of the interpreter. It is of course very little
real-world, but conversely makes you know immediately where a performance
regression has happened. (by contrast, if you witness a regression in a
high-level benchmark, you still have a lot of investigation to do to find out
where exactly something bad happened)

Perhaps someone should start maintaining a suite of benchmarks, high-level and
low-level; we currently have them all scattered around (pybench, pystone,
stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not
to mention other third-party stuff that can be found in e.g. the Computer
Language Shootout).



Already in the works :)

As part of the common standard library and test suite that we agreed
on at the PyCon language summit last week, we're going to include a
common benchmark suite that all Python implementations can share. This
is still some months off, though, so there'll be plenty of time to
bikeshed^Wrationally discuss which benchmarks should go in there.
  
Where is the right place for us to discuss this common benchmark and 
test suite?


As the benchmark is developed I would like to ensure it can run on 
IronPython.


The test suite changes will need some discussion as well - Jython and 
IronPython (and probably PyPy) have almost identical changes to tests 
that currently rely on deterministic finalisation (reference counting) 
so it makes sense to test changes on both platforms and commit a single 
solution.


Michael


Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Michael Foord]

Collin Winter wrote:

On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord
 wrote:
  

Collin Winter wrote:


As part of the common standard library and test suite that we agreed
on at the PyCon language summit last week, we're going to include a
common benchmark suite that all Python implementations can share. This
is still some months off, though, so there'll be plenty of time to
bikeshed^Wrationally discuss which benchmarks should go in there.

  

Where is the right place for us to discuss this common benchmark and test
suite?

As the benchmark is developed I would like to ensure it can run on
IronPython.

The test suite changes will need some discussion as well - Jython and
IronPython (and probably PyPy) have almost identical changes to tests that
currently rely on deterministic finalisation (reference counting) so it
makes sense to test changes on both platforms and commit a single solution.



I believe Brett Cannon is the best person to talk to about this kind
of thing. I don't know that any common mailing list has been set up,
though there may be and Brett just hasn't told anyone yet :)

Collin
  

Which begs the question of whether we *should* have a separate mailing list.

I don't think we discussed this specific point in the language summit - 
although it makes sense. Should we have a list specifically for the test 
/ benchmarking or would a more general implementations-sig be appropriate?


And is it really Brett who sets up mailing lists? My understanding is 
that he is pulling out of stuff for a while anyway, so that he can do 
Java / Phd type things... ;-)


Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Going "offline" for three months

[Michael Foord]

Brett Cannon wrote:
In order to hunker down and get my thesis proposal done by its due 
date, I am disabling mail delivery for myself for all mail.python.org 
 mailing lists for three months (sans 
python-committers so I don't accidentally commit when I shouldn't). If 
something comes up I should know about you can always email or IM me 
directly.


See you all on July 1. Here is to hoping I don't suffer any withdrawal.


We'll miss you. Hope you don't end up preferring Java. ;-)

Michael


-Brett


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] core python tests

[Michael Foord]

Antoine Pitrou wrote:

Nick Coghlan  gmail.com> writes:
  

C. Titus Brown wrote:


I vote for a separate mailing list -- 'python-tests'? -- but I don't
know exactly how splintered to make the conversation.  It probably
belongs at python.org but if you want me to host it, I can.
  

If too many things get moved off to SIGs there won't be anything left
for python-dev to talk about ;)



There is already an stdlib-sig, which has been almost unused.

  
stdlib-sig isn't *quite* right (the testing and benchmarking are as much 
about core python as the stdlib) - although we could view the benchmarks 
and tests themselves as part of the standard library...


Either way we should get it underway. Collin and Jeffrey - happy to use 
stdlib-sig?


Michael


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] core python tests

[Michael Foord]

Jeffrey Yasskin wrote:

On Sat, Apr 4, 2009 at 11:52 AM, Collin Winter  wrote:
  

On Sat, Apr 4, 2009 at 7:33 AM, Michael Foord  wrote:


Antoine Pitrou wrote:
  

Nick Coghlan  gmail.com> writes:



C. Titus Brown wrote:

  

I vote for a separate mailing list -- 'python-tests'? -- but I don't
know exactly how splintered to make the conversation.  It probably
belongs at python.org but if you want me to host it, I can.



If too many things get moved off to SIGs there won't be anything left
for python-dev to talk about ;)

  

There is already an stdlib-sig, which has been almost unused.




stdlib-sig isn't *quite* right (the testing and benchmarking are as much
about core python as the stdlib) - although we could view the benchmarks and
tests themselves as part of the standard library...

Either way we should get it underway. Collin and Jeffrey - happy to use
stdlib-sig?
  

Works for me.



Me too.

bcc python-dev, -> stdlib-sig

First question: Do people want the unladen-swallow performance tests
in the CPython repository until the whole library gets moved out? If
so, where? Tools/performance? Lib/test/benchmarks?
  


I'm +1 on including them (so long as they run under trunk of course) but 
agnostic on location.


Maybe better not in test as it might be expected that a full regrtest 
would then run them?


I'm keeping Python-dev cc'd as it is a Python-dev decision and bcc 
messages require individual admin approval.


Michael


Jeffrey
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial?

[Michael Foord]

Nick Coghlan wrote:

Dirkjan Ochtman wrote:
  

Another thing that I discussed with Georg last night would be a setup
where changesets get pushed to a gateway repo that runs the tests and
only pushes to an "official" repo if everything's still green. That
should probably be a topic discussed separately, though.



That was one of the post-switch workflow enhancements that Barry was
advocating - it's still a good idea, even if Barry's preferred flavour
of DVCS wasn't chosen :)

  


Gated checkins can work fine but can also have many problems. For 
example if we have a spuriously failing test then if you are working on 
an unrelated issue it will be entirely up to chance as to whether you 
can checkin...


Building the docs would be another thing we could check, although it can 
take a while.


If we have a queue then it could be the case that you do a commit - and 
then discover half an hour later that it conflicts with something that 
was ahead of you in the queue.


Michael


Cheers,
Nick.

  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Shorter float repr in Python 3.1?

[Michael Foord]

Mark Dickinson wrote:

[snip...]
  
Discussion points

=

(1) Any objections to including this into py3k?  If there's
controversy, then I guess we'll need a PEP.
  


Big +1

(2) Should other Python implementations (Jython,
IronPython, etc.) be expected to use short float repr, or should
it just be considered an implementation detail of CPython?
I propose the latter, except that all implementations should
be required to satisfy eval(repr(x)) == x for finite floats x.
  
Short float repr should be an implementation detail, so long as 
eval(repr(x)) == x still holds.


Michael Foord

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] decorator module in stdlib?

[Michael Foord]

Guido van Rossum wrote:

On Wed, Apr 8, 2009 at 9:31 PM, Michele Simionato
 wrote:
  

Then perhaps you misunderstand the goal of the decorator module.
The raison d'etre of the module is to PRESERVE the signature:
update_wrapper unfortunately *changes* it.

When confronted with a library which I do not not know, I often run
over it pydoc, or sphinx, or a custom made documentation tool, to extract the
signature of functions.



Ah, I see. Personally I rarely trust automatically extracted
documentation -- too often in my experience it is out of date or
simply absent. Extracting the signatures in theory wouldn't lie, but
in practice I still wouldn't trust it -- not only because of what
decorators might or might not do, but because it might still be
misleading. Call me old-fashioned, but I prefer to read the source
code.
  


If you auto-generate API documentation by introspection (which we do at 
Resolver Systems) then preserving signatures can also be important. 
Interactive use (support for help), and more straightforward tracebacks 
in the event of usage errors are other reasons to want to preserve 
signatures and function name.



 For instance, if I see a method
  

get_user(self, username) I have a good hint about what it is supposed
to do. But if the library (say a web framework) uses non signature-preserving
decorators, my documentation tool says to me that there is function
get_user(*args, **kwargs) which frankly is not enough [this is the
optimistic case, when the author of the decorator has taken care
to preserve the name of the original function].



But seeing the decorator is often essential for understanding what
goes on! Even if the decorator preserves the signature (in truth or
according inspect), many decorators *do* something, and it's important
to know how a function is decorated. For example, I work a lot with a
small internal framework at Google whose decorators can raise
exceptions and set instance variables; they also help me understand
under which conditions a method can be called.
  


Having methods renamed to 'wrapped' and their signature changed to 
*args, **kwargs may tell you there *is* a decorator but doesn't give you 
any useful information about what it does. If you look at the code then 
the decorator is obvious (whether or not it mangles the method)...

[+1]

But I feel strongly about
the possibility of being able to preserve (not change!) the function
signature.



That could be added to functools if enough people want it.

  


+1

Michael


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Email-SIG] Dropping bytes "support" in json

[Michael Foord]

Glenn Linderman wrote:
On approximately 4/10/2009 9:56 AM, came the following characters from 
the keyboard of Barry Warsaw:

On Apr 10, 2009, at 1:19 AM, gl...@divmod.com wrote:

On 02:38 am, ba...@python.org wrote:
So, what I'm really asking is this.  Let's say you agree that there 
are use cases for accessing a header value as either the raw 
encoded bytes or the decoded unicode.  What should this return:


>>> message['Subject']

The raw bytes or the decoded unicode?


My personal preference would be to just get deprecate this API, and 
get rid of it, replacing it with a slightly more explicit one.


  message.headers['Subject']
  message.bytes_headers['Subject']


This is pretty darn clever Glyph.  Stop that! :)

I'm not 100% sure I like the name .bytes_headers or that .headers 
should be the decoded header (rather than have .headers return the 
bytes thingie and say .decoded_headers return the decoded thingies), 
but I do like the general approach.


If one name has to be longer than the other, it should be the bytes 
version.  Real user code is more likely to want to use the text 
version, and hopefully there will be more of that type of code than 
implementations using bytes.


Of course, one could use message.header and message.bythdr and they'd 
be the same length.




Shouldn't headers always be text?

Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)

[Michael Foord]

Antoine Pitrou wrote:

Mart Sõmermaa  gmail.com> writes:
  

On Mon, Apr 13, 2009 at 12:56 AM, Antoine Pitrou  pitrou.net>


wrote:
  

Mart Sõmermaa  gmail.com> writes:


Proposal: add add_query_params() for appending query parameters to an URL
  

to
  

urllib.parse and urlparse.
Is there anything to /remove/ a query parameter?

I'd say this is outside the scope of add_query_params().



Given the name of the proposed function, sure. But it sounds a bit weird to
have a function dedicated to adding parameters and nothing to remove them.

  


Weird or not, is there actually a *need* to remove query parameters?

Michael


You could e.g. rename the function to update_query_params() and decide that
every parameter whose specified value is None must atcually be removed from
the URL.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)

[Michael Foord]

Bill Janssen wrote:

Mart Sõmermaa  wrote:

  

On Sun, Apr 19, 2009 at 2:06 AM, Nick Coghlan  wrote:


That said, I'm starting to wonder if an even better option may be to
just drop the kwargs support from the function and require people to
always supply a parameters dictionary. That would simplify the signature
to the quite straightforward:

 def add_query_params(url, params, allow_dups=True, sep='&')
  


Or even better, stop trying to use a mapping, and just make the "params"
value a list of (name, value) pairs.  That way you can stop fiddling
around with "allow_dups" and just get rid of it.
  


Reluctant +1, it seems the best solution. You can always use {}.items() 
if you still want to store the params in a mapping.


Michael

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #!/usr/bin/env python --> python3 where applicable

[Michael Foord]

Jared Grubb wrote:


On 19 Apr 2009, at 02:17, Stephen J. Turnbull wrote:

Nick Coghlan writes:

3. Change the shebang lines in Python standard library scripts to be
version specific and update release.py to fix them all when bumping the
version number in the source tree.


+1

I think that it's probably best to leave "python", "python2", and
"python3" for the use of downstream distributors.  ISTR that was what
Guido concluded, in the discuss that led to Python 3 defaulting to
altinstall---it wasn't just convenient because Python 3 is a major
change, but that experience has shown that deciding which Python is
going to be "The python" on somebody's system just isn't a decision
that Python should make.


Ok, so if I understand, the situation is:
* python points to 2.x version
* python3 points to 3.x version
* need to be able to run certain 3k scripts from cmdline (since we're 
talking about shebangs) using Python3k even though "python" points to 2.x


So, if I got the situation right, then do these same scripts 
understand that PYTHONPATH and PYTHONHOME and all the others are also 
probably pointing to 2.x code?

IIRC the proposal was to also create PYTHON3PATH and PYTHON3HOME.

Michael



Jared
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Two proposed changes to float formatting

[Michael Foord]

Steven D'Aprano wrote:

On Sun, 26 Apr 2009 08:06:56 pm Mark Dickinson wrote:
  

I'd like to propose two minor changes to float and complex
formatting, for 3.1.  I don't think either change should prove
particularly disruptive.

(1) Currently, '%f' formatting automatically changes to '%g'
formatting for numbers larger than 1e50.


...
  

I propose removing this feature for 3.1



No objections from me. +1

  

I propose changing the complex str and repr to behave like the
float version.  That is, repr(4. + 10.j) should be "(4.0 + 10.0j)"
rather than "(4+10j)".



No objections here either. +0



  
Doing it sooner rather than later means that it is less likely to 
disrupt anyone relying on the representation (i.e. doctests).


Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 383: Non-decodable Bytes in System C haracter Interfaces

[Michael Foord]

Stephen J. Turnbull wrote:

Antoine Pitrou writes:

 > > or (better for 2.x, where bytes are strings as far as most
 > > programmers are concerned) as a new data type,
 > 
 > I'm -1 on any new string-like type (for file paths or whatever

 > else) with custom encoding/decoding semantics. It's the best way to
 > ruin the clean str/bytes separation that 3.x introduced.

Excuse me, but I can't see a scheme that encodes bytes as Unicodes but
only sometimes as a "clean separation".  It's a dirty hack that makes
life a lot easier for Windows programmers and a little easier for many
Unix programmers.  Practicality beats purity, true, but at the cost of
the purity.

  


The problem you don't address, which is still the reality for most 
programmers (especially Mac OS X where filesystem encoding is UTF 8), is 
that programmers *are* going to treat filenames as strings.


The proposed PEP allows that to work for them - whatever platform their 
program runs on.


Michael


 > Besides, the goal is also to makes things easier for the
 > programmer. Otherwise, we'll have the same situation as in 2.x
 > where many English-centric programmers produced code that was
 > incapable of dealing with non-ASCII input, because they didn't care
 > about the distinction between str and unicode.

So what you'll get here, AFAICS, is a new situation where many
Windows-centric programmers will produce code that's incapable of
dealing with non-Unicode input because they don't have to care about
the distinction between Unicode and bytes.

That's an improvement, but we can do still better and not at huge
expense to programmers.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can not run under python 2.6

[Michael Foord]

Jianchun Zhou wrote:

Hi, there:

I am new to python, and now I got a trouble:

I have an application named canola, it is written under python 2.5, 
and can run normally under python 2.5


But when it comes under python 2.6, problem up, it says:

Traceback (most recent call last):
  File 
"/usr/lib/python2.6/site-packages/terra/core/plugin_manager.py", line 
151, in _load_plugins

classes = plg.load()
  File 
"/usr/lib/python2.6/site-packages/terra/core/plugin_manager.py", line 
94, in load

mod = self._ldr.load()
  File "/usr/lib/python2.6/site-packages/terra/core/module_loader.py", 
line 42, in load

mod = __import__(modpath, fromlist=[mod_name])
ImportError: Import by filename is not supported.

Any body any idea what should I do?


The Python-Dev mailing list is for the development of Python and not 
with Python. You will get a much better response asking on the 
comp.lang.python (python-list) or python-tutor newsgroups / mailing 
lists. comp.lang.python has both google groups and gmane gateways and so 
is easy to post to.


For the particular problem you mention it is an intentional change and 
so the code in canola will need to be modified in order to run under 
Python 2.6.


All the best,

Michael Foord



--
Best Regards


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces

[Michael Foord]

Paul Moore wrote:

2009/4/28 Antoine Pitrou :
  

Paul Moore  gmail.com> writes:


I've yet to hear anyone claim that they would have an actual problem
with a specific piece of code they have written.
  

Yep, that's the problem. Lots of theoretical problems noone has ever encountered
brought up against a PEP which resolves some actual problems people encounter on
a regular basis.

For the record, I'm +1 on the PEP being accepted and implemented as soon as
possible (preferably before 3.1).



In case it's not clear, I am also +1 on the PEP as it stands.
  


Me 2

Michael

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed: add support for UNC paths to all functions in ntpath

[Michael Foord]

Larry Hastings wrote:


I've written a patch for Python 3.1 that changes os.path so it handles 
UNC paths on Windows:


  http://bugs.python.org/issue5799


+1 for the feature. I have to deal with Windows networks from time to 
time and this would be useful.


Michael



In a Windows path string, a UNC path functions *exactly* like a drive
letter.  This patch means that the Python path split/join functions
treats them as if they were.

For instance:
   >>> splitdrive("A:\\FOO\\BAR.TXT")
   ("A:", "\\FOO\\BAR.TXT")

With this patch applied:
   >>> splitdrive("HOSTNAME\\SHARE\\FOO\\BAR.TXT")
   ("HOSTNAME\\SHARE", "\\FOO\\BAR.TXT")

This methodology only breaks down in one place: there is no "default
directory" for a UNC share point.  E.g. you can say
   >>> os.chdir("c:")
or
   >>> os.chdir("c:foo\\bar")
but you can't say
   >>> os.chdir("hostname\\share")
But this is irrelevant to the patch.

Here's what my patch changes:
* Modify join, split, splitdrive, and ismount to add explicit support
 for UNC paths.  (The other functions pick up support from these four.)
* Simplify isabs and normpath, now that they don't need to be delicate
 about UNC paths.
* Modify existing unit tests and add new ones.
* Document the changes to the API.
* Deprecate splitunc, with a warning and a documentation remark.

This patch adds one subtle change I hadn't expected.  If you call
split() with a drive letter followed by a trailing slash, it returns the
trailing slash as part of the "head" returned.  E.g.
   >>> os.path.split("\\")
   ("\\", "")
   >>> os.path.split("A:\\")
   ("A:\\", "")
This is mentioned in the documentation, as follows:
   Trailing slashes are stripped from head unless it is the root
   (one or more slashes only).

For some reason, when os.path.split was called with a UNC path with only
a trailing slash, it stripped the trailing slash:
   >>> os.path.split("hostname\\share\\")
   ("hostname\\share", "")
My patch changes this behavior; you would now see:
   >>> os.path.split("hostname\\share\\")
   ("hostname\\share\\", "")
I think it's an improvement--this is more consistent.  Note that this
does *not* break the documented requirement that
os.path.join(os.path.split(path)) == path; that continues to work fine.


In the interests of full disclosure: I submitted a patch providing this
exact behavior just over ten years ago.  GvR accepted it into Python
1.5.2b2 (marked "*EXPERIMENTAL*") and removed it from 1.5.2c1.

You can read GvR's commentary upon removing it; see comments in
Misc/HISTORY  
dated "Tue Apr  6 19:38:18 1999".  If memory serves

correctly, the problems cited were only on Cygwin.  At the time Cygwin
used "ntpath", and it supported "//a/foo" as an alias for "A:\\FOO". 
You can see how this would cause Cygwin problems.


In the intervening decade, two highly relevant things have happened:
* Python no longer uses ntpath for os.path on Cygwin.  Instead it uses
 posixpath.
* Cygwin removed the "//a/foo" drive letter hack.  In fact, I believe it
 now support UNC paths.
Therefore this patch will have no effect on Cygwin users.


What do you think?


/larry/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 383 and GUI libraries

[Michael Foord]

Zooko O'Whielacronx wrote:

[snip...]
Would it be possible for Python unicode objects to have a flag
indicating whether the 'python-escape' error handler was present?  That
would serve the same purpose as my "failed_decode" flag above, and would
basically allow me to use the Python APIs directory and make all this
work-around code disappear.

Failing that, I can't see any way to use the os.listdir() in its
unicode-oriented mode to satisfy Tahoe's requirements.

If you take the above code and then add the fact that you want to use
the failed_decode flag when *encoding* the d argument to os.listdir(),
then you get this code: [2].

Oh, I just realized that I *could* use the PEP 383 os.listdir(), like
this:

def listdir(d):
fse = sys.getfilesystemencoding()
if fse == 'utf-8b':
fse = 'utf-8'
ns = []
for fn in os.listdir(d):
bytes = fn.encode(fse, 'python-escape')
try:
ns.append(FName(bytes.decode(fse, 'strict')))
except UnicodeDecodeError:
ns.append(FName(fn.decode('utf-8', 'python-escape'),
  failed_decode=True))
return ns

(And I guess I could define listdir() like this only on the
non-unicode-safe platforms, as above.)

However, that strikes me as even more horrible than the previous
"listdir()" work-around, in part because it means decoding, re-encoding,
and re-decoding every name, so I think I would stick with the previous
version.
  


The current unicode mode would skip the filenames you are interested 
(those that fail to decode correctly) - so you would have been forced to 
use the bytes mode. If you need access to the original bytes then you 
should continue to do this. PEP-383 is entirely neutral for your use 
case as far as I can see.


Michael


Oh, one more note: for Tahoe's purposes you can, in all of the code
above, replace ".decode('utf-8', 'python-replace')" with
".decode('windows-1252')" and it works just as well.  While UTF-8b seems
like a really cool hack, and it would produce more legible results if
utf-8-encoded strings were partially corrupted, I guess I should just
use 'windows-1252' which is already implemented in Python 2 (as well as
in all other software in the world).

I guess this means that PEP 383, which I have approved of and liked so
far in this discussion, would actually not help Tahoe at all and would
in fact harm Tahoe -- I would have to remember to detect and work-around
the automatic 'utf-8b' filesystem encoding when porting Tahoe to Python
3.

If anyone else has a concrete, real use case which would be helped by
PEP 383, I would like to hear about it.  Perhaps Tahoe can learn
something from it.

Oh, if this PEP could be extended to add a flag to each unicode object
indicating whether it was created with the python-escape handler or not,
then it would be useful to me.

Regards,

Zooko

[1] http://mail.python.org/pipermail/python-dev/2009-April/089020.html
[2] http://allmydata.org/trac/tahoe/attachment/ticket/534/fsencode.3.py
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Oddity PEP 0 key

[Michael Foord]

MRAB wrote:

Benjamin Peterson wrote:

2009/5/1 MRAB :

I've just noticed an oddity in the key in PEP 0. Most letters are used
more than once. Wouldn't it be clearer if different letters were used
for "Accepted" and "Active" instead of them both being 'A', for 
example?


-> A - Accepted proposal
-> R - Rejected proposal
  W - Withdrawn proposal
-> D - Deferred proposal
  F - Final proposal
-> A - Active proposal
-> D - Draft proposal
-> R - Replaced proposal


Yes, that makes more sense. Would you like to submit a patch against
the PEP 0 generator? (It's in peps/pep0)


I'm still trying to think which letters to use!


P for Proposal (to replace Active Proposal)? Every active PEP is a 
proposal...


Michael


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] special method lookup: how much do we care?

[Michael Foord]

Nick Coghlan wrote:

Benjamin Peterson wrote:
  

A while ago, Guido declared that all special method lookups on
new-style classes bypass __getattr__ and __getattribute__. This almost
completely consistent now, and I've been working on patching up a few
incorrect cases. I've know hit __enter__ and __exit__. The compiler
generates LOAD_ATTR instructions for these, so it uses the normal
lookup. The only way I can see to fix this is add a new opcode which
uses _PyObject_LookupSpecial, but I don't think we really care this
much. Opinions?



As Georg pointed out, the expectation was that we would eventually add a
SETUP_WITH opcode that used the special method lookup (and hopefully
speed with statements up to a point where they're competitive with
writing out the associated try statement directly). The current code is
the way it is because there is no "LOAD_SPECIAL" opcode and adding type
dereferencing logic to the expansion would have been difficult without a
custom opcode.

For other special methods that are looked up from Python code, the
closest we can ever get is to bypass the instance (i.e. using
"type(obj).__method__(obj, *args)") to avoid metaclass confusion. The
type slots are even *more* special than that because they bypass
__getattribute__ and __getattr__ even on the metaclass for speed reasons.

There's a reason the docs already say that for a guaranteed override you
*must* actually define the special method on the class rather than
merely making it accessible via __getattr__ or even __getattribute__.

The PyPy guys are right to think that some developer somewhere is going
to rely on these implementation details in CPython at some point.
However lots of developers rely on CPython ref counting as well, no
matter how many times they're told not to do that if they want to
support alternative interpreters.
  


It's actually very annoying for things like writing Mock or proxy 
objects when this behaviour is inconsistent (sorry should have spoken up 
earlier).


The Python interpreter bases some of its decisions on whether these 
methods exist at all - and when you have objects that provide methods 
through __getattr__ then you can accidentally get screwed if magic 
method lookup returns an object unexpectedly when it should have raised 
an AttributeError.


Of course for proxy objects it might be more convenient if *all* 
attribute access did go through __getattr__ - but with that not the case 
it is much better for it to be consistent rather than have to put in 
specific workaround code.


All the best,

Michael



Cheers,
Nick.

  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LZW support in tarfile ?

[Michael Foord]

Antoine Pitrou wrote:

Tarek Ziadé  gmail.com> writes:
  

But I was wondering if we should we add a LZW support in tarinfo,
besides gzip and bzip2 ?

Although this compression standard doesn't seem very used these days,



It would be more useful to add LZMA / xz support.
I don't think compress is used anymore, except perhaps on old legacy systems.
On my Linux system, I have lots of .gz, .bz2 and .lzma files, but absolutely no
.Z file.
  


I've seen the occasional .Z file in recent years, but never that I 
recall for a Python package.


As plugging in external compression tools is less likely to work 
cross-platform wouldn't it be both easier and better to deprecate (and 
not replace) the compress support.


If there is a huge outcry adding LZW support to tarfile can be reconsidered.

Michael Foord


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384: Defining a Stable ABI

[Michael Foord]

Martin v. Löwis wrote:

Dino Viehland wrote:
  

Dirkjan Ochtman wrote:


It would seem to me that optimizations are likely to require data
structure changes, for exactly the kind of core data structures that
you're talking about locking down. But that's just a high-level view,
I might be wrong.

  

In particular I would guess that ref counting is the biggest issue here.
I would think not directly exposing the field and having inc/dec ref
Functions (real methods, not macros) for it would give a lot more
ability to change the API in the future.



In the context of optimization, I'm skeptical that introducing functions
for the reference counting would be useful. Making the INCREF/DECREF
macros functions just in case the reference counting goes away is IMO
an unacceptable performance cost.

Instead, such a change should go through the regular deprecation
procedure and/or cause the release of Python 4.0.

  

It also might make it easier for alternate implementations to support
the same API so some modules could work cross implementation - but I
suspect that's a non-goal of this PEP :).



Indeed :-) I'm also skeptical that this would actually allow
cross-implementation modules to happen. The list of functions that
an alternate implementation would have to provide is fairly long.

  


Just in case you're unaware of it; the company I work for has an open 
source project called Ironclad. This *is* a reimplementation of the 
Python C API and gives us binary compatibility with [some subset of] 
Python C extensions for use from IronPython.


http://www.resolversystems.com/documentation/index.php/Ironclad.html

It's an ambitious project but it is now at the stage where 1000s of the 
Numpy and Scipy tests pass when run from IronPython. I don't think this 
PEP impacts the project, but it is not completely unfeasible for the 
alternative implementations to do this.


In particular we have had to address the issue of the GIL and extensions 
(IronPython has no GIL) and reference counting (which IronPython also 
doesn't) use.


Michael Foord



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384: Defining a Stable ABI

[Michael Foord]

Martin v. Löwis wrote:

It also might make it easier for alternate implementations to support
the same API so some modules could work cross implementation - but I
suspect that's a non-goal of this PEP :).



Indeed :-) I'm also skeptical that this would actually allow
cross-implementation modules to happen. The list of functions that
an alternate implementation would have to provide is fairly long.

  
  

Just in case you're unaware of it; the company I work for has an open
source project called Ironclad.



I was unaware indeed; thanks for pointing this out.

IIUC, it's not just an API emulation, but also an ABI emulation.

  


Correct.


In particular we have had to address the issue of the GIL and extensions
(IronPython has no GIL) and reference counting (which IronPython also
doesn't) use.



I think this somewhat strengthens the point I was trying to make: An
alternate implementation that tries to be API compatible has to consider
so many things that it is questionable whether making Py_INCREF/DECREF
functions would be any simplification.
  


It would actually have been helpful for us, but I understand that it 
would be a big performance hit. The Ironclad garbage collection 
mechanism is described here:


http://www.voidspace.org.uk/python/weblog/arch_d7_2009_01_24.shtml#e1055

We artificially inflate the refcount of all objects that Ironclad 
creates to 2 and hold a reference to them on the .NET side to make them 
ineligible for garbage collection.


Because we can't always know when objects have been decreffed back down 
to 1, there are some circumstances when we have to scan all the objects 
we are holding onto. If their refcount is only 1 then we no longer need 
to hold a reference them. When nothing is using them on the IronPython 
side either normal .NET garbage collection kicks in and the IronPython 
proxy object has a destructor that calls back into Ironclad and uses the 
CPython dealloc method.



So I just ask:
a) Would it help IronClad if it could restrict itself to PEP 384
   compatible modules?
b) Would further restrictions in the PEP help that cause?
  


I've forwarded these questions to the lead developer of Ironclad 
(William Reade) along with a link to the PEP. He isn't on Python-dev so 
I may have to be a proxy for him in discussion. His initial response was 
"looks pretty sweet".


Michael


Regards,
Martin
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] IronPython specific code in inspect module

[Michael Foord]

Hello all,

The inspect module (inspect.get_argspec etc) work fine for Python 
functions and classes in IronPython, but they don't work on .NET types 
which don't have the Python function attributes like im_func etc.


I have IronPython specific versions of several of these functions which 
use .NET reflection and inspect could fallback to if sys.platform == 
'cli'. Would it be ok for me to add these to the inspect module? 
Obviously the tests would only run on IronPython... The behaviour for 
CPython would be unaffected.


All the best,

Michael Foord

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] syntax warnings don't go through warnings.warn?

[Michael Foord]

Matthew Wilkes wrote:


On 1 Jun 2009, at 17:50, Dino Viehland wrote:

I’m just a little surprised by this - Is there a reason why syntax 
warnings are special and untrappable via warnings.warn?


Why should this work?  From the docs... "Python programmers issue 
warnings by calling the warn() function defined in this module. (C 
programmers use PyErr_WarnEx; see Exception Handling for details)."


Check out the warnings.catch_warnings context manager, but if you have 
any further questions please direct them to the normal Python mailing 
list, this is for development _of_ Python only.


Dino is developing Python - he's one of the core developers of 
IronPython and I suspect he is asking whether this is intentional, and 
IronPython should implement the same behaviour, or whether it is a bug.


Michael



Matthew
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr)

[Michael Foord]

Paul Moore wrote:

2009/6/3  :
  

So, here are my recommendations:

 1. Use the tracker for discussing tickets, so that it's easy to refer back
to a previous point in the discussion, and so that people working on those
tickets can easily find your commentary.
 2. Use the mailing list for drawing attention to these discussions if they
are of general interest, especially if the discussion is time- critical.  In
this case, an announcement "You have six weeks to review ipaddr now until
its inclusion is permanent, anyone interested please look at issue 3959."
 3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at the
top of your message, so that it's easy for newcomers to the discussion to
get a general feel.



Mostly, I agree, but I definitely disagree, I'm afraid, on the use of
the tracker for discussions. To keep track of discussions on a ticket,
I have to personally keep a list of the tickets I'm interested in,
check back regularly to see if there's anything new,


Not true - if you are added as nosy on a tracker item (which happens 
when you make a comment or you can do yourself) then you get emailed 
about new comments. The email contains the body of the comment so you 
can follow discussions completely by email only going to the tracker to 
add responses.


Michael


 and keep a mental
note of where I've read up to so I know what's new. RSS would make
this simpler, certainly, but I'm not sure about how I'd use it (it's
not how I currently use RSS, so I'd have to mess round with my current
setup to make it appropriate).

Email is delivered to me by default - I get anything new in my
python-dev folder, and I can skip or read the discussion as I choose.
I don't have to take action just to monitor things. (In other words,
the default is for people to see the discussions, rather than the
other way around.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of 2.7 and 3.2

[Michael Foord]

R. David Murray wrote:

[snip...]

By the policy you propose, we could not have released 2.6 in October
2008, which we really really wanted to because Apple wanted us to.


I don't think the 2.6 release date is relevant to this discussion,
since 3.x hadn't been released at all at that point.  What I want to
avoid is people writing new code for 2.7 instead of 3.1 because they
want to take advantage of a nifty new feature that 3.1 doesn't have.

But 3.1 is in feature freeze (py3k trunk) and it is not possible to 
check-in code for 3.2. Following this policy it would mean a feature 
freeze on trunk for an indefinite period of time.



Michael



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of 2.7 and 3.2

[Michael Foord]

Terry Reedy wrote:

[snip...]


I don't think that's a novel idea though - I'm pretty sure it was
suggested (and met with general approval) when the idea of a short
release cycle for 3.1 was first brought up.


I presume because it has been stated before.

In addition to the question above, I am also trying to provoke thought 
on the nature and purpose of 2.7.  Backporting features 'if someone 
feels like it' seems pretty haphazard.  For someone wanting to 
maintain compatibility across multiple 2.x releases, a random new 
features may be nearly useless.




The "What's new in Python 2.7" list is already very impressive. :-)

Michael


Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type

[Michael Foord]

Dino Viehland wrote:

Guido wrote:
  

I should add that this policy is also forced somewhat by the existence
of the "multiple interpreters in one address space" feature, which is
used e.g. by mod_python. This feature attempts to provide isolation
between interpreters to the point that each one can have a completely
different set of modules loaded and can be working on a totally
different application. The implementation of CPython shares built-in
types between multiple interpreters (and it wouldn't be easy to change
this); if you were able to modify a built-in type from one
interpreter, all other interpreters would see that same modification.



IronPython is in the exact same boat here - we share built-in types
Across multiple Python engines as well.


  


And indeed it is needed - if you are working with multiple interpreters 
(engines in IronPython) you don't want isinstance(something, dict) to 
fail because it is a dictionary from a different interpreter...


Michael


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [IronPython] Exception for setting attributes of built-in type

[Michael Foord]

Guido van Rossum wrote:

On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord wrote:
  

Dino Viehland wrote:


Guido wrote:
  

I should add that this policy is also forced somewhat by the existence
of the "multiple interpreters in one address space" feature, which is
used e.g. by mod_python. This feature attempts to provide isolation
between interpreters to the point that each one can have a completely
different set of modules loaded and can be working on a totally
different application. The implementation of CPython shares built-in
types between multiple interpreters (and it wouldn't be easy to change
this); if you were able to modify a built-in type from one
interpreter, all other interpreters would see that same modification.



  

IronPython is in the exact same boat here - we share built-in types
Across multiple Python engines as well.
  


  

And indeed it is needed - if you are working with multiple interpreters
(engines in IronPython) you don't want isinstance(something, dict) to fail
because it is a dictionary from a different interpreter...



Ah, but that suggests you have sharing between different interpreters.
If you're doing that, perhaps you shouldn't be using multiple
interpreters, but instead multiple threads?

  
Well, in our use case we use multiple engines to provide an isolated 
execution context for every document (the Resolver One spreadsheet 
written in IronPython). Each of these has their own calculation thread 
as well - but the engine per document structure is nice and clean and 
means each document can have its own set of modules loaded without 
affecting the other documents (although they share a core set of modules).


Once we move these engines into their own app domains we can completely 
isolate each document and apply separate security permissions to each 
one. That might mean each document effectively paying the 
not-insubstantial startup time hit and we haven't begun to look at how 
to mitigate that.


Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen()

[Michael Foord]

Steven D'Aprano wrote:

On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote:

  

I don't think all
pythons do immediate ref-counted GC.



Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen 
Swallow, etc.



  

PyPy doesn't, Unladen Swallow won't.

Michael

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] draft pep: backwards compatibility

[Michael Foord]

gl...@divmod.com wrote:

[snip...]
For example, I think it was wrong to change the name of a 
test-skipping unittest

method just so that it wouldn't clash with a method created by Twisted
subclassing unittest (besides, self.skip() was much nicer than 
self.skipTest()
;-)). Just because some class is public shouldn't prevent us to add 
new public

methods or attributes to it.


I think it would be wrong to have a blanket prohibition on such 
changes, by which I mean additions of names to public namespaces.  
Twisted's own compatibility possibility would not forbid a similar 
change.  But in that specific case I think it was the right thing to 
do.  Like it or not, a lot of people use Twisted, a lot of people run 
tests with Trial, and we beat stdlib unittest to the punch on the 
'skip' testing feature by a good 5 years.  We caught the change well 
before the release, we reported it and discussed it.


Just to note that Twisted (along with Bazaar) are one of the few 'good 
citizens' in the Python community running their tests on Python trunk. 
Both projects have caught incompatibilities *before* release of new 
versions which is greatly preferable to discovering them after a 
release. Thanks for this.


Michael Foordt



IMHO this is the best way to deal with incompatible changes, 
especially in the case of superclasses, given Python's subtle and 
complex inheritance semantics.  It's not feasible to provide a policy 
that somehow prohibits subclasses from adding names which may 
eventually be used on a superclass.


Projects which notice test failures with new versions of Python should 
report them so that the features can be adjusted to be compatible, 
assuming the project in question hasn't done anything in egregious 
violation of the compatibility policy (like invoking a private 
method). This lets users, system administrators, and application 
authors upgrade components individually, without worrying about the 
components further down the stack flaking out on them.  It also 
provides a feedback loop for the compatibility policy: if there are 
things that initially seem reasonable, but projects report 
compatibility issues when they are changed, they might need to be 
added later.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] draft pep: backwards compatibility

[Michael Foord]

Antoine Pitrou wrote:

 divmod.com> writes:
  
This is a false dichotomy; for core developers, the list needs to be 
exhaustive.  Everything that can change needs to be described as either 
compatible or incompatible.



How do you enumerate "everything that can change"? It does not look like a
finite set to me (but perhaps I'm wrong); and certainly not like a set of a size
reasonable enough to be enumerated in a human-readable way :)

  


And this is why expressing a finite list of things we guarantee won't 
change is a virtually impossible task - unless one of you is 
volunteering to write an official spec for Python and its libraries... 
:-) (Something that would not be bad IMO - just a long and difficult 
task, *especially* if you include the library along with language 
semantics and APIs).


Michael



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] draft pep: backwards compatibility

[Michael Foord]

Benjamin Peterson wrote:

Backwards compatibility seems to be an issue that arises a lot here. I
think we all have an idea of it is, but we need some hard place to
point to. So here's my attempt:
  


Should this PEP include a note about features which require new syntax, 
particularly  new keywords?


The policy (AFAICT) is that if new keywords are created they are enabled 
with a future import (with a warning?) in the version they are 
introduced and then enabled by default in the next version.


All the best,

Michael Foord



PEP: 387
Title: Backwards Compatibility Policy
Version: $Revision$
Last-Modified: $Date$
Author: Benjamin Peterson 
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 18-Jun-2009


Abstract


This PEP outlines Python's backwards compatibility policy.


Rationale
=

As one of the most used programming languages today [#tiobe]_, the Python core
language and its standard library play a critcal role in thousands of
applications and libraries.  This is fantastic; it is probably one of a language
designer's most wishful dreams.  However, it means the development team must be
very careful not to break this existing 3rd party code with new releases.


Backwards Compatibility Rules
=

This policy applys to all public APIs.  These include the C-API, the standard
library, and the core language including syntax and operation as defined by the
reference manual.

This is the basic policy for backwards compatibility:

* The behavior of an API *must* not change between any two consecutive releases.

* A feature cannot be removed without notice between any two consecutive
  releases.

* Addition of a feature which breaks 3rd party libraries or applications should
  have a large benefit to breakage ratio, and/or the incompatibility should be
  trival to fix in broken code.


Making Incompatible Changes
===

It's a fact: design mistakes happen.  Thus it is important to be able to change
APIs or remove misguided features.  This is accomplished through a gradual
process over several releases:

1. Discuss the change.  Depending on the size of the incompatibility, this could
   be on the bug tracker, python-dev, python-list, or the appropriate SIG.  A
   PEP or similar document may be written.  Hopefully users of the affected API
   will pipe up to comment.

2. Add a warning [#warnings_]_.  If behavior is changing, a the API may gain a
   new function or method to perform the new behavior; old usage should raise
   the warning.  If an API is being removed, simply warn whenever it is entered.
   DeprecationWarning is the usual warning category to use, but
   PendingDeprecationWarning may be used in special cases were the old and new
   versions of the API will coexist for many releases.

3. Wait for a release.

4. See if there's any feedback.  Users not involved in the original discussions
   may comment now after seeing the warning.  Perhaps reconsider.

5. The behavior change or feature removal may now be made default or permanent
   in the next release.  Remove the old version and warning.


References
==

.. [#tiobe] TIOBE Programming Community Index

   http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

.. [#warnings] The warnings module

   http://docs.python.org/library/warnings.html


Copyright
=

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com