Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 10:02 AM, Glyph wrote: > On Mar 4, 2013, at 11:13 PM, Robert Collins > wrote: > > In principle maybe. Need to talk with the trial developers, nose > developers, py.test developers etc - to get consensus on a number of > internal API friction points. > > > Some of trial's lessons might be also useful for the stdlib going forward, > given the hope of doing some event-loop stuff in the core. > > But, I feel like this might be too much to cover at the language summit; > there could be a test frameworks summit of its own, of about equivalent > time and scope, and we'd still have a lot to discuss. > > Is there a unit testing SIG someone from Twisted ought to be a member of, > to represent Trial, and to get consensus on these points going forward? > > The testing-in-python list is pretty much where most test tool authors hang out, see http://lists.idyll.org/listinfo/testing-in-python Also, maybe related, i am heading the "tox" effort which many people use to have a frontend for their testing process, see http://tox.testrun.org -- it has a somewhat different focus in that it sets up virtualenv and install test specific dependencies. However, positional arguments (often used to select tests) can be configured to be passed on to the test runner of choice. I was considering extending tox to directly support nose, pytest, unittest and trial "drivers" and offer a unified (minimal) command line API. Am open to collaboration on that. cheers, holger > -glyph > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/holger.krekel%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
On Tue, Feb 9, 2010 at 7:29 PM, Olemis Lang wrote: > On Tue, Feb 9, 2010 at 11:42 AM, Michael Foord > wrote: >> Hello all, >> >> Several >> authors of other Python testing frameworks spoke up *against* them, but >> several *users* of test frameworks spoke up in favour of them. ;-) >> > > +1 for having something like that included in unittest hey Olemis, aren't you a test tool author as well? :) >> I'm pretty sure I can introduce setUpClass and setUpModule without breaking >> compatibility with existing unittest extensions or backwards compatibility >> issues > > Is it possible to use the names `BeforeClass` and `AfterClass` (just > to be make it look similar to JUnit naming conventions ;o) ? > >> - with the possible exception of test sorting. Where you have a class >> level setUp (for example creating a database connection) you don't want the >> tearDown executed a *long* time after the setUp. In the presence of class or >> module level setUp /tearDown (but only if they are used) I would expect test >> sorting to only sort within the class or module [1]. I will introduce the >> setUp and tearDown as new 'tests' - so failures are reported separately, > > Perhaps I am missing something, but could you please mention what will > happen if a failure is raised inside class-level `tearDown` ? > >> and >> all tests in the class / module will have an explicit skip in the event of a >> setUp failure. >> I think reporting tests as skipped when the setup failed is a bad idea. Out of several years of practise with skips and large test suites (and talking/experiencing many users :) i recommend to reserve skips for platform/dependency/environment mismatches. A Setup Error should just error or fail all the tests in its scope. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
On Tue, Feb 9, 2010 at 10:57 PM, Ben Finney wrote: > Michael Foord writes: > >> The next 'big' change to unittest will (may?) be the introduction of >> class and module level setUp and tearDown. This was discussed on >> Python-ideas and Guido supported them. They can be useful but are also >> very easy to abuse (too much shared state, monolithic test classes and >> modules). Several authors of other Python testing frameworks spoke up >> *against* them, but several *users* of test frameworks spoke up in >> favour of them. ;-) > > I think the perceived need for these is from people trying to use the > ‘unittest’ API for test that are *not* unit tests. > > That is, people have a need for integration tests (test this module's > interaction with some other module) or system tests (test the behaviour > of the whole running system). They then try to crowbar those tests into > ‘unittest’ and finding it lacking, since ‘unittest’ is designed for > tests of function-level units, without persistent state between those > test cases. > > Is there a better third-party framework for use in these cases? As > Olemis points out later in this thread, I don't think it's good for the > ‘unittest’ module to keep growing for uses that aren't focussed on unit > tests (as contrasted with other kinds of tests). My general view these days is that for unit tests there is practically not much of a a difference in using unittest, nose or py.test (give or take reporting niceness and flexibility). However, Functional and integration tests involve more complex fixture management and i came to find the setup/teardown on classes and modules lacking. Which is why there is testresources from Rob and funcargs in py.test. The latter allow to setup and teardown resources from a fixture factory which can determine the setup/teardown scope and perform whole-session caching without changing test code. In my Pycon testing tutorial (http://tinyurl.com/ya6b3vr ) i am going to exercise it in depth with beginners and here are docs: http://pytest.org/funcargs.html One nice bit is that you can for a given test module issue "py.test --funcargs" and get a list of resources you can use in your test function - by simply specifying them in the test function. In principle it's possible to port this approach to the stdlib - actually i consider to do it for the std-unittest- running part of py.test because people asked for it - if that proves useful i can imagine to refine it and offer it for inclusion. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
Hi Guido, On Thu, Feb 11, 2010 at 7:11 PM, Guido van Rossum wrote: > On Tue, Feb 9, 2010 at 8:42 AM, Michael Foord > wrote: >> The next 'big' change to unittest will (may?) be the introduction of class >> and module level setUp and tearDown. This was discussed on Python-ideas and >> Guido supported them. They can be useful but are also very easy to abuse >> (too much shared state, monolithic test classes and modules). Several >> authors of other Python testing frameworks spoke up *against* them, but >> several *users* of test frameworks spoke up in favour of them. ;-) > > Hi Michael, > > I have skimmed this thread (hence this reply to the first rather than > the last message), but in general I am baffled by the hostility of > testing framework developers towards their users. The arguments > against class- and module-level seUp/tearDown functions seems to be > inspired by religion or ideology more than by the zen of Python. What > happened to Practicality Beats Purity? Hostility against users? I have not heart that feedback from my users yet - or am i missing some meaning of your words? > The potential for abuse in and of itself should not be an argument > against a feature; it must always be weighed against the advantages. sure. > The argument that a unittest framework shouldn't be "abused" for > regression tests (or integration tests, or whatever) is also bizarre > to my mind. Surely if a testing framework applies to multiple kinds of > testing that's a good thing, not something to be frowned upon? If an approach has known limitations it's also good to point them out. Also ok to disregard them and still consider something useful enough. > There are several alternative testing frameworks available outside the > standard library. The provide useful competition with the stlib's > unittest and doctest modules, and useful inspiration for potential new > features. They also, by and large, evolve much faster than a stdlib > module ever could, and including anyone of these in the stdlib might > well be the death of it (just as unittest has evolved much slower > since it was included). Fully agreed :) > But unittest *is* still evolving, and there is no reason not to keep > adding features along the lines of your module/class setUp/tearDown > proposal (or extra assertions like assertListEqual, which I am happy > to see has been added). > > On the other hand, I think we should be careful to extend unittest in > a consistent way. I shuddered at earlier proposals (on python-ideas) > to name the new functions (variations of) set_up and tear_down "to > conform with PEP 8" (this would actually have violated that PEP, which > explicitly prefers local consistency over global consistency). If that was me you refer to - i followed PEP8 5 years ago when introducing setup_class/module and i still stand by it, it was supposed to be a more pythonic alternative and i consider PEP8 as part of that. But i agree - introducing it to std-unittest now makes not much sense due to local consistency reasons. I appreciate Michael's effort to help advance testing - we have a good private discussion currently btw - and i am happy to collaborate with him on future issues, setupClass or not :) cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
On Fri, Feb 12, 2010 at 12:00 AM, Robert Kern wrote: > On 2010-02-11 16:20 PM, Ben Finney wrote: >> >> Guido van Rossum writes: > >>> The argument that a unittest framework shouldn't be "abused" for >>> regression tests (or integration tests, or whatever) is also bizarre >>> to my mind. Surely if a testing framework applies to multiple kinds of >>> testing that's a good thing, not something to be frowned upon? >> >> To my mind, an API should take a stand on the “right” way to use it, >> rather than being a kitchen-sink of whatever ideas had any support. >> Doing something the right way should be easy, and doing something the >> wrong way should be awkward. > > setUpClass and setUpModule are the "right" way to do many types of > integration and functional tests. Integration and functional tests are vital > tasks to perform, and unittest provides a good framework otherwise for > implementing such tests. Ben just expressed his opinion about API design and you claim some truth about testing in general. In my experience, integration and functional testing is a complex and evolving topic, usually requiring more from the tool or framework than classic unit-testing. To name a few issues: * creating tempdirs and files * setting up base environments * starting and stopping servers * mocking components * replaying individual tests * reacting to timeouts * handling asynchronicity * web-javascript integration support * configuring fixtures from config files * CI tool integration and multi-platform deployment * running variations of the same tests across different base configs * ... much much more It's true that you can go and extend unittest for that but a) unittest is just a tiny bit of what is involved for satisfying the needs b) what you are doing then is mostly using the fact that a setup function (or chain) is invoked and a test function is invoked and that python has some builtin modules for handling the above issues. And you are using Python - and Python is nice and (potentially) concise for writing tests, sure. That's not wholly the fault of the unittest module, though :) So. Doing fixtures via static encoding in class and module setup functions is a way to provide a generic framing for writing tests. The "right" way? In many cases and for the about 6 different people i interacted with (and on actual RL code) in the last 2 weeks it does not help incredibly much. There is experiences from other test tool authors indicating similar experiences. I will say that module/class can be helpful and you can do some useful things with it and thus it makes some sense to add it for std-unittest but claiming this is great and most of what you need for "many types" of functional testing is misleading and plays down the many good things you can do with Testing and Python. best, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 proposed changes for basic plugins support
On Mon, Aug 2, 2010 at 6:57 PM, Ian Bicking wrote: > Just to add a general opinion in here: > > Having worked with Setuptools' entry points, and a little with some Zope > pluginish systems (Products.*, which I don't think anyone liked much, and > some ways ZCML is used is pluginish), I'm not very excited about these. The > plugin system that causes the least confusion and yet seems to accomplish > everything it needs is just listing objects in configuration -- nothing gets > activated implicitly with installation, and names are Python package/object > names without indirection. This makes it a two-step process to use a plugin: install it and then configure it correctly to have it used. I'd much prefer a one-step process and rather provide a way to not-use a plugin even if installed. The difference is e.g. with py.test that i can point users to e.g. pip install pytest-figleaf py.test --figleaf instead of also having to explain a configuration file, its location and exact content or some magic attribute variables on some classes. TBH, i'd like to have pip handle plugins, based on metadata (especially some data signaling something is a plugin of otherthing). This way i only once have to teach about "pip" and people leverage their knowledge for installing/managing plugins. best, holger > The only thing I'd want to add is the ability to > also point to files, as a common use for plugins is adding ad hoc > functionality to an application, and the overhead of package creation isn't > always called for. hg for example seems both simple and general enough, and > it doesn't use anything fancy. > > Purely for the purpose of discovery and documentation it might be helpful to > have APIs, then some tool could show available plugins (especially if PyPI > had a query interface for this), or at least installed plugins, with the > necessary code to invoke them. > > *Maybe* it would make sense to generalize the discovery of plugin types, so > that you can simply refer to an object and the application can determine > what kind of plugin it is. But having described this, it actually doesn't > seem like a useful thing to generalize. > > -- > Ian Bicking | http://blog.ianbicking.org > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/holger.krekel%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 proposed changes for basic plugins support
On Mon, Aug 2, 2010 at 8:12 PM, Michael Foord wrote:
> On 02/08/2010 19:05, Holger Krekel wrote:
>>
>> On Mon, Aug 2, 2010 at 6:57 PM, Ian Bicking wrote:
>>
>>>
>>> Just to add a general opinion in here:
>>>
>>> Having worked with Setuptools' entry points, and a little with some Zope
>>> pluginish systems (Products.*, which I don't think anyone liked much, and
>>> some ways ZCML is used is pluginish), I'm not very excited about these.
>>> The
>>> plugin system that causes the least confusion and yet seems to accomplish
>>> everything it needs is just listing objects in configuration -- nothing
>>> gets
>>> activated implicitly with installation, and names are Python
>>> package/object
>>> names without indirection.
>>>
>>
>> This makes it a two-step process to use a plugin: install it and then
>> configure it correctly to have it used.
>>
>> I'd much prefer a one-step process and rather provide a way to not-use
>> a plugin even if installed. The difference is e.g. with py.test that i
>> can
>> point users to e.g.
>>
>> pip install pytest-figleaf
>> py.test --figleaf
>>
>
> How do you achieve this currently?
it uses setuptools entrypoints, so with a setup.py param like
entry_points = {'pytest11': ['pytest_figleaf = pytest_figleaf'],},
py.test's pluginmanager.py does:
for ep in pkg_resources.iter_entry_points('pytest11'):
# ... ep.name == "pytest_figleaf" <- left side of above "*=*"
plugin = ep.load() # <- right side of above "*=*"
# ...
best,
holger
> Michael
>
>> instead of also having to explain a configuration file, its location
>> and exact content or some magic attribute variables on some
>> classes.
>>
>> TBH, i'd like to have pip handle plugins, based on metadata
>> (especially some data signaling something is a plugin of otherthing).
>> This way i only once have to teach about "pip" and people leverage
>> their knowledge for installing/managing plugins.
>>
>> best,
>> holger
>>
>>
>>>
>>> The only thing I'd want to add is the ability to
>>> also point to files, as a common use for plugins is adding ad hoc
>>> functionality to an application, and the overhead of package creation
>>> isn't
>>> always called for. hg for example seems both simple and general enough,
>>> and
>>> it doesn't use anything fancy.
>>>
>>> Purely for the purpose of discovery and documentation it might be helpful
>>> to
>>> have APIs, then some tool could show available plugins (especially if
>>> PyPI
>>> had a query interface for this), or at least installed plugins, with the
>>> necessary code to invoke them.
>>>
>>> *Maybe* it would make sense to generalize the discovery of plugin types,
>>> so
>>> that you can simply refer to an object and the application can determine
>>> what kind of plugin it is. But having described this, it actually
>>> doesn't
>>> seem like a useful thing to generalize.
>>>
>>> --
>>> Ian Bicking | http://blog.ianbicking.org
>>>
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>>
>>> http://mail.python.org/mailman/options/python-dev/holger.krekel%40gmail.com
>>>
>>>
>>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>>
>
>
> --
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
> READ CAREFULLY. By accepting and reading this email you agree, on behalf of
> your employer, to release me from all obligations and waivers arising from
> any and all NON-NEGOTIATED agreements, licenses, terms-of-service,
> shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure,
> non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have
> entered into with your employer, its partners, licensors, agents and
> assigns, in perpetuity, without prejudice to my ongoing rights and
> privileges. You further represent that you have the authority to release me
> from any BOGUS AGREEMENTS on behalf of your employer.
>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 proposed changes for basic plugins support
On Mon, Aug 2, 2010 at 8:48 PM, Michael Foord wrote: > On 02/08/2010 19:45, Holger Krekel wrote: [...] >>>> I'd much prefer a one-step process and rather provide a way to not-use >>>> a plugin even if installed. The difference is e.g. with py.test that i >>>> can point users to e.g. >>>> >>>> pip install pytest-figleaf >>>> py.test --figleaf >>> >>> How do you achieve this currently? >>> >> >> it uses setuptools entrypoints, so with a setup.py param like > > Right. I can't use that for unittest. With the new proposal we *could* > automatically use all available plugins, but I think I prefer to only have > plugins active that the user has chosen. This sounds like a situation where a user has more installed than he actually asked for. I guess i am a big fan of "use virtualenv, avoid global installations" and thus don't see the point to have more installed than needed. best, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] pypy's interpreter/highlevel backend features
Hello Python-dev! we have a document on PyPy's interpreter and translation features that might be of interest to you - we target it at language implementors in general: Includes a discussion on our .NET and also the emerging Java backends, as well as our RPython -> Javascript webapp generating experiments, on security relevant taint propagation and transparent proxies ... IOW quite a bag :) We'd be very happy about feedback and opinions/questions (preferably until Monday, 19th March) http://codespeak.net/pypy/extradoc/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-interim-2007-03-12.pdf best, Holger -- merlinux GmbH Steinbergstr. 4231139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] the role of assert in the standard library ?
On Thu, Apr 28, 2011 at 6:59 PM, Guido van Rossum wrote: > On Thu, Apr 28, 2011 at 12:54 AM, Tarek Ziadé wrote: >> In my opinion assert should be avoided completely anywhere else than >> in the tests. If this is a wrong statement, please let me know why :) > > I would turn that around. The assert statement should not be used in > unit tests; unit tests should use self.assertXyzzy() always. FWIW this is only true for the unittest module/pkg policy for writing and organising tests. There are other popular test frameworks like nose and pytest which promote using plain asserts within writing unit tests and also allow to write tests in functions. And judging from my tutorials and others places many people appreciate the ease of using asserts as compared to learning tons of new methods. YMMV. Holger > regular code, assert should be about detecting buggy code. It should > not be used to test for error conditions in input data. (Both these > can be summarized as "if you still want the test to happen with -O, > don't use assert.) > > -- > --Guido van Rossum (python.org/~guido) > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/holger.krekel%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] the role of assert in the standard library ?
On Fri, Apr 29, 2011 at 12:31 AM, Raymond Hettinger wrote: > > On Apr 28, 2011, at 3:07 PM, Guido van Rossum wrote: > >> On Thu, Apr 28, 2011 at 2:53 PM, Raymond Hettinger >> wrote: >>> >>> On Apr 28, 2011, at 1:27 PM, Holger Krekel wrote: >>> >>>> On Thu, Apr 28, 2011 at 6:59 PM, Guido van Rossum wrote: >>>>> On Thu, Apr 28, 2011 at 12:54 AM, Tarek Ziadé >>>>> wrote: >>>>>> In my opinion assert should be avoided completely anywhere else than >>>>>> in the tests. If this is a wrong statement, please let me know why :) >>>>> >>>>> I would turn that around. The assert statement should not be used in >>>>> unit tests; unit tests should use self.assertXyzzy() always. >>>> >>>> FWIW this is only true for the unittest module/pkg policy for writing and >>>> organising tests. There are other popular test frameworks like nose and >>>> pytest >>>> which promote using plain asserts within writing unit tests and also allow >>>> to >>>> write tests in functions. And judging from my tutorials and others places >>>> many >>>> people appreciate the ease of using asserts as compared to learning tons >>>> of new methods. YMMV. >>> >>> I've also observed that people appreciate using asserts with nose.py and >>> py.test. >> >> They must not appreciate -O. :-) > > It might be nice if there were a pragma or module variable to selectively > enable asserts for a given test module so that -O would turn-off asserts in > the production code but leave them on in a test_suite. A way to tell Python "if you are going to compile this module/path, don't turn off asserts, no matter what" would be great. Then nose/py.test and whichever tools/apps could fine-tune the handling of asserts. (This would probably be better than marking things inline for those use cases). Then testing with "-O" would work nicely as well which would be appreciated :) best, holger > Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: __source__ proposal
Hi Stelios, [Stelios Xanthakis Fri, Dec 03, 2004 at 11:54:25AM +0200] > Abstract > > This PEP suggests the implementation of __source__ attribute for > functions and classes. The attribute is a read-only string which > is generated by the parser and is a copy of the original source > code of the function/class (including comments, indentation and > whitespace). I've had similar ideas in the past as we are doing dynamic code generation in PyPy, as well as in other projects. After some discussion with Armin i think there is another possibility for storing "source" or any other such meta information with code/module objects: make __file__ and co_filename instances of a subclass of 'str' providing an extra attribute. For a simple example, they could have a 'source' attribute, which could be tried first by appropriate inspect functions and traceback related functionality. We are about to test out this approach with the py lib (http://codespeak.net/py) and want to have it work for for Python 2.2, 2.3. and 2.4. I suspect there may be some issues lurking (also in your proposed PEP) especially with respect to encodings. Also we have some use cases where we want to retrieve source code from non-local locations and want to integrate this seemlessly with the introspection facilities of Python which obviously is an important part of the equation. I can report back if there is interest. cheers, holger ___ Python-Dev mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: __source__ proposal
[Stelios Xanthakis Fri, Dec 03, 2004 at 11:59:30PM +0200] > On Fri, 3 Dec 2004, holger krekel wrote: > >We are about to test out this approach with the py lib > >(http://codespeak.net/py) and want to have it work for > >for Python 2.2, 2.3. and 2.4. > > Do you plan hacking python ? > It appears that tok_nextc() is the best place to > catch all the source passed to the interpreter. Well, as we want to have the library work on past python versions modifying CPython 2.5 does not make much sense. It's more about (like Martin pointed out) organizing dynamic code generation so that Python's introspect and traceback logic works as much as possible - with tiny runtime "monkey" patches if needed. Now Martin also correctly pointed out that you can store source code before/after you pass it to compile/parse. We are doing this already with an external dictionary. This has multithreading issues, though. So we think that hooking onto code's objects co_filename or a module's __file__ might be an interesting idea. cheers, holger ___ Python-Dev mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.4 news reaches interesting places
[Guido van Rossum Wed, Dec 08, 2004 at 02:18:48PM -0800]
> I was pleasantly surprised to find a pointer to this article in a news
> digest that the ACM emails me regularly (ACM TechNews).
>
> http://gcn.com/vol1_no1/daily-updates/28026-1.html
>
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?
What about doing a survey on c.l.py asking questions like:
what do you find runs slowly in Python
that should run faster?
Could you help with some concrete -
preferably real life examples with speed problems?".
If python-dev takes the time to seriously debate (and debunk :-)
upcoming code and suggestions then such a process could
be very useful for all sides and also for PR purposes.
IMO the biggest PR problem is that python programmers themselves [*]
tend to say that Python is slow and it's interesting to find
out why they think so, document and discuss the "answers" if any.
I am not saying that such questioning/discussion doesn't already
happen sometimes. But doing a survey in a more systematic way
might let us find out how pervasive the perception of "Python
is too slow" really is. Maybe it turns out that not many people
have concrete problems to offer? Anyway, this would probably also
be interesting for the various alternative Python implementations
currently in the works.
just an idea,
holger
[*] For example, something i stumbled upon today:
http://unununium.org/articles/languages
where it states (without providing any details!):
But what about that fast system? Python isn't a slow
language; it just has a slow implementation. There are many
projects underway to correct this situation: Psyco, PyPy,
Starkiller, IronPython, and Parrotcode are among them. It's
likely these projects will be nearing completion when the time
comes for Unununium to look at optimizations.
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 310 and exceptions
Hi all, probably unsuprisingly i am still pondering the idea of having an optional __except__ hook on block handlers. The PEP says this about this: An extension to the protocol to include an optional __except__ handler, which is called when an exception is raised, and which can handle or re-raise the exception, has been suggested. It is not at all clear that the semantics of this extension can be made precise and understandable. For example, should the equivalent code be try ... except ... else if an exception handler is defined, and try ... finally if not? How can this be determined at compile time, in general? In fact, i think the translation even to python code is not that tricky: x = X(): ... basically translates to: if hasattr(x, '__enter__'): x.__enter__() try: ... except: if hasattr(x, '__except__'): x.__except__(...) else: x.__exit__() else: x.__exit__() this is the original definition from the PEP with the added except clause. Handlers are free to call 'self.__exit__()' from the except clause. I don't think that anything needs to be determined at compile time. (the above can probably be optimized at the bytecode level but that is a side issue). Moreover, i think that there are more than the "transactional" use cases mentioned in the PEP. For example, a handler may want to log exceptions to some tracing utility or it may want to swallow certain exceptions when its block does IO operations that are ok to fail. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 310 and exceptions
On Fri, Apr 22, 2005 at 19:03 -0700, Josiah Carlson wrote: > [EMAIL PROTECTED] (holger krekel) wrote: > > basically translates to: > > > > if hasattr(x, '__enter__'): > > x.__enter__() > > try: > > ... > > except: > > if hasattr(x, '__except__'): x.__except__(...) > > else: x.__exit__() > > else: > > x.__exit__() > > Nope... > > >>> def foo(): > ... try: > ... print 1 > ... return > ... except: > ... print 2 > ... else: > ... print 3 > ... > >>> foo() > 1 > >>> doh! of course, you are right. So it indeeds better translates to a nested try-finally/try-except when transformed to python code. Nick Coghlan points at the correct ideas below in this thread. At the time i was implementing things by modifying ceval.c rather than by just a compiling addition, i have to admit. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
__except__ use cases (was: Re: [Python-Dev] PEP 310 and exceptions)
On Sat, Apr 23, 2005 at 13:41 +1000, Nick Coghlan wrote: > Nick Coghlan wrote: > In light of Alex's comments, I'd actually like to suggest the below as a > potential new definition for PEP 310 (making __exit__ optional, and adding > an __else__ handler): > > if hasattr(x, '__enter__'): > x.__enter__() > try: > try: > # Contents of 'with' block > except: > if hasattr(x, '__except__'): > if not x.__except__(*sys.exc_info()): # [1] > raise On a side note, I don't see too much point in having __except__ return something when it is otherwise easy to say: def __except__(self, typ, val, tb): self.abort_transaction() raise typ, val, tb But actually i'd like to to mention some other than transaction-use cases for __except__, for example with class MyObject: def __except__(self, typ, val, tb): if isinstance(val, KeyboardInterrupt): raise # process exception and swallow it you can use it like: x = MyObject(): # do some long running stuff and MyObject() can handle internal problems appropriately and present clean Exceptions to the outside without changing the "calling side". With my implementation i also played with little things like: def __getattr__(self, name): Key2AttributeError: return self._cache[key] ... with an obvious __except__() implementation for Key2AttributeError. Similar to what Alex points out i generally think that being able to define API/object specific exception handling in *one* place is a great thing. I am willing to help with the PEP and implementation, btw. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: __except__ use cases
Hi Nick, On Sun, Apr 24, 2005 at 12:40 +1000, Nick Coghlan wrote: > Seeing this example has convinced me of something. PEP 310 should use the > 'with' keyword, and 'expression block' syntax should be used to denote the > 'default object' semantics proposed for Python 3K. For example: While that may be true, i don't care too much about the syntax yet but more about the idea and semantics of an __except__ hook. I simply followed the syntax that Guido currently seems to prefer. holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Hi Guido, On Mon, May 02, 2005 at 17:55 -0700, Guido van Rossum wrote: > These are the loose ends on the PEP (apart from filling in some > missing sections): > > 1. Decide on a keyword to use, if any. I just read the PEP340 basically the first time so bear with me. First i note that introducing a keyword 'block' would break lots of programs, among it half of PyPy. Unlike many other keywords 'block' is a pretty common variable name. For invoking blocktemplates i like the no-keyword approach, instead. However, i would find it much clearer if *defining* blocktemplates used a new keyword, like: blocktemplate opening(filename, mode="r"): ... because this immediately tells me what the purpose and semantics of the folowing definition is. The original overloading of 'def' to mean generators if the body contains a yield statement was already a matter of discussion (ASFAIK). When i came to Python it was at 2.2 and i remember wondering about this "def" oddity. Extending poor old 'def' functions now to possibly mean block templates gives me semantical overload even if it is justified from an implementation point of view. I am talking purely about (my sense of) code readability here not about implementation. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
[Guido] > [Holger] > > However, i would find it much clearer if *defining* blocktemplates > > used a new keyword, like: > > > > blocktemplate opening(filename, mode="r"): > > ... > > > > because this immediately tells me what the purpose and semantics > > of the folowing definition is. The original overloading of 'def' to > > mean generators if the body contains a yield statement was already a > > matter of discussion (ASFAIK). When i came to Python it was at 2.2 > > and i remember wondering about this "def" oddity. > > > > Extending poor old 'def' functions now to possibly mean block > > templates gives me semantical overload even if it is justified > > from an implementation point of view. I am talking purely > > about (my sense of) code readability here not about implementation. > > Hm... Maybe you also want to have separate function and procedure > keywords? Or static typing? 'def' can be used to define all sorts of > things, that is Python's beauty! Sure, 'def' is nice and i certainly wouldn't introduce a new keyword for adding e.g. static typing to function 'defs'. But for my taste, blocktemplates derive enough from the old-style function/sub-routine notion that many people still think of when seing a 'def'. When (new) people would see something like 'blocktemplate ...:' they know they have to look it up in the language documentation or in some book under 'blocktemplate' instead of trying to figure out (what the hell) this "function" or "generator" does and how they can use it. Or they might simply think they can invoke it from a for-loop which - as far as i understand - could lead to silent errors, no? Let me add that with the growing number of Python programmers (as stated in your Pycon2005 keynote) it seems to make sense to increase emphasis on how new syntax/concepts will be viewed/used by possibly 100'dreds of thousands of programmers already familiar with (some version of) Python. But i also see your point of confronting people with the fact that Python has a nice unified 'def' statement so i guess it's a balancing act. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3000 and new style classes
On Fri, Sep 09, 2005 at 11:31 -0700, Russell E. Owen wrote: > In article <[EMAIL PROTECTED]>, > Tristan Seligmann <[EMAIL PROTECTED]> wrote: > > > > Why does it matter if the single statement you insert is spelled > > " metaclass = type" instead of "from future import whatever"? > > Remember, unlike the division example, you would only have to insert one > > statement, as opposed to changing every use of integer division. > > It matters because "metaclass = type" is completely obscure. How would > any non-expert have a clue what it means? How would this non-expert have a clue what "from __future__ import new_style_classes" means? holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
