Re: [Python-Dev] Assignment to __class__ of module? (Autoloading? (Making Queue.Queue easier to use))
Phillip J. Eby wrote: > I meant that just changing its class is a mutation, and since immutables > can be shared or cached, that could lead to problems. So I do think > it's a reasonable implementation limit to disallow changing the > __class__ of an immutable. That's a fair point. Although I was actually thinking recently of a use case for changing the class of a tuple, inside a Pyrex module for database access. The idea was that the user would be able to supply a custom subclass of tuple for returning the records. To avoid extra copying of the data, I was going to create a normal uninitialised tuple, stuff the data into it, and then change its class to the user-supplied one. But seeing as all this would be happening in Pyrex where the normal restrictions don't apply anyway, I suppose it wouldn't matter if user code wasn't allowed to do this. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)
Sokolov Yura wrote:
> May be allow modules to define __getattr__ ?
I think I like the descriptor idea better. Besides
being more in keeping with modern practice, it would
allow for things like
from autoloading import autoload
Foo = autoload('foomodule', 'Foo')
Blarg = autoload('blargmodule', 'Blarg')
where autoload is defined as a suitable descriptor
subclass.
I guess we could do with a PEP on this...
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode charmap decoders slow
Tony Nelson wrote:
> I have written my fastcharmap decoder and encoder. It's not meant to be
> better than the patch and other changes to come in a future version of
> Python, but it does work now with the current codecs.
It's an interesting solution.
> To use, hook each codec to be speed up:
>
> import fastcharmap
> help(fastcharmap)
> fastcharmap.hook('name_of_codec')
> u = unicode('some text', 'name_of_codec')
> s = u.encode('name_of_codec')
>
> No codecs were rewritten. It took me a while to learn enough to do this
> (Pyrex, more Python, some Python C API), and there were some surprises.
> Hooking in is grosser than I would have liked. I've only used it on Python
> 2.3 on FC3.
Indeed, and I would claim that you did not completely achieve your "no
changes necessary" goal: you still have to install the hooks explicitly.
I also think overriding codecs.charmap_{encode,decode} is really ugly.
Even if this could be simplified if you would modify the existing
codecs, I still don't think supporting changes to the encoding dict
is worthwhile. People will probably want to update the codecs in-place,
but I don't think we need to make a guarantee that that such an approach
works independent of the Python version. People would be much better off
writing their own codecs if they think the distributed ones are
incorrect.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
Phillip J. Eby wrote: > A more > compact scheme is possible, by using two tables - a bytecode->line > number table, and a line number-> file table. > > If you have to encode multiple files, you just offset their line numbers > by the size of the other files, More straightforwardly, the second table could just be a bytecode -> file number mapping. The filename is likely to change much less often than the line number, so this file would contain far fewer entries than the line number table. In the case of only one file, it would contain just a single entry, so it probably wouldn't even be worth the bother of special-casing that. You could save a bit more by having two kinds of line number table, "small" (16-bit entries) and "large" (32-bit entries) depending on the size of the code object and range of line numbers. The small one would be sufficient for almost all code objects, so the most common case would use only about 4 bytes per line of code. That's only twice as much as the current scheme uses. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Early PEP draft (For Python 3000?)
Calvin Spealman <[EMAIL PROTECTED]> wrote: > > On 10/11/05, Eyal Lotem <[EMAIL PROTECTED]> wrote: > > locals()['x'] = 1 # Quietly fails! > > Replaced by: > > frame.x = 1 # Raises error > > What about the possibility of making this hypothetic frame object an > indexable, such that frame[0] is the current scope, frame[1] is the > calling scope, etc.? On the same lines, what about closure[0] for the > current frame, while closure[1] resolves to the closure the function > was defined in? These would ensure that you could reliably access any > namespace you would need, without nasty stack tricks and such, and > would make working around some of the limitation of the closures, when > you have such a need. One might even consider a __resolve__ to be > defined in any namespace, allowing all the namespace resolution rules > to be overridden by code at any level. -1000 If you want a namespace, create one and pass it around. If the writer of a function or method wanted you monkeying around with a namespace, they would have given you one to work with. As for closure monkeywork, you've got to be kidding. Closures in Python are a clever and interesting way of keeping around certain things, but are actually unnecessary with the existance of class and instance namespaces. Every example of a closure can be re-done as a class/instance, and many end up looking better. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
On 10/13/05, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > Indeed. I should've threatened to kill the AST branch long ago! :) :-) I decreased a lot of the memory leaks. Here are some more to work on. I doubt this list is complete, but it's a start: PyObject_Malloc (obmalloc.c:717) _PyObject_DebugMalloc (obmalloc.c:1014) compiler_enter_scope (newcompile.c:1204) compiler_mod (newcompile.c:1894) PyAST_Compile (newcompile.c:471) Py_CompileStringFlags (pythonrun.c:1240) builtin_compile (bltinmodule.c:391) Tuple (Python-ast.c:907) ast_for_testlist (ast.c:1782) ast_for_classdef (ast.c:2677) ast_for_stmt (ast.c:2758) PyAST_FromNode (ast.c:233) PyParser_ASTFromFile (pythonrun.c:1291) parse_source_module (import.c:762) load_source_module (import.c:886) new_arena (obmalloc.c:500) PyObject_Malloc (obmalloc.c:699) PyObject_Realloc (obmalloc.c:837) _PyObject_DebugRealloc (obmalloc.c:1077) PyNode_AddChild (node.c:95) shift (parser.c:112) PyParser_AddToken (parser.c:244) parsetok (parsetok.c:165) PyParser_ParseFileFlags (parsetok.c:89) PyParser_ASTFromFile (pythonrun.c:1288) parse_source_module (import.c:762) load_source_module (import.c:886) Lambda (Python-ast.c:610) ast_for_lambdef (ast.c:859) ast_for_expr (ast.c:1443) ast_for_testlist (ast.c:1776) ast_for_expr_stmt (ast.c:1845) ast_for_stmt (ast.c:2716) PyAST_FromNode (ast.c:233) PyParser_ASTFromString (pythonrun.c:1271) Py_CompileStringFlags (pythonrun.c:1237) builtin_compile (bltinmodule.c:391) BinOp (Python-ast.c:557) ast_for_binop (ast.c:1389) ast_for_expr (ast.c:1531) ast_for_testlist (ast.c:1776) ast_for_expr_stmt (ast.c:1845) ast_for_stmt (ast.c:2716) PyAST_FromNode (ast.c:233) PyParser_ASTFromString (pythonrun.c:1271) Py_CompileStringFlags (pythonrun.c:1237) builtin_compile (bltinmodule.c:391) Name (Python-ast.c:865) ast_for_atom (ast.c:1201) ast_for_expr (ast.c:1555) ast_for_testlist (ast.c:1776) ast_for_expr_stmt (ast.c:1798) ast_for_stmt (ast.c:2716) PyAST_FromNode (ast.c:233) PyParser_ASTFromString (pythonrun.c:1271) Py_CompileStringFlags (pythonrun.c:1237) builtin_compile (bltinmodule.c:391) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AST branch update
On 10/14/05, Neal Norwitz <[EMAIL PROTECTED]> wrote: > > I decreased a lot of the memory leaks. Here are some more to work on. > I doubt this list is complete, but it's a start: Oh and since I fixed the memory leaks in a generated file Python/Python-ast.c, the changes still need to be implemented in the right place (ie, Parser/asdl_c.py). Valgrind didn't report any invalid uses of memory, though there is also a lot potentially leaked memory. It seemed a lot higher than what I remembered, so I'm not sure if it's an issue or not. I'll look into that after we get the definite memory leaks plugged. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
"Phillip J. Eby" <[EMAIL PROTECTED]> writes: > Even better if the lines for a particular piece of code don't have > to all come from the same file. This seems _fairly_ esoteric to me. Why do you need it? I can think of two uses for lnotab information: printing source lines and locating source lines on the filesystem. For both, I think I'd rather see some kind of defined protocol (methods on the code object, maybe?) rather than inventing some kind of insane too-general-for-the-common-case data structure. Cheers, mwh -- 42. You can measure a programmer's perspective by noting his attitude on the continuing vitality of FORTRAN. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
At 08:07 PM 10/14/2005 +1300, Greg Ewing wrote: >Phillip J. Eby wrote: > > > A more > > compact scheme is possible, by using two tables - a bytecode->line > > number table, and a line number-> file table. > > > > If you have to encode multiple files, you just offset their line numbers > > by the size of the other files, > >More straightforwardly, the second table could just be a >bytecode -> file number mapping. That would use more space in any case involving multiple files. >In the case of only one file, it would contain just a single >entry, so it probably wouldn't even be worth the bother of >special-casing that. A line->file mapping would also have only one entry in that case. >You could save a bit more by having two kinds of line number >table, "small" (16-bit entries) and "large" (32-bit entries) >depending on the size of the code object and range of line >numbers. The small one would be sufficient for almost all >code objects, so the most common case would use only about >4 bytes per line of code. That's only twice as much as the >current scheme uses. That'd probably work. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
At 09:23 AM 10/14/2005 +0100, Michael Hudson wrote: >"Phillip J. Eby" <[EMAIL PROTECTED]> writes: > > > Even better if the lines for a particular piece of code don't have > > to all come from the same file. > >This seems _fairly_ esoteric to me. Why do you need it? Compilers that inline function calls, but want the code to still be debuggable. AOP tools that weave bytecode. Overloaded functions implemented by combining bytecode. Okay, those are fairly esoteric use cases, I admit. :) However, PyPy already has some inlining capability in its optimizer, so it's not all that crazy of an idea that Python in general will need it. >I can think of two uses for lnotab information: printing source lines >and locating source lines on the filesystem. For both, I think I'd >rather see some kind of defined protocol (methods on the code object, >maybe?) rather than inventing some kind of insane >too-general-for-the-common-case data structure. Certainly a protocol would be nice; right now one is forced to interpret the data structure directly. Being able to say, "give me the file and line number for a given byte offset" would be handy in any case. However, since you can't subclass code objects, the capability would have to be part of the core. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
"Phillip J. Eby" <[EMAIL PROTECTED]> writes: > At 09:23 AM 10/14/2005 +0100, Michael Hudson wrote: >>"Phillip J. Eby" <[EMAIL PROTECTED]> writes: >> >> > Even better if the lines for a particular piece of code don't have >> > to all come from the same file. >> >>This seems _fairly_ esoteric to me. Why do you need it? > > Compilers that inline function calls, but want the code to still be > debuggable. AOP tools that weave bytecode. Overloaded functions > implemented by combining bytecode. Err... > Okay, those are fairly esoteric use cases, I admit. :) However, PyPy > already has some inlining capability in its optimizer, so it's not all that > crazy of an idea that Python in general will need it. Um. Well, _I_ still think it's pretty crazy. >>I can think of two uses for lnotab information: printing source lines >>and locating source lines on the filesystem. For both, I think I'd >>rather see some kind of defined protocol (methods on the code object, >>maybe?) rather than inventing some kind of insane >>too-general-for-the-common-case data structure. > > Certainly a protocol would be nice; right now one is forced to interpret > the data structure directly. Being able to say, "give me the file and line > number for a given byte offset" would be handy in any case. > > However, since you can't subclass code objects, the capability would have > to be part of the core. Clearly, but any changes to co_lnotab would have to be part of the core too. Let's not make a complicated situation _worse_. Something I didn't say was that a protocol like this would also let us remove the horrors of functions like inspect.getsourcelines() (see SF bugs passim). Cheers, mwh -- There's an aura of unholy black magic about CLISP. It works, but I have no idea how it does it. I suspect there's a goat involved somewhere. -- Johann Hibschman, comp.lang.scheme ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode charmap decoders slow
Martin v. Löwis wrote:
> Tony Nelson wrote:
>
>> I have written my fastcharmap decoder and encoder. It's not meant to be
>> better than the patch and other changes to come in a future version of
>> Python, but it does work now with the current codecs.
>
> It's an interesting solution.
I like the fact that encoding doesn't need a special data structure.
>> To use, hook each codec to be speed up:
>>
>> import fastcharmap
>> help(fastcharmap)
>> fastcharmap.hook('name_of_codec')
>> u = unicode('some text', 'name_of_codec')
>> s = u.encode('name_of_codec')
>>
>> No codecs were rewritten. It took me a while to learn enough to do this
>> (Pyrex, more Python, some Python C API), and there were some surprises.
>> Hooking in is grosser than I would have liked. I've only used it on
>> Python
>> 2.3 on FC3.
>
> Indeed, and I would claim that you did not completely achieve your "no
> changes necessary" goal: you still have to install the hooks explicitly.
> I also think overriding codecs.charmap_{encode,decode} is really ugly.
>
> Even if this could be simplified if you would modify the existing
> codecs, I still don't think supporting changes to the encoding dict
> is worthwhile. People will probably want to update the codecs in-place,
> but I don't think we need to make a guarantee that that such an approach
> works independent of the Python version. People would be much better off
> writing their own codecs if they think the distributed ones are
> incorrect.
Exacty. If you need another codec write your own insteaad of patching an
existing one on the fly!
Of course we can't accept Pyrex code in the Python core, so it would be
great to rewrite the encoder as a patch to PyUnicode_EncodeCharmap().
This version must be able to cope with encoding tables that are random
strings without crashing.
We've already taken care of decoding. What we still need is a new
gencodec.py and regenerated codecs.
Bye,
Walter Dörwald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode charmap decoders slow
Walter Dörwald wrote: > We've already taken care of decoding. What we still need is a new > gencodec.py and regenerated codecs. I'll take care of that; just haven't gotten around to it yet. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 14 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Autoloading? (Making Queue.Queue easier to use)
Sokolov Yura wrote: > May be allow modules to define __getattr__ ? > > def __getattr__(thing): > try: > return __some_standart_way__(thing) > except AttributeError: > if thing=="Queue": >import sys >from Queue import Queue >setattr(sys.modules[__name__],"Queue",Queue) >return Queue > raise I proposed something like this in the RFE tracker a while ago, but no one commented on it. Reinhold -- Mail address is perfectly valid! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
Hi! Phillip J. Eby wrote: [snip] > Okay, those are fairly esoteric use cases, I admit. :) However, PyPy > already has some inlining capability in its optimizer, so it's not all that > crazy of an idea that Python in general will need it. It's kind of strange to argue with PyPy's inlining capabilities, since inlining in PyPy happens on a completely different level, that has nothing at all to do with Python code objects any more. So your proposed changes would not make a difference for PyPy (not even to speak about benefits). [snip] cheers, Carl Friedrich Bolz ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
> >> > Even better if the lines for a particular piece of code don't have > >> > to all come from the same file. > >> > >>This seems _fairly_ esoteric to me. Why do you need it? > > > > Compilers that inline function calls, but want the code to still be > > debuggable. AOP tools that weave bytecode. Overloaded functions > > implemented by combining bytecode. > > Err... > > > Okay, those are fairly esoteric use cases, I admit. :) However, PyPy > > already has some inlining capability in its optimizer, so it's not all > that > > crazy of an idea that Python in general will need it. > > Um. Well, _I_ still think it's pretty crazy. YAGNI Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] LOAD_SELF and SELF_ATTR opcodes
I ran across an interesting paper about some VM optimizations yesterday: http://www.object-arts.com/Papers/TheInterpreterIsDead.PDF One thing mentioned was that saving even one cycle in their 'PUSH_SELF' opcode improved interpreter performance by 5%. I thought that was pretty cool, and then I realized CPython doesn't even *have* a PUSH_SELF opcode. So, today, I took a stab at implementing one, by converting "LOAD_FAST 0" calls to a "LOAD_SELF" opcode. Pystone and Parrotbench improved by about 2% or so. That wasn't great, so I added a "SELF_ATTR" opcode that combines a LOAD_SELF and a LOAD_ATTR in the same opcode while avoiding extra stack and refcount manipulation. This raised the total improvement for pystone to about 5%, but didn't seem to improve parrotbench any further. I guess parrotbench doesn't do much self.attr stuff in places that really count, and looking at the code it indeed seems that most self.* stuff is done at higher levels of the parsing benchmark, not the innermost loops. Indeed, even pystone doesn't do much attribute access on the first argument of most of its functions, especially not those in inner loops. Only Proc1() and the Record.copy() method do anything that would be helped by SELF_ATTR. But it seems to me that this is very unusual for object-oriented code, and that more common uses of Python should be helped a lot more by this. Do we have any benchmarks that don't use 'foo = self.foo' type shortcuts in their inner loops? Anyway, my main question is, do these sound like worthwhile optimizations? The code isn't that complex; the only tricky thing I did was having the opcodes' error case (unbound local) fall through to the LOAD_FAST opcode so as not to duplicate the error handling code, in the hopes of keeping the eval loop size down. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
[Michael Hudson] > "Phillip J. Eby" <[EMAIL PROTECTED]> writes: > > Even better if the lines for a particular piece of code don't have > > to all come from the same file. > This seems _fairly_ esoteric to me. Why do you need it? For when Python code is generated from more than one original source file (referring to the `#line' directive message, a little earlier this week). For example, think include files. -- François Pinard http://pinard.progiciels-bpi.ca ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
[Raymond Hettinger] > > >> > Even better if the lines for a particular piece of code don't > > >> > have to all come from the same file. > YAGNI I surely needed it, more than once. Don't be so assertive. :-) -- François Pinard http://pinard.progiciels-bpi.ca ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify lnotab? (AST branch update)
At 02:41 PM 10/14/2005 -0400, Raymond Hettinger wrote: >YAGNI If the feature were there, I'd have used it already, so I wouldn't consider it YAGNI. In the cases where I would've used it, I instead split generated code into separate functions so I could compile() each one with a different filename. Also, I notice that the peephole optimizer contains stuff to avoid making co_lnotab "too complex", although I haven't looked at it to be sure it'd actually benefit from an expanded lnotab format. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LOAD_SELF and SELF_ATTR opcodes
Phillip> Indeed, even pystone doesn't do much attribute access on the Phillip> first argument of most of its functions, especially not those Phillip> in inner loops. Only Proc1() and the Record.copy() method do Phillip> anything that would be helped by SELF_ATTR. But it seems to me Phillip> that this is very unusual for object-oriented code, and that Phillip> more common uses of Python should be helped a lot more by this. Phillip> Do we have any benchmarks that don't use 'foo = self.foo' type Phillip> shortcuts in their inner loops? (Just thinking out loud...) Maybe we should create an alternate "object-oriented" version of pystone as a way to inject more attribute access into a convenient benchmark. Even if it's completely artificial and has no connection to other versions of the Drhystone benchmark, it might be useful for testing improvements to attribute access. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Unicode charmap decoders slow
Walter Dörwald wrote: > Of course we can't accept Pyrex code in the Python core, so it would be > great to rewrite the encoder as a patch to PyUnicode_EncodeCharmap(). > This version must be able to cope with encoding tables that are random > strings without crashing. I don't think this will be necessary. I personally dislike the decoding tables, as I think a straight-forward trie will do better than a hashtable. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LOAD_SELF and SELF_ATTR opcodes
Phillip J. Eby wrote: > Anyway, my main question is, do these sound like worthwhile > optimizations? In the past, I think the analysis was always "no". It adds an opcode, so increases the size of the switch, causing more pressure on the cache, with an overall questionable effect. As for measuring the effect of the change: how often does that pattern occur in the standard library? (compared to what total number of LOAD_ATTR) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LOAD_SELF and SELF_ATTR opcodes
At 12:33 AM 10/15/2005 +0200, Martin v. Löwis wrote: >Phillip J. Eby wrote: > > Anyway, my main question is, do these sound like worthwhile > > optimizations? > >In the past, I think the analysis was always "no". It adds >an opcode, so increases the size of the switch, causing >more pressure on the cache, with an overall questionable >effect. Hm. I'd have thought 5% pystone and 2% pybench is nothing to sneeze at, for such a minor change. I thought Skip's peephole optimizer originally only produced a 5% or so speedup. In any case, in relation to this specific kind of optimization, this is the only thing I found: http://mail.python.org/pipermail/python-dev/2002-February/019854.html which is a proposal by Guido to do the same thing, but also speeding up the actual attribute lookup. I didn't find any follow-up suggesting that anybody tried this, but perhaps it was put on hold pending the AST branch? :) >As for measuring the effect of the change: how often >does that pattern occur in the standard library? >(compared to what total number of LOAD_ATTR) [EMAIL PROTECTED] src]$ grep 'self\.[A-Za-z_]' Lib/*.py | wc -l 9919 [EMAIL PROTECTED] src]$ grep '[a-zA-Z_][a-zA-Z_0-9]*\.[a-zA-Z_]' Lib/*.py | wc -l 19804 So, something like 50% of lines doing an attribute access include a 'self' attribute access. This very rough estimate may be thrown off by: * Import statements (causing an error in favor of more non-self attributes) * Functions whose first argument isn't 'self' (error in favor of non-self attributes) * Comments or docstrings talking about attributes or modules (could go either way) * Multiple attribute accesses on the same line (could go either way) The parrotbench code shows a similar ratio of self to non-self attribute usage, but nearly all of parrotbench's self-attribute usage is in b0.py, and not called in the innermost loop. That also suggests that the volume of usage of 'self.' isn't the best way to determine the performance impact, because pystone has almost no 'self.' usage at all, but still got a 5% total boost. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LOAD_SELF and SELF_ATTR opcodes
>> Phillip J. Eby wrote: >> > Anyway, my main question is, do these sound like worthwhile >> > optimizations? >> >> In the past, I think the analysis was always "no". It adds an opcode, >> so increases the size of the switch, causing more pressure on the >> cache, with an overall questionable effect. Phillip> Hm. I'd have thought 5% pystone and 2% pybench is nothing to Phillip> sneeze at, for such a minor change. We've added lots of new opcodes over the years. CPU caches have grown steadily in that time as well, from maybe 128KB-256KB in the early 90's to around 1MB today. I suspect cache size has kept up with the growth of Python's VM inner loop. At any rate, each change has to be judged on its own merits. If it speeds things up and is uncontroversial implementation-wise, I see no reason it should be rejected out-of-hand. (Send it to Raymond H. He'll probably sneak it in when Martin's not looking. ) Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - offtopic
Josiah Carlson wrote: >Sokolov Yura <[EMAIL PROTECTED]> wrote: > > >>Offtopic: >> >>Microsoft Windows [Version 5.2.3790] >>(C) Copyright 1985-2003 Microsoft Corp. >> >>G:\Working\1>c:\Python24\python >>Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on >>win32 >>Type "help", "copyright", "credits" or "license" for more information. >> >>> from os import fork >>Traceback (most recent call last): >> File "", line 1, in ? >>ImportError: cannot import name fork >> >>> >> >> > >Python for Windows, if I remember correctly, has never supported forking. >This is because the underlying process execution code does not have >support for the standard copy-on-write semantic which makes unix fork >fast. > >Cygwin Python does support fork, but I believe this is through a literal >copying of the memory space, which is far slower than unix fork. > >Until Microsoft adds kernel support for fork, don't expect standard >Windows Python to support it. > > - Josiah > > > > > That is what i mean... sorry for being noisy... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LOAD_SELF and SELF_ATTR opcodes
Sorry I can't reply to the message (I'm at home, and don't currently have python-dev sent there). I have a version of Raymond's constant binding recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940 that also binds all attribute accesses all the way down into a single constant call e.g. LOAD_FAST 0 LOAD_ATTR 'a' LOAD_ATTR 'b' LOAD_ATTR 'c' LOAD_ATTR 'd' is bound to a single constant: LOAD_CONST 5 where constant 5 is the object obtained from `self.a.b.c.d`. Unfortunately, I think it's at work - don't seem to have a copy here :( Obviously, this isn't applicable to as many cases, but it might be interesting to compare what kind of results this produces compared to LOAD_SELF/SELF_ATTR. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Divorcing str and unicode (no more implicit conversions).
On 10/3/05, Michael Hudson <[EMAIL PROTECTED]> wrote:
> Martin Blais <[EMAIL PROTECTED]> writes:
>
> > How hard would that be to implement?
>
> import sys
> reload(sys)
> sys.setdefaultencoding('undefined')
Hmmm any particular reason for the call to reload() here?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
