[Cython] Speedup module-level lookup
I tried to optimize module lookups (__pyx_m) by caching internal PyDict state. In this example bar() is 1.6 time faster (500us against 842us): C = 123 def foo(a): return C * adef bar(): for i in range(1): foo(i) Here is proof of concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9 So the question is: does it worth it? -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
I think the right thing to do here is make all module-level globals into "cdef public" attributes, i.e. C globals with getters and setters for Python space. I'm not sure whether this would best be done by creating a custom dict or module subclass, but it would probably be cleaner and afford much more than a 1.6x speedup. - Robert On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov wrote: > I tried to optimize module lookups (__pyx_m) by caching internal PyDict state. > > In this example bar() is 1.6 time faster (500us against 842us): > > C = 123 > def foo(a): > return C * adef bar(): > for i in range(1): foo(i) > Here is proof of > concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9 > > So the question is: does it worth it? > > -- > vitja. > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
AFAIK, a module's dict is readonly, so I don't believe a dict subclass will
work there (I could be wrong) unless you hack up the module object from C.
You can do it with descriptors on a ModuleType however, which should be
plenty fast from Cython-land.
In [16]: class AGetter(object):
: def __get__(self, obj, cls):
: return obj.a
: def __set__(self, obj, val):
: obj.a = val
:
In [17]: class MyMod(types.ModuleType):
: b = AGetter()
:
In [18]: mmod = MyMod('my_mod')
In [20]: mmod.__dict__['a'] = 42
In [21]: mmod.a
Out[21]: 42
In [22]: mmod.b
Out[22]: 42
In [23]: mmod.b = 87
In [24]: mmod.a
Out[24]: 87
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
2012/1/19 Robert Bradshaw :
> I think the right thing to do here is make all module-level globals
> into "cdef public" attributes, i.e. C globals with getters and setters
> for Python space. I'm not sure whether this would best be done by
> creating a custom dict or module subclass, but it would probably be
> cleaner and afford much more than a 1.6x speedup.
>
> - Robert
>
> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov
> wrote:
>> I tried to optimize module lookups (__pyx_m) by caching internal PyDict
>> state.
>>
>> In this example bar() is 1.6 time faster (500us against 842us):
>>
>> C = 123
>> def foo(a):
>> return C * adef bar():
>> for i in range(1): foo(i)
>> Here is proof of
>> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>>
>> So the question is: does it worth it?
>>
Yes, nice idea.
It's possible to subclass PyModuleObject and I didn't find any use of
PyModule_CheckExact() in CPython's sources:
import types
import sys
global_foo = 1234
class CustomModule(types.ModuleType):
def __init__(self, name):
types.ModuleType.__init__(self, name)
sys.modules[name] = self
@property
def foo(self):
return global_foo
@foo.setter
def foo(self, value):
global global_foo
global_foo = value
CustomModule('foo')
import foo
print foo.foo
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
2012/1/19 Vitja Makarov :
> 2012/1/19 Robert Bradshaw :
>> I think the right thing to do here is make all module-level globals
>> into "cdef public" attributes, i.e. C globals with getters and setters
>> for Python space. I'm not sure whether this would best be done by
>> creating a custom dict or module subclass, but it would probably be
>> cleaner and afford much more than a 1.6x speedup.
>>
>> - Robert
>>
>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov
>> wrote:
>>> I tried to optimize module lookups (__pyx_m) by caching internal PyDict
>>> state.
>>>
>>> In this example bar() is 1.6 time faster (500us against 842us):
>>>
>>> C = 123
>>> def foo(a):
>>> return C * adef bar():
>>> for i in range(1): foo(i)
>>> Here is proof of
>>> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>>>
>>> So the question is: does it worth it?
>>>
>
> Yes, nice idea.
> It's possible to subclass PyModuleObject and I didn't find any use of
> PyModule_CheckExact() in CPython's sources:
>
> import types
> import sys
>
> global_foo = 1234
>
> class CustomModule(types.ModuleType):
> def __init__(self, name):
> types.ModuleType.__init__(self, name)
> sys.modules[name] = self
>
> @property
> def foo(self):
> return global_foo
>
> @foo.setter
> def foo(self, value):
> global global_foo
> global_foo = value
>
> CustomModule('foo')
>
> import foo
> print foo.foo
>
But this seems to break globals().
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
