Re: [Cython] Speedup module-level lookup
On Wed, Jan 18, 2012 at 11:53 PM, Vitja Makarov wrote:
> 2012/1/19 Vitja Makarov :
>> 2012/1/19 Robert Bradshaw :
>>> I think the right thing to do here is make all module-level globals
>>> into "cdef public" attributes, i.e. C globals with getters and setters
>>> for Python space. I'm not sure whether this would best be done by
>>> creating a custom dict or module subclass, but it would probably be
>>> cleaner and afford much more than a 1.6x speedup.
>>>
>>> - Robert
>>>
>>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov
>>> wrote:
I tried to optimize module lookups (__pyx_m) by caching internal PyDict
state.
In this example bar() is 1.6 time faster (500us against 842us):
C = 123
def foo(a):
return C * adef bar():
for i in range(1): foo(i)
Here is proof of
concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
So the question is: does it worth it?
>>
>> Yes, nice idea.
>> It's possible to subclass PyModuleObject and I didn't find any use of
>> PyModule_CheckExact() in CPython's sources:
>>
>> import types
>> import sys
>>
>> global_foo = 1234
>>
>> class CustomModule(types.ModuleType):
>> def __init__(self, name):
>> types.ModuleType.__init__(self, name)
>> sys.modules[name] = self
>>
>> @property
>> def foo(self):
>> return global_foo
>>
>> @foo.setter
>> def foo(self, value):
>> global global_foo
>> global_foo = value
>>
>> CustomModule('foo')
>>
>> import foo
>> print foo.foo
>>
>
> But this seems to break globals().
How so? We have to hack globals() to get it to work for a Cython
module anyways. (I wonder if this must return a dict, or would any
mapping (or subclass of dict) be sufficient...)
- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
On 19 January 2012 08:00, Robert Bradshaw wrote:
> On Wed, Jan 18, 2012 at 11:53 PM, Vitja Makarov
> wrote:
>> 2012/1/19 Vitja Makarov :
>>> 2012/1/19 Robert Bradshaw :
I think the right thing to do here is make all module-level globals
into "cdef public" attributes, i.e. C globals with getters and setters
for Python space. I'm not sure whether this would best be done by
creating a custom dict or module subclass, but it would probably be
cleaner and afford much more than a 1.6x speedup.
- Robert
On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov
wrote:
> I tried to optimize module lookups (__pyx_m) by caching internal PyDict
> state.
>
> In this example bar() is 1.6 time faster (500us against 842us):
>
> C = 123
> def foo(a):
> return C * adef bar():
> for i in range(1): foo(i)
> Here is proof of
> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>
> So the question is: does it worth it?
>
>>>
>>> Yes, nice idea.
>>> It's possible to subclass PyModuleObject and I didn't find any use of
>>> PyModule_CheckExact() in CPython's sources:
>>>
>>> import types
>>> import sys
>>>
>>> global_foo = 1234
>>>
>>> class CustomModule(types.ModuleType):
>>> def __init__(self, name):
>>> types.ModuleType.__init__(self, name)
>>> sys.modules[name] = self
>>>
>>> @property
>>> def foo(self):
>>> return global_foo
>>>
>>> @foo.setter
>>> def foo(self, value):
>>> global global_foo
>>> global_foo = value
>>>
>>> CustomModule('foo')
>>>
>>> import foo
>>> print foo.foo
>>>
>>
>> But this seems to break globals().
>
> How so? We have to hack globals() to get it to work for a Cython
> module anyways. (I wonder if this must return a dict, or would any
> mapping (or subclass of dict) be sufficient...)
>
> - Robert
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
You'd also want this to work from python space using module.__dict__
or vars(module). I think the custom dict could solve this. Or would
you make __dict__ a property as well? (I don't know if vars() would
still break).
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
If it doesn't pass PyDict_CheckExact you won't be able to use it as the
globals to eval or exec. That's assuming you hack the module type so you
can change its __dict__.
On Jan 19, 2012 2:01 AM, "Robert Bradshaw"
wrote:
> On Wed, Jan 18, 2012 at 11:53 PM, Vitja Makarov
> wrote:
> > 2012/1/19 Vitja Makarov :
> >> 2012/1/19 Robert Bradshaw :
> >>> I think the right thing to do here is make all module-level globals
> >>> into "cdef public" attributes, i.e. C globals with getters and setters
> >>> for Python space. I'm not sure whether this would best be done by
> >>> creating a custom dict or module subclass, but it would probably be
> >>> cleaner and afford much more than a 1.6x speedup.
> >>>
> >>> - Robert
> >>>
> >>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov <
> vitja.maka...@gmail.com> wrote:
> I tried to optimize module lookups (__pyx_m) by caching internal
> PyDict state.
>
> In this example bar() is 1.6 time faster (500us against 842us):
>
> C = 123
> def foo(a):
> return C * adef bar():
> for i in range(1):foo(i)
> Here is proof of
> concept:
> https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>
> So the question is: does it worth it?
>
> >>
> >> Yes, nice idea.
> >> It's possible to subclass PyModuleObject and I didn't find any use of
> >> PyModule_CheckExact() in CPython's sources:
> >>
> >> import types
> >> import sys
> >>
> >> global_foo = 1234
> >>
> >> class CustomModule(types.ModuleType):
> >>def __init__(self, name):
> >>types.ModuleType.__init__(self, name)
> >>sys.modules[name] = self
> >>
> >>@property
> >>def foo(self):
> >>return global_foo
> >>
> >>@foo.setter
> >>def foo(self, value):
> >>global global_foo
> >>global_foo = value
> >>
> >> CustomModule('foo')
> >>
> >> import foo
> >> print foo.foo
> >>
> >
> > But this seems to break globals().
>
> How so? We have to hack globals() to get it to work for a Cython
> module anyways. (I wonder if this must return a dict, or would any
> mapping (or subclass of dict) be sufficient...)
>
> - Robert
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
On Thu, Jan 19, 2012 at 12:18 AM, Chris Colbert wrote:
> If it doesn't pass PyDict_CheckExact you won't be able to use it as the
> globals to eval or exec.
:(. I wonder how many other places have similar restrictions, perhaps
even implicitly. In particular, this would mean that an eval statement
modifying globals() would be difficult to efficiently detect. Still,
if this can be done at all, the massive speedup for the common case
could make it worth it.
> That's assuming you hack the module type so you can
> change its __dict__.
I don't think that's near as big of a hurdle (from C).
> On Jan 19, 2012 2:01 AM, "Robert Bradshaw"
> wrote:
>>
>> On Wed, Jan 18, 2012 at 11:53 PM, Vitja Makarov
>> wrote:
>> > 2012/1/19 Vitja Makarov :
>> >> 2012/1/19 Robert Bradshaw :
>> >>> I think the right thing to do here is make all module-level globals
>> >>> into "cdef public" attributes, i.e. C globals with getters and setters
>> >>> for Python space. I'm not sure whether this would best be done by
>> >>> creating a custom dict or module subclass, but it would probably be
>> >>> cleaner and afford much more than a 1.6x speedup.
>> >>>
>> >>> - Robert
>> >>>
>> >>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov
>> >>> wrote:
>> I tried to optimize module lookups (__pyx_m) by caching internal
>> PyDict state.
>>
>> In this example bar() is 1.6 time faster (500us against 842us):
>>
>> C = 123
>> def foo(a):
>> return C * adef bar():
>> for i in range(1): foo(i)
>> Here is proof of
>>
>> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>>
>> So the question is: does it worth it?
>>
>> >>
>> >> Yes, nice idea.
>> >> It's possible to subclass PyModuleObject and I didn't find any use of
>> >> PyModule_CheckExact() in CPython's sources:
>> >>
>> >> import types
>> >> import sys
>> >>
>> >> global_foo = 1234
>> >>
>> >> class CustomModule(types.ModuleType):
>> >> def __init__(self, name):
>> >> types.ModuleType.__init__(self, name)
>> >> sys.modules[name] = self
>> >>
>> >> @property
>> >> def foo(self):
>> >> return global_foo
>> >>
>> >> @foo.setter
>> >> def foo(self, value):
>> >> global global_foo
>> >> global_foo = value
>> >>
>> >> CustomModule('foo')
>> >>
>> >> import foo
>> >> print foo.foo
>> >>
>> >
>> > But this seems to break globals().
>>
>> How so? We have to hack globals() to get it to work for a Cython
>> module anyways. (I wonder if this must return a dict, or would any
>> mapping (or subclass of dict) be sufficient...)
>>
>> - Robert
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Cython crash: C++ class with missing default constructor
Hi, the following very short Cython code crashes the Cython compiler (v0.15.1): ---8<--- cdef extern from "foo.h": cdef cppclass foo: pass foo() ---8<--- The stack trace is attached. Best regards Simon Traceback (most recent call last): File "/home/anders/tmp/Cython-0.15.1/cython.py", line 17, in main(command_line = 1) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Main.py", line 662, in main result = compile(sources, options) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Main.py", line 637, in compile return compile_multiple(source, options) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Main.py", line 609, in compile_multiple result = run_pipeline(source, options) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Main.py", line 473, in run_pipeline err, enddata = Pipeline.run_pipeline(pipeline, source) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Pipeline.py", line 278, in run_pipeline data = phase(data) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Visitor.py", line 272, in __call__ return super(CythonTransform, self).__call__(node) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Visitor.py", line 255, in __call__ return self._visit(root) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Visitor.py", line 163, in _visit return handler_method(obj) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/ParseTreeTransforms.py", line 1614, in visit_ModuleNode node.body.analyse_expressions(node.scope) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Nodes.py", line 337, in analyse_expressions stat.analyse_expressions(env) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/Nodes.py", line 3630, in analyse_expressions self.expr.analyse_expressions(env) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/ExprNodes.py", line 326, in analyse_expressions self.analyse_types(env) File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/ExprNodes.py", line 3039, in analyse_types if self.analyse_as_type_constructor(env): File "/home/anders/tmp/Cython-0.15.1/Cython/Compiler/ExprNodes.py", line 2948, in analyse_as_type_constructor self.function = RawCNameExprNode(self.function.pos, constructor.type) AttributeError: 'NoneType' object has no attribute 'type' ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] non-template nested C++ classes
Hi, I'm currently experimenting with Cython's support for nested C++ classes and might havce encountered a bug. I have attempted to strip down the example from the documentation to its bare minimum. This here works fine: ---8<--- cdef extern from "foo": cdef cppclass outer[T]: cppclass inner: pass cdef outer[int].inner foo ---8<--- Next, I remove the template parameter as well. After all, not every outer class containing an inner class is a template class. ---8<--- cdef extern from "foo": cdef cppclass outer: cppclass inner: pass cdef outer.inner foo ---8<--- Now, I get this error message: 'outer' is not a cimported module It seems that without the square brackets, Cython no longer recognizes 'outer' as a class name and thinks it must be a module because it is followed by a dot. I suppose this is not what should happen, right? Best regards Simon ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] nested C++ classes, the third: default constructor
Hi, sorry for spreading these issues into three mails, but I cannot quite figure out whether they are related or not. So, here is the third installment of my adventures with nested classes. Consider the following code which compiles fine: ---8<--- cdef extern from "foo": cdef cppclass outer[T]: outer( ) cppclass inner: pass cdef outer[int].inner bar ---8<--- If I change 'outer' to lose its default constructor, the last line, which instantiates 'inner', causes an error: ---8<--- cdef extern from "foo": cdef cppclass outer[T]: outer( int ) cppclass inner: pass cdef outer[int].inner bar ---8<--- The only change is that the constructor to 'outer' now takes an argument. Now, the last line causes this error: "C++ class must have a default constructor to be stack allocated". However, 'inner' does have a default constructor, only 'outer' does not. Could it be that Cython looks for the constructor for the outer class where it should look for the constructor for the inner class? Cheers Simon ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
