[Cython] More typed constants (was: Fix integer width constant names in stdint.pxd)
Hello again, Attached is a patch that continues with the idea of declaring constants using their corresponding type. Great work on Cython, by the way. It's very useful. Mansour From 4f537f477d845468f36ac7b9370185124250520a Mon Sep 17 00:00:00 2001 From: Mansour Moufid Date: Fri, 20 Jan 2012 13:48:41 -0500 Subject: [PATCH] Continue defining constants using corresponding types. Mimic glibc when uncertain. For the new wchar.pxd: - wint_t is defined as typedef unsigned int in glibc. For stdint.pxd: - ptrdiff_t, wchar_t, and size_t are defined in stddef.pxd; - wint_t is defined in wchar.pxd; and - sig_atomic_t is defined in signal.pxd. For limits.pxd: - CHAR_BIT is always 8; and - MB_LEN_MAX is 16 in glibc. --- Cython/Includes/libc/limits.pxd | 36 ++-- Cython/Includes/libc/stdint.pxd | 32 ++-- Cython/Includes/libc/wchar.pxd |5 + 3 files changed, 41 insertions(+), 32 deletions(-) create mode 100644 Cython/Includes/libc/wchar.pxd diff --git a/Cython/Includes/libc/limits.pxd b/Cython/Includes/libc/limits.pxd index 3951dde..17cb640 100644 --- a/Cython/Includes/libc/limits.pxd +++ b/Cython/Includes/libc/limits.pxd @@ -1,29 +1,29 @@ # 5.2.4.2.1 Sizes of integer types -cdef extern from "limits.h": +cdef extern from "limits.h" nogil: enum: CHAR_BIT enum: MB_LEN_MAX -enum: CHAR_MIN -enum: CHAR_MAX +char CHAR_MIN +char CHAR_MAX -enum: SCHAR_MIN -enum: SCHAR_MAX -enum: UCHAR_MAX +signed char SCHAR_MIN +signed char SCHAR_MAX +unsigned char UCHAR_MAX -enum: SHRT_MIN -enum: SHRT_MAX -enum: USHRT_MAX +short SHRT_MIN +short SHRT_MAX +unsigned short USHRT_MAX -enum:INT_MIN -enum:INT_MAX -enum: UINT_MAX +int INT_MIN +int INT_MAX +unsigned int UINT_MAX -enum: LONG_MIN -enum: LONG_MAX -enum: ULONG_MAX +long int LONG_MIN +long int LONG_MAX +unsigned long int ULONG_MAX -enum: LLONG_MIN -enum: LLONG_MAX -enum: ULLONG_MAX +long long int LLONG_MIN +long long int LLONG_MAX +unsigned long long int ULLONG_MAX diff --git a/Cython/Includes/libc/stdint.pxd b/Cython/Includes/libc/stdint.pxd index 4d791a1..1237eb8 100644 --- a/Cython/Includes/libc/stdint.pxd +++ b/Cython/Includes/libc/stdint.pxd @@ -1,6 +1,10 @@ # Longness only used for type promotion. # Actual compile time size used for conversions. +from libc.stddef cimport ptrdiff_t, wchar_t, size_t +from libc.wchar cimport wint_t +from libc.signal cimport sig_atomic_t + # 7.18 Integer types cdef extern from "stdint.h" nogil: @@ -80,26 +84,26 @@ cdef extern from "stdint.h" nogil: uint_fast32_t UINT_FAST32_MAX uint_fast64_t UINT_FAST64_MAX #7.18.2.4 Limits of integer types capable of holding object pointers -enum: INTPTR_MIN -enum: INTPTR_MAX -enum: UINTPTR_MAX +intptr_t INTPTR_MIN +intptr_t INTPTR_MAX +uintptr_t UINTPTR_MAX # 7.18.2.5 Limits of greatest-width integer types -enum: INTMAX_MAX -enum: INTMAX_MIN -enum: UINTMAX_MAX +intmax_t INTMAX_MAX +intmax_t INTMAX_MIN +uintmax_t UINTMAX_MAX # 7.18.3 Limits of other integer types # ptrdiff_t -enum: PTRDIFF_MIN -enum: PTRDIFF_MAX +ptrdiff_t PTRDIFF_MIN +ptrdiff_t PTRDIFF_MAX # sig_atomic_t -enum: SIG_ATOMIC_MIN -enum: SIG_ATOMIC_MAX +sig_atomic_t SIG_ATOMIC_MIN +sig_atomic_t SIG_ATOMIC_MAX # size_t size_t SIZE_MAX # wchar_t -enum: WCHAR_MIN -enum: WCHAR_MAX +wchar_t WCHAR_MIN +wchar_t WCHAR_MAX # wint_t -enum: WINT_MIN -enum: WINT_MAX +wint_t WINT_MIN +wint_t WINT_MAX diff --git a/Cython/Includes/libc/wchar.pxd b/Cython/Includes/libc/wchar.pxd new file mode 100644 index 000..4c01390 --- /dev/null +++ b/Cython/Includes/libc/wchar.pxd @@ -0,0 +1,5 @@ +cdef extern from "wchar.h" nogil: + +ctypedef unsigned int wint_t + +wint_t WEOF -- 1.7.5.4 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
Chris Colbert, 19.01.2012 09:18:
> If it doesn't pass PyDict_CheckExact you won't be able to use it as the
> globals to eval or exec.
What makes you say that? I tried and it worked for me, all the way back to
Python 2.4:
Python 2.4.6 (#2, Jan 21 2010, 23:45:25)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class MyDict(dict): pass
>>> eval('1+1', MyDict())
2
>>> exec '1+1' in MyDict()
>>>
I only see a couple of calls to PyDict_CheckExact() in CPython's sources
and they usually seem to be related to special casing for performance
reasons. Nothing that should impact a module's globals.
Besides, Cython controls its own language usages of eval and exec.
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
Vitja Makarov, 19.01.2012 08:49:
> 2012/1/19 Robert Bradshaw:
>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov wrote:
>>> I tried to optimize module lookups (__pyx_m) by caching internal PyDict
>>> state.
>>>
>>> In this example bar() is 1.6 time faster (500us against 842us):
>>>
>>> C = 123
>>> def foo(a):
>>> return C * adef bar():
>>> for i in range(1):foo(i)
>>> Here is proof of
>>> concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
>>>
>>> So the question is: does it worth it?
>>
>> I think the right thing to do here is make all module-level globals
>> into "cdef public" attributes, i.e. C globals with getters and setters
>> for Python space. I'm not sure whether this would best be done by
>> creating a custom dict or module subclass, but it would probably be
>> cleaner and afford much more than a 1.6x speedup.
>
> Yes, nice idea.
> It's possible to subclass PyModuleObject and I didn't find any use of
> PyModule_CheckExact() in CPython's sources:
>
> import types
> import sys
>
> global_foo = 1234
>
> class CustomModule(types.ModuleType):
> def __init__(self, name):
> types.ModuleType.__init__(self, name)
> sys.modules[name] = self
>
> @property
> def foo(self):
> return global_foo
>
> @foo.setter
> def foo(self, value):
> global global_foo
> global_foo = value
>
> CustomModule('foo')
>
> import foo
> print foo.foo
The one thing I don't currently see is how to get the module subtype
instantiated in a safe and portable way.
The normal way to create the module in Python 2.x is a call to
Py_InitModule*(), which internally does a PyImport_AddModule(). We may get
away with creating and registering the module object before calling into
Py_InitModule*(), so that PyImport_AddModule() finds it there. At least,
the internal checks on modules seem to use PyModule_Check() and not
PyModule_CheckExact(), so someone seems to have already thought about this.
In Python 3.x, the situation is different. There is no lookup involved and
the module is always newly instantiated. That may mean that we have to copy
the module creation code into Cython. But that doesn't look like a huge
drawback (except for compatibility to potential future changes), because we
already do most of the module initialisation ourselves anyway, especially
now that we have CyFunction.
I start feeling a bit like Linus Torvalds when he broke his minix
installation and went: "ok, what else do I need to add to this terminal
emulator in order to make it an operating system?"
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
On Fri, Jan 20, 2012 at 11:58 PM, Stefan Behnel wrote:
> Chris Colbert, 19.01.2012 09:18:
> > If it doesn't pass PyDict_CheckExact you won't be able to use it as the
> > globals to eval or exec.
>
> What makes you say that? I tried and it worked for me, all the way back to
> Python 2.4:
>
>
Ah, you're right. I was mixing up issues I'd dealt with recently. The issue
with the eval/exec is that the globals must be a dict (or dict subclass)
whereas the locals can be a mapping. However, if you subclass dict for your
globals, any overridden __getitem__ or __setitem__ will not be called.
There is this line for eval/exec in ceval.c
if (!PyDict_Check(globals)) {
PyErr_SetString(PyExc_TypeError,
"exec: arg 2 must be a dictionary or None");
return -1;
}
This allows the eval loop to use PyDict_GetItem on the globals instead of
PyObject_GetItem, so no subclass method overrides will be called.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Speedup module-level lookup
On Fri, Jan 20, 2012 at 10:00 PM, Stefan Behnel wrote:
> Vitja Makarov, 19.01.2012 08:49:
>> 2012/1/19 Robert Bradshaw:
>>> On Wed, Jan 18, 2012 at 12:30 PM, Vitja Makarov wrote:
I tried to optimize module lookups (__pyx_m) by caching internal PyDict
state.
In this example bar() is 1.6 time faster (500us against 842us):
C = 123
def foo(a):
return C * adef bar():
for i in range(1): foo(i)
Here is proof of
concept:https://github.com/vitek/cython/commit/1d134fe54a74e6fc6d39d09973db499680b2a8d9
So the question is: does it worth it?
>>>
>>> I think the right thing to do here is make all module-level globals
>>> into "cdef public" attributes, i.e. C globals with getters and setters
>>> for Python space. I'm not sure whether this would best be done by
>>> creating a custom dict or module subclass, but it would probably be
>>> cleaner and afford much more than a 1.6x speedup.
>>
>> Yes, nice idea.
>> It's possible to subclass PyModuleObject and I didn't find any use of
>> PyModule_CheckExact() in CPython's sources:
>>
>> import types
>> import sys
>>
>> global_foo = 1234
>>
>> class CustomModule(types.ModuleType):
>> def __init__(self, name):
>> types.ModuleType.__init__(self, name)
>> sys.modules[name] = self
>>
>> @property
>> def foo(self):
>> return global_foo
>>
>> @foo.setter
>> def foo(self, value):
>> global global_foo
>> global_foo = value
>>
>> CustomModule('foo')
>>
>> import foo
>> print foo.foo
>
> The one thing I don't currently see is how to get the module subtype
> instantiated in a safe and portable way.
>
> The normal way to create the module in Python 2.x is a call to
> Py_InitModule*(), which internally does a PyImport_AddModule(). We may get
> away with creating and registering the module object before calling into
> Py_InitModule*(), so that PyImport_AddModule() finds it there. At least,
> the internal checks on modules seem to use PyModule_Check() and not
> PyModule_CheckExact(), so someone seems to have already thought about this.
>
> In Python 3.x, the situation is different. There is no lookup involved and
> the module is always newly instantiated. That may mean that we have to copy
> the module creation code into Cython. But that doesn't look like a huge
> drawback (except for compatibility to potential future changes), because we
> already do most of the module initialisation ourselves anyway, especially
> now that we have CyFunction.
Or swap out its ob_type pointer after it's created... It's going to be
messy unless we can directly add hooks into its __dict__ though.
> I start feeling a bit like Linus Torvalds when he broke his minix
> installation and went: "ok, what else do I need to add to this terminal
> emulator in order to make it an operating system?"
>
> Stefan
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
