> On 04/10/2012 02:11 AM, Travis Oliphant wrote:
>> Hi all,
>>
>> Some of you are aware of Numba. Numba allows you to create the equivalent
>> of C-function's dynamically from Python. One purpose of this system is to
>> allow NumPy to take these functions and use them in operations like ufuncs,
>> generalized ufuncs, file-reading, fancy-indexing, and so forth. There are
>> actually many use-cases that one can imagine for such things.
>>
>> One question is how do you pass this function pointer to the C-side. On
>> the Python side, Numba allows you to get the raw integer address of the
>> equivalent C-function pointer that it just created out of the Python code.
>> One can think of this as a 32- or 64-bit integer that you can cast to a
>> C-function pointer.
>>
>> Now, how should this C-function pointer be passed from Python to NumPy?
>> One approach is just to pass it as an integer --- in other words have an API
>> in C that accepts an integer as the first argument that the internal
>> function interprets as a C-function pointer.
>>
>> This is essentially what ctypes does when creating a ctypes function pointer
>> out of:
>>
>> func = ctypes.CFUNCTYPE(restype, *argtypes)(integer)
>>
>> Of course the problem with this is that you can easily hand it integers
>> which don't make sense and which will cause a segfault when control is
>> passed to this "function"
>>
>> We could also piggy-back on-top of Ctypes and assume that a ctypes
>> function-pointer object is passed in. This allows some error-checking at
>> least and also has the benefit that one could use ctypes to access a
>> c-function library where these functions were defined. I'm leaning towards
>> this approach.
>>
>> Now, the issue is how to get the C-function pointer (that npy_intp integer)
>> back and hand it off internally. Unfortunately, ctypes does not make it
>> very easy to get this address (that I can see). There is no ctypes C-API,
>> for example. There are two potential options:
>>
>> 1) Create an API for such Ctypes function pointers in NumPy and use the
>> ctypes object structure. If ctypes were to ever change it's object
>> structure we would have to adapt this API.
>>
>> Something like this is what is envisioned here:
>>
>> typedef struct {
>> PyObject_HEAD
>> char *b_ptr;
>> } _cfuncptr_object;
>>
>> then the function pointer is:
>>
>> (*((void **)(((_sp_cfuncptr_object *)(obj))->b_ptr)))
>>
>> which could be wrapped-up into a nice little NumPy C-API call like
>>
>> void * Npy_ctypes_funcptr(obj)
>>
>>
>> 2) Use the Python API of ctypes to do the same thing. This has the
>> advantage of not needing to mirror the simple _cfuncptr_object structure in
>> NumPy but it is *much* slower to get the address. It basically does the
>> equivalent of
>>
>> ctypes.cast(obj, ctypes.c_void_p).value
>>
>>
>> There is working code for this in the ctypes_callback branch of my
>> scipy fork on github.
>>
>>
>> I would like to propose two things:
>>
>> * creating a Npy_ctypes_funcptr(obj) function in the C-API of NumPy and
>> * implement it with the simple pointer dereference above (option #1)
>>
>>
>> Thoughts?
>
> I really hope we can find some project-neutral common ground, so that lots of
> tools (Cython, f2py, numba, C extensions in NumPy and SciPy) can agree on how
> to "unbox callables".
>
> A new extension type in NumPy would not fit this bill I feel. I've created a
> specification for this; if a number of projects (the ones mentioned above)
> agree on this or something similar and implement support, we could propose a
> PEP and do it properly once it has proven itself.
>
> http://wiki.cython.org/enhancements/cep1000
>
> In Cython, this may take the form
>
> def call_callback(object func):
> cdef double (*typed_func)(int)
> typed_func = func
> return typed_func(4)
>
> ...it would be awesome if passing a Numba-compiled function just worked in
> this example.
Yes, I think we should go the Python PEP route. However, it will take some
time to see that to completion (especially with ctypes already in existence).
Dag, this would be a very good thing for you to champion however ;-)
In the mean-time, I think we could do as Robert essentially suggested and just
use Capsule Objects around an agreed-upon simple C-structure:
int id /* Some number that can be used as a "type-check" */
void *func;
char *string;
We can then just create some nice functions to go to and from this form in
NumPy ctypeslib and then use this while the Python PEP gets written and adopted.
-Travis
>
> Dag
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
