Re: [Cython] Julialang
On 22 April 2012 08:10, Robert Bradshaw wrote: > Yes, Julia looks really cool. It's been on my radar for a while, but I > haven't had a chance to really try it out for anything yet. But I > hadn't thought about low-level Python/Cython <-> Julia integration. > That sounds very interesting. I wonder if Jython could give any > insight into to the tight interaction between two languages that are > usually used in isolation but have been made to call each other > (though there are a lot of differences too, e.g. we're not targeting > replacing the CPython interpreter (on first pass at least...)). > Are you all aware that "calling C" actually means a ctypes-like functionality based in dlopen()/dlsym() ? http://julialang.org/manual/calling-c-and-fortran-code/. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] test crash in Py3.2 (NumPy/memoryview/refcounting related?)
Hi, I keep seeing test crashes in Py3.2 debug builds on Jenkins with the latest master, referring to ref-counting problems. Here, for example: https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py32-ext/328/console and likewise in the series of builds starting at #312. I'm not sure what changed in that corner, it looks like one of those problems that was there for a while and suddenly starts showing when you touch that innocent looking brick at the other end of the house that was holding the balance, until now. In this case, that brick was this commit: https://github.com/cython/cython/commit/c8f61a668d0ce8af1020f520253e1b5f623cf349 I reverted it here and that fixed the problem at least temporarily: https://github.com/cython/cython/commit/5aac8caf1ba21933cc85a51af3319c78fc08d675 but it seems to be back now (after my refnanny optimisations). Before reverting my changes, I was able to reproduce it somewhat reliably on sage.math by running these three tests together (non-forking): memslice numpy_bufacc numpy_memoryview None of the tests shows the problem when run by itself. I can't tell if it's also in the latest py3k because I don't have a NumPy lying around that works there. So 3.2 is basically the latest Python version this can be tested with, and it doesn't occur in the 3.1 tests. The tests use NumPy 1.6.1, i.e. the latest official release. Mark, Dag, could any of you take a look to see if it appears in any way more obvious to you than to me? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] test crash in Py3.2 (NumPy/memoryview/refcounting related?)
On 22 April 2012 11:57, Stefan Behnel wrote: > Hi, > > I keep seeing test crashes in Py3.2 debug builds on Jenkins with the latest > master, referring to ref-counting problems. Here, for example: > > https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py32-ext/328/console > > and likewise in the series of builds starting at #312. > > I'm not sure what changed in that corner, it looks like one of those > problems that was there for a while and suddenly starts showing when you > touch that innocent looking brick at the other end of the house that was > holding the balance, until now. In this case, that brick was this commit: > > https://github.com/cython/cython/commit/c8f61a668d0ce8af1020f520253e1b5f623cf349 > > I reverted it here and that fixed the problem at least temporarily: > > https://github.com/cython/cython/commit/5aac8caf1ba21933cc85a51af3319c78fc08d675 > > but it seems to be back now (after my refnanny optimisations). Before > reverting my changes, I was able to reproduce it somewhat reliably on > sage.math by running these three tests together (non-forking): > > memslice numpy_bufacc numpy_memoryview > > None of the tests shows the problem when run by itself. I can't tell if > it's also in the latest py3k because I don't have a NumPy lying around that > works there. So 3.2 is basically the latest Python version this can be > tested with, and it doesn't occur in the 3.1 tests. The tests use NumPy > 1.6.1, i.e. the latest official release. > > Mark, Dag, could any of you take a look to see if it appears in any way > more obvious to you than to me? > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel Hm, I think you can try to disable the numpy_memoryview test and just continue development as normal. numpy_memoryview has a testcase function which inserts the test into __test__, so you could just comment out that line. The test seems to fail before it runs though? Is it possible to obtain a backtrace? ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] test crash in Py3.2 (NumPy/memoryview/refcounting related?)
mark florisson, 22.04.2012 13:54: > On 22 April 2012 11:57, Stefan Behnel wrote: >> I keep seeing test crashes in Py3.2 debug builds on Jenkins with the latest >> master, referring to ref-counting problems. Here, for example: >> >> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py32-ext/328/console >> >> and likewise in the series of builds starting at #312. >> >> I'm not sure what changed in that corner, it looks like one of those >> problems that was there for a while and suddenly starts showing when you >> touch that innocent looking brick at the other end of the house that was >> holding the balance, until now. In this case, that brick was this commit: >> >> https://github.com/cython/cython/commit/c8f61a668d0ce8af1020f520253e1b5f623cf349 >> >> I reverted it here and that fixed the problem at least temporarily: >> >> https://github.com/cython/cython/commit/5aac8caf1ba21933cc85a51af3319c78fc08d675 >> >> but it seems to be back now (after my refnanny optimisations). Before >> reverting my changes, I was able to reproduce it somewhat reliably on >> sage.math by running these three tests together (non-forking): >> >>memslice numpy_bufacc numpy_memoryview >> >> None of the tests shows the problem when run by itself. I can't tell if >> it's also in the latest py3k because I don't have a NumPy lying around that >> works there. So 3.2 is basically the latest Python version this can be >> tested with, and it doesn't occur in the 3.1 tests. The tests use NumPy >> 1.6.1, i.e. the latest official release. >> >> Mark, Dag, could any of you take a look to see if it appears in any way >> more obvious to you than to me? > > Hm, I think you can try to disable the numpy_memoryview test and just > continue development as normal. numpy_memoryview has a testcase > function which inserts the test into __test__, so you could just > comment out that line. ... as long as we remember to put it back in ;) > The test seems to fail before it runs though? Is it possible to obtain > a backtrace? When I reproduced it in the Jenking workspace, it crashed while trying to clean up objects to free memory, specifically in the deallocation visitor function of one of the memory views of numpy_memoryview (IIRC). That didn't really tell me where that object was created or what happened to it to get it to crash. Needs some more investigation. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] test crash in Py3.2 (NumPy/memoryview/refcounting related?)
On 22 April 2012 13:34, Stefan Behnel wrote: > mark florisson, 22.04.2012 13:54: >> On 22 April 2012 11:57, Stefan Behnel wrote: >>> I keep seeing test crashes in Py3.2 debug builds on Jenkins with the latest >>> master, referring to ref-counting problems. Here, for example: >>> >>> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py32-ext/328/console >>> >>> and likewise in the series of builds starting at #312. >>> >>> I'm not sure what changed in that corner, it looks like one of those >>> problems that was there for a while and suddenly starts showing when you >>> touch that innocent looking brick at the other end of the house that was >>> holding the balance, until now. In this case, that brick was this commit: >>> >>> https://github.com/cython/cython/commit/c8f61a668d0ce8af1020f520253e1b5f623cf349 >>> >>> I reverted it here and that fixed the problem at least temporarily: >>> >>> https://github.com/cython/cython/commit/5aac8caf1ba21933cc85a51af3319c78fc08d675 >>> >>> but it seems to be back now (after my refnanny optimisations). Before >>> reverting my changes, I was able to reproduce it somewhat reliably on >>> sage.math by running these three tests together (non-forking): >>> >>> memslice numpy_bufacc numpy_memoryview >>> >>> None of the tests shows the problem when run by itself. I can't tell if >>> it's also in the latest py3k because I don't have a NumPy lying around that >>> works there. So 3.2 is basically the latest Python version this can be >>> tested with, and it doesn't occur in the 3.1 tests. The tests use NumPy >>> 1.6.1, i.e. the latest official release. >>> >>> Mark, Dag, could any of you take a look to see if it appears in any way >>> more obvious to you than to me? >> >> Hm, I think you can try to disable the numpy_memoryview test and just >> continue development as normal. numpy_memoryview has a testcase >> function which inserts the test into __test__, so you could just >> comment out that line. > > ... as long as we remember to put it back in ;) > > >> The test seems to fail before it runs though? Is it possible to obtain >> a backtrace? > > When I reproduced it in the Jenking workspace, it crashed while trying to > clean up objects to free memory, specifically in the deallocation visitor > function of one of the memory views of numpy_memoryview (IIRC). That didn't > really tell me where that object was created or what happened to it to get > it to crash. Needs some more investigation. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel Hm, I can't reproduce the issue, but that assertion triggers only for update_refs and subtract_refs (when the gc is trying to determine which object may be potentially unreachable). I think the problem here is that a single memoryview object is traversed multiple times through different traverse functions, and that the refcount doesn't match the number of traverses. Indeed, the refcount is only one, as the actual count is the acquisition count. So we shouldn't traverse the memoryview objects in memoryview slices, i.e. not _memoryviewslice.from_slice.memview. I'll come up with a commit shortly, would you be willing to test it? ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] test crash in Py3.2 (NumPy/memoryview/refcounting related?)
On 22 April 2012 15:31, mark florisson wrote: > On 22 April 2012 13:34, Stefan Behnel wrote: >> mark florisson, 22.04.2012 13:54: >>> On 22 April 2012 11:57, Stefan Behnel wrote: I keep seeing test crashes in Py3.2 debug builds on Jenkins with the latest master, referring to ref-counting problems. Here, for example: https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py32-ext/328/console and likewise in the series of builds starting at #312. I'm not sure what changed in that corner, it looks like one of those problems that was there for a while and suddenly starts showing when you touch that innocent looking brick at the other end of the house that was holding the balance, until now. In this case, that brick was this commit: https://github.com/cython/cython/commit/c8f61a668d0ce8af1020f520253e1b5f623cf349 I reverted it here and that fixed the problem at least temporarily: https://github.com/cython/cython/commit/5aac8caf1ba21933cc85a51af3319c78fc08d675 but it seems to be back now (after my refnanny optimisations). Before reverting my changes, I was able to reproduce it somewhat reliably on sage.math by running these three tests together (non-forking): memslice numpy_bufacc numpy_memoryview None of the tests shows the problem when run by itself. I can't tell if it's also in the latest py3k because I don't have a NumPy lying around that works there. So 3.2 is basically the latest Python version this can be tested with, and it doesn't occur in the 3.1 tests. The tests use NumPy 1.6.1, i.e. the latest official release. Mark, Dag, could any of you take a look to see if it appears in any way more obvious to you than to me? >>> >>> Hm, I think you can try to disable the numpy_memoryview test and just >>> continue development as normal. numpy_memoryview has a testcase >>> function which inserts the test into __test__, so you could just >>> comment out that line. >> >> ... as long as we remember to put it back in ;) >> >> >>> The test seems to fail before it runs though? Is it possible to obtain >>> a backtrace? >> >> When I reproduced it in the Jenking workspace, it crashed while trying to >> clean up objects to free memory, specifically in the deallocation visitor >> function of one of the memory views of numpy_memoryview (IIRC). That didn't >> really tell me where that object was created or what happened to it to get >> it to crash. Needs some more investigation. >> >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > > Hm, I can't reproduce the issue, but that assertion triggers only for > update_refs and subtract_refs (when the gc is trying to determine > which object may be potentially unreachable). I think the problem here > is that a single memoryview object is traversed multiple times through > different traverse functions, and that the refcount doesn't match the > number of traverses. Indeed, the refcount is only one, as the actual > count is the acquisition count. So we shouldn't traverse the > memoryview objects in memoryview slices, i.e. not > _memoryviewslice.from_slice.memview. I'll come up with a commit > shortly, would you be willing to test it? A fix is here: https://github.com/markflorisson88/cython/commit/cd32184f3f782b6d7275cf430694b59801ce642a Lets see what jenkins has to say :) BTW, tp_clear calls Py_CLEAR on Py_buffer.obj, shouldn't it call releasebuffer instead? ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Cython 0.16: "eval" problem
With Cython 0.15, the following works with Python 2.7:
### start file: prob.pyx
def f(x):
cdef int* p
return eval(x)
### end file
>>> import pyximport; pyximport.install()
>>> import prob
>>> prob.f("5")
5
but with Cython 0.16 it doesn't even compile:
>>> import prob
Error compiling Cython file:
...
def f(x):
cdef int* p
return eval(x)
^
prob.pyx:3:15: Cannot convert 'int *' to Python object
If I comment out the (unused) line "cdef int* p" then it works with Cython
0.16. The issue is the pointer declaration; something like:
def f(x):
cdef int p
p = eval(x)
return p*p
works fine with Cython 0.16.
Thanks,
Nathan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
On Sat, Apr 21, 2012 at 2:48 PM, Stefan Behnel wrote: > Dimitri Tcaciuc, 21.04.2012 21:17: >> On a somewhat relevant node, have you considered enabling Issues page on >> Github? > > It was discussed, but the drawback of having two separate bug trackers is > non-negligible. Ok. I was wondering since it would make it much easier to connect issue/patch/discussion together without, say, me needlessly adding to the development mailing list and/or manually registering for trac and sending htpasswd digest over the mail. Here's something to consider if you ever want to migrate over from trac: https://github.com/adamcik/github-trac-ticket-import Cheers, Dimitri. > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16: "eval" problem
2012/4/22 Nathan Dunfield :
> With Cython 0.15, the following works with Python 2.7:
>
> ### start file: prob.pyx
>
> def f(x):
> cdef int* p
> return eval(x)
>
> ### end file
>
import pyximport; pyximport.install()
import prob
prob.f("5")
> 5
>
> but with Cython 0.16 it doesn't even compile:
>
import prob
>
> Error compiling Cython file:
>
> ...
> def f(x):
> cdef int* p
> return eval(x)
> ^
>
>
> prob.pyx:3:15: Cannot convert 'int *' to Python object
>
> If I comment out the (unused) line "cdef int* p" then it works with Cython
> 0.16. The issue is the pointer declaration; something like:
>
> def f(x):
> cdef int p
> p = eval(x)
> return p*p
>
> works fine with Cython 0.16.
>
> Thanks,
>
> Nathan
>
Oops, it seems to be a problem with locals() dict creation. Perhaps it
should ignore variables that can't be converted to PyObject
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16: "eval" problem
On Apr 22, 2012, at 1:22 PM, Vitja Makarov wrote:
> Oops, it seems to be a problem with locals() dict creation.
Yes it does. The following variants of my original example both work:
## prob.pyx version 1
def cy_eval(s):
return eval(s)
def f(x):
cdef int* p
return cy_eval(x)
## prob.pyx version 2
def f(x):
cdef int* p
return eval(x, {})
Best,
Nathan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16: "eval" problem
2012/4/22 Nathan Dunfield :
> On Apr 22, 2012, at 1:22 PM, Vitja Makarov wrote:
>> Oops, it seems to be a problem with locals() dict creation.
>
> Yes it does. The following variants of my original example both work:
>
> ## prob.pyx version 1
>
> def cy_eval(s):
> return eval(s)
>
> def f(x):
> cdef int* p
> return cy_eval(x)
>
> ## prob.pyx version 2
>
> def f(x):
> cdef int* p
> return eval(x, {})
>
> Best,
>
> Nathan
>
I've fixed it here:
https://github.com/vitek/cython/commit/6dc132731b8f3f7eaabf55e51d89bcbc7b8f4eb7
Now waiting for jenkins, then I'll push it into upstream. As a
workaround you can manually pass locals dictionary, e.g.:
eval(x, None, {'a: a})
--
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
On 21 April 2012 20:17, Dimitri Tcaciuc wrote:
> Hey everyone,
>
> Congratulations on shipping 0.16! I think I found a problem which
> seems pretty straight forward. Say I want to factor out inner part of
> some N^2 loops over a flow array, I write something like
>
> cdef inline float _inner(size_t i, size_t j, float[:] x):
> cdef float d = x[i] - x[j]
> return sqrtf(d * d)
>
> In 0.16, this actually compiles (as opposed to 0.15 with ndarray) and
> function is declared as inline, which is great. However, the
> memoryview structure is passed by value:
>
> static CYTHON_INLINE float __pyx_f_3foo__inner(size_t __pyx_v_i,
> size_t __pyx_v_j, __Pyx_memviewslice __pyx_v_x) {
> ...
>
> This seems to hinder compiler's (in my case, GCC 4.3.4) ability to
> perform efficient inlining (although function does in fact get
> inlined). If I manually inline that distance calculation, I get 3x
> speedup. (in my case 0.324020147324 vs 1.43209195137 seconds for 10k
> elements). When I manually modified generated .c file to pass memory
> view slice by pointer, slowdown was eliminated completely.
>
> On a somewhat relevant node, have you considered enabling Issues page on
> Github?
>
>
> Thanks!
>
>
> Dimitri.
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
Although it is neither documented nor tested, it works if you just
take the address of the memoryview. You can then index it using
memoryview_pointer[0][i]. One should be careful, as taking the pointer
and passing that around means that pointer is not acquisition counted,
and will point to invalid memory if the memoryview goes out of scope
(e.g. if it's a local variable, when you return).
Cython could manually inline functions though, which could greatly
reduce argument passing and unpacking overhead in some situations
(like buffers).
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
On 22 April 2012 19:14, Dimitri Tcaciuc wrote: > On Sat, Apr 21, 2012 at 2:48 PM, Stefan Behnel wrote: >> Dimitri Tcaciuc, 21.04.2012 21:17: >>> On a somewhat relevant node, have you considered enabling Issues page on >>> Github? >> >> It was discussed, but the drawback of having two separate bug trackers is >> non-negligible. > > Ok. I was wondering since it would make it much easier to connect > issue/patch/discussion together without, say, me needlessly adding to > the development mailing list and/or manually registering for trac and > sending htpasswd digest over the mail. Here's something to consider if > you ever want to migrate over from trac: > https://github.com/adamcik/github-trac-ticket-import > > Cheers, > > Dimitri. I haven't heard very good things about github issues, but I like to have everything in one place, and I'm not too fond of trac in any regard. It's also quite a barrier to get trac access, so I'd be in favour of moving tickets. >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
On Sun, Apr 22, 2012 at 1:20 PM, mark florisson
wrote:
> On 21 April 2012 20:17, Dimitri Tcaciuc wrote:
>> Hey everyone,
>>
>> Congratulations on shipping 0.16! I think I found a problem which
>> seems pretty straight forward. Say I want to factor out inner part of
>> some N^2 loops over a flow array, I write something like
>>
>> cdef inline float _inner(size_t i, size_t j, float[:] x):
>> cdef float d = x[i] - x[j]
>> return sqrtf(d * d)
>>
>> In 0.16, this actually compiles (as opposed to 0.15 with ndarray) and
>> function is declared as inline, which is great. However, the
>> memoryview structure is passed by value:
>>
>> static CYTHON_INLINE float __pyx_f_3foo__inner(size_t __pyx_v_i,
>> size_t __pyx_v_j, __Pyx_memviewslice __pyx_v_x) {
>> ...
>>
>> This seems to hinder compiler's (in my case, GCC 4.3.4) ability to
>> perform efficient inlining (although function does in fact get
>> inlined). If I manually inline that distance calculation, I get 3x
>> speedup. (in my case 0.324020147324 vs 1.43209195137 seconds for 10k
>> elements). When I manually modified generated .c file to pass memory
>> view slice by pointer, slowdown was eliminated completely.
>>
>> On a somewhat relevant node, have you considered enabling Issues page on
>> Github?
>>
>>
>> Thanks!
>>
>>
>> Dimitri.
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
> Although it is neither documented nor tested, it works if you just
> take the address of the memoryview. You can then index it using
> memoryview_pointer[0][i]. One should be careful, as taking the pointer
> and passing that around means that pointer is not acquisition counted,
> and will point to invalid memory if the memoryview goes out of scope
> (e.g. if it's a local variable, when you return).
Nice, passing by pointer did the trick! As an observation, I tried
using `cython.operator.dereference(x)` and in this case it's way less
efficient than `x[0]`. Dereferencing actually allocates an empty
memory view slice and copies the contents of `x`, even if the
`dereference(x)` result is never assigned anywhere and is only a
temporary value in the expression.
Dimitri.
> Cython could manually inline functions though, which could greatly
> reduce argument passing and unpacking overhead in some situations
> (like buffers).
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
mark florisson, 22.04.2012 22:21: > On 22 April 2012 19:14, Dimitri Tcaciuc wrote: >> On Sat, Apr 21, 2012 at 2:48 PM, Stefan Behnel wrote: >>> Dimitri Tcaciuc, 21.04.2012 21:17: On a somewhat relevant node, have you considered enabling Issues page on Github? >>> >>> It was discussed, but the drawback of having two separate bug trackers is >>> non-negligible. >> >> Ok. I was wondering since it would make it much easier to connect >> issue/patch/discussion together without, say, me needlessly adding to >> the development mailing list and/or manually registering for trac and >> sending htpasswd digest over the mail. Here's something to consider if >> you ever want to migrate over from trac: >> https://github.com/adamcik/github-trac-ticket-import > > I haven't heard very good things about github issues I find them nicely accessible from user side, but hardly usable for the developers. All you get is basically a blog style comment system with bare tag support. Sure, you can build many of the necessary features on top of tags, but trac (or any other real issue tracker) already provides a lot more. Pull request tracking works well in github, but I consider their general issue tracker a last resort if you don't have anything else. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
mark florisson, 22.04.2012 22:20:
> On 21 April 2012 20:17, Dimitri Tcaciuc wrote:
>> Say I want to factor out inner part of
>> some N^2 loops over a flow array, I write something like
>>
>> cdef inline float _inner(size_t i, size_t j, float[:] x):
>> cdef float d = x[i] - x[j]
>> return sqrtf(d * d)
>>
>> In 0.16, this actually compiles (as opposed to 0.15 with ndarray) and
>> function is declared as inline, which is great. However, the
>> memoryview structure is passed by value:
>>
>> static CYTHON_INLINE float __pyx_f_3foo__inner(size_t __pyx_v_i,
>> size_t __pyx_v_j, __Pyx_memviewslice __pyx_v_x) {
>> ...
>>
>> This seems to hinder compiler's (in my case, GCC 4.3.4) ability to
>> perform efficient inlining (although function does in fact get
>> inlined). If I manually inline that distance calculation, I get 3x
>> speedup. (in my case 0.324020147324 vs 1.43209195137 seconds for 10k
>> elements). When I manually modified generated .c file to pass memory
>> view slice by pointer, slowdown was eliminated completely.
>
> Although it is neither documented nor tested, it works if you just
> take the address of the memoryview. You can then index it using
> memoryview_pointer[0][i].
Are you advertising this an an actual feature here? I'm just asking because
supporting hacks can be nasty in the long run. What if we ever want to make
a change to the internal way memoryviews work that would break this?
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
