[Cython] Cannot assign type 'set &' to 'set'
Hello everyone,
Here's a small test case I'm trying to compile. I'm trying to pass a
STL set reference to a method in a template class.
x.pyx:
from libcpp.set cimport set as cpp_set
cdef extern from "x.hh":
cdef cppclass Foo [T]:
Foo()
void set_x(cpp_set[size_t] & x)
cpdef func():
cdef Foo[int] foo
cdef cpp_set[size_t] x
cdef cpp_set[size_t] & xref = x
foo.set_x(xref)
x.hh:
#include
template
struct Foo {
void set_x(const std::set & x) { /* do nothing */ }
};
To compile,
bash $ cython --cplus x.pyx
Which results in
foo.set_x(xref)
^
x.pyx:15:18: Cannot assign type 'set &' to 'set'
However, if I remove the template parameter from Foo, everything works.
y.pyx:
from libcpp.set cimport set as cpp_set
cdef extern from "y.hh":
cdef cppclass Foo:
Foo()
void set_x(cpp_set[size_t] & x)
cpdef func():
cdef Foo foo
cdef cpp_set[size_t] x
cdef cpp_set[size_t] & xref = x
foo.set_x(xref)
y.hh:
#include
struct Foo {
void set_x(const std::set & x) { /* do nothing */ }
};
>From what I can tell, the CppClassType instance the CReferenceType is
pointing to has the correct name "set", however it's a
different class instance. The particular failing expression is in
`ExprNode.coerce_to`
if not (str(src.type) == str(dst_type) or
dst_type.assignable_from(src_type))
I wish I could suggest a patch, but unfortunately I'm a complete
newbie to Cython internals. Perhaps someone could give a few pointers
as to what should be done to fix this?
Thanks,
Dimitri
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cannot assign type 'set &' to 'set'
Hi folks,
Since the original question, I've created a pull request with a
failing test and tried to get some discussion going, but so far no
answer. I'm a tad discouraged, since obviously there's movement on
mail list and with pull requests. Is a pull request a proper way to do
this? I completely understand if you guys don't have enough available
time to deal with it right now, however at least some acknowledgement
and feedback would be much appreciated.
Thanks,
Dimitri.
On Sun, Dec 18, 2011 at 8:17 PM, Dimitri Tcaciuc wrote:
> Hello everyone,
>
> Here's a small test case I'm trying to compile. I'm trying to pass a
> STL set reference to a method in a template class.
>
> x.pyx:
>
> from libcpp.set cimport set as cpp_set
>
> cdef extern from "x.hh":
>
> cdef cppclass Foo [T]:
> Foo()
> void set_x(cpp_set[size_t] & x)
>
> cpdef func():
> cdef Foo[int] foo
>
> cdef cpp_set[size_t] x
> cdef cpp_set[size_t] & xref = x
>
> foo.set_x(xref)
>
> x.hh:
>
> #include
>
> template
> struct Foo {
> void set_x(const std::set & x) { /* do nothing */ }
> };
>
> To compile,
>
> bash $ cython --cplus x.pyx
>
> Which results in
>
> foo.set_x(xref)
> ^
>
> x.pyx:15:18: Cannot assign type 'set &' to 'set'
>
>
> However, if I remove the template parameter from Foo, everything works.
>
>
> y.pyx:
>
> from libcpp.set cimport set as cpp_set
>
> cdef extern from "y.hh":
>
> cdef cppclass Foo:
> Foo()
> void set_x(cpp_set[size_t] & x)
>
> cpdef func():
> cdef Foo foo
>
> cdef cpp_set[size_t] x
> cdef cpp_set[size_t] & xref = x
>
> foo.set_x(xref)
>
> y.hh:
>
> #include
>
> struct Foo {
> void set_x(const std::set & x) { /* do nothing */ }
> };
>
>
> From what I can tell, the CppClassType instance the CReferenceType is
> pointing to has the correct name "set", however it's a
> different class instance. The particular failing expression is in
> `ExprNode.coerce_to`
>
> if not (str(src.type) == str(dst_type) or
> dst_type.assignable_from(src_type))
>
>
> I wish I could suggest a patch, but unfortunately I'm a complete
> newbie to Cython internals. Perhaps someone could give a few pointers
> as to what should be done to fix this?
>
> Thanks,
>
>
> Dimitri
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] distutils extension pxd problem
Hey everyone,
I bumped into an issue where my .pyx file doesn't see its matching
.pxd file. Here's a build test to show the problem If I change my
target package from `b.a` to just `a`, it works as expected. Running
`cython src/a.pyx` works as expected as well, but not the Extension.
PYTHON setup.py build_ext --inplace
PYTHON -c "from b import a"
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension("b.a", ["src/a.pyx"])
]
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
b/__init__.py
src/a.pxd
cdef class X:
cdef object foo
src/a.pyx
cdef class X:
def __cinit__(self):
self.foo = 1
x = X()
Traceback (most recent call last):
File "", line 1, in
File "a.pyx", line 7, in init b.a (src/a.c:793)
File "a.pyx", line 5, in b.a.X.__cinit__ (src/a.c:488)
AttributeError: 'b.a.X' object has no attribute 'foo'
Any idea what's going on here?
Thanks,
Dimitri.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] distutils extension pxd problem
Ok, so I narrowed the problem down to
https://github.com/cython/cython/blob/master/Cython/Compiler/Main.py#L223.
At this point, it looks like if target extension name is `x.y.z`, the
pxd must either be called `x.y.z.pxd` and be located in project root
(I believe this is Pyrex convention?) or be in the exact x/y/z.pxd
directory structure, and each of the parents have to be a package (ie.
contain __init__.[py,pyx,pyd]).
Again, this looks like a problem only if module name is nested. If
this is along the right lines, I'd be happy to make some
clarifications to
http://docs.cython.org/src/userguide/sharing_declarations.html#search-paths-for-definition-files.
It looks like there's a conflict between the Extension name parameter
(which also says where the module gets installed in package tree) and
the name of the actual .so file, which sometimes one needs to
customize (eg. I need to compile x/y/z.pyx to x/y/_z.so since I'd like
to have z.py with some extra bits in them. In this case `cythonize`
seems to ignore the extension name, goes ahead and names the output
`z.so`)
I recon you guys probably had plenty of discussions on this topic. Is
there a general direction where you're taking the whole include/pxd
discovery system or is it staying where it is right now?
Dimitri.
On Wed, Feb 1, 2012 at 4:53 PM, Dimitri Tcaciuc wrote:
> Hey everyone,
>
> I bumped into an issue where my .pyx file doesn't see its matching
> .pxd file. Here's a build test to show the problem If I change my
> target package from `b.a` to just `a`, it works as expected. Running
> `cython src/a.pyx` works as expected as well, but not the Extension.
>
>
>
> PYTHON setup.py build_ext --inplace
> PYTHON -c "from b import a"
>
> setup.py
>
> from distutils.core import setup
> from distutils.extension import Extension
> from Cython.Distutils import build_ext
>
> ext_modules = [
> Extension("b.a", ["src/a.pyx"])
> ]
>
> setup(
> cmdclass = {'build_ext': build_ext},
> ext_modules = ext_modules
> )
>
> b/__init__.py
>
> src/a.pxd
>
> cdef class X:
> cdef object foo
>
> src/a.pyx
>
> cdef class X:
>
> def __cinit__(self):
> self.foo = 1
>
> x = X()
>
>
>
> Traceback (most recent call last):
> File "", line 1, in
> File "a.pyx", line 7, in init b.a (src/a.c:793)
> File "a.pyx", line 5, in b.a.X.__cinit__ (src/a.c:488)
> AttributeError: 'b.a.X' object has no attribute 'foo'
>
>
>
> Any idea what's going on here?
>
> Thanks,
>
>
> Dimitri.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] distutils extension pxd problem
> Date: Thu, 02 Feb 2012 08:11:16 +0100
> From: Stefan Behnel
> To: Core developer mailing list of the Cython compiler
>
> Subject: Re: [Cython] distutils extension pxd problem
> Message-ID: <4f2a3714.7000...@behnel.de>
> Content-Type: text/plain; charset=UTF-8
>
> Dimitri Tcaciuc, 02.02.2012 01:53:
>> I bumped into an issue where my .pyx file doesn't see its matching
>> .pxd file. Here's a build test to show the problem If I change my
>> target package from `b.a` to just `a`, it works as expected. Running
>> `cython src/a.pyx` works as expected as well, but not the Extension.
>>
>>
>>
>> PYTHON setup.py build_ext --inplace
>> PYTHON -c "from b import a"
>>
>> setup.py
>>
>> from distutils.core import setup
>> from distutils.extension import Extension
>> from Cython.Distutils import build_ext
>>
>> ext_modules = [
>> Extension("b.a", ["src/a.pyx"])
>> ]
>>
>> setup(
>> cmdclass = {'build_ext': build_ext},
>> ext_modules = ext_modules
>> )
>>
>> b/__init__.py
>>
>> src/a.pxd
>>
>> cdef class X:
>> cdef object foo
>>
>> src/a.pyx
>>
>> cdef class X:
>>
>> def __cinit__(self):
>> self.foo = 1
>>
>> x = X()
>>
>>
>>
>> Traceback (most recent call last):
>> File "", line 1, in
>> File "a.pyx", line 7, in init b.a (src/a.c:793)
>> File "a.pyx", line 5, in b.a.X.__cinit__ (src/a.c:488)
>> AttributeError: 'b.a.X' object has no attribute 'foo'
>>
>>
>
> Any reason you cannot rename "src" to "b"? Because that would fix your
> problem. Cython basically uses the same algorithm for finding package
> content as Python, i.e. it will look inside the package "b" when looking
> for "b.a". And "a.pxd" is not in "b" in your setup.
>
> Stefan
>
This certainly looks like the cleanest solution now. I have some mixed
C++ bits in src/ so I thought to keep compiled/interpreted bits
separate, if possible.
Dimitri.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OpenCL support
Mark, Couple of thoughts based on some experience with OpenCL... 1. This may be going outside the proposed purpose, but some algorithms such as molecular simulations can benefit from a fairly large amount of constant data loaded at the beginning of the program and persisted in between invocations of a function. If I understand the proposal, entire program would need to be within one `with` block, which would certainly be limiting to the architecture. Eg. # run.py from cython_module import Evaluator # Arrays are loaded into device memory here x = Evaluator(params...) for i in range(N): # Calculations are performed with # mostly data in the device memory data_i = x.step() ... 2. AFAIK, given a device, OpenCL basically takes it over (which would be eg. 8 cores on 2 CPU x 4 cores machine), so I'm not sure how `num_cores` parameter would work here. There's the fission extension that allows you to selectively run on a portion of the device, but the idea is that you're still dedicating entire device to your process, but merely giving more organization to your processing tasks, where you have to specify the core numbers you want to use. I may very well be wrong here, bashing is welcome :) 3. Does it make sense to make OpenCL more explicit? Heuristics and automatic switching between, say, CPU and GPU is great for eg. Sage users, but maybe not so much if you know exactly what you're doing with your machine resources. E.g just having a library with thin cython-adapted wrappers would be awesome. I imagine this can be augmented by arrays having a knowledge of device-side/client-side (which would go towards addressing the issue 1. above) Cheers, Dimitri. On Sun, Feb 5, 2012 at 1:57 PM, mark florisson wrote: > Hey, > > I created a CEP for opencl support: http://wiki.cython.org/enhancements/opencl > What do you think? > > Mark > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OpenCL support
On Mon, Feb 6, 2012 at 2:21 AM, mark florisson wrote: > On 6 February 2012 07:22, Stefan Behnel wrote: >> mark florisson, 06.02.2012 00:12: >>> On 5 February 2012 22:39, Dimitri Tcaciuc wrote: >>>> 3. Does it make sense to make OpenCL more explicit? Heuristics and >>>> automatic switching between, say, CPU and GPU is great for eg. Sage >>>> users, but maybe not so much if you know exactly what you're doing >>>> with your machine resources. E.g just having a library with thin >>>> cython-adapted wrappers would be awesome. I imagine this can be >>>> augmented by arrays having a knowledge of device-side/client-side >>>> (which would go towards addressing the issue 1. above) >>> >>> Hm, there are several advantages to supporting this in the language. >> >> ... and there's always the obvious disadvantage of making the language too >> complex and magic to learn and understand. Worth balancing. > > Definitely. This would however introduce very minor changes to the > language (no new syntax at least, just a few memoryview methods), but > more major changes to the compiler. The support would mostly be > transparent. > Clyther (http://srossross.github.com/Clyther/) is a related project, > which does a similar thing by compiling python (bytecode) to opencl. > What I want for Cython is something even more transparent, the user > wouldn't perhaps even know opencl was involved, and the compiler has > more control over how data is handled. What I'm absolutely certain of is that sort of complete transparency will eventually start getting edge cases and from there on additional development and design will have to be made, so I it's better to plan not-as-transparent elements and user-side control right form the start. I think another reason I would go for a less automatic solution is because I imagine the alternative would inevitably complicate Cython internals. I think keeping that as simple as possible is huge advantage in the long run, which is arguably as important as reducing amount of code a language user has to write (hence me initially suggesting a more library-like Cython integration, although pyopencl did work quite well already :). Dimitri. > >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OpenCL support
On Tue, Feb 7, 2012 at 5:52 AM, Sturla Molden wrote: > On 05.02.2012 23:39, Dimitri Tcaciuc wrote: > >> 3. Does it make sense to make OpenCL more explicit? > > > No, it takes the usefuness of OpenCL away, which is that kernels are text > strings and compiled at run-time. I'm not sure I understand you, maybe you could elaborate on that? By "explicit" I merely meant that the user will explicitly specify that they're working on OpenCL-enabled array or certain bit of Cython code will get compiled into OpenCL program etc. > >> Heuristics and >> automatic switching between, say, CPU and GPU is great for eg. Sage >> users, but maybe not so much if you know exactly what you're doing >> with your machine resources. E.g just having a library with thin >> cython-adapted wrappers would be awesome. I imagine this can be >> augmented by arrays having a knowledge of device-side/client-side >> (which would go towards addressing the issue 1. above) > > > Just use PyOpenCL and manipulate kernels as text. Python is excellent for > that - Cython is not needed. If you think using Cython instead of Python > (PyOpenCL and NumPy) will be important, you don't have a CPU bound problem > that warrants the use of OpenCL. Again, not sure what you mean here. As I mentioned in the thread, PyOpenCL worked quite fine, however if Cython is getting OpenCL support, I'd much rather use that than keeping a dependency on another library. > Sturla > > > > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OpenCL support
On Wed, Feb 8, 2012 at 6:46 AM, Dag Sverre Seljebotn wrote: > On 02/05/2012 10:57 PM, mark florisson wrote: > > I don't really know how good the Intel and AMD CPU drivers are w.r.t. this > -- I have seen the Intel driver emit "vectorizing" and "could not > vectorize", but didn't explore the circumstances. For our project, we've tried both Intel and AMD (previously ATI) backends. The AMD experience somewhat mirrors what this developer described (http://www.msoos.org/2012/01/amds-opencl-heaven-and-hell/), although not as bad in terms of silent failures (or maybe I just havent caught any!). Intel backend was great and clearly better in terms of performance, sometimes by about 20-30%. However, when ran on older AMD-based machine as opposed to Intel one, the resulting kernel simply segfaulted without any warning about an unsupported architecture (I think its because it didn't have SSE3 support). > > Dag Sverre > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel I know Intel is working with LLVM/Clang folks to introduce their vectorization additions, at least to some degree, and LLVM seems to be consistently improving in this regard (eg http://blog.llvm.org/2011/12/llvm-31-vector-changes.html). I suppose if Cython emitted vectorization-friendly numerical loops, then appropriate C/C++ compiler should take care of this automatically, if used. Intel C++ can already do certain stuff like that (see http://software.intel.com/en-us/articles/a-guide-to-auto-vectorization-with-intel-c-compilers/), and GCC as well AFAIK. Dimitri. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.16 and ndarray fields deprecation
On Fri, Mar 2, 2012 at 8:29 AM, mark florisson wrote: > On 1 March 2012 16:18, Dag Sverre Seljebotn > wrote: >> On 03/01/2012 04:03 AM, mark florisson wrote: >>> >>> On 29 February 2012 17:57, Dag Sverre Seljebotn >>> wrote: On 02/29/2012 09:42 AM, Stefan Behnel wrote: > > > Dag Sverre Seljebotn, 29.02.2012 18:06: >> >> >> I'm wondering what the best course of action for deprecating the shape >> field in numpy.pxd is. >> >> The thing is, currently "shape" really gets in the way. In most >> situations >> it is OK with slow access to shape through the Python layer, and >> "arr.shape[0]" is often just fine, but currently one is in a situation >> where one must either write "(arr).shape[0])" or >> "np.PyArray_DIMS(arr)[0]", or be faced with code that isn't >> forward-compatible with NumPy. > > > > Can Cython emulate this at the C layer? And even your work-around for > the > Python object access looks more like a Cython bug to me. I wouldn't know > why that can't "just work". It usually works for other undeclared Python > attributes of "anything", so it might just as well be made to work here. Well, the problem is that shape is currently declared as a C field. It is also available as a Python attribute. Usually the user doesn't care which one is used, but the C field is declared for the few cases where access is speed-critical. Though even with current NumPy, I find myself doing "print (arr).shape" in order to get a tuple rather than a Py_ssize_t*... >> It would really be good to do the transition as fast as possible, so >> that >> all Cython code eventually becomes ready for upcoming NumPy releases. > > > > But it previously worked, right? It's just no longer supported in newer > NumPy versions IIUC? If that's the case, deleting it would break > otherwise > working code. No-one forces you to switch to the latest NumPy version, > after all, and certainly not right now. A warning is much better. It previously worked, but it turns out that it was always frowned-upon. I didn't know that when I added the fields, and it was a convenient way of speeding things up... >>> >>> >>> I would personally prefer either cdef nogil extension class properties >>> (needs compiler support) or just special-casing in the compiler, which >>> shouldn't be too hard I think. Warnings would be a first step, but the >>> linkage of ndarray attributes to C attributes is really an >>> implementation detail, so it's better to keep supporting the >>> attributes correctly. >> >> >> So you are saying we (somehow) stick with supporting "arr.shape[0]" in the >> future, and perhaps even support "print arr.shape"? (+ arr.dim, >> arr.strides). Exactly how we could figure out at PyCon. > > To remain consistent with previous versions the former should be > supported and the latter would be a bonus (and it wouldn't be too hard > anyway). > >> I'm anyway leaning towards deprecating arr.data, as it's too different from >> what the Python attribute does. > > +1 for that, just write &arr[0] as Sturla mentioned. The transition > should be trivial. If there's a confusion due to .data already having a certain meaning with the python attribute, perhaps it would make sense to have an attribute with a different name, eg. .ptr or .voidptr ? IMHO writing &arr[0] looks like a workaround of some kind. Like, when in C you had something like a 2d array and you'd need to interpret it as a 1d array you'd write &arr[0][0], but C array syntax doesn't support attributes which you can add here. Unless of course the idea is to make arrays to behave and look exactly like C counterparts. Dimitri. > >> Reason I'm asking is that I'm giving a talk on Saturday, and I don't want to >> teach people bad habits -- so we must figure out what the bad habits are :-) >> (I think this applies for the PyCon poster as well...) >> >> [1] PyData workshop at Google's offices in Mountain View; the event was open >> for all but now it is full with a long waiting list, which is why I didn't >> announce it. http://pydataworkshop.eventbrite.com/ >> >> >> Dag >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] `cdef inline` and typed memory views
Hey everyone,
Congratulations on shipping 0.16! I think I found a problem which
seems pretty straight forward. Say I want to factor out inner part of
some N^2 loops over a flow array, I write something like
cdef inline float _inner(size_t i, size_t j, float[:] x):
cdef float d = x[i] - x[j]
return sqrtf(d * d)
In 0.16, this actually compiles (as opposed to 0.15 with ndarray) and
function is declared as inline, which is great. However, the
memoryview structure is passed by value:
static CYTHON_INLINE float __pyx_f_3foo__inner(size_t __pyx_v_i,
size_t __pyx_v_j, __Pyx_memviewslice __pyx_v_x) {
...
This seems to hinder compiler's (in my case, GCC 4.3.4) ability to
perform efficient inlining (although function does in fact get
inlined). If I manually inline that distance calculation, I get 3x
speedup. (in my case 0.324020147324 vs 1.43209195137 seconds for 10k
elements). When I manually modified generated .c file to pass memory
view slice by pointer, slowdown was eliminated completely.
On a somewhat relevant node, have you considered enabling Issues page on Github?
Thanks!
Dimitri.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
On Sat, Apr 21, 2012 at 2:48 PM, Stefan Behnel wrote: > Dimitri Tcaciuc, 21.04.2012 21:17: >> On a somewhat relevant node, have you considered enabling Issues page on >> Github? > > It was discussed, but the drawback of having two separate bug trackers is > non-negligible. Ok. I was wondering since it would make it much easier to connect issue/patch/discussion together without, say, me needlessly adding to the development mailing list and/or manually registering for trac and sending htpasswd digest over the mail. Here's something to consider if you ever want to migrate over from trac: https://github.com/adamcik/github-trac-ticket-import Cheers, Dimitri. > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] `cdef inline` and typed memory views
On Sun, Apr 22, 2012 at 1:20 PM, mark florisson
wrote:
> On 21 April 2012 20:17, Dimitri Tcaciuc wrote:
>> Hey everyone,
>>
>> Congratulations on shipping 0.16! I think I found a problem which
>> seems pretty straight forward. Say I want to factor out inner part of
>> some N^2 loops over a flow array, I write something like
>>
>> cdef inline float _inner(size_t i, size_t j, float[:] x):
>> cdef float d = x[i] - x[j]
>> return sqrtf(d * d)
>>
>> In 0.16, this actually compiles (as opposed to 0.15 with ndarray) and
>> function is declared as inline, which is great. However, the
>> memoryview structure is passed by value:
>>
>> static CYTHON_INLINE float __pyx_f_3foo__inner(size_t __pyx_v_i,
>> size_t __pyx_v_j, __Pyx_memviewslice __pyx_v_x) {
>> ...
>>
>> This seems to hinder compiler's (in my case, GCC 4.3.4) ability to
>> perform efficient inlining (although function does in fact get
>> inlined). If I manually inline that distance calculation, I get 3x
>> speedup. (in my case 0.324020147324 vs 1.43209195137 seconds for 10k
>> elements). When I manually modified generated .c file to pass memory
>> view slice by pointer, slowdown was eliminated completely.
>>
>> On a somewhat relevant node, have you considered enabling Issues page on
>> Github?
>>
>>
>> Thanks!
>>
>>
>> Dimitri.
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
> Although it is neither documented nor tested, it works if you just
> take the address of the memoryview. You can then index it using
> memoryview_pointer[0][i]. One should be careful, as taking the pointer
> and passing that around means that pointer is not acquisition counted,
> and will point to invalid memory if the memoryview goes out of scope
> (e.g. if it's a local variable, when you return).
Nice, passing by pointer did the trick! As an observation, I tried
using `cython.operator.dereference(x)` and in this case it's way less
efficient than `x[0]`. Dereferencing actually allocates an empty
memory view slice and copies the contents of `x`, even if the
`dereference(x)` result is never assigned anywhere and is only a
temporary value in the expression.
Dimitri.
> Cython could manually inline functions though, which could greatly
> reduce argument passing and unpacking overhead in some situations
> (like buffers).
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Julialang
I may be misuderstanding the intent here, but here it goes. If the main idea is to be able to call functions that are written in Julia or other languages, I think an effort to create an LLVM backend for Cython would go a long way towards inter-language connections as the one discussed here. It should be possible to take Cython- and Julia- produced LLVM bytecode and assemble it all together, applying whatever bytecode optimizers that are available (eg. SSE vectorization). A big advantage of that approach is that there's no need for one language to know syntax conventions of the other one (or at least not to full extent). Continuing the effort, it should be possible to eliminate the need for writing an intermediate .c/.cpp file if Clang compiler is used, which is also LLVM based. Dimitri. On Mon, Apr 23, 2012 at 9:30 AM, Robert Bradshaw wrote: > On Sun, Apr 22, 2012 at 1:59 AM, Lisandro Dalcin wrote: >> On 22 April 2012 08:10, Robert Bradshaw wrote: >>> Yes, Julia looks really cool. It's been on my radar for a while, but I >>> haven't had a chance to really try it out for anything yet. But I >>> hadn't thought about low-level Python/Cython <-> Julia integration. >>> That sounds very interesting. I wonder if Jython could give any >>> insight into to the tight interaction between two languages that are >>> usually used in isolation but have been made to call each other >>> (though there are a lot of differences too, e.g. we're not targeting >>> replacing the CPython interpreter (on first pass at least...)). >>> >> >> Are you all aware that "calling C" actually means a ctypes-like >> functionality based in dlopen()/dlsym() ? >> http://julialang.org/manual/calling-c-and-fortran-code/. > > Yes, with all its drawbacks, but the fact that it's JIT'ed at least > cuts into the overhead issues. > > - Robert > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Julialang
On Mon, Apr 23, 2012 at 10:09 AM, Dimitri Tcaciuc wrote: > I may be misuderstanding the intent here, but here it goes. > > If the main idea is to be able to call functions that are written in > Julia or other languages, I think an effort to create an LLVM backend > for Cython would go a long way towards inter-language connections as > the one discussed here. It should be possible to take Cython- and > Julia- produced LLVM bytecode and assemble it all together, applying > whatever bytecode optimizers that are available (eg. SSE > vectorization). A big advantage of that approach is that there's no > need for one language to know syntax conventions of the other one (or > at least not to full extent). Continuing the effort, it should be > possible to eliminate the need for writing an intermediate .c/.cpp > file if Clang compiler is used, which is also LLVM based. I should clarify myself to avoid a gross mistake; a .c module would still be necessary for CPython integration. > Dimitri. > > On Mon, Apr 23, 2012 at 9:30 AM, Robert Bradshaw wrote: >> On Sun, Apr 22, 2012 at 1:59 AM, Lisandro Dalcin wrote: >>> On 22 April 2012 08:10, Robert Bradshaw wrote: >>>> Yes, Julia looks really cool. It's been on my radar for a while, but I >>>> haven't had a chance to really try it out for anything yet. But I >>>> hadn't thought about low-level Python/Cython <-> Julia integration. >>>> That sounds very interesting. I wonder if Jython could give any >>>> insight into to the tight interaction between two languages that are >>>> usually used in isolation but have been made to call each other >>>> (though there are a lot of differences too, e.g. we're not targeting >>>> replacing the CPython interpreter (on first pass at least...)). >>>> >>> >>> Are you all aware that "calling C" actually means a ctypes-like >>> functionality based in dlopen()/dlsym() ? >>> http://julialang.org/manual/calling-c-and-fortran-code/. >> >> Yes, with all its drawbacks, but the fact that it's JIT'ed at least >> cuts into the overhead issues. >> >> - Robert >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Anyone for a doc cleanup before the release?
Here's my go at it: https://github.com/cython/cython/pull/141 Cheers! Dimitri. On Tue, Aug 7, 2012 at 12:34 AM, Stefan Behnel wrote: > Hi, > > building the documentation currently yields a number of Sphinx warnings: > > https://sage.math.washington.edu:8091/hudson/job/cython-docs/lastSuccessfulBuild/warnings1Result/ > > It would be nice if someone found the time to clean some of them up. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
