[Cython] aritmetic with arrays in Cython
Hi all, I noticed that some time ago there was a pull request ( https://github.com/cython/cython/pull/144) open that was trying to implement basic arithmetic operations with arrays. This seems to have also been proposed in CEP 518 ( https://github.com/cython/cython/wiki/enhancements-simd). What is the status on this? This is a feature I'd really like to see in Cython. I'd be happy to help with it, though I don't currently know enough about the internals of Cython to do it without some guidance. What would be the best way to do this? I've had three ideas thus far that might allow something like this to work, but I'd really like to get some second opinions before I get too far with any of them. 1. Make these array operations available as a part of Cython's memory view objects. 2. Write a Cython wrapper around a NumPy-like C++ library. https://github.com/ContinuumIO/libdynd seems to be a good candidate, though I'm not as familiar with it. (This doesn't strike me as something that would be a part of the main Cython project, but it could be a good way to make this sort of functionality available. It seems like a good way to avoid a lot of duplicate work though.) 3. Write a Fortran backend for Cython that uses Fortran's arrays wherever memory views are used. (This could be a bit crazy, but with the iso_c_binding module, it might be possible. It could also allow better optimizations for arrays to be applied by the compiler.) What would be the best way to go about this? Thanks! -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] aritmetic with arrays in Cython
Message: 3 Date: Tue, 05 Aug 2014 21:38:38 +0200 From: Julian Taylor To: Core developer mailing list of the Cython compiler Subject: Re: [Cython] aritmetic with arrays in Cython Message-ID: <53e132be.3070...@googlemail.com> Content-Type: text/plain; charset="windows-1252" >On 04.08.2014 21:11, Ian Henriksen wrote: >> Hi all, >> I noticed that some time ago there was a pull request >> (https://github.com/cython/cython/pull/144) open that was trying to >> implement basic arithmetic operations with arrays. This seems to have >> also been proposed in CEP 518 >> (https://github.com/cython/cython/wiki/enhancements-simd). What is the >> status on this? >> >> This is a feature I'd really like to see in Cython. I'd be happy to help >> with it, though I don't currently know enough about the internals of >> Cython to do it without some guidance. What would be the best way to do >> this? I've had three ideas thus far that might allow something like this >> to work, but I'd really like to get some second opinions before I get >> too far with any of them. >> >> 1. Make these array operations available as a part of Cython's memory >> view objects. >> >> 2. Write a Cython wrapper around a NumPy-like C++ >> library. https://github.com/ContinuumIO/libdynd seems to be a good >> candidate, though I'm not as familiar with it. (This doesn't strike me >> as something that would be a part of the main Cython project, but it >> could be a good way to make this sort of functionality available. It >> seems like a good way to avoid a lot of duplicate work though.) >> >> 3. Write a Fortran backend for Cython that uses Fortran's arrays >> wherever memory views are used. >> (This could be a bit crazy, but with the iso_c_binding module, it might >> be possible. It could also allow better optimizations for arrays to be >> applied by the compiler.) >> >> What would be the best way to go about this? >> > >4. Use numpy itself. numpy since 1.8 has vectorized base math functions, >though for large arrays you have to apply manual blocking and inplace >operations to see significant gains. That is where cython can help. >Possibly numpy is also open to expose the raw data buffer functions in >its api to make blocking more efficient. Thank you for pointing that out. Using the machinery that is already built in to numpy would be ideal. I hadn't realized that was a part of numpy's API. If I understand correctly, you're referring to https://github.com/numpy/numpy/blob/master/doc/neps/generalized-ufuncs.rst. Is that the right portion of the API? I'll try and figure out how to use it. Any working examples would be greatly appreciated. Other suggestions are also still welcome Thanks! -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] aritmetic with arrays in Cython
Nice module! That seems to implement most of the basic functionality I've seen lacking. Ideally, I'm still trying to find a way to be able to do something syntactically like this: cpdef whateverfunction(double[:] a, float[:] b): return a + a * b + .5 It would be ideal if that sort of thing were already a part of Cython. All things considered though, most of what I'm looking for seems to be there. Thanks! -Ian Henriksen On Thu, Aug 7, 2014 at 1:58 AM, Matěj Laitl wrote: > On Monday 04 of August 2014 13:11:13 Ian Henriksen wrote: > > Hi all, > > I noticed that some time ago there was a pull request ( > > https://github.com/cython/cython/pull/144) open that was trying to > > implement basic arithmetic operations with arrays. This seems to have > also > > been proposed in CEP 518 ( > > https://github.com/cython/cython/wiki/enhancements-simd). What is the > > status on this? > > > > This is a feature I'd really like to see in Cython. I'd be happy to help > > with it, though I don't currently know enough about the internals of > Cython > > to do it without some guidance. What would be the best way to do this? > I've > > had three ideas thus far that might allow something like this to work, > but > > I'd really like to get some second opinions before I get too far with any > > of them. > > Hi, > you may also check out https://github.com/strohel/Ceygen if it would suit > your > needs. > > Cheers, > Matěj > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] aritmetic with arrays in Cython
I'd like to work on it. That was why I asked. It's been bothering me for some time that we don't have this feature yet. This will take me some time though since I'm not at all familiar with Cython's internals. I've used it in relatively small settings for speeding up array operations and wrapping external functions, but I've never had occasion to dig through much of what it does internally. Some help navigating Cython's internals would be greatly appreciated. I'll also have to take some time to become more familiar with eigen itself. Most of my work has used numpy arrays. Eigen seems to be ideal for this since it allows us to offload the expression analysis to another package where it is already well-implemented. It looks like they have an array class for n-dimensional array operations as well. My guess is that that would allow a fairly natural translation from memory views to eigen arrays. For now, I'll work on learning some of the details of how to use eigen. Where would be a good place to start from there? Would it be more helpful for me to start working on Cython bindings for eigen, or to work on adding this directly as a part of Cython? Where should I look in Cython's source code to do that? What kinds of modifications would be necessary? Thanks! -Ian On Fri, Aug 8, 2014 at 4:28 AM, Stefan Behnel wrote: > Matěj Laitl schrieb am 08.08.2014 um 11:07: > > On Friday 08 of August 2014 07:05:08 Stefan Behnel wrote: > >> Eigen can apparently do these things already: > >> > >> http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html > >> > >> It might not be all that much work to integrate this with Cython to > enable > >> syntax support. Or have some kind of preprocessor, but I guess that's > not > >> going to be simpler than have it in Cython as an optional AST transform. > >> Basically, whenever an arithmetic expression consists of a mixture of > >> memory views and (optionally) scalars, it could be mapped to the kind of > >> code that Ceygen uses, just at an arbitrarily complex expression length. > > > > Exactly! That was one of my ideas when I decided to create Ceygen, but I > > resorted to take the easier route and made uglier static API because I > didn't > > (and still don't) have enough knowledge of Cython internals and Mark's > "array > > expressions" still looked like they are going to be merged. > > There are a couple of points in favour of Eigen/Ceygen. It's a way simpler > approach than moving the entire vectorised code generator into Cython. And > it lets Eigen do the hard work, leaving only the interfacing part to > Cython. Basically, both tools can do what they're good at. We're always > happy that there's a C compiler cleaning up after us to which we can > delegate the low level optimisation and tuning stuff. The same would apply > to Eigen. It's a much more cythonic approach than what's implemented in the > pie-in-the-sky array expressions branch. > > It requires a C++ compiler, but I think that's an acceptable dependency > these days in places where array computation performance matters. People > even accept a dependency on huge tool stacks like Numba/LLVM for the same > kind of tasks. An array expression syntax is just so much simpler than > writing dumb nested loops. > > > > Unfortunately, I no longer work on the project (my master's thesis) that > > triggered work on Ceygen, and now I can invest only little maintenance > work to > > it. > > I don't see me being the one to implement this, either. :) > > > > *However*, if such project is found worth to tacking by Cython devs, I > can > > imagine in my wild dreams that there may be a student at my former > university > > interested in doing it as their project. (this is rather optimistic > statement, > > please don't get too excited yet) > > I'm sure there are many people who would benefit from this. Some of them > might even be able to provide funding, if someone wants to pick up this > idea. But past experience shows that we shouldn't count on this either. > It's definitely a nice project to tackle for someone who likes coding. > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] aritmetic with arrays in Cython
On Fri, Aug 8, 2014 at 12:02 PM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > On 08.08.2014 19:36, Stefan Behnel wrote: > > Hi, > > > > please don't top-post. > > > > Ian Henriksen schrieb am 08.08.2014 um 18:47: > >> I'd like to work on it. That was why I asked. It's been bothering me for > >> some time that we don't have this feature yet. This will take me some > time > >> though since I'm not at all familiar with Cython's internals. I've used > it > >> in relatively small settings for speeding up array operations and > wrapping > >> external functions, but I've never had occasion to dig through much of > what > >> it does internally. Some help navigating Cython's internals would be > >> greatly appreciated. > > > > Here's a little intro. > > > > https://github.com/cython/cython/wiki/HackerGuide#getting-started > > > > I recommend to start by writing a simple test that does some integer > > arithmetic and enable code generation traces for AST nodes in the > generated > > C code. "cython -a" should make this quite navigable. > > > > > >> I'll also have to take some time to become more familiar with eigen > itself. > >> Most of my work has used numpy arrays. > >> > >> Eigen seems to be ideal for this since it allows us to offload the > >> expression analysis to another package where it is already > >> well-implemented. It looks like they have an array class for > n-dimensional > >> array operations as well. My guess is that that would allow a fairly > >> natural translation from memory views to eigen arrays. > > > > You should dig into the code that Ceygen uses for the mapping. That's > most > > likely the best start you can get. > > > > > >> Would it be more helpful for me to start working on Cython bindings for > >> eigen, or to work on adding this directly as a part of Cython? Where > should > >> I look in Cython's source code to do that? What kinds of modifications > >> would be necessary? > > > > My guess is that this is best implemented with a set of matrix specific > AST > > nodes for the arithmetic operators. There is a recursive tree processing > > phase in Cython's pipeline called "AnalyseExpressionsTransform" or type > > analysis, where it figures out what variables reference memory views, > what > > the result type of an arithmetic expression is, etc. This phase can > easily > > replace nodes by returning new nodes from the analyse_types() methods of > > AST nodes. Currently, arithmetic with memory views is not allowed, so > this > > needs to be enabled in the corresponding NumBinopNode AST subclasses > > (AddNode, MatMulNode, etc. in ExprNodes.py). Write a test and debug it to > > see where it fails. > > If someone wants to work on an ast transformer for array expressions I > would not restrict its output to something ceygen can parse but also numpy. > E.g. transforming: > > cpdef whateverfunction(double[:] a, float[:] b): > return a + a * b + .5 > > to > > cpdef whateverfunction(double[:] a, float[:] b): > bs = 1 > r = empty_like(a) > for i in range(0, a.size, bs): > u = min(i + bs, a.size) > r[i:u] = a[i:u] * b[i:u] > r[i:u] += a[i:u] > r[i:u] += 0.5 > return r > > this is already very efficient in numpy. > > Sorry about the top post. Stefan, thanks for the direction. I think I know how to proceed now. It'll probably take me some time to learn how to do all this, but I'll start reading/working on it now. Julian, from a dependency standpoint I'd really prefer to use numpy. As near as I can tell, the people most likely to use this feature already use numpy. That would also remove the C++ dependency for this. I think the main reason for using eigen is that it will be easy to interface with and will eliminate temporaries automatically. In the long term, it could be good to offer multiple backends for these sorts of operations. Numpy would be a primary candidate for something like that. I'll spend some time working with numpy's ufunc api to see if I can figure out a good way to use that instead. Thanks! -Ian ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] aritmetic with arrays in Cython
On Fri, Aug 8, 2014 at 3:09 PM, Ian Henriksen < insertinterestingnameh...@gmail.com> wrote: > On Fri, Aug 8, 2014 at 12:02 PM, Julian Taylor < > jtaylor.deb...@googlemail.com> wrote: > >> On 08.08.2014 19:36, Stefan Behnel wrote: >> > Hi, >> > >> > please don't top-post. >> > >> > Ian Henriksen schrieb am 08.08.2014 um 18:47: >> >> I'd like to work on it. That was why I asked. It's been bothering me >> for >> >> some time that we don't have this feature yet. This will take me some >> time >> >> though since I'm not at all familiar with Cython's internals. I've >> used it >> >> in relatively small settings for speeding up array operations and >> wrapping >> >> external functions, but I've never had occasion to dig through much of >> what >> >> it does internally. Some help navigating Cython's internals would be >> >> greatly appreciated. >> > >> > Here's a little intro. >> > >> > https://github.com/cython/cython/wiki/HackerGuide#getting-started >> > >> > I recommend to start by writing a simple test that does some integer >> > arithmetic and enable code generation traces for AST nodes in the >> generated >> > C code. "cython -a" should make this quite navigable. >> > >> > >> >> I'll also have to take some time to become more familiar with eigen >> itself. >> >> Most of my work has used numpy arrays. >> >> >> >> Eigen seems to be ideal for this since it allows us to offload the >> >> expression analysis to another package where it is already >> >> well-implemented. It looks like they have an array class for >> n-dimensional >> >> array operations as well. My guess is that that would allow a fairly >> >> natural translation from memory views to eigen arrays. >> > >> > You should dig into the code that Ceygen uses for the mapping. That's >> most >> > likely the best start you can get. >> > >> > >> >> Would it be more helpful for me to start working on Cython bindings for >> >> eigen, or to work on adding this directly as a part of Cython? Where >> should >> >> I look in Cython's source code to do that? What kinds of modifications >> >> would be necessary? >> > >> > My guess is that this is best implemented with a set of matrix specific >> AST >> > nodes for the arithmetic operators. There is a recursive tree processing >> > phase in Cython's pipeline called "AnalyseExpressionsTransform" or type >> > analysis, where it figures out what variables reference memory views, >> what >> > the result type of an arithmetic expression is, etc. This phase can >> easily >> > replace nodes by returning new nodes from the analyse_types() methods of >> > AST nodes. Currently, arithmetic with memory views is not allowed, so >> this >> > needs to be enabled in the corresponding NumBinopNode AST subclasses >> > (AddNode, MatMulNode, etc. in ExprNodes.py). Write a test and debug it >> to >> > see where it fails. >> >> If someone wants to work on an ast transformer for array expressions I >> would not restrict its output to something ceygen can parse but also >> numpy. >> E.g. transforming: >> >> cpdef whateverfunction(double[:] a, float[:] b): >> return a + a * b + .5 >> >> to >> >> cpdef whateverfunction(double[:] a, float[:] b): >> bs = 1 >> r = empty_like(a) >> for i in range(0, a.size, bs): >> u = min(i + bs, a.size) >> r[i:u] = a[i:u] * b[i:u] >> r[i:u] += a[i:u] >> r[i:u] += 0.5 >> return r >> >> this is already very efficient in numpy. >> >> > Sorry about the top post. > > Stefan, thanks for the direction. I think I know how to proceed now. It'll > probably take me some time to learn how to do all this, but I'll start > reading/working on it now. > > Julian, from a dependency standpoint I'd really prefer to use numpy. As > near as I can tell, the people most likely to use this feature already use > numpy. That would also remove the C++ dependency for this. I think the main > reason for using eigen is that it will be easy to interface with and will > eliminate temporaries automatically. In the long term, it could be good to &
Re: [Cython] aritmetic with arrays in Cython
On Sun, Aug 10, 2014 at 12:41 PM, Sturla Molden wrote: > Ian Henriksen > wrote: > > > Maybe I should clarify a little about why eigen is a good place to start. > > According to > href="http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html";> > http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html > > it > > already takes care of things like the elimination of temporary variables > > and common subexpression reduction at compile time. This should make it > > possible to compile array expressions in Cython without having to > > re-implement those sorts of optimizations. Ideally we would just have to > > map memory view operations to corresponding equivalents from eigen. It's > > not yet clear to me how to do things with arbitrary-dimensional arrays or > > broadcasting, but, given some more time, a solution may present itself. > > -Ian > > cilkplus is what you want, not Eigen. > > But if you are serious about number crunching, learn Fortran 95. > > > Sturla > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > Cilk Plus would also work really nicely for this. Thanks for the suggestion. Fortran is a really great language for this sort of thing, but I don't think I'm ready to tackle the difficulties of using it as a backend for array arithmetic in Cython. It would be a really great feature to have later on though. Thanks! -Ian ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] aritmetic with arrays in Cython
On Fri, Oct 24, 2014 at 2:36 AM, Stefan Behnel wrote: > Ian Henriksen schrieb am 09.08.2014 um 00:15: > > On Fri, Aug 8, 2014 at 3:09 PM, Ian Henriksen wrote: > >> On Fri, Aug 8, 2014 at 12:02 PM, Julian Taylor wrote: > >> Stefan, thanks for the direction. I think I know how to proceed now. > It'll > >> probably take me some time to learn how to do all this, but I'll start > >> reading/working on it now. > >> > >> Julian, from a dependency standpoint I'd really prefer to use numpy. As > >> near as I can tell, the people most likely to use this feature already > use > >> numpy. That would also remove the C++ dependency for this. I think the > main > >> reason for using eigen is that it will be easy to interface with and > will > >> eliminate temporaries automatically. In the long term, it could be good > to > >> offer multiple backends for these sorts of operations. Numpy would be a > >> primary candidate for something like that. I'll spend some time working > >> with numpy's ufunc api to see if I can figure out a good way to use that > >> instead. > > > > Maybe I should clarify a little about why eigen is a good place to start. > > According to http://eigen.tuxfamily.org/dox/TopicLazyEvaluation.html it > > already takes care of things like the elimination of temporary variables > > and common subexpression reduction at compile time. This should make it > > possible to compile array expressions in Cython without having to > > re-implement those sorts of optimizations. Ideally we would just have to > > map memory view operations to corresponding equivalents from eigen. It's > > not yet clear to me how to do things with arbitrary-dimensional arrays or > > broadcasting, but, given some more time, a solution may present itself. > > Did anything come out of this yet? > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > Unfortunately, nothing directly on an syntax tree transformer. I've been working on a related scipy pull request to expose their blas and lapack wrappers to cython. There are a few things left to add before it is merged there, but it should be a helpful start to making linear algebra operations more available in cython. See https://github.com/scipy/scipy/pull/4021. I'm a little overwhelmed this semester, so that is coming slower than I'd like, but I'm hoping to finish in the next two weeks or so. >From there, I've done a bit more reading, and it seems the best approach is to work on the wrappers for continuum's dynd array library. It already has a design similar to numpy and it's bindings are already written in cython. See https://github.com/ContinuumIO/libdynd and https://github.com/ContinuumIO/dynd-python, particularly https://github.com/ContinuumIO/libdynd/blob/master/docs/DESIGN.txt Getting n-dimensional array arithmetic should just be a matter of exposing their cython bindings to external packages properly. They don't have the buffer interface from pep 3118 set up yet though, so there is more work to be done there. I'd definitely welcome your input on that plan, or on anything I've done thus far. Thank you for your ideas and feedback. -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: New function (pointer) syntax.
On Thursday, November 6, 2014 9:48:53 PM UTC-7, 1989lzhh wrote: > > > > > > 在 Nov 7, 2014，02:56，Robert Bradshaw > > 写道： > > > > [Cc'ing elsewhere for more feedback. Also top-posting for initial > > impressions before the discussion.] > > > > Here's some proposed function pointer syntaxes; which are the most > > obvious to understand/read/remember? Can you figure them out? > > > >cdef float (*F)(float) > >cdef float (*G)(float (*)(float), float, float) > >cdef float ((*H)(char*))(float (*)(float), float, float) > > > > vs > > > >cdef float -> float F > >cdef (float -> float, float, float) -> float G > >cdef (char*) -> (float -> float, float, float) -> float H > > > > vs > > > >cdef lambda float: float F > >cdef lambda (lambda float: float), float, float: float G > >cdef lambda (char*): lambda: (lambda float: float), float, float: > float H > > > > > > If you want a hint, the last is something that returns numerical > > integration algorithm given a string name. Yes, you could use > > typedefs, but you shouldn't have to. especially for the first. > > >Here are numba kind function annotation, I guess it may fit in here. > cdef float(float) F > cdef float(float(float), float, float) G > cdef float(float(float), float, float)(char*) H > I personally feel this kind of annotation is more packed that using ->. > > Regards, > Liu zhenhai > Here are my thoughts, for what they're worth. I actually really like the arrow syntax. It matches the way that the domains and ranges of functions are often written mathematically. On the other hand, it does conflict with the syntax from C. The syntax from Numba is a really good way to do this as well. It is just as clear and doesn't conflict with C. Strictly speaking, it isn't Python, but it would be nice to have similarities in the syntax for the different packages. The C syntax is not intuitive. The only benefit there is the overlap with C. For something this simple, learning the new syntax is easy enough. It will also The lambda syntax strikes me as an intuitive solution, but it isn't particularly clean or concise. If the syntax is going to change, it would be good to change it to something nicer than that. I don't think this one would be worth making the change. All things considered, the Numba syntax looks like the best idea. The arrows are intuitive as well, but the overlap with the syntax from C could be a bad thing. Thanks for looking into this! -Ian Henriksen > > > >> On Thu, Nov 6, 2014 at 9:59 AM, Stefan Behnel > wrote: > >> Robert Bradshaw schrieb am 06.11.2014 um 18:15: > >>>> On Thu, Nov 6, 2014 at 12:29 AM, Stefan Behnel wrote: > >>>> Robert Bradshaw schrieb am 06.11.2014 um 08:34: > >>>>> I'd like to propose a more pythonic way to declare function pointer > >>>>> types, namelye > >>>>> > >>>>>type0 (*[ident])(type1, type2, type3) > >>>>> > >>>>> would instead become > >>>>> > >>>>>(type1, type2, type3) -> type0 [ident] > >>>> > >>>> Not convinced. Looks quite magic, very different from the C that > people > >>>> would know, and takes a while reading from left to right until you > see the > >>>> "->" which is essentially the only indication of what is happening > here. > >>> > >>> I suppose it depends on whether one knows just C, is familiar with a > >>> wide variety of other (more functional) languages. I think it's a lot > >>> easier for people to read who only know Python, or just a smattering > >>> of C. > >> > >> The thing is, in order to use Cython, you currently only need to > understand > >> Python, some C/C++ and a minor bit of special Cython syntax. The more > new > >> syntax we add outside of Python/C/C++, the higher the entry bar is > raised > >> for both the "mostly Python" and "I know C" users. > > > > True. However, " -> return_annotation" is still Python syntax. > > > >>>> Also, "->" has a widely known meaning in C/C++ which makes the above > very > >>>> confusing. I think in C when thinking about function pointers, and > there is > >>>> no "def ..." keyword indication here that would switch me back to &
Re: [Cython] a C++ wrapping wishlist
On Thu, May 21, 2015 at 2:13 AM Stefan Behnel wrote: > Someone wrote a list of shortcomings after wrapping some C++ code: > > http://blog.marcus-brinkmann.de/2014/07/31/cython-trouble/ > > A couple of these issues are due to misunderstandings (specifically the > "imports" section) or now-fixed bugs (post is almost a year old), but it > seems that some of them are still valid and might be low-hanging fruit. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel That's a very well-thought-out list. Overloading assignment and defining custom type coercions would be useful features to have. An idiomatic way of writing signatures for variadic templates and letting functions returning references be used as left-values are two other things that would be very nice to see. Thanks! -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython produces invalid code for operator()
On Thu, Jun 11, 2015 at 10:09 AM Michael Enßlin wrote:
> Hi guys,
>
> have a look at this:
>
> $ cat bar.pyx
> cdef extern from "foo.h":
> cdef cppclass Foo:
> int operator() (int arg)
> int do_call (int arg)
>
>
> cdef int bar(int arg):
> cdef Foo foo
> foo.do_call(arg)
> return foo(arg)
>
> $ cython3 --cplus bar.pyx
> $ cat bar.cpp
>
> (...)
>
> static int __pyx_f_3bar_bar(int __pyx_v_arg) {
> Foo __pyx_v_foo;
> int __pyx_r;
> __Pyx_RefNannyDeclarations
> __Pyx_RefNannySetupContext("bar", 0);
>
> /* "bar.pyx":9
> * cdef int bar(int arg):
> * cdef Foo foo
> * foo.do_call(arg) # <<<<<<<<<<<<<<
> * return foo(arg)
> */
> __pyx_v_foo.do_call(__pyx_v_arg);
>
> /* "bar.pyx":10
> * cdef Foo foo
> * foo.do_call(arg)
> * return foo(arg) # <<<<<<<<<<<<<<
> */
> __pyx_r = operator()(__pyx_v_arg);
> goto __pyx_L0;
>
> /* "bar.pyx":7
> *
> *
> * cdef int bar(int arg): # <<<<<<<<<<<<<<
> * cdef Foo foo
> * foo.do_call(arg)
> */
>
> /* function exit code */
> __pyx_L0:;
> __Pyx_RefNannyFinishContext();
> return __pyx_r;
> }
>
> (...)
>
>
>
> Note how the function invocation for "do_call" is generated correctly,
> but the invocation of operator() is nonsensical.
>
> The correct line would be this:
>
> __pyx_r = __pyx_v_foo(__pyx_v_arg);
>
> Happy debugging :D
>
> ~ mic_e
>
> _______
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
There was a patch for this in
https://mail.python.org/pipermail/cython-devel/2015-February/004328.html.
I'm not sure what ever became of it though. I wrote up a workaround on SO
at http://stackoverflow.com/a/25955546/1935144 a while ago.
Best of luck!
-Ian Henriksen
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] operator() bug in cython
On Tue, Feb 17, 2015 at 2:51 AM Ulrich Dobramysl wrote:
> On Fri Feb 13 2015 at 11:07:39 PM Greg Ewing
> wrote:
>
>> Ulrich Dobramysl wrote:
>>
>> > Thanks! I haven't figured out how to call heap allocated objects, as the
>> > code
>> > ---
>> > cdef OperatorTest *t = new OperatorTest()
>> > t()
>> > ---
>> > is not translatable by Cython.
>>
>> Have you tried:
>>
>> t[0]()
>>
>> ?
>>
>
> Thanks, I didn't think about the t[0] trick!
>
>
>> I haven't been following the development of Cython's internals,
>> so I may be speaking naively here, but it looks wrong to me that
>> the cname of that entry should be 'operator()' rather than the
>> c-level name of the variable that the NameNode refers to.
>>
>>
> The cname of the entry is correct IMO, because it cannot have any
> knowledge about the cname of the object it is being called as later. The
> logic dealing with the special case of an operator() member function of a
> c++ class in SimpleCallNode.analyse_c_function_call() needs to be changed
> slightly to not treat operator() as an overloaded entry. Rather, the
> function type of the SimpleCallNode needs to be set to the type of the
> looked-up operator() entry. Then, at the code generation stage, the correct
> cname of the SimpleCallNode entry is used and not the entry of operator().
> With the patch below my test case compiles and runs without problems:
>
> diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py
> index f99ec6e..88522c9 100644
> --- a/Cython/Compiler/ExprNodes.py
> +++ b/Cython/Compiler/ExprNodes.py
> @@ -4569,11 +4569,13 @@ class SimpleCallNode(CallNode):
> args = self.args
>
> if func_type.is_cpp_class:
> -overloaded_entry =
> self.function.type.scope.lookup("operator()")
> -if overloaded_entry is None:
> +call_operator = self.function.type.scope.lookup("operator()")
> +if call_operator is None:
> self.type = PyrexTypes.error_type
> self.result_code = ""
> return
> +self.function.type = call_operator.type
> +overloaded_entry = None
> elif hasattr(self.function, 'entry'):
> overloaded_entry = self.function.entry
> elif (isinstance(self.function, IndexNode) and
> @@ -4603,7 +4605,7 @@ class SimpleCallNode(CallNode):
> else:
> entry = None
> func_type = self.function_type()
> -if not func_type.is_cfunction:
> +if not (func_type.is_cfunction or func_type.is_cpp_class):
> error(self.pos, "Calling non-function type '%s'" %
> func_type)
> self.type = PyrexTypes.error_type
> self.result_code = ""
>
> What do you think? I don't know a lot about cython's internals, so there
> might be use cases which break this code.
>
> Ulrich
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
What is the status on this? The most recent patch appears to work
if the first two changed lines are ignored. If it would help move this
along, I can open a PR with the modified patch and a commit that still
attributes the patch to the original author.
Thanks!
-Ian Henriksen
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] operator() bug in cython
On Mon, Jun 22, 2015 at 3:13 PM Ian Henriksen <
insertinterestingnameh...@gmail.com> wrote:
> On Tue, Feb 17, 2015 at 2:51 AM Ulrich Dobramysl wrote:
>
>> On Fri Feb 13 2015 at 11:07:39 PM Greg Ewing
>> wrote:
>>
>>> Ulrich Dobramysl wrote:
>>>
>>> > Thanks! I haven't figured out how to call heap allocated objects, as
>>> the
>>> > code
>>> > ---
>>> > cdef OperatorTest *t = new OperatorTest()
>>> > t()
>>> > ---
>>> > is not translatable by Cython.
>>>
>>> Have you tried:
>>>
>>> t[0]()
>>>
>>> ?
>>>
>>
>> Thanks, I didn't think about the t[0] trick!
>>
>>
>>> I haven't been following the development of Cython's internals,
>>> so I may be speaking naively here, but it looks wrong to me that
>>> the cname of that entry should be 'operator()' rather than the
>>> c-level name of the variable that the NameNode refers to.
>>>
>>>
>> The cname of the entry is correct IMO, because it cannot have any
>> knowledge about the cname of the object it is being called as later. The
>> logic dealing with the special case of an operator() member function of a
>> c++ class in SimpleCallNode.analyse_c_function_call() needs to be changed
>> slightly to not treat operator() as an overloaded entry. Rather, the
>> function type of the SimpleCallNode needs to be set to the type of the
>> looked-up operator() entry. Then, at the code generation stage, the correct
>> cname of the SimpleCallNode entry is used and not the entry of operator().
>> With the patch below my test case compiles and runs without problems:
>>
>> diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py
>> index f99ec6e..88522c9 100644
>> --- a/Cython/Compiler/ExprNodes.py
>> +++ b/Cython/Compiler/ExprNodes.py
>> @@ -4569,11 +4569,13 @@ class SimpleCallNode(CallNode):
>> args = self.args
>>
>> if func_type.is_cpp_class:
>> -overloaded_entry =
>> self.function.type.scope.lookup("operator()")
>> -if overloaded_entry is None:
>> +call_operator = self.function.type.scope.lookup("operator()")
>> +if call_operator is None:
>> self.type = PyrexTypes.error_type
>> self.result_code = ""
>> return
>> +self.function.type = call_operator.type
>> +overloaded_entry = None
>> elif hasattr(self.function, 'entry'):
>> overloaded_entry = self.function.entry
>> elif (isinstance(self.function, IndexNode) and
>> @@ -4603,7 +4605,7 @@ class SimpleCallNode(CallNode):
>> else:
>> entry = None
>> func_type = self.function_type()
>> -if not func_type.is_cfunction:
>> +if not (func_type.is_cfunction or func_type.is_cpp_class):
>> error(self.pos, "Calling non-function type '%s'" %
>> func_type)
>> self.type = PyrexTypes.error_type
>> self.result_code = ""
>>
>> What do you think? I don't know a lot about cython's internals, so there
>> might be use cases which break this code.
>>
>> Ulrich
>>
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
> What is the status on this? The most recent patch appears to work
> if the first two changed lines are ignored. If it would help move this
> along, I can open a PR with the modified patch and a commit that still
> attributes the patch to the original author.
> Thanks!
> -Ian Henriksen
>
Alright, I've put up a pull request at
https://github.com/cython/cython/pull/400.
I hope this makes things a bit easier.
-Ian Henriksen
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Use of long type for intermediate integral variables
On Thu, Jul 2, 2015 at 1:08 PM Robert McGibbon wrote: > Right, okay. I think I understand. > > -Robert > > On Thu, Jul 2, 2015 at 12:58 AM, Stefan Behnel > wrote: > >> Robert McGibbon schrieb am 02.07.2015 um 09:49: >> >> "libc.stdint.int64_t" is hand-wavingly declared as "long" >> > >> > There are some deeper issues in the rest of your message, but as a >> > preliminary matter, isn't this a clear error for linux-32 and windows? >> >> No, it's not. That's just what Cython sees. The C compiler then sees the >> exact platform specific type. And as the vast amount of Cython code out >> there shows, it's usually not a problem in practice. Cython is designed to >> handle most of these "platform specific type" issues at C compile time >> rather than C code generation time. But as your example shows, it can't >> always hide the details entirely. And there can be bugs. >> >> Stefan >> >> ___ >> cython-devel mailing list >> cython-devel@python.org >> https://mail.python.org/mailman/listinfo/cython-devel >> > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel This is an interesting discussion. Thanks. For the particular case of int64_t, wouldn't it still be wise to make it a typedef of long long rather than just long so as to consistently get the correct size on platforms where long is only 32 bits? Thanks! -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Use of long type for intermediate integral variables
On Thu, Jul 2, 2015 at 9:50 PM Robert Bradshaw wrote: > To clarify, any choice of int64_t + unsigned long will be wrong on one > platform or the others, but getting the wrong sign seems preferable to > getting the wrong number of bits. > That makes sens. Thanks for looking at it! -Ian Henriksen > > On Thu, Jul 2, 2015 at 8:45 PM, Robert Bradshaw > wrote: > > I've fixed this particular case by making our integer ranking more > > consistent: signedness is ignored for anything but chars and typedefs > > are always preferred (as a last tiebreaker) over non-typedefs (rather > > than arbitrarily picking the first or second argument). > > > https://github.com/cython/cython/commit/61ee8e544bd8802dfc313d832f5da97baf755c3d > > > > I've also upgraded the definition of int64_t to be long long to avoid > > issues with int64_t + unsigned long. There's still an issue of int32_t > > + unsigned int for any platforms where sizeof(int) == 16. > > > > -Robert > > > > On Thu, Jul 2, 2015 at 1:11 PM, Ian Henriksen > > wrote: > >> On Thu, Jul 2, 2015 at 1:08 PM Robert McGibbon > wrote: > >>> > >>> Right, okay. I think I understand. > >>> > >>> -Robert > >>> > >>> On Thu, Jul 2, 2015 at 12:58 AM, Stefan Behnel > >>> wrote: > >>>> > >>>> Robert McGibbon schrieb am 02.07.2015 um 09:49: > >>>> >> "libc.stdint.int64_t" is hand-wavingly declared as "long" > >>>> > > >>>> > There are some deeper issues in the rest of your message, but as a > >>>> > preliminary matter, isn't this a clear error for linux-32 and > windows? > >>>> > >>>> No, it's not. That's just what Cython sees. The C compiler then sees > the > >>>> exact platform specific type. And as the vast amount of Cython code > out > >>>> there shows, it's usually not a problem in practice. Cython is > designed > >>>> to > >>>> handle most of these "platform specific type" issues at C compile time > >>>> rather than C code generation time. But as your example shows, it > can't > >>>> always hide the details entirely. And there can be bugs. > >>>> > >>>> Stefan > >>>> > >>>> ___ > >>>> cython-devel mailing list > >>>> cython-devel@python.org > >>>> https://mail.python.org/mailman/listinfo/cython-devel > >>> > >>> > >>> ___ > >>> cython-devel mailing list > >>> cython-devel@python.org > >>> https://mail.python.org/mailman/listinfo/cython-devel > >> > >> > >> This is an interesting discussion. Thanks. > >> For the particular case of int64_t, wouldn't it still be wise to > >> make it a typedef of long long rather than just long so as to > consistently > >> get the correct size on platforms where long > >> is only 32 bits? > >> Thanks! > >> -Ian Henriksen > >> > >> ___ > >> cython-devel mailing list > >> cython-devel@python.org > >> https://mail.python.org/mailman/listinfo/cython-devel > >> > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Multidimensional indexing of C++ objects
Does anyone have any better ideas? My preference would definitely be toward allowing C++ calls returning references to be used as lvalues, but I'd really appreciate any alternative solutions. Thanks! -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Multidimensional indexing of C++ objects
On Sat, Jul 4, 2015 at 12:43 AM Stefan Behnel wrote: > Hi Ian! > > Ian Henriksen schrieb am 04.07.2015 um 00:43: > > I'm a GSOC student working to make a Cython API for DyND. DyND > > <https://github.com/libdynd/libdynd> is a relatively new n-dimensional > > array library in C++ that is based on NumPy. A full set of Python > bindings > > (created using Cython) are provided as a separate package. The goal of my > > project is to make it so that DyND arrays can be used easily within > Cython > > so that an n-dimensional array object can be used without any of the > > corresponding Python overhead. > > > > Currently, there isn't a good way to assign to multidimensional slices > > within Cython. Since the indexing operator in C++ is limited to a single > > argument, we use the call operator to represent multidimensional > indexing, > > and then use a proxy class to perform assignment to a slice. > > Currently, in C++, assigning to a slice along the second axis of a DyND > > array looks like this: > > > > a(irange(), 1).vals() = 0; > > > > Unfortunately, in Cython, only the index operator can be used for > > assignment, so following the C++ syntax isn't currently possible. Does > > anyone know of a good way to address this? > > Just an idea, don't know how feasible this is, but we could allow inline > special methods in C++ class declarations that implement Python protocols. > Example: > > cdef extern from ...: > cppclass Array2D: >int operator[] except + >int getItemAt(ssize_t x, ssize_t y) except + > >cdef inline __getitem__(self, Py_ssize_t x, Py_ssize_t y): >return self.getItemAt(x, y) > > def test(): > cdef Array2D a > return a[1, 2] > > Cython could then translate an item access on an Array2D instance into the > corresponding special "method" call. > > Drawbacks: > > 1) The example above would conflict with the C++ [] operator, so it would > be ambiguous which one is being used in Cython code. Not sure if there's a > use case for making both available to Cython code, but that would be > difficult to achieve if the need arises. > > 2) It doesn't solve the general problem of assigning to C++ expressions, > especially because it does not extend the syntax allowed by Cython which > would still limit what you can do in these fake special methods. > > Regarding your proposals, I'd be happy if we could avoid adding syntax > support for assigning to function calls. And I agree that the cname > assignment hack is really just a big hack. It shouldn't be relied on. > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel Yes, both this idea and the modified version that redefines operator[] are similar to the idea I had about respecting the cname entries for operator[]. This method would certainly expose a more flexible API for modules that want to do this. It may work in my case, but I worry that getting this into Cython would further complicate the (already lengthy) indexing logic. I'm still uneasy about exporting an API that is fundamentally different from the existing Python and C++ APIs, but making a way to use Python's syntax could help with that. Is there a good way to make a method like this accept Python-like indexing syntax? It would be confusing to put a code definition like this inside an extern block too. Could this syntax be adapted to work outside the extern block while still showing its connection to the original cppclass? Thanks for looking at it! -Ian ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Handling of cascaded assignment
I came across this while working on allowing proper overloading of the
assignment operator.
The problem is, essentially, that Cython does not follow the C/C++
ordering for cascaded assignment, even when dealing with objects at
the C/C++ level. For example, calling the test function from the
following Cython file and running the following C++ file should
both give the same output:
def test():
cdef:
long long a = (2LL << 36) - 257
int b
char c
b = c = a
print a, b, c
#include
int main(){
long long a = (2LL << 36) - 257;
int b;
char c;
b = c = a;
std::cout << a << std::endl;
std::cout << b << std::endl;
std::cout << (int)c << std::endl;
}
The Cython version prints 137438953215 -257 -1
and the C++ version prints 137438953215 -1 -1
To mimic C/C++, it should go through the arguments from right to left
and perform each assignment operation in that order, i.e.
c = a
b = c
As it stands, the current code does:
b = a
c = a
This does appear to follow what Python does. For example, the
following gives the same output as the Cython version:
import numpy as np
a = np.array([(2 << 36) - 257], np.int64)
b = np.empty(1, np.int32)
c = np.empty(1, np.int8)
b[:] = c[:] = a
print a[0], b[0], c[0]
Is this a deliberate design decision? If this weren't already included in
the language, I'd be inclined to not allow cascaded assignment to C
objects at all since it can give such surprising results. On the other
hand I don't see any particularly clean way to deal with this
incompatibility between C/C++ and Python.
Thanks!
-Ian Henriksen
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Multidimensional indexing of C++ objects
On Wed, Jul 8, 2015 at 11:20 AM Stefan Behnel wrote: > Ian Henriksen schrieb am 08.07.2015 um 03:50: > > On Sat, Jul 4, 2015 at 12:43 AM Stefan Behnel wrote: > >> Ian Henriksen schrieb am 04.07.2015 um 00:43: > >>> I'm a GSOC student working to make a Cython API for DyND. DyND > >>> <https://github.com/libdynd/libdynd> is a relatively new n-dimensional > >>> array library in C++ that is based on NumPy. A full set of Python > >> bindings > >>> (created using Cython) are provided as a separate package. The goal of > my > >>> project is to make it so that DyND arrays can be used easily within > >> Cython > >>> so that an n-dimensional array object can be used without any of the > >>> corresponding Python overhead. > >>> > >>> Currently, there isn't a good way to assign to multidimensional slices > >>> within Cython. Since the indexing operator in C++ is limited to a > single > >>> argument, we use the call operator to represent multidimensional > >> indexing, > >>> and then use a proxy class to perform assignment to a slice. > >>> Currently, in C++, assigning to a slice along the second axis of a DyND > >>> array looks like this: > >>> > >>> a(irange(), 1).vals() = 0; > >>> > >>> Unfortunately, in Cython, only the index operator can be used for > >>> assignment, so following the C++ syntax isn't currently possible. Does > >>> anyone know of a good way to address this? > >> > >> Just an idea, don't know how feasible this is, but we could allow inline > >> special methods in C++ class declarations that implement Python > protocols. > >> Example: > >> > >> cdef extern from ...: > >> cppclass Array2D: > >>int operator[] except + > >>int getItemAt(ssize_t x, ssize_t y) except + > >> > >>cdef inline __getitem__(self, Py_ssize_t x, Py_ssize_t y): > >>return self.getItemAt(x, y) > >> > >> def test(): > >> cdef Array2D a > >> return a[1, 2] > >> > >> Cython could then translate an item access on an Array2D instance into > the > >> corresponding special "method" call. > >> > >> Drawbacks: > >> > >> 1) The example above would conflict with the C++ [] operator, so it > would > >> be ambiguous which one is being used in Cython code. Not sure if > there's a > >> use case for making both available to Cython code, but that would be > >> difficult to achieve if the need arises. > >> > >> 2) It doesn't solve the general problem of assigning to C++ expressions, > >> especially because it does not extend the syntax allowed by Cython which > >> would still limit what you can do in these fake special methods. > >> > >> Regarding your proposals, I'd be happy if we could avoid adding syntax > >> support for assigning to function calls. And I agree that the cname > >> assignment hack is really just a big hack. It shouldn't be relied on. > > > > Yes, both this idea and the modified version that redefines operator[] > are > > similar to the idea I had about respecting the cname entries for > > operator[]. This method would certainly expose a more flexible API for > > modules that want to do this. It may work in my case, but I worry that > > getting this into Cython would further complicate the (already lengthy) > > indexing logic. > > The main problem with the logic in IndexNode is that it predates the > infrastructure change that allows node replacements in the analyse_types() > methods. It should eventually be split into separate nodes that do > different things, e.g. integer indexing into C arrays, Python object item > access, C++ operator[] usage, buffer/memory view indexing, memory view > slicing, you name it. > > In any case, adding new functionality can now be done by creating a new > node rather than complicating the type analysis code. And any further > refactoring would be warmly appreciated. :) > > > > I'm still uneasy about exporting an API that is > > fundamentally different from the existing Python and C++ APIs, but > making a > > way to use Python's syntax could help with that. Is there a good way to > > make a method like this accept Python-like indexing syntax? It would be > > confusing to put a code definition like this inside an extern block too. &g
[Cython] Non-type template parameters
Hi all, I've spent a little time working on adding support for non-type template parameters. In doing this, it has been very easy to add support for doing something like the following: cdef extern from "add_const.hpp" nogil: int myfunc1[i](int) def test(): print myfunc1[2](a) The downside of this is that it does not let the Cython compiler distinguish between different kinds of template parameters. Stricter checking could be added using a syntax like this: cdef extern from "add_const.hpp" nogil: int myfunc1[int i](int) def test(): print myfunc1[2](a) The downsides here are that the syntax doesn't really match the existing template syntax. It will also complicate the Cython codebase since we'll have to go to greater lengths to allow or disallow all the different special cases for templates. Which version would be preferable? On a similar note, for variadic templates, would we prefer something like cdef extern from "my_variadic.hpp" nogil: T myfunc2[T,...](T, ...) or something like: cdef extern from "my_variadic.hpp" nogil: T myfunc2[T, Types...](T, Types... args) Again, the latter syntax is more explicit, but it will require much more complicated code in Cython. It also doesn't match the existing syntax very well. The former syntax matches the existing syntax for templates better, but will make it hard for Cython to raise errors early on in compilation. I'd greatly appreciate any input on the best syntax for either use-case. Regards, -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Non-type template parameters
On Tue, Jul 28, 2015 at 1:36 AM Robert Bradshaw wrote: > Sorry for not getting back sooner... responses below. > > On Mon, Jul 20, 2015 at 2:06 PM, Ian Henriksen > wrote: > > Hi all, > > I've spent a little time working on adding support for non-type > > template parameters. In doing this, it has been very easy to add > > support for doing something like the following: > > > > cdef extern from "add_const.hpp" nogil: > > int myfunc1[i](int) > > def test(): > > print myfunc1[2](a) > > > > The downside of this is that it does not let the Cython compiler > > distinguish between different kinds of template parameters. > > > > Stricter checking could be added using a syntax like this: > > > > cdef extern from "add_const.hpp" nogil: > > int myfunc1[int i](int) > > def test(): > > print myfunc1[2](a) > > > > The downsides here are that the syntax doesn't really match the > > existing template syntax. It will also complicate the Cython codebase > > since we'll have to go to greater lengths to allow or disallow all the > > different special cases for templates. > > > > Which version would be preferable? > > I think I'd prefer the [int i] syntax. Hopefully it shouldn't > complicate things too much. > Okay, I think I see a way to make that work. On the other hand, since there weren't any replies here, I've already nearly finished implementing the first syntax. I'll spend another hour or two finishing it off later today and submit a PR so you can look it over. I originally favored the first syntax because it minimizes the number of fancy template features (SFINAE, for example) we have to worry about on the Cython end. I'm still open to discuss it though. > > > On a similar note, for variadic templates, would we prefer something > > like > > > > cdef extern from "my_variadic.hpp" nogil: > > T myfunc2[T,...](T, ...) > > > > or something like: > > > > cdef extern from "my_variadic.hpp" nogil: > > T myfunc2[T, Types...](T, Types... args) > > > > Again, the latter syntax is more explicit, but it will require much more > > complicated code in Cython. It also doesn't match the existing syntax > > very well. The former syntax matches the existing syntax for templates > > better, but will make it hard for Cython to raise errors early on in > > compilation. > > Hmm... this is a tougher call. Let's go with the former for now. > I like the former a lot more. It will keep the syntax simpler on our end and I haven't been able to find any case that it doesn't cover. This will also be significantly easier to implement. I'll take a look at it soon. Thanks! -Ian Henriksen > > > I'd greatly appreciate any input on the best syntax for either use-case. > > > > Regards, > > > > -Ian Henriksen > > > > ___ > > cython-devel mailing list > > cython-devel@python.org > > https://mail.python.org/mailman/listinfo/cython-devel > > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Non-type template parameters
On Tue, Jul 28, 2015 at 12:37 PM Robert Bradshaw wrote: > On Tue, Jul 28, 2015 at 11:18 AM, Ian Henriksen > wrote: > > > > On Tue, Jul 28, 2015 at 1:36 AM Robert Bradshaw > wrote: > >> > >> Sorry for not getting back sooner... responses below. > >> > >> On Mon, Jul 20, 2015 at 2:06 PM, Ian Henriksen > >> wrote: > >> > Hi all, > >> > I've spent a little time working on adding support for non-type > >> > template parameters. In doing this, it has been very easy to add > >> > support for doing something like the following: > >> > > >> > cdef extern from "add_const.hpp" nogil: > >> > int myfunc1[i](int) > >> > def test(): > >> > print myfunc1[2](a) > >> > > >> > The downside of this is that it does not let the Cython compiler > >> > distinguish between different kinds of template parameters. > >> > > >> > Stricter checking could be added using a syntax like this: > >> > > >> > cdef extern from "add_const.hpp" nogil: > >> > int myfunc1[int i](int) > >> > def test(): > >> > print myfunc1[2](a) > >> > > >> > The downsides here are that the syntax doesn't really match the > >> > existing template syntax. It will also complicate the Cython codebase > >> > since we'll have to go to greater lengths to allow or disallow all the > >> > different special cases for templates. > >> > > >> > Which version would be preferable? > >> > >> I think I'd prefer the [int i] syntax. Hopefully it shouldn't > >> complicate things too much. > > > > Okay, I think I see a way to make that work. On the other hand, since > there > > weren't any replies here, I've already nearly finished implementing the > > first > > syntax. I'll spend another hour or two finishing it off later today and > > submit a PR > > so you can look it over. I originally favored the first syntax because it > > minimizes > > the number of fancy template features (SFINAE, for example) we have to > worry > > about on the Cython end. I'm still open to discuss it though. > > I think this falls into the "explicit is better than implicit" bucket. > That and getting obscure template errors that could have been caught > at Cython compile time will be very nice. > > >> > On a similar note, for variadic templates, would we prefer something > >> > like > >> > > >> > cdef extern from "my_variadic.hpp" nogil: > >> > T myfunc2[T,...](T, ...) > >> > > >> > or something like: > >> > > >> > cdef extern from "my_variadic.hpp" nogil: > >> > T myfunc2[T, Types...](T, Types... args) > >> > > >> > Again, the latter syntax is more explicit, but it will require much > more > >> > complicated code in Cython. It also doesn't match the existing syntax > >> > very well. The former syntax matches the existing syntax for templates > >> > better, but will make it hard for Cython to raise errors early on in > >> > compilation. > >> > >> Hmm... this is a tougher call. Let's go with the former for now. > > > > > > I like the former a lot more. It will keep the syntax simpler on our end > and > > I > > haven't been able to find any case that it doesn't cover. This will also > be > > significantly easier to implement. I'll take a look at it soon. > > Sounds good. > > There's also the question of default arguments. Let's adopt the [type > T = *] syntax for that. > > >> > I'd greatly appreciate any input on the best syntax for either > use-case. > >> > > >> > Regards, > >> > > >> > -Ian Henriksen > >> > > >> > _______ > >> > cython-devel mailing list > >> > cython-devel@python.org > >> > https://mail.python.org/mailman/listinfo/cython-devel > >> > > >> ___ > >> cython-devel mailing list > >> cython-devel@python.org > >> https://mail.python.org/mailman/listinfo/cython-devel > > > > > > ___ > > cython-devel mailing list > > cython-devel@python.org > > https://mail.python.org/mailman/listinfo/cython-devel > > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel Yep, I can see why that syntax makes sense. I'm a little worried I may not be able to finish adding all of this in the near future. Adding all the of this type checking for templates goes a long way toward implementing the full syntax for declaring them. That said, it seems sensible to catch errors early if possible. Thanks! -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Integer division
Since Cython already tries to make division of C integers behave like Python division, I'd expect python-like behavior (mimicking python 2 or 3 as needed) by default and unchanged C integer division when the cdivision directive is set. Best, -Ian Henriksen On Mon, Sep 7, 2015 at 9:55 PM Robert Bradshaw wrote: > On Mon, Sep 7, 2015 at 5:42 PM, Antony Lee > wrote: > > My 2c (not cc'ed to cython-devel, as I am not registered): > > > > - language_level=3 implies __future__ division. > > - if __future__ division is set: int / int => float (whether variables > are > > untyped (= typed at runtime by python) or typed (including by the > > inferencer)) > > - if __future__ division is not set: int / int => int, with > > opposite-sign-case and zero-divisor-case handled depending on cdivision > > - in both cases, int // int => int, with opposite-sign-case and > > zero-divisor-case handled depending on cdivision (also for % and divmod). > > That's what I'm leaning towards as well, but it is backwards > incompatible. Anyone else have any opinions? > > > 2015-09-07 17:15 GMT-07:00 Robert Bradshaw : > >> > >> Or perhaps we should respect float division even with cdivision... > >> Thoughts? > >> > >> On Mon, Sep 7, 2015 at 5:13 PM, Robert Bradshaw > >> wrote: > >> > Oh, that's another issue. I'll clarify this in the documentation. > >> > > >> > On Mon, Sep 7, 2015 at 3:59 PM, Antony Lee > >> > wrote: > >> >> Actually I realized that it's another problem; setting cdivision to > >> >> True (as > >> >> I did in the directive of the example I gave, because I basically > >> >> always set > >> >> this to True) overrides __future__ division and restores the C > division > >> >> semantic that int / int = int. The documentation actually says that > >> >> the > >> >> cdivision directive should only affect the case of operands with > >> >> different > >> >> signs and of division by zero, not whether int / int = int. > >> >> Antony > >> >> > >> >> 2015-09-07 15:50 GMT-07:00 Robert Bradshaw : > >> >>> > >> >>> On Mon, Sep 7, 2015 at 3:46 PM, Robert Bradshaw > > >> >>> wrote: > >> >>> > On Mon, Sep 7, 2015 at 3:29 PM, Antony Lee < > antony@berkeley.edu> > >> >>> > wrote: > >> >>> >> Sorry, I intended the entire discussion to apply to the case of > >> >>> >> Python3 > >> >>> >> semantics. Indeed, even with language_level=3 and from > __future__ > >> >>> >> import > >> >>> >> division, foo.decl(3, 3) prints "1, 2, 0, 1" whereas > foo.nodecl(3, > >> >>> >> 3) > >> >>> >> prints > >> >>> >> "1.0, 0.5, 2.0, 1.0". > >> >>> > > >> >>> > from __future__ import division behaves as expected; I just added > >> >>> > some > >> >>> > even more explicit tests: > >> >>> > > >> >>> > > >> >>> > > https://github.com/cython/cython/commit/e61da2eb8a292bc34285c895aade523b6d353414 > >> >>> > > >> >>> > I'm surprised language_level=3 doesn't automatically set this > >> >>> > directive. > >> >>> > >> >>> Are you sure you're setting it correctly? > >> >>> > https://github.com/cython/cython/blob/master/tests/run/cython3.pyx#L30 > >> >>> > >> >>> >> 2015-09-07 15:07 GMT-07:00 Robert Bradshaw : > >> >>> >>> > >> >>> >>> On Mon, Sep 7, 2015 at 11:59 AM, Antony Lee > >> >>> >>> > >> >>> >>> wrote: > >> >>> >>> > I would like to suggest that division of integers with "/" > >> >>> >>> > prints a > >> >>> >>> > warning > >> >>> >>> > at compile time. In my opinion, "/" should be restricted to > >> >>> >>> > float > >> >>> >>> > division, > >> >>> >>> > and "//" should be used when rounding semantics are required. > >> >>
Re: [Cython] Non-type template parameters
> > I'd like to see this feature merged rather sooner than later. Would it help > if we only require a type to be there (so that people get the syntax right > in their code) but do not validate it for now? > > Stefan > > Stefan, Sorry for the delay, I've been mulling over what to do about this and haven't been fully satisfied with any of the answers. Not requiring the checks for now would definitely make this faster to implement though. My biggest concern is that, in C++, the types can actually play a significant role in the implementation of a given operation, and many of the things that go into determining what type may be returned are things that we don't want to have visible in Cython. Features like SFINAE and enable_if may be difficult for us to support, but syntactically, they don't use much fancy syntax. At some point, it seems to be in the best interest of our users if we let the C++ compiler finish off the type checking rather than requiring that all the type information be present in Cython. As near as I can tell, adding type specifiers to non-type template parameters won't put us into much murky water though, so I'll take another shot at adding the new syntax. Thanks for following up here. Best, -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython alpha 0.24
On Mon, Mar 21, 2016 at 6:29 PM Martin Bammer wrote: > Hi, > > I just saw that PyrexTypes.py seems to have a typo in lines 3740 and 3742: > > def __cmp__(self, other): > if isinstance(other, TemplatePlaceholderType): > return cmp(self.name, other.name) > else: > return cmp(type(self), type(other)) > > > PyCharm is complaining that "cmp" is an unresolved reference. > > Regards, > > Martin > > AFAICT, that whole function isn't really necessary. It defines the behavior for when you call `cmp` on an instance of that particular class. In Python 3 that particular builtin function doesn't exist. It'll work fine in Python 2, but since the codebase supports both it's not clear when that would actually be useful. Best, -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Question about how best require compiler options for C sources
ule objects. Provide a header for the Python module that exports that interface. cimports: Use capsule objects and parsing of pxd files to share things like external declarations, header includes, inline Cython functions, and Cython functions exported by modules between Cython modules. The public and api use cases are essentially the same most of the time, but api declarations use capsules rather than the OS's linker. There are still some trade-offs between public and api functions. Technically, the api functions require that the initialization routine exported in the api header be called for each translation unit that uses them. The public api just requires that the module already be initialized. In cases where no Python functionality is used in a public function, you may be able to get away with using the function without initializing the module, though I really wouldn't recommend that. There are some more subtle issues here though. The reason api functions need to be initialized on a per-translation unit basis is that things exported as api declarations are exported as translation-unit-local (static) function pointers. They aren't shared by the different translation units within a module built from multiple source files. I think that's a mistake. It'd be ideal if we could have api interfaces (or something like them) provide things with shared object local visibility rather than translation unit local visibility. This would require that the API headers have more carefully structured ifdef directives so that a macro could be set in a given translation unit to designate when to emit the actual declarations for the needed pointers rather than just forward declaring them. It would also require that the main generated c/cpp file define the pointers it uses as shared-object-local rather rather than static. In dynd-python we currently solve this problem by defining shared object local wrappers for the api exported function pointers and then using those instead, but I'm not a huge fan of that approach. It works well, but results in another unnecessary layer of indirection through the source files to connect the C++ code back to its Python bindings. With regards to the dllexporting/dllimporting of things: given that public declarations are already designed to export things through the shared object interface, we may as well fix the current setup to export the right things. It's a bad design that probably ought to be deprecated or at least documented better so people know not to use it unless their case actually requires sidestepping best practices. On the other hand, it's also a supported interface, so there's value in "fixing" it. I think the best way to do that is the following: - mark public symbols as dllimport unless a given (module specific) preprocessor define is set. - people using the public header outside of the module exporting the symbols should not have to set the define at all. - people using the public header to compile other source files that are linked in to the same Python module should set the preprocessor flag for that module. On top of that, at some point we still need to fix our api and public headers so that they still work when included into the translation unit for the main Cython-generated c/cpp file. This use-case should just forward-declare everything since the needed symbols are all defined later on in the Cython module. Since static variables cannot be forward declared in C, this will require that api declarations use shared object local symbols or that the main generated c/cpp file use some ifdef guards when it initializes the various pointers in question. As far as making an additional header goes, I personally prefer the extra preprocessor define. On the other hand, if people think an additional header is easier to use, then why not make it do something like #define USE_DLLEXPORT_NOT_DLLIMPORT_FOR_PARTICULAR_MODULE #include I think, that'd cover all the use cases better. Anyway, sorry for the length of my remarks on this. There are several issues here that have been bothering me for quite some time. Best, -Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Question about how best require compiler options for C sources
On Mon, Apr 11, 2016 at 11:50 AM Ian Henriksen < insertinterestingnameh...@gmail.com> wrote: > To answer the original question about define macros, it appears that the > canonical > way to pass preprocessor defines through distutils is to use the > define_macros > keyword when constructing your Extension class. You should also be able to > do > this within a Cython source file by including a directive like: > > # distutils: define_macros = MY_DEFINE, MY_DEFINE_2=2 > > Unfortunately, it looks like there's a bug in that that's making it so > that these > macros are undef'ed rather than being defined, so, for now, just pass the > appropriate flags to your Extension object. > > Small update on this, it looks like the issue with undef/define only applies when a define is specified this way without a value, so I either haven't gotten the syntax quite right, or that's not supported yet. Specifying an actual value for the macro works fine. Best, Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Question about how best require compiler options for C sources
On Tue, Apr 12, 2016 at 2:35 AM Erik Bray wrote: > On Mon, Apr 11, 2016 at 7:49 PM, Ian Henriksen > wrote: > > That aside, I agree with Nathaniel that exporting public declarations as > a > > part of the > > shared object interface was a design mistake. That aside, however, using > an > > api > > declaration lets you get equivalent results without exposing anything as > a > > part of > > the shared object API. Here's how this all works: > > I don't know if I would outright call it a "mistake", but in > retrospect I think we're all in agreement now that maybe the behavior > of "public" should be curtailed? Or should we still try to keep its > current behavior and "fix" it as I've been trying to do? > Great question, I don't know for sure. Input from others here would be nice. My thoughts are basically just that we should do one or the other. If it's supported and we want to keep supporting it, let's make it work right on platforms other than Linux. If we want to stop supporting it, let's deprecate it. Best, -Ian ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Question about how best require compiler options for C sources
On Tue, Apr 12, 2016 at 2:35 AM Erik Bray wrote: > On Mon, Apr 11, 2016 at 7:49 PM, Ian Henriksen wrote: > > With regards to the dllexporting/dllimporting of things: given that > public > > declarations > > are already designed to export things through the shared object > interface, > > we may > > as well fix the current setup to export the right things. It's a bad > design > > that > > probably ought to be deprecated or at least documented better so people > know > > not > > to use it unless their case actually requires sidestepping best > practices. > > On the > > other hand, it's also a supported interface, so there's value in "fixing" > > it. > > > > I think the best way to do that is the following: > > - mark public symbols as dllimport unless a given (module specific) > > preprocessor > > define is set. > > - people using the public header outside of the module exporting the > symbols > > should not have to set the define at all. > > - people using the public header to compile other source files that are > > linked in to > > the same Python module should set the preprocessor flag for that module. > > That's the approach I was trying to take originally, yes. But I think > it's a burden to require developers to set that preprocessor > flag--instead it should happen automatically, (or not at all, with > preference instead for using a completely separate header for > intra-module use). > > > On top of that, at some point we still need to fix our api and public > > headers so that > > they still work when included into the translation unit for the main > > Cython-generated > > c/cpp file. This use-case should just forward-declare everything since > the > > needed > > symbols are all defined later on in the Cython module. Since static > > variables > > cannot be forward declared in C, this will require that api declarations > use > > shared > > object local symbols or that the main generated c/cpp file use some ifdef > > guards > > when it initializes the various pointers in question. > > Sounds good--I see no problem with this. > > > As far as making an additional header goes, I personally prefer the extra > > preprocessor define. On the other hand, if people think an additional > header > > is > > easier to use, then why not make it do something like > > > > #define USE_DLLEXPORT_NOT_DLLIMPORT_FOR_PARTICULAR_MODULE > > #include > > > > I think, that'd cover all the use cases better. > > That would be fine too and reduce duplication. I think such a header > should still be generated though and it should be documented when to > use which header. > > After thinking this over a bit more, it seems like we're discussing things that really have more to do with the API chosen for a particular module than anything else. Maybe these options should all be made available as Compiler flags to Cython instead. As far as I can tell there are two separate options here: - Whether to provide distinct headers for defining the needed importing functionality, or declaring it as extern - Whether to make the preprocessor define flag for a given module's header work as an opt-in or an opt-out for defining the symbols needed to import and use the module. (numpy's headers use an opt-out system. DyND doesn't currently have this set up all of the way, but we'd like to use an opt-in model for working with our headers). Thoughts? Best, -Ian ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Question about how best require compiler options for C sources
On Mon, Apr 11, 2016 at 12:36 PM Ian Henriksen < insertinterestingnameh...@gmail.com> wrote: > On Mon, Apr 11, 2016 at 11:50 AM Ian Henriksen < > insertinterestingnameh...@gmail.com> wrote: > >> To answer the original question about define macros, it appears that the >> canonical >> way to pass preprocessor defines through distutils is to use the >> define_macros >> keyword when constructing your Extension class. You should also be able >> to do >> this within a Cython source file by including a directive like: >> >> # distutils: define_macros = MY_DEFINE, MY_DEFINE_2=2 >> >> Unfortunately, it looks like there's a bug in that that's making it so >> that these >> macros are undef'ed rather than being defined, so, for now, just pass the >> appropriate flags to your Extension object. >> >> > Small update on this, it looks like the issue with undef/define only > applies when a > define is specified this way without a value, so I either haven't gotten > the syntax quite > right, or that's not supported yet. Specifying an actual value for the > macro works fine. > > Best, > Ian Henriksen > Should be fixed in https://github.com/cython/cython/pull/509. Best, Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cygwin: Handling missing C99 long double functions
On Tue, Apr 26, 2016 at 2:56 PM Robert Bradshaw wrote: > On Tue, Apr 26, 2016 at 8:36 AM, Erik Bray wrote: > >> On Tue, Apr 26, 2016 at 5:16 PM, Dima Pasechnik >> wrote: >> > Hi, >> > certainly we did something with Sage on cygwin to work around these... >> > Just in case, >> >> From what I can tell there are several places where Sage has hacked >> around this issue in different packages, but it's not doing anything >> specifically with it for Cython. Sage uses the Cephes math lib to >> support these functions on FreeBSD--and apparently used to use it on >> Cygwin too, but disabled that for some reason. >> >> Regardless, Cython should ultimately do something sensible in these cases. >> >> My general thinking is that in cases where Cython generates code >> containing C math functions, it ought to support a fallback. This >> will require some feature checks so that Cython can generate wrappers, >> when necessary, around the double versions of those functions (as >> Numpy currently does). >> > > Let's make things concrete. You're complaining that something like > > cdef extern from "math.h": > long double sqrtl(long double) > > def foo(long double x): > return sqrtl(x) > > Doesn't work on Cygwin? > > The same is true for *any* C function that you use that's not totally > portable (this is the bane of trying to use C). I don't think Cython should > be detecting this and substituting a (less accurate) sqrt for sqrtl in this > case. If you want do do this, write your own headers that (conditionally) > define these things however you want. > > Or, are you complaining that Cython's test suite doesn't pass on some > Cygwin because there are tests of features not available on Cygwin? (Your > original email isn't clear.) If so, the test framework can be set up to > exclude these tests on that platform. > > Right, this sounds like a good place to exclude some tests. long double's behavior is pretty platform dependent. I wouldn't expect to be able to write platform independent code that uses it. There's not much Cython can do to change the situation either. Best, -Ian ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter
> > > Remember - cython is "the way" that comes up for how to glue C/C++ into > python these days, not just the program in cish python frontend for speed. > > There's some truth to the issues you raise, but there are a lot of nasty details behind the scenes. Do you have some more concrete details for what you're proposing? In particular, Cython mangles all of its variable and function names. How do you want to work around that? How should exception forwarding and throwing work? You're right that it's unrealistic to ever expect Cython to support a full set of C++ like features, but, at the same time, often C++ libraries use many of the fancier features to create higher level interfaces that, conceptually, have a much simpler interface than we give them credit for. In many cases, the metaprogramming, though a part of the "interface" in C++ really is more of an implementation detail and is hidden from Cython. I wouldn't expect Cython to ever properly implement things like SFINAE, expression SFINAE, or even perfect forwarding, but it often still makes sense for it to interact with interfaces that are written using these features. Given that not all features actually need to be implemented in Cython, the question becomes more about what is good enough to support most interfaces nicely. Regardless of exactly which C++ features need to be supported right now, here are a few ways you can do things like this right now. - You can define inline functions, macros, or templates in separate headers. Though this does introduce additional files, it routes things through fairly standard code paths and makes things pretty reasonable. This is what I prefer to do when need more C++ features that Cython provides. - Tell Cython how the interface you're using is used, not how it is constructed. C style variadic function calls are especially good for this since they essentially bypass all type checking on a given function's arguments. - You can currently exploit user-specified C names of functions and variables to inject expressions into compiled code. This is an awful hack that happens to combine supported features in a useful way. You can inject a wide variety of expressions into the generated C++ code, but it's painful enough that doing something else is almost always preferable. It totally kills readability. All that said, I really think that there are more useful and concrete things to work on at the moment. There's plenty of work to do to make Cython up to scratch for wrapping arbitrary (reasonably well designed) C++ interfaces. Inlining C++ code into Cython seems like a nice idea up front, but in practice it's less clear what that would mean and how exactly all the accompanying semantics would work. Thanks for raising a lot of these issues though. Interfacing between Python and modern C++ is definitely worth the discussion. Best, Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.26 release
On Mon, Jun 19, 2017 at 4:27 PM Robert Bradshaw wrote: > I'd like to officially start off the release process with an alpha > candidate. This does necessarily contain everything that will be in > the final release, but given the amount of time that has passed I > would recommend everyone test this out with their projects sooner > rather than later. > > Download: https://github.com/cython/cython/archive/0.26.alpha0.zip > Initial changelog: > https://github.com/cython/cython/blob/0.26.alpha0/CHANGES.rst > > Please feel free to respond to this thread or file bugs as necessary. > > Thanks, > Robert > I ran a quick test against dynd-python with master this morning. Everything looks fine there. Thanks for managing the release. Best, Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Preparations for Cython 0.26.1 and 0.27
On Thu, Aug 3, 2017 at 10:05 AM Matthew Brett wrote: > Hi, > > On Thu, Aug 3, 2017 at 2:12 PM, Andy wrote: > > One more thing: If there was a nightly wheel like for scipy and numpy, > > we could run CI with Cython master. That might lead to finding issues > > earlier. > > We probably don't want to build cython from source on CI, as I expect > that > > would > > take too much time. > > That's a good idea - I will set that up, and email back here with details. > > Cheers, > > Matthew > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel FWIW, depending on your CI needs using the no-cython-compile option can speed things up pretty drastically. Cython files don't compile quite as quickly with that option set, but installing Cython itself is much faster. Best of luck, Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] [Scipy-Dev] FFT from Cython
Actually, this seems like a pretty reasonable SciPy feature request. Feel free to open an issue there. Thanks, Ian Henriksen On Mon, Jul 30, 2018 at 3:18 PM 'Chris Barker' via cython-users < cython-us...@googlegroups.com> wrote: > BTW, > > Please don't spam so many lists with the same question. cython-users is > the right one for this. > > -CHB > > > > On Mon, Jul 30, 2018 at 10:50 AM, Chris Barker > wrote: > >> On Sun, Jul 29, 2018 at 6:17 PM, Eleftherios Garyfallidis < >> garyfalli...@gmail.com> wrote: >> >>> For a project in DIPY (http://dipy.org) we need to be calling 1D and 2D >>> FFTs and iFFTs very often. For this reason the most efficient way seems to >>> be through Cython. >>> >>> It would be great if we could call scipy's fft or numpy's fft without >>> the overhead. >>> >> >> have you profiled the overhead? Is it really significant? I"d expect the >> answer would only be yes if you were doing a lot of really small arrays -- >> is that the case? >> >> -CHB >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R(206) 526-6959 voice >> 7600 Sand Point Way NE >> <https://maps.google.com/?q=7600+Sand+Point+Way+NE&entry=gmail&source=g> >> (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> chris.bar...@noaa.gov >> > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE > <https://maps.google.com/?q=7600+Sand+Point+Way+NE&entry=gmail&source=g> > (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > > -- > > --- > You received this message because you are subscribed to the Google Groups > "cython-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to cython-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] longintrepr.h issues with MinGW
It's been a while since I've used MinGW to compile any extensions. The CRT incompatibility is a hassle to worry about unless you are using a custom Python build and MSVC 2015 and 2017 are pretty good. That said, there is a workaround for this one. See https://stackoverflow.com/a/30881190/1935144. Best, Ian On Wed, Oct 24, 2018, 1:02 AM Jeroen Demeyer wrote: > On 2018-10-24 07:58, Jeroen Demeyer wrote: > > If you care about MinGW, there is also this CPython issue which makes it > > basically impossible to compile any Cython extension on MinGW: > > > > https://bugs.python.org/issue11566 > > I forgot to say that this makes it impossible to compile any *C++* > Cython extension out of the box on MinGW. > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
