Re: [Cython] gsoc: array expressions
On Mon, May 28, 2012 at 8:49 AM, mark florisson wrote: > On 25 May 2012 21:53, Frédéric Bastien wrote: >> - About pickling theano, we currently can't pick Theano function. It >> could be made to work in some cases, but not for all cases as there is >> hardware dependent optimization in the Theano function. Currently it >> is mostly CPU vs GPU operation. So if we stay on the CPU, we could do >> some pickling, but we should make sure that the compiled c code into >> python module are still there when we unpickle or recompile them. >> >> - I think it make sense to make a theano graph from cython ast, >> optimize and redo a cython ast from the optimized graph. This would >> allow using Theano optimizations. > > Ok, the important thing is that the graph can be pickled, it should be > pretty straightforward to generate code to build the function again > from the loaded graph. We can pickle not compiled graph. So no problem here. >> - It also make sense to do the code generation in Theano and reuse it >> in Cython. But this would make the Theano dependency much stronger. >> I'm not sure you want this. >> >> >> - Another point not raised, theano need to know at compile time is the >> dtype, number of dimensions and witch dimensions are broadcastable for >> each variable. I think that the last one could cause problem, but if >> you use specialization for the dtype, the same can be done for the >> broadcsatability of a dimensions. > > Hm, that would lead to kind of an explosion of combinations. I think > we could specialize only on no broadcasting at all (except for > operands with lesser dimensionality). I expect that in normal user script they won't user all the combination :) So I won't worry about it at first. If there is a need, we could parametrise Theano op (especially the Elemwise op) so that when a dimensions is marked as not broadcasted, it also work when it is broadcasted. In the case of Elemwise, it is probably just the error checking code that will need to change. >> - The compyte(gpu nd array) project do collapsing of dimensions. This >> is an important optimization on the GPU as doing the index computation >> in parallel is costlier. I think on the CPU we could probably do >> collapsing just of the inner dimensions to make it faster. >> >> - Theano don't generate intrinsect or assembly, but we suppose that >> g++ will generate vectorized operation for simple loop. Recent version >> of gcc/g++ do this. > > Right, the aim is definitely to specialize for contiguous arrays, > where you collapse everything. Specializing statically for anything > more would be unfeasible, and better handled by a runtime compiler I > think. For the C backend, I'll start by generating simple C loops and > see if the compilers vectorize that already. I was under the impression you where doing run time code generation. I mixed the ongoing project. But collapsing the inner dimensions could still be useful as if you don't write explicitly all the loop, you will call a function or make a loop over the number of dimensions. It will reduice this number of looping. If the inner dimensions is small(ex matrix of shape (1, 3)) this can be useful. But that is less important that the default contiguous case. >> - Our generated code for element-wise operation take care a little >> about the memory access pattern. We swap dimensions to iterate on the >> dimensions with the smallest strides. But we don't go further. >> >> - What do you mean by CSE? Constant optimization? > > Yes, common subexpression elimination and also hoisting of unchanging > expressions outside the loop. Theano do CSE in the merge optimization. As for lifting expression outside of loop, we do it for Theano Scan(our loop), but they are not normal loop. It is much better to use tensor expression then scan if possible. > I started a new project, https://github.com/markflorisson88/minivect , > which currently features a simple C code generator. The specializer > and astbuilder do most of the work of creating the right AST, so the > code generator only has to implement code generation functions for > simple expressions. Depending on how it progresses I will look at > incorporating Theano's optimizations into it and having Theano use it > as a C backend for compatible expressions. Great, when you think it is a good time for me to look at it, tell me. Do it mimic cython internal? If so, is there doc about it so that I look at it? Fred ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: How to use Cython to wrap a C++ template class for use in Python?
On Tue, May 29, 2012 at 5:42 PM, Paul Leopardi wrote: > > On Wednesday, 30 May 2012 02:40:47 UTC+10, Chris Barker wrote: >> >> >> Well, the third option is to use your own home-made templatesto >> auto-generate cython code for each type. That's actually not as >> painful as it sounds -- there are lot's of templating systems for >> python -- Cheetah, for instance is designed for "any" use, not just >> html (though I'm never used it for Cython). >> >> http://www.cheetahtemplate.org/ >> >> You might want to look at the "bottleneck" project -- they did >> something like this -- not for calling a C++ templates, but the >> principle is the same. >> >> http://berkeleyanalytics.com/bottleneck/ >> > > Chris, thanks for your suggestions. I will take a look at Cheetah, but I'm > afraid it would add extra complication to an already very complicated build > process for PyClical. What I would like is a way for Cython to make > instantiations of my C++ template classes visible to Python. Cython supports > C++ templates, and Cython now has fused types, but as far as I can tell, > these two template concepts within Cython do not work with each other in any > meaningful, documented way, let alone support what I am proposing. Maybe I > am just too late to propose another Google Summer of Code project, or > perhaps the whole idea is all too hard? Surely there must be a wider > use-case for this idea than just me and my one library? Should I repost to > the core developers list? I think there are several issues with why Cython does not (yet?) have these capabilities, primarily: (1) Metaprogramming done Right can be very nice, but done wrong is disastrous, (2) The AST of Cython is really not that nice to work with, and (3) If it's just about specifying types, template preprocessing and, eventually, using JITs may be sufficient, more flexible, and certainly better than a half-baked solution. The thread you linked to also has some good discussion. We actually had a lot of discussion about this issue at the Cython Days workshop last year, and never hit upon a metaprogramming framework that seemed Right. Consensus was that until a clean proposal was put forward, we would focus on making it easy to use your templating engine of choice and implement fused types which would cover 85% or more of the need for metaproramming (especially tight loops over numeric types, as opposed to more generic cases where Python objects and vtables are often good enough). The fact that fused types don't work with C++ specializations is certainly a bug, though this still limits us to a fixed number of instantiations (and separate Python classes) in Python space due to the nature of C++ templates. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: How to use Cython to wrap a C++ template class for use in Python?
On 30 May 2012 19:49, Robert Bradshaw wrote: > On Tue, May 29, 2012 at 5:42 PM, Paul Leopardi > wrote: >> >> On Wednesday, 30 May 2012 02:40:47 UTC+10, Chris Barker wrote: >>> >>> >>> Well, the third option is to use your own home-made templatesto >>> auto-generate cython code for each type. That's actually not as >>> painful as it sounds -- there are lot's of templating systems for >>> python -- Cheetah, for instance is designed for "any" use, not just >>> html (though I'm never used it for Cython). >>> >>> http://www.cheetahtemplate.org/ >>> >>> You might want to look at the "bottleneck" project -- they did >>> something like this -- not for calling a C++ templates, but the >>> principle is the same. >>> >>> http://berkeleyanalytics.com/bottleneck/ >>> >> >> Chris, thanks for your suggestions. I will take a look at Cheetah, but I'm >> afraid it would add extra complication to an already very complicated build >> process for PyClical. What I would like is a way for Cython to make >> instantiations of my C++ template classes visible to Python. Cython supports >> C++ templates, and Cython now has fused types, but as far as I can tell, >> these two template concepts within Cython do not work with each other in any >> meaningful, documented way, let alone support what I am proposing. Maybe I >> am just too late to propose another Google Summer of Code project, or >> perhaps the whole idea is all too hard? Surely there must be a wider >> use-case for this idea than just me and my one library? Should I repost to >> the core developers list? > > I think there are several issues with why Cython does not (yet?) have > these capabilities, primarily: > > (1) Metaprogramming done Right can be very nice, but done wrong is disastrous, > (2) The AST of Cython is really not that nice to work with, and > (3) If it's just about specifying types, template preprocessing and, > eventually, using JITs may be sufficient, more flexible, and certainly > better than a half-baked solution. > > The thread you linked to also has some good discussion. > > We actually had a lot of discussion about this issue at the Cython > Days workshop last year, and never hit upon a metaprogramming > framework that seemed Right. Consensus was that until a clean proposal > was put forward, we would focus on making it easy to use your > templating engine of choice and implement fused types which would > cover 85% or more of the need for metaproramming (especially tight > loops over numeric types, as opposed to more generic cases where > Python objects and vtables are often good enough). Definitely, fused types are far from ideal, even if cdef class fused attributes would be supported. > The fact that fused types don't work with C++ specializations is > certainly a bug, though this still limits us to a fixed number of > instantiations (and separate Python classes) in Python space due to > the nature of C++ templates. > > - Robert If there really is a bug, then please report the use case (anyone who can find one). C++ and fused types *do* work together (unless there really is a bug), but the thing that is not supported (for anything) is fused types for cdef class attributes. One, entirely terrible but working, way to get around from Python space is the metaclass hack I posted earlier. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
