On Tue, Feb 21, 2012 at 8:02 PM, Robert Bradshaw <rober...@math.washington.edu> wrote: > On Tue, Feb 21, 2012 at 9:19 AM, mark florisson > <markflorisso...@gmail.com> wrote: >> On 21 February 2012 04:42, Robert Bradshaw <rober...@math.washington.edu> >> wrote: >>> Python bytecode -> LLVM is a great idea for creating ufuncs, the >>> overhead of Cython + GCC is atrocious for stuff like this. (I think >>> Cython could make a good frontent as well, especially if we generated >>> just the .c code for the function rather than a full extension module >>> and used a good compiler to generate machine code in-memory >>> on-the-fly.) >> >> I want elementwise functions and reductions for Cython as well, but >> they would be specialized according to the types you use them for, so >> it wouldn't be atrocious bloat (at least, no more then you'd already >> be having). >> I've been giving profile guided optimizations some thoughts as well, I >> think they are most neatly handled through a python tracer that is >> installed e.g. through sitecustomize which collects data about types, >> time spent in which code regions etc during testing and development. >> You can then tell Cython to compile the module, so e.g. if it >> registered "a numpy array" it could generate some specializations for >> dtypes either statically or dynamically through OpenCL. The >> alternative approach using code instrumentation would be more of a >> chore but would work just as well for Cython code with object-typed >> arguments. That's all rather far off, and this looks like an >> interesting development. > > One of the reasons JIT compilers can do so well is that they can use > runtime profile information to write specialized code. It'd be great > to be able to incorporate this kind of profiling into a Cython compile > too. One of the big missing features here is a "fallback mode" where > we can insert guards and fall back to generic code if they're not met, > which I think would open us up to a huge number of optimizations > without violating Python semantics.
A really neat first-step would be to write a straight LLVM backend for Cython -- instead of generating code, one could generate equivalent in-memory IR. (The really hacky way to do this would be to throw the output of the current C code generator through Clang.) That seems achievable (at least as pie-in-the-sky projects go), and even if no-one ever got around to adding fancier profile-guided optimizations, it would still be useful. For instance, it would be possible to not just import .pyx modules directly, but reload() them after editing (!!). Or have a cython.inline() function that works like weave.inline, but more awesome. -- Nathaniel _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
