Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter
On Sat, Aug 20, 2016 at 1:19 AM, Stefan Behnel wrote: > > Especially with respect to C++, we've always tried to make things safe and > helpful that Cython can support directly, and to make things possible that > are too complex to handle safely. But I agree with Robert that allowing > arbitrary C++ code snippets inside of Cython code is not a good idea. Why is allowing arbitrary code inside not a good idea? We're not talking something necessarily like eval here and the reputation it got, we're talking C/C++ glue and for the scale of Cython's development effort, proper C++ support is insurmountable simply because the language is that hard to work with on the compiler side. You can't just go and take something that is readily in the process of evolving (as newer standards actively come out), that very slowly achieves full compiler support in it's own domain, and pretend/aspire you can work like that though high level constructs in your language - this project and none before it have not accomplished anywhere near that and simply don't have the man-years/decades to contribute. I truly think it is irresponsible to think a) we should do nothing or b) delude ourselfs that we can make high level language level features to get around this. > > If there are code constructs that we can add to improve the C++ integration > then we should do that, even if it means that Cython would (initially?) not > be able to validate that code. But we should otherwise keep the language > such that it allows us to get back to a state where Cython can validate and > understand what it sees. > I'm not going to argue against you making C++ developer life easier but I have no idea how you could wrangle the beast in another way than letting the C++ compiler handle things. I don't think it's important that Cython compiler can validate / understand everything it sees - I'm not saying that by default, but just like the various flags you can use with cpdef, strategic sacrifices are acceptable for consenting adults. Otherwise you are still saying you will limit capability, which many can't agree with and avoid like the plague. Remember - cython is "the way" that comes up for how to glue C/C++ into python these days, not just the program in cish python frontend for speed. > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter
> > > Remember - cython is "the way" that comes up for how to glue C/C++ into > python these days, not just the program in cish python frontend for speed. > > There's some truth to the issues you raise, but there are a lot of nasty details behind the scenes. Do you have some more concrete details for what you're proposing? In particular, Cython mangles all of its variable and function names. How do you want to work around that? How should exception forwarding and throwing work? You're right that it's unrealistic to ever expect Cython to support a full set of C++ like features, but, at the same time, often C++ libraries use many of the fancier features to create higher level interfaces that, conceptually, have a much simpler interface than we give them credit for. In many cases, the metaprogramming, though a part of the "interface" in C++ really is more of an implementation detail and is hidden from Cython. I wouldn't expect Cython to ever properly implement things like SFINAE, expression SFINAE, or even perfect forwarding, but it often still makes sense for it to interact with interfaces that are written using these features. Given that not all features actually need to be implemented in Cython, the question becomes more about what is good enough to support most interfaces nicely. Regardless of exactly which C++ features need to be supported right now, here are a few ways you can do things like this right now. - You can define inline functions, macros, or templates in separate headers. Though this does introduce additional files, it routes things through fairly standard code paths and makes things pretty reasonable. This is what I prefer to do when need more C++ features that Cython provides. - Tell Cython how the interface you're using is used, not how it is constructed. C style variadic function calls are especially good for this since they essentially bypass all type checking on a given function's arguments. - You can currently exploit user-specified C names of functions and variables to inject expressions into compiled code. This is an awful hack that happens to combine supported features in a useful way. You can inject a wide variety of expressions into the generated C++ code, but it's painful enough that doing something else is almost always preferable. It totally kills readability. All that said, I really think that there are more useful and concrete things to work on at the moment. There's plenty of work to do to make Cython up to scratch for wrapping arbitrary (reasonably well designed) C++ interfaces. Inlining C++ code into Cython seems like a nice idea up front, but in practice it's less clear what that would mean and how exactly all the accompanying semantics would work. Thanks for raising a lot of these issues though. Interfacing between Python and modern C++ is definitely worth the discussion. Best, Ian Henriksen ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter
On Fri, Aug 19, 2016 at 9:34 AM, Kevin Thornton wrote: > Hi Robert, > > Related to this point, I think there are three important features that > Cython would benefit from, as far as C++11 and newer are concerned. In > order of what I'm guessing to be increased complexity are: > > 1. Non-type template parameters. This is actually probably the hardest :). There is a PR that's stalled out, probably needs some attention by both the author and us; it'd be great to get this in. > 2. "rvalue" references for standalone functions and class member functions: > > void foo[T]( T && t ) >From a callers perspective, one can just declare an extern foo as foo(T) in Cython, it doesn't care if the actual call is made by value or by reference--the calling code is exactly the same. Actually implementing it for Cython-defined functions (as the callee) would require additional support, but would probably be pretty easy (implement a new type, with parsing, which generally behaves the same as a reference or value from Python's perspective--the actual heavy lifting happens in the C compiler). Want to file an issue on github for this? > 3. Variadic templates (member and non-member). More generally, "parameter > packs": > > http://en.cppreference.com/w/cpp/language/parameter_pack This'd be interesting (especially as to how it relates to changing the arity of functions for concrete instantiations). There's also a question of syntax. Maybe void foo[A, int N, ... Types](A a, Types t): ... It'd likely require #1, e.g. to be able to declare things like tuple_element::type such that std::tuple.get would be correctly typed. Unfortunately seems declaring the (somewhat intuitive) API of std::tuple requires being able to declare a fair amount of its implementation. Still likely feasible though, and I think the notion of variadic templates is natural enough in a Python setting (which has had heterogeneously typed tuples since its inception, though of course not statically typed). > #2 is important for efficiency reasons--it isn't just a convenience feature. > Without it, Cython cannot support all the member function of std::vector, > etc. The implication is that some expensive operations, like moving a big > object into a vector, require extra copies in Cython while a C++ program > would just move the object into the new location. It's interesting to call this an essential feature, given that C++ itself didn't have it until recently :). Not that it can't be a big win, and libraries will more and more probably be designed assuming it. > #3 seems like the hardest, but would enable std::tuple, std::bind, and other > handy things to be used in a .pyx file. > > Support for "auto" would be nice, but perhaps unrealistic. Just leave the cdef out and you basically get auto, i.e. my_var = expr gives my_var the type of expr. If multiple assignments are made to my_var, it attempts to find a suitable union type that can hold all assignments. The one exception is for integers, where it infers the type to be object iff my_var is used in arithmetic expressions (that it can't deduce would not overflow with the finite-sized C integer type). > I know that PRs have been submitted, and since stalled out, on a couple of > these features. I've tried to look at the Cython parser myself, but I'm not > able to follow it, sadly. > > Thanks for all your hard work, > > Kevin > > On Fri, Aug 19, 2016 at 2:39 AM Robert Bradshaw wrote: >> >> >> Though, as you discovered, there are some basic things like non-type >> template arguments that we would like to have. If there are other >> specific C++ constructs that are commonly used but impossible to >> express in Cython it'd be useful to know. >> >> >> ___ >> cython-devel mailing list >> cython-devel@python.org >> https://mail.python.org/mailman/listinfo/cython-devel > > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
