[Cython] Feature request: generate signature information for use by "inspect"
Python's "inspect" module is a great help to get valuable information about a package. Many higher level tools (e.g. the "help" builtin and "pydoc") are based on it. I have just recognized a deficiency of "cython" generated modules with respect to "inspect" support: "inspect" cannot determine the signatures for Python functions defined in "Cython" source. I understand that this might be a limitation of Python's "C" interface. In this case, I suggest to enhance the function's docstring by signature information. I now transform manually my docstrings def (): """ """ into: def (): """() -> : """ and would be happy to get something similar automatically. -- Dieter ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Feature request: generate signature information for use by "inspect"
Dieter Maurer, 28.06.2012 09:04: > Python's "inspect" module is a great help to get valuable information > about a package. Many higher level tools (e.g. the "help" builtin > and "pydoc") are based on it. > > I have just recognized a deficiency of "cython" generated > modules with respect to "inspect" support: > > "inspect" cannot determine the signatures for Python functions > defined in "Cython" source. > > I understand that this might be a limitation of Python's "C" > interface. Correct, although Cython goes to great length to enable introspection of Cython implemented functions and classes (admittedly, we could still do more...) > In this case, I suggest to enhance the > function's docstring by signature information. > > I now transform manually my docstrings > > def (): > """ > > > """ > > into: > > def (): > """() -> : > > > """ > > and would be happy to get something similar automatically. And the time machine strikes again. You can use the "embedsignature" compiler option for that. http://docs.cython.org/src/reference/compilation.html?highlight=embedsignature#compiler-directives Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Feature request: generate signature information for use by "inspect"
Stefan Behnel wrote at 2012-6-28 09:25 +0200: >Dieter Maurer, 28.06.2012 09:04: >> ... >> In this case, I suggest to enhance the >> function's docstring by signature information. >> >> I now transform manually my docstrings >> >> def (): >>""" >> >> >>""" >> >> into: >> >> def (): >>"""() -> : >> >> >>""" >> >> and would be happy to get something similar automatically. > >And the time machine strikes again. You can use the "embedsignature" >compiler option for that. > >http://docs.cython.org/src/reference/compilation.html?highlight=embedsignature#compiler-directives Thank you! I missed this part of the documentation. -- Dieter ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Automatic C++ conversions
I've been looking how painful it is to constantly convert between Python objects and string in C++. Yes, it's easy to write a utility, but this should be as natural (if not more so, as the length is explicit) than bytes <-> char*. Several other of the libcpp classes (vector, map) have natural Python analogues too. What would people think about making it possible to declare these in a C++ file? Being able to make arbitrary mappings anywhere between types is contextless global state that I'd rather avoid, but perhaps special methods defined on the class such as cdef extern from "" namespace "std": cdef cppclass string: def __object__(sting s): return s.c_str()[s.size()] def __create__(object o): return string(o, len(o)) ... (names open to suggestions) Then one could write cdef extern from *: string c_func(string) def f(x): return c_func(x) - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Automatic C++ conversions
Robert Bradshaw, 28.06.2012 10:59: > I've been looking how painful it is to constantly convert between > Python objects and string in C++. You mean std::string (as I think it's called)? Can't we just special case that in the same way that we special case char* and friends? Basically just one type more in that list. And it would give you efficient encoding/decoding more or less for free. I mean, well, it would likely break existing code to start doing that (in the same way that we broke code by enabling type inference for convertible pointers), but as long as it helps more than it breaks ... > Yes, it's easy to write a utility, > but this should be as natural (if not more so, as the length is > explicit) than bytes <-> char*. Several other of the libcpp classes > (vector, map) have natural Python analogues too. And you would want to enable coercion to those, too? Have a vector copy into a Python list automatically? (Although that's trivially done with a list comprehension, maybe the other way is more interesting...) I think, as long as there is one obvious mapping for a given type, I wouldn't mind letting Cython apply it automatically. > What would people think about making it possible to declare these in a > C++ file? Being able to make arbitrary mappings anywhere between types > is contextless global state that I'd rather avoid, but perhaps special > methods defined on the class such as > > cdef extern from "" namespace "std": > cdef cppclass string: > def __object__(sting s): > return s.c_str()[s.size()] > def __create__(object o): > return string(o, len(o)) > ... > > (names open to suggestions) Then one could write > > cdef extern from *: > string c_func(string) > > def f(x): > return c_func(x) Admittedly, it fits somewhat more naturally into C++ classes than generally into C, although we could allow the same thing in ctypedefs. However, I'm reluctant to introduce something like this as long as we can get away with built-in auto-coercion. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Automatic C++ conversions
On Thu, Jun 28, 2012 at 2:54 AM, Stefan Behnel wrote: > Robert Bradshaw, 28.06.2012 10:59: >> I've been looking how painful it is to constantly convert between >> Python objects and string in C++. > > You mean std::string (as I think it's called)? Can't we just special case > that in the same way that we special case char* and friends? Yes, we could. If we do that it'd make sense to special case list and vector and pair and and map and set as well, though perhaps those are special enough to hard code them, and it makes the language simpler to not have more special methods. > Basically just > one type more in that list. And it would give you efficient > encoding/decoding more or less for free. > > I mean, well, it would likely break existing code to start doing that (in > the same way that we broke code by enabling type inference for convertible > pointers), but as long as it helps more than it breaks ... I don't think it'd be backwards compatible, currently it's just an error. >> Yes, it's easy to write a utility, >> but this should be as natural (if not more so, as the length is >> explicit) than bytes <-> char*. Several other of the libcpp classes >> (vector, map) have natural Python analogues too. > > And you would want to enable coercion to those, too? Have a vector copy > into a Python list automatically? (Although that's trivially done with a > list comprehension, maybe the other way is more interesting...) > > I think, as long as there is one obvious mapping for a given type, I > wouldn't mind letting Cython apply it automatically. > > >> What would people think about making it possible to declare these in a >> C++ file? Being able to make arbitrary mappings anywhere between types >> is contextless global state that I'd rather avoid, but perhaps special >> methods defined on the class such as >> >> cdef extern from "" namespace "std": >> cdef cppclass string: >> def __object__(sting s): >> return s.c_str()[s.size()] >> def __create__(object o): >> return string(o, len(o)) >> ... >> >> (names open to suggestions) Then one could write >> >> cdef extern from *: >> string c_func(string) >> >> def f(x): >> return c_func(x) > > Admittedly, it fits somewhat more naturally into C++ classes than generally > into C, although we could allow the same thing in ctypedefs. > > However, I'm reluctant to introduce something like this as long as we can > get away with built-in auto-coercion. > > Stefan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Automatic C++ conversions
Robert Bradshaw, 28.06.2012 12:07: > On Thu, Jun 28, 2012 at 2:54 AM, Stefan Behnel wrote: >> Robert Bradshaw, 28.06.2012 10:59: >>> I've been looking how painful it is to constantly convert between >>> Python objects and string in C++. >> >> You mean std::string (as I think it's called)? Can't we just special case >> that in the same way that we special case char* and friends? > > Yes, we could. Then I think it makes sense to do that. Basically, the std::string type would set its is_string flag and then we need the actual coercion code for it. > If we do that it'd make sense to special case list and > vector and pair and and map and set as well, though perhaps those are > special enough to hard code them, and it makes the language simpler to > not have more special methods. Ok, then it's std::string <=> bytes std::vector <=> list std::map <=> dict std::set <=> set Potentially also: std::pair => tuple (maybe 2-tuple => std::pair with a runtime length test?) What about allowing list() etc.? As long as the item type can be coerced at compile time, this should be doable: => Python iterator and it would even be easy to implement in Cython code using a generator function. The other direction (Python iterator => ) would be trickier but could also be made to work when the C++ item type on the LHS of the assignment that triggers the coercion is known at compile time. We might want to look for a way to make these coercions a "thing" in the code (maybe through a registry or dedicated class) rather than adding special casing code everywhere. I think a CEP would be a good way to specify the above coercions. I also think that this is large enough a feature to openly ask for sponsorship. >> Basically just >> one type more in that list. And it would give you efficient >> encoding/decoding more or less for free. >> >> I mean, well, it would likely break existing code to start doing that (in >> the same way that we broke code by enabling type inference for convertible >> pointers), but as long as it helps more than it breaks ... > > I don't think it'd be backwards compatible, currently it's just an error. Ah, right, sorry. I got confused. Assignments to an untyped variable inherit the type of the RHS, so only typed assignments would be impacted, and those are currently errors, sure. Nothing in the way then. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] C++: how to handle failures of 'new'?
[moving this to cython-devel as it's getting technical] Robert Bradshaw, 28.06.2012 21:46: > On Thu, Jun 28, 2012 at 11:38 AM, Stefan Behnel wrote: >> currently, when I write "new CppClass()" in Cython, it generates a straight >> call to the "new" operator. It doesn't do any error handling. And the >> current documentation doesn't even mention this case. >> >> Is there a "standard" way to handle this? It seems that C++ has different >> ways to deal with failures here but raises an exception by default. Would >> you declare the constructor(s) with an "except +MemoryError"? Is there a >> reason Cython shouldn't be doing this automatically (if nothing else was >> declared) ? > > I think it certainly makes sense to declare the default constructor as > "except +" (and std::bad_alloc should become MemoryError), Right. The code in the constructor can raise other exceptions that must also be handled properly. An explicit "except +" will handle that. > but whether > to implicitly annotate declared constructors is less clear, especially > as there's no way to un-annotate them. I agree, but sadly, it's the default behaviour that is wrong. I'm sure we made lots of users run into this trap already. I fixed the documentation for now, but the bottom line is that we require users to take care of proper declarations themselves. Otherwise, the code that we generate is incorrect, although it's 100% certain that an allocation error can occur, even if the constructor code doesn't raise any exceptions itself. Apparently, changing the behaviour of the "new" operator requires a special annotation "std::nothrow", which then returns NULL on allocation failures. You can pass that from Cython by hacking up a cname, e.g. Rectangle "(std::nothrow) Rectangle" (int w, int h) I'm sure there are users out there who figured this out (I mean, I did...) and use it in their code, so I agree that this isn't easy to handle because Cython simply wouldn't know what the actual error behaviour is for a given constructor and how to correctly detect an error. This problem applies only to heap allocation in that form. However, stack allocation and the new exttype field allocation suffer from similar problems when the default constructor raises an exception. Exttype fields are a particularly nasty case because the user has no control over the allocation. A C++ exception in the C++ class constructor would terminate the exttype constructor unexpectedly and thus leak resources (in the best case - no idea how CPython reacts if you throw a C++ exception through its type instantiation code). Similarly, a C++ exception in the constructor of a stack allocated object would then originate from the function entry code and potentially hit the Python function wrapper etc. Again, potentially leaking resources or worse. To me, this sounds like we should do something about it. At least for the implicit calls to the default constructor, we should generate "except +" code automatically because there is no other way to handle them safely. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
