On Sat, Feb 19, 2011 at 11:22 AM, W. Trevor King <wk...@drexel.edu> wrote:
>> > If you try to override anything in a .so compiled module at runtime, >> > you'd get the same kind of error you currently do trying to rebind a >> > compiled class' method. >> >> That's the desired behavior for statically-bound globals, but >> implementing it is not so trivial. > > It's currently implemented for classes. Are modules that different > from classes? > > >>> import types > >>> types.ModuleType.__class__.__mro__ > (<type 'type'>, <type 'object'>) > > So you've got your standard __setattr__ to override with an > error-message generator. What is the implementation difficulty? You can't just add a __setattr__ to a module and have it work, it's a class-level slot. And you don't want to modify all module classes. To do this you have to subclass module itself and insert that in place during import. > I am also unclear about the distinction between cdef and cpdef > (perhaps this should be a new thread?). > > cpdef means "I'm declaring something with C and Python interfaces. > The Python interface is a thin wrapper which can be rebound to an > object of any type, leaving the static C interface inaccessible from > Python." No, you can't re-bind cdef class methods. Despite Cython's attempt to homogenize things for the user, extension classes are quite different than "normal" Python classes. This is a Python C/API issue. > cdef [private] means "I'm declaring something that only has a C > interface." > cdef public means "I'm declaring something with C and Python > interfaces backed by C data. Python code can alter the C data." > cdef readonly means "I'm declaring something with C and Python > interfaces backed by C data. Python code cannot alter the C data." cdef means "back this by a C variable" > This seems to be broken in Cython at the module level, since I can > rebind a cdef-ed class but not a cpdef-ed method: Yes, we don't currently control the module's set/getattr. And classes are "public" by default. > $ cat square.pyx > cdef class A (object): > cdef public int value > > cpdef square(self): > return self.value**2 > $ python -c 'import pyximport; pyximport.install(); import xx; > a = xx.A(); a.value = 3; print a.square(); > a.square = lambda self: self.value' > Traceback (most recent call last): > File "<string>", line 3, in <module> > AttributeError: 'square.A' object attribute 'square' is read-only > $ python -c 'import pyximport; pyximport.install(); import square; > square.A = object; print square.A; a = square.A(); a.value = 3' > <type 'object'> > Traceback (most recent call last): > File "<string>", line 2, in <module> > AttributeError: 'object' object has no attribute 'value' > > So the cdef-ed A currently has a rebindable Python interface (acting > like a hypothetical cpdef-ed class), but it's square method is not > rebindable (acting like a hypothetical `cdef readonly`-ed method). > >> >>>> (and where would we do it--on the first import of a cimporting >> >>>> module?) >> >>> >> >>> Compilation is an issue. I think that .pxd files should be able to be >> >>> cythoned directly, since then they Cython can build any wrappers they >> >>> request. If the file has a matching .pyx file, cythoning either one >> >>> should compile both together, since they'll produce a single Python >> >>> .so module. >> >> >> >> ... >> > >> > Under the mantra "explicit is better than implicit", we could have >> > users add something like >> > >> > cdef module "modname" >> > >> > to any .pxd files that should be inflated into Python modules. .pxd >> > files without such a tag would receive the current treatment, error on >> > any cpdef, etc. The drawback of this approach is that it makes Cython >> > more complicated, but if both behaviors are reasonable, there's >> > probably no getting around that. >> >> The other drawback is that it subverts the usual filename <-> module >> name convention that one usually expects. > > I've been convinced that the `cimport .pyx file` route is a better way > to go. > > However, the filename <-> module mapping is troublesome for backing > externally-implemented Python modules (e.g. numpy). If you wanted to > write a .pxd file backing numpy.random, how would you go about getting > your module installed in Cython/Includes/numpy/random.pxd or another > path that cython would successfully match with `cimport numpy.random`? Note that extern blocks (by definition) declare where things come from. > On Sat, Feb 19, 2011 at 10:24:05AM +0100, Stefan Behnel wrote: >> Robert Bradshaw, 18.02.2011 23:08: >> > On Thu, Feb 17, 2011 at 8:38 PM, W. Trevor King wrote: >> >> On Thu, Feb 17, 2011 at 3:53 PM, Robert Bradshaw wrote: >> >>> On Thu, Feb 17, 2011 at 3:12 PM, W. Trevor King wrote: >> >>>>>> A side effect of this cpdef change would be that now even bare .pxd >> >>>>>> files (no matching .pyx) would have a Python presence, >> >>>>> >> >>>>> Where would it live? Would we just create this module (in essence, >> >>>>> acting as if there was an empty .pyx file sitting there as well)? On >> >>>>> this note, it may be worth pursuing the idea of a "cython helper" >> >>>>> module where common code and objects could live. >> >>>> >> >>>> I'm not sure exactly what you mean by "cython helper", but this sounds >> >>>> like my 'bare .pyx can create a Python .so module idea above. >> >>> >> >>> I'm thinking of a place to put, e.g. the generator and bind-able >> >>> function classes, which are now re-implemented in every module that >> >>> uses them. I think there will be more cases like this in the future >> >>> rather than less. C-level code could be #included and linked from >> >>> "global" stores as well. However, that's somewhat tangential. >> >> If you generate more than one file from a .pyx, including files that are >> shared between compiler runs (or even readily built as .so files), you'd >> quickly end up in dependency hell. > > I disagree here, but I like your cimportable .pyx better, so it > doesn't matter ;). > >> >>>>>> Unions don't really have a Python parallel, >> >>>>> >> >>>>> They can be a cdef class wrapping the union type. >> >>>> >> >>>> But I would think coercion would be difficult. Unions are usually (in >> >>>> my limited experience) for "don't worry about the type, just make sure >> >>>> it fits in X bytes". How would union->Python conversion work? >> >>> >> >>> There would be a wrapping type, e.g. >> >>> >> >>> cdef class MyUnion: >> >>> cdef union_type value >> >> Wouldn't that have to be a pointer to the real thing instead? > > Do you mean `cdef union_type *value`? Why would the above version not > work? The union type has a well defined size and a number of well > defined interpretations, so I don't see the problem. > >> >>> with a bunch of setters/getters for the values, just like there are >> >>> for structs. (In fact the same code would handle structs and unions). >> >>> >> >>> This is getting into the wrapper-generator territory, but I'm starting >> >>> to think for simple things that might be worth it. >> >> >> >> I think that if Cython will automatically generate a wrapper for >> >> >> >> cdef public int x >> >> >> >> it should generate a wrapper for >> >> >> >> cdef struct X: cdef public int x >> > >> > Or >> > >> > cdef public struct X: >> > int x >> > readonly int z >> > private int z >> > >> > I would perhaps say that non-Pythonable non-private members in public >> > structs would be a compile error. >> >> +1, keep it safe at the beginning. > > -1, keep the code clean and the interface consistent ;). I think the > struct syntax should be identical to the class syntax, with the > exception that you can't bind methods to structs. That's the only > real difference between structs and classes, isn't it? In C++, the only difference between structs and classes is that struct members are public by default. (Not saying that C++ is always the model to follow, but it gives precedent). And structs can have function members, that's how to do OOP in C. > If safety with a new feature is a concern, a warning like > "EXPERIMENTAL FEATURE" in the associated docs and compiler output > should be sufficient. I think the point of "safe" is to start out with a compiler error, and we can change our minds later, which is better than trying to make legal statements illegal in the future. >> >> There really aren't that metatypes in C, so it doesn't seem like a >> >> slippery slope to me. Maybe I'm just missing something... >> >> >> >>>> Ok, I think we're pretty much agreed ;). I think that the next step >> >>>> is to start working on implementations of: >> >>>> >> >>>> * Stand alone .pxd -> Python module >> >>> >> >>> I'm not sure we're agreed on this one. >> >> Same from here. To me, that doesn't make much sense for code that wraps a >> library. And if it doesn't wrap a library, there isn't much benefit in >> writing a stand-alone .pxd in the first place. A .pyx is much more explicit >> and obvious in this case. Especially having some .pxd files that generate >> .so files and others that don't will make this very ugly. >> >> I'd prefer adding support for cimporting from .pyx files instead, >> potentially with an automated caching generation of corresponding .pxd >> files (maybe as ".pxdg" files to make them easier to handle for users). >> However, cyclic dependencies would be tricky to handle automatically then. > > That's fine with me, since I just checked and you can do > cdef extern from 'somelib.h': > in .pyx files. > > Why would cyclic dependencies be difficult? The contents of .pxdg > files should only depend on the paired .pyx file, so there is no need > to parse another .pyx/.pxd file when generating a .pxdg. After you've > created a .pxdg file, you're in the same situation that Cython > presumably already handles well, with a .pyx/.pxd(g) pair. Cyclic dependancies might require some special work, but they already do. >> >>>> * Extending class cdef/cdpef/public/readonly handling to cover enums, >> >>>> stucts, and possibly unions. >> >>> >> >>> This seems like the best first step. >> >> +1 > > Working on it... > >> >>>> * I don't know how to handle things like dummy enums (perhaps by >> >>>> requiring all cdef-ed enums to be named). >> >>> >> >>> All enums in C are named. >> >> >> >> But my Cython declaration (exposing a C `#define CONST_A 1`): >> >> >> >> cdef extern from 'mylib.h': >> >> enum: CONST_A >> >> >> >> is not a named enum. >> > >> > Ah, yes. Maybe we require a name (that would only be used in Python space). >> >> ... require it for cpdef enums, you mean? >> >> OTOH, the "enum: NAME" scheme is ugly by itself. There should be a way to >> declare external constants correctly. After all, we loose all type >> information that way. I just saw that in math.pxd things like "M_PI" are >> declared as plain "enum" instead of "const double" or something. The type >> inferencer can't do anything with that. It might even draw the completely >> wrong conclusions. > > Something like: > > [cdef|cpdef] extern [public|readonly] <type> <name> > > For example: > > cdef extern readonly double M_PI > > That would be nice, since the C compiler would (I think) raise an error > when you try to use an invalid <type> for macro value. Const is different than readonly, as readonly specifies the python-level accessibility. - Robert _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
