I agree that some of the more powerful elements of Cython would be/are difficult or impossible to pickle in a general way. I have a counter use-case where auto-pickling would be very useful.
I have cdef classes that have many (think 50-100) members of double and long types, as well as other cdef classes that are also only using Cython-friendly data types. I have implemented pickling for these classes and it sucks. Trying to maintain the list of parameters that need to be pickled is a major hassle. The biggest problem is that if you add add a parameter to the class but forget to add it to the pickling function, you end up with the parameter not getting reloaded on unpickling. This is not good! Yes it is up the programmer to do this right, but I could use some help. What I have ended up doing for my other classes is define a __cdict__ function that returns a dictionary with the values and parameters as defined in the cdef class. In this way, pickling and unpickling is a little bit more pleasant. But it would be fantastic if the __cdict__ could be baked into the class automatically. It would be fine for me if only the things that could be easily handled would be returned, and a second list of attributes that require more work to be serialized so that the user/coder could be sure that all the attributes are being handled and none get forgotten. Also, the docs on pickling of cdef classes is very sparse, a simple example would be very beneficial. The is one in the examples in the source, but it is not easy to find Ian On Tue, Sep 25, 2012 at 6:44 AM, Sturla Molden <stu...@molden.no> wrote: > On 20.09.2012 21:11, Ian Bell wrote: > > Auto-pickling would be tremendously helpful as pickling and unpickling >> is one of the most annoying features of working with threads and >> processes in python. >> > > How should Cython interfere how to pickle a C pointer? > > cdef class foobar: > cdef double *data > > A C object can be anything. Cython does not know anything about size, > offset or strides, or even if it's safe to take a copy. > > Example: How to pickle a shared memory buffer? Surely we cannot take a > copy, because that would defeat the purpose of "shared" memory. And even if > could take a copy, how many bytes should be copied? Do you think an > autopickler could have figured this out? > > https://github.com/**sturlamolden/sharedmem-numpy/**blob/master/sharedmem/ > **sharedmemory_sysv.pyx<https://github.com/sturlamolden/sharedmem-numpy/blob/master/sharedmem/sharedmemory_sysv.pyx> > > https://github.com/**sturlamolden/sharedmem-numpy/**blob/master/sharedmem/ > **sharedmemory_win.pyx<https://github.com/sturlamolden/sharedmem-numpy/blob/master/sharedmem/sharedmemory_win.pyx> > > On yes, the code is different on Unix and Windows, something the > auto-pickler could not possibly know either. > > Auto-pickling cdef classes is not doable, IMHO. > > > And by the way, implementing a __reduce__ method manually is not very > difficult either. > > > Sturla Molden > > ______________________________**_________________ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/**mailman/listinfo/cython-devel<http://mail.python.org/mailman/listinfo/cython-devel> >
_______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
