On Sun, Sep 21, 2014 at 5:50 PM, Stephan Hoyer <sho...@gmail.com> wrote:
> pandas has some hacks to support custom types of data for which numpy > can't handle well enough or at all. Examples include datetime and > Categorical [1], and others like GeoArray [2] that haven't make it into > pandas yet. > > Most of these look like numpy arrays but with custom dtypes and type > specific methods/properties. But clearly nobody is particularly excited > about writing the the C necessary to implement custom dtypes [3]. Nor is do > we need the ndarray ABI. > > In many cases, writing C may not actually even be necessary for > performance reasons, e.g., categorical can be fast enough just by wrapping > an integer ndarray for the internal storage and using vectorized > operations. And even if it is necessary, I think we'd all rather write > Cython than C. > > It's great for pandas to write its own ndarray-like wrappers (*not* > subclasses) that work with pandas, but it's a shame that there isn't a > standard interface like the ndarray to make these arrays useable for the > rest of the scientific Python ecosystem. For example, pandas has loads of > fixes for np.datetime64, but nobody seems to be up for porting them to > numpy (I doubt it would be easy). > > I know these sort of concerns are not new, but I wish I had a sense of > what the solution looks like. Is anyone actively working on these issues? > Does the fix belong in numpy, pandas, blaze or a new project? I'd love to > get a sense of where things stand and how I could help -- without writing > any C :). > > I haven't thought much about this myself, but others (Nathaniel?) have, and it would be good to explore the topic and maybe put together some examples/templates to make this approach easier. Input from someone with some experience would be *much* appreciated. The datetime problem persists and I've thinking it would be nice to replace the current implementation with something simpler that can be stolen from elsewhere. It would be nice to hear how someone else dealt with the problem. Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
