On Mon, Apr 23, 2012 at 6:18 PM, Ralf Gommers <[email protected]> wrote: > > > On Mon, Apr 23, 2012 at 12:15 AM, Nathaniel Smith <[email protected]> wrote: >> >> We need to decide what to do with the NA masking code currently in >> master, vis-a-vis the 1.7 release. While this code is great at what it >> is, we don't actually have consensus yet that it's the best way to >> give our users what they want/need -- or even an appropriate way. So >> we need to figure out how to release 1.7 without committing ourselves >> to supporting this design in the future. >> >> Background: what does the code currently in master do? >> -------------------------------------------- >> >> It adds 3 pointers at the end of the PyArrayObject struct (which is >> better known as the numpy.ndarray object). These new struct members, >> and some accessors for them, are exposed as part of the public API. >> There are also a few additions to the Python-level API (mask= argument >> to np.array, skipna= argument to ufuncs, etc.) >> >> What does this mean for compatibility? >> ------------------------------------------------ >> >> The change in the ndarray struct is not as problematic as it might >> seem, compatibility-wise, since Python objects are almost always >> referred to by pointers. Since the initial part of the struct will >> continue to have the same memory layout, existing source and binary >> code that works with PyArrayObject *pointers* will continue to work >> unchanged. >> >> One place where the actual struct size matters is for any C-level >> ndarray subclasses, which will have their memory layout change, and >> thus will need to be recompiled. (Python-level ndarray subclasses will >> have their memory layout change as well -- e.g., they will have >> different __dictoffset__ values -- but it's unlikely that any existing >> Python code depends on such details.) >> >> What if we want to change our minds later? >> ------------------------------------------------------- >> >> For the same reasons as given above, any new code which avoids >> referencing the new struct fields referring to masks, or using the new >> masking APIs, will continue to work even if the masking is later >> removed. >> >> Any new code which *does* refer to the new masking APIs, or references >> the fields directly, will break if masking is later removed. >> Specifically, source will fail to compile, and existing binaries will >> silently access memory that is past the end of the PyArrayObject >> struct, which will have unpredictable consequences. (Most likely >> segfaults, but no guarantees.) This applies even to code which simply >> tries to check whether a mask is present. >> >> So I think the preconditions for leaving this code as-is for 1.7 are >> that we must agree: >> * We are willing to require a recompile of any C-level ndarray >> subclasses (do any exist?) > > > As long as it's only subclasses I think this may be OK. Not 100% sure on > this one though. > >> >> * We are willing to make absolutely no guarantees about future >> compatibility for code which uses APIs marked "experimental" > > > That is what I understand "experimental" to mean. Could stay, could change - > no guarantees.
Earlier you said it meant "some changes are to be expected, but not complete removal", which seems different from "absolutely no guarantees": http://www.mail-archive.com/[email protected]/msg36833.html So I just wanted to double-check whether you're revising that earlier opinion, or...? >> * We are willing for this breakage to occur in the form of random >> segfaults > > > This is not OK of course. But it shouldn't apply to the Python API, which I > think is the most important one here. Right, this part is specifically about ABI compatibility, not API compatibility -- segfaults would only occur for extension libraries that were compiled against one version of numpy and then used with a different version. - N _______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
