[Numpy-discussion] record arrays with char*?

2014-02-10 Thread Christopher Jordan-Squire
I'm trying to wrap some C code using cython. The C code can take inputs in two modes: dense inputs and sparse inputs. For dense inputs the array indexing is naive. I have wrappers for that. In the sparse case the matrix entries are typically indexed via names. So, for example, the library documenta

Re: [Numpy-discussion] Inheriting from ndarray Was: deprecate numpy.matrix

2014-02-10 Thread Charles R Harris
On Mon, Feb 10, 2014 at 9:01 PM, Alexander Belopolsky wrote: > > On Mon, Feb 10, 2014 at 11:31 AM, Nathaniel Smith wrote: > >> And in the long run, I >> think the goal is to move people away from inheriting from np.ndarray. >> > > This is music to my ears, but what is the future of numpy.ma? I >

[Numpy-discussion] Inheriting from ndarray Was: deprecate numpy.matrix

2014-02-10 Thread Alexander Belopolsky
On Mon, Feb 10, 2014 at 11:31 AM, Nathaniel Smith wrote: > And in the long run, I > think the goal is to move people away from inheriting from np.ndarray. > This is music to my ears, but what is the future of numpy.ma? I understand that numpy.oldnumeric.ma (the older version written without inh

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 7:39 PM, Pauli Virtanen wrote: > The issue here is semantics for basic linear algebra operations, such as > matrix multiplication, that work for different matrix objects, including > ndarrays. I'll see if I can restate my suggestion in another way, because I do not feel you are respo

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Charles R Harris
On Mon, Feb 10, 2014 at 6:11 PM, Charles R Harris wrote: > > > > On Mon, Feb 10, 2014 at 5:39 PM, Pauli Virtanen wrote: > >> 11.02.2014 01:39, josef.p...@gmail.com kirjoitti: >> [clip] >> > Almost all the code in scipy.stats and statsmodels starts with >> np.asarray. >> > The numpy doc standard

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Charles R Harris
On Mon, Feb 10, 2014 at 5:39 PM, Pauli Virtanen wrote: > 11.02.2014 01:39, josef.p...@gmail.com kirjoitti: > [clip] > > Almost all the code in scipy.stats and statsmodels starts with > np.asarray. > > The numpy doc standard has the term `array_like` to indicate things that > > can be converted to

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Pauli Virtanen
11.02.2014 01:39, josef.p...@gmail.com kirjoitti: [clip] > Almost all the code in scipy.stats and statsmodels starts with np.asarray. > The numpy doc standard has the term `array_like` to indicate things that > can be converted to a usable object by ndasarray. > > ducktyping could be restricted to

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 6:39 PM, wrote: > > > On Mon, Feb 10, 2014 at 6:29 PM, Pauli Virtanen wrote: > >> 11.02.2014 00:31, Alan G Isaac kirjoitti: >> > On 2/10/2014 5:11 PM, Pauli Virtanen wrote: >> >> The existence of np.matrix messes up the general agreement on ndarray >> >> semantics in Pyth

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 6:29 PM, Pauli Virtanen wrote: > 11.02.2014 00:31, Alan G Isaac kirjoitti: > > On 2/10/2014 5:11 PM, Pauli Virtanen wrote: > >> The existence of np.matrix messes up the general agreement on ndarray > >> semantics in Python. The meaning of very basic code such as > >> > >>

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Pauli Virtanen
11.02.2014 00:31, Alan G Isaac kirjoitti: > On 2/10/2014 5:11 PM, Pauli Virtanen wrote: >> The existence of np.matrix messes up the general agreement on ndarray >> semantics in Python. The meaning of very basic code such as >> >> A * B >> A.sum(0) >> A[0] >> >> where A and B are NxN

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 2:33 PM, Pauli Virtanen wrote: > 11.02.2014 00:17, Matthew Brett kirjoitti: > [clip] >> That is a very convincing argument. >> >> What would be the problems (apart from code compatibility) in making >> scipy.sparse use the ndarray semantics? > > I'd estimate the effort

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Pauli Virtanen
11.02.2014 00:17, Matthew Brett kirjoitti: [clip] > That is a very convincing argument. > > What would be the problems (apart from code compatibility) in making > scipy.sparse use the ndarray semantics? I'd estimate the effort it would take to convert scipy.sparse to ndarray semantics is about a

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 5:11 PM, Pauli Virtanen wrote: > The existence of np.matrix messes up the general agreement on ndarray > semantics in Python. The meaning of very basic code such as > > A * B > A.sum(0) > A[0] > > where A and B are NxN matrices of some sort now depends on the types >

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 2:11 PM, Pauli Virtanen wrote: > 10.02.2014 23:40, Alan G Isaac kirjoitti: >> On 2/10/2014 4:28 PM, Pauli Virtanen wrote: >>> Starting with asarray won't work: sparse matrices are not >>> subclasses of ndarray. >> >> I was focused on the `matrix` object. For this objec

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Chris Barker
On Mon, Feb 10, 2014 at 1:13 PM, Alan G Isaac wrote: > Do you mean one must start out with an 'asarray'? > Or more than that? > maybe np.asanyarray() It's nice, at least in principle for duck-typed functions to return the type they were handed. And this really is the primary issu ewith np.matr

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread alex
On Mon, Feb 10, 2014 at 4:42 PM, Alan G Isaac wrote: > On 2/10/2014 4:40 PM, alex wrote: >> I really want to remove it > > Can you articulate the problem created by its existence > that leads you to this view? In my opinion, Pauli has articulated these problems well in this thread. __

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Pauli Virtanen
10.02.2014 23:40, Alan G Isaac kirjoitti: > On 2/10/2014 4:28 PM, Pauli Virtanen wrote: >> Starting with asarray won't work: sparse matrices are not >> subclasses of ndarray. > > I was focused on the `matrix` object. For this object, an initial > asarray is all it takes to use array code. (Or ...

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 1:52 PM, Charles R Harris wrote: > > > > On Mon, Feb 10, 2014 at 2:28 PM, Pauli Virtanen wrote: >> >> 10.02.2014 23:13, Alan G Isaac kirjoitti: >> > On 2/10/2014 4:03 PM, Pauli Virtanen wrote: >> >> What sparked this discussion (on Github) is that it is not >> >> poss

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Charles R Harris
On Mon, Feb 10, 2014 at 2:28 PM, Pauli Virtanen wrote: > 10.02.2014 23:13, Alan G Isaac kirjoitti: > > On 2/10/2014 4:03 PM, Pauli Virtanen wrote: > >> What sparked this discussion (on Github) is that it is not > >> possible to write duck-typed code that works correctly for: > > > > Do you mean o

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 4:40 PM, alex wrote: > I really want to remove it Can you articulate the problem created by its existence that leads you to this view? Alan Isaac ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/l

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread alex
On Mon, Feb 10, 2014 at 3:47 PM, Matthew Brett wrote: > Hi, > > On Mon, Feb 10, 2014 at 12:44 PM, Charles R Harris > wrote: >> >> >> >> On Mon, Feb 10, 2014 at 1:23 PM, Alan G Isaac wrote: >>> >>> On 2/10/2014 3:04 PM, Matthew Brett wrote: >>> > I teach psychologists and neuroscientists mainly >

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 4:28 PM, Pauli Virtanen wrote: > Starting with asarray won't work: sparse matrices are not subclasses > of ndarray. I was focused on the `matrix` object. For this object, an initial asarray is all it takes to use array code. (Or ... not?) And it is a view, not a copy. I don't have

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Pauli Virtanen
10.02.2014 23:13, Alan G Isaac kirjoitti: > On 2/10/2014 4:03 PM, Pauli Virtanen wrote: >> What sparked this discussion (on Github) is that it is not >> possible to write duck-typed code that works correctly for: > > Do you mean one must start out with an 'asarray'? Or more than > that? Starting

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 4:08 PM, Matthew Brett wrote: > I think the active questions here are: > * Should we collect the discussion in coherent form somewhere? > * Should we add something to the np.matrix docstring and if so what? > * (Pauli's point): to what extent should we try to emulate the np.matrix API.

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 4:03 PM, Pauli Virtanen wrote: > What sparked this discussion (on Github) is that it is not possible to > write duck-typed code that works correctly for: Do you mean one must start out with an 'asarray'? Or more than that? As I detailed in past discussion, the one thing I really do n

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 12:58 PM, wrote: > > > On Mon, Feb 10, 2014 at 3:45 PM, alex wrote: >> >> On Mon, Feb 10, 2014 at 2:36 PM, Matthew Brett >> wrote: >> > Hi, >> > >> > On Mon, Feb 10, 2014 at 6:26 AM, Nathaniel Smith wrote: >> >> On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: >> >>> H

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Pauli Virtanen
10.02.2014 22:23, Alan G Isaac kirjoitti: [clip] > As far as I have been able to discern, the underlying > motivation for eliminating the matrix class is that > some developers want to stop supporting in any form > the subclassing of numpy arrays. Do I have that right? What sparked this discussio

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 3:39 PM, wrote: > > > On Mon, Feb 10, 2014 at 3:04 PM, Matthew Brett wrote: > >> Hi, >> >> On Mon, Feb 10, 2014 at 7:09 AM, Alan G Isaac >> wrote: >> [snip] >> > Just to forestall the usual "just start them with arrays, eventually >> they'll >> > be grateful" reply, I wou

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 3:45 PM, alex wrote: > On Mon, Feb 10, 2014 at 2:36 PM, Matthew Brett > wrote: > > Hi, > > > > On Mon, Feb 10, 2014 at 6:26 AM, Nathaniel Smith wrote: > >> On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: > >>> Hello list, > >>> > >>> I wrote this mini-nep for numpy but I've

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 12:39 PM, wrote: > > > On Mon, Feb 10, 2014 at 3:04 PM, Matthew Brett > wrote: >> >> Hi, >> >> On Mon, Feb 10, 2014 at 7:09 AM, Alan G Isaac >> wrote: >> [snip] >> > Just to forestall the usual "just start them with arrays, eventually >> > they'll >> > be grateful"

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 12:44 PM, Charles R Harris wrote: > > > > On Mon, Feb 10, 2014 at 1:23 PM, Alan G Isaac wrote: >> >> On 2/10/2014 3:04 PM, Matthew Brett wrote: >> > I teach psychologists and neuroscientists mainly >> >> >> I must suspect that notebook was not for >> **undergraduate**

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread alex
On Mon, Feb 10, 2014 at 2:36 PM, Matthew Brett wrote: > Hi, > > On Mon, Feb 10, 2014 at 6:26 AM, Nathaniel Smith wrote: >> On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: >>> Hello list, >>> >>> I wrote this mini-nep for numpy but I've been advised it is more >>> appropriate for discussion on the li

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Charles R Harris
On Mon, Feb 10, 2014 at 1:23 PM, Alan G Isaac wrote: > On 2/10/2014 3:04 PM, Matthew Brett wrote: > > I teach psychologists and neuroscientists mainly > > > I must suspect that notebook was not for > **undergraduate** psychology students. > At least, not the ones I usually meet. > > SymPy is grea

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 3:04 PM, Matthew Brett wrote: > Hi, > > On Mon, Feb 10, 2014 at 7:09 AM, Alan G Isaac > wrote: > [snip] > > Just to forestall the usual "just start them with arrays, eventually > they'll > > be grateful" reply, I would want to hear that suggestion only from > someone > > w

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 12:23 PM, Alan G Isaac wrote: > On 2/10/2014 3:04 PM, Matthew Brett wrote: >> I teach psychologists and neuroscientists mainly > > I must suspect that notebook was not for > **undergraduate** psychology students. > At least, not the ones I usually meet. Well - in this

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/10/2014 3:04 PM, Matthew Brett wrote: > I teach psychologists and neuroscientists mainly I must suspect that notebook was not for **undergraduate** psychology students. At least, not the ones I usually meet. SymPy is great but for those without background it is at best awkward. It certainl

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 7:09 AM, Alan G Isaac wrote: [snip] > Just to forestall the usual "just start them with arrays, eventually they'll > be grateful" reply, I would want to hear that suggestion only from someone > who has used it successfully with undergraduates in the social sciences. I

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 2:49 PM, Matthew Brett wrote: > Hi, > > On Mon, Feb 10, 2014 at 11:44 AM, wrote: > > > > > > On Mon, Feb 10, 2014 at 2:12 PM, eat wrote: > >> > >> > >> > >> > >> On Mon, Feb 10, 2014 at 9:08 PM, alex wrote: > >>> > >>> On Mon, Feb 10, 2014 at 2:03 PM, eat wrote: > >>>

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Skipper Seabold
On Mon, Feb 10, 2014 at 2:49 PM, Matthew Brett wrote: > Hi, > > On Mon, Feb 10, 2014 at 11:44 AM, wrote: > > > > > > On Mon, Feb 10, 2014 at 2:12 PM, eat wrote: > >> > >> > >> > >> > >> On Mon, Feb 10, 2014 at 9:08 PM, alex wrote: > >>> > >>> On Mon, Feb 10, 2014 at 2:03 PM, eat wrote: > >>>

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 10:09 AM, Alan G Isaac wrote: > On 2/9/2014 5:55 PM, alex wrote: > > I'm working on the same kinds of problems in scipy development > > (functions involving sparse matrices and abstract linear operators) > > > And how is numpy's matrix object getting in your way? > Your in

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 11:44 AM, wrote: > > > On Mon, Feb 10, 2014 at 2:12 PM, eat wrote: >> >> >> >> >> On Mon, Feb 10, 2014 at 9:08 PM, alex wrote: >>> >>> On Mon, Feb 10, 2014 at 2:03 PM, eat wrote: >>> > Rhetorical or not, but FWIW I'll prefer to take singular value >>> > decompositi

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 2:12 PM, eat wrote: > > > > On Mon, Feb 10, 2014 at 9:08 PM, alex wrote: > >> On Mon, Feb 10, 2014 at 2:03 PM, eat wrote: >> > Rhetorical or not, but FWIW I'll prefer to take singular value >> decomposition >> > (u, s, vt= svd(x)) and then based on the singular values s

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthew Brett
Hi, On Mon, Feb 10, 2014 at 6:26 AM, Nathaniel Smith wrote: > On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: >> Hello list, >> >> I wrote this mini-nep for numpy but I've been advised it is more >> appropriate for discussion on the list. >> >> """ >> The ``numpy.matrix`` API provides a low barrier

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread eat
On Mon, Feb 10, 2014 at 9:08 PM, alex wrote: > On Mon, Feb 10, 2014 at 2:03 PM, eat wrote: > > Rhetorical or not, but FWIW I'll prefer to take singular value > decomposition > > (u, s, vt= svd(x)) and then based on the singular values s I'll estimate > a > > "numerically feasible rank" r. Thus t

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread alex
On Mon, Feb 10, 2014 at 2:03 PM, eat wrote: > Rhetorical or not, but FWIW I'll prefer to take singular value decomposition > (u, s, vt= svd(x)) and then based on the singular values s I'll estimate a > "numerically feasible rank" r. Thus the diagonal of such hat matrix would be > (u[:, :r]** 2).su

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread eat
On Mon, Feb 10, 2014 at 7:00 PM, alex wrote: > On Mon, Feb 10, 2014 at 11:27 AM, wrote: > > How do we calculate the diagonal of the hat matrix without using N by N > > matrices? > > Not sure if this was a rhetorical question or what, but this seems to work > leverages = np.square(scipy.linalg.q

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread alex
On Mon, Feb 10, 2014 at 12:16 PM, Nathaniel Smith wrote: > On Mon, Feb 10, 2014 at 12:02 PM, Matthieu Brucher > wrote: >> Yes, but these will be scipy.sparse matrices, nothing to do with numpy >> (dense) matrices. > > Unfortunately when scipy.sparse matrices interact with dense ndarrays > (e.g.,

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Nathaniel Smith
On Mon, Feb 10, 2014 at 12:02 PM, Matthieu Brucher wrote: > Yes, but these will be scipy.sparse matrices, nothing to do with numpy > (dense) matrices. Unfortunately when scipy.sparse matrices interact with dense ndarrays (e.g., sparse matrix * dense vector), then you always get back np.matrix obj

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Matthieu Brucher
Yes, but these will be scipy.sparse matrices, nothing to do with numpy (dense) matrices. Cheers, Matthieu 2014-02-10 Dinesh Vadhia : > Scipy sparse uses matrices - I was under the impression that scipy sparse > only works with matrices or have things moved on? > > > > ___

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread alex
On Mon, Feb 10, 2014 at 11:27 AM, wrote: > How do we calculate the diagonal of the hat matrix without using N by N > matrices? Not sure if this was a rhetorical question or what, but this seems to work leverages = np.square(scipy.linalg.qr(X, mode='economic')[0]).sum(axis=1) http://www4.ncsu.edu

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Dinesh Vadhia
Scipy sparse uses matrices - I was under the impression that scipy sparse only works with matrices or have things moved on? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Nathaniel Smith
On Mon, Feb 10, 2014 at 11:16 AM, Alexander Belopolsky wrote: > > On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: >> >> On the other hand, it really needs to be deprecated. > > > While numpy.matrix may have its problems, a NEP should list a better > rationale than the above to gain acceptance. > > Pe

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread josef . pktd
On Mon, Feb 10, 2014 at 10:09 AM, Alan G Isaac wrote: > On 2/9/2014 5:55 PM, alex wrote: > > I'm working on the same kinds of problems in scipy development > > (functions involving sparse matrices and abstract linear operators) > > > And how is numpy's matrix object getting in your way? > Your in

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alexander Belopolsky
On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: > On the other hand, it really needs to be deprecated. While numpy.matrix may have its problems, a NEP should list a better rationale than the above to gain acceptance. Personally, I decided not to use numpy.matrix in production code about 10 years a

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Alan G Isaac
On 2/9/2014 5:55 PM, alex wrote: > I'm working on the same kinds of problems in scipy development > (functions involving sparse matrices and abstract linear operators) And how is numpy's matrix object getting in your way? Your initial post simply treated the desirability of deprecation as a given

Re: [Numpy-discussion] deprecate numpy.matrix

2014-02-10 Thread Nathaniel Smith
On Sun, Feb 9, 2014 at 4:59 PM, alex wrote: > Hello list, > > I wrote this mini-nep for numpy but I've been advised it is more > appropriate for discussion on the list. > > """ > The ``numpy.matrix`` API provides a low barrier to using Python > for linear algebra, just as the pre-3 Python ``input`