Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Dag Sverre Seljebotn
On 06/28/2011 11:52 PM, Matthew Brett wrote: > Hi, > > On Tue, Jun 28, 2011 at 5:38 PM, Charles R Harris > wrote: >> Nathaniel, an implementation using masks will look *exactly* like an >> implementation using na-dtypes from the user's point of view. Except that >> taking a masked view of an unma

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 6:57 PM, Pierre GM wrote: > > On Jun 29, 2011, at 1:39 AM, Mark Wiebe wrote: > > > On Tue, Jun 28, 2011 at 5:20 PM, Matthew Brett > wrote: > > Hi, > > > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > > ... > > > (You might think, what difference does it make

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 6:56 PM, Pierre GM wrote: > > On Jun 29, 2011, at 1:37 AM, Mark Wiebe wrote: > > > On Tue, Jun 28, 2011 at 3:45 PM, Pierre GM wrote: > > ... > > > > I think that would really take care of the missing data part in a > consistent and non-ambiguous way. > > However, I unders

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 6:55 PM, Nathaniel Smith wrote: > On Tue, Jun 28, 2011 at 4:37 PM, Mark Wiebe wrote: > > I've nearly finished this parameter, and decided to call it 'where' > instead, > > because it is operating like an SQL where clause. Here if neither a nor b > > are masked array it wi

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Pierre GM
On Jun 29, 2011, at 1:39 AM, Mark Wiebe wrote: > On Tue, Jun 28, 2011 at 5:20 PM, Matthew Brett > wrote: > Hi, > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > ... > > (You might think, what difference does it make if you *can* unmask an > > item? Us missing data folks could just

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Pierre GM
On Jun 29, 2011, at 1:37 AM, Mark Wiebe wrote: > On Tue, Jun 28, 2011 at 3:45 PM, Pierre GM wrote: > ... > > I think that would really take care of the missing data part in a consistent > and non-ambiguous way. > However, I understand that if a choice would be made, this approach would be >

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Nathaniel Smith
On Tue, Jun 28, 2011 at 4:37 PM, Mark Wiebe wrote: > I've nearly finished this parameter, and decided to call it 'where' instead, > because it is operating like an SQL where clause. Here if neither a nor b > are masked array it will only modify those values of b where the 'where' > parameter has t

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 6:00 PM, Matthew Brett wrote: > Hi, > > On Tue, Jun 28, 2011 at 11:40 PM, Jason Grout > wrote: > > On 6/28/11 5:20 PM, Matthew Brett wrote: > >> Hi, > >> > >> On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > >> ... > >>> (You might think, what difference does it

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 5:20 PM, Matthew Brett wrote: > Hi, > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > ... > > (You might think, what difference does it make if you *can* unmask an > > item? Us missing data folks could just ignore this feature. But: > > whatever we end up imple

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 3:45 PM, Pierre GM wrote: > All, > I'm not sure I understand some aspects of Mark's new proposal, sorry (blame > the lack of sleep). > I'm pretty excited with the idea of built-in NA like > np.dtype(NA['float64']), provided we can come with some shortcuts like > np.nafloat

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 2:41 PM, Eric Firing wrote: > On 06/28/2011 07:26 AM, Nathaniel Smith wrote: > > On Tue, Jun 28, 2011 at 9:38 AM, Charles R Harris > > wrote: > >> Nathaniel, an implementation using masks will look *exactly* like an > >> implementation using na-dtypes from the user's poi

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Mark Wiebe
On Tue, Jun 28, 2011 at 10:06 AM, Nathaniel Smith wrote: > On Mon, Jun 27, 2011 at 2:03 PM, Mark Wiebe wrote: > > On Mon, Jun 27, 2011 at 12:18 PM, Matthew Brett > > > wrote: > >> You won't get complaints, you'll just lose a group of users, who will, > >> I suspect, stick to NaNs, unsatisfactor

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread eat
Hi, On Wed, Jun 29, 2011 at 1:40 AM, Jason Grout wrote: > On 6/28/11 5:20 PM, Matthew Brett wrote: > > Hi, > > > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > > ... > >> (You might think, what difference does it make if you *can* unmask an > >> item? Us missing data folks could jus

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Matthew Brett
Hi, On Tue, Jun 28, 2011 at 11:40 PM, Jason Grout wrote: > On 6/28/11 5:20 PM, Matthew Brett wrote: >> Hi, >> >> On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith  wrote: >> ... >>> (You might think, what difference does it make if you *can* unmask an >>> item? Us missing data folks could just ign

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Jason Grout
On 6/28/11 5:20 PM, Matthew Brett wrote: > Hi, > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > ... >> (You might think, what difference does it make if you *can* unmask an >> item? Us missing data folks could just ignore this feature. But: >> whatever we end up implementing is someth

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Matthew Brett
Hi, On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: ... > (You might think, what difference does it make if you *can* unmask an > item? Us missing data folks could just ignore this feature. But: > whatever we end up implementing is something that I will have to > explain over and over to

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Matthew Brett
Hi, On Tue, Jun 28, 2011 at 8:41 PM, Eric Firing wrote: > On 06/28/2011 07:26 AM, Nathaniel Smith wrote: >> On Tue, Jun 28, 2011 at 9:38 AM, Charles R Harris >>  wrote: >>> Nathaniel, an implementation using masks will look *exactly* like an >>> implementation using na-dtypes from the user's poi

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Matthew Brett
Hi, On Tue, Jun 28, 2011 at 5:38 PM, Charles R Harris wrote: > Nathaniel, an implementation using masks will look *exactly* like an > implementation using na-dtypes from the user's point of view. Except that > taking a masked view of an unmasked array allows ignoring values without > destroying o

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Pierre GM
All, I'm not sure I understand some aspects of Mark's new proposal, sorry (blame the lack of sleep). I'm pretty excited with the idea of built-in NA like np.dtype(NA['float64']), provided we can come with some shortcuts like np.nafloat64. I think that would really take care of the missing data p

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Pierre GM
On Jun 28, 2011, at 9:41 PM, Eric Firing wrote: > > One of the real frustrations of the present masked array is that there > is no savez/load support. I could roll my own by using a convention > like saving the mask of xxx as xxx__mask__, and then reversing the > process in a modified load; b

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Nathaniel Smith
On Tue, Jun 28, 2011 at 12:41 PM, Eric Firing wrote: > I think you are exaggerating some of the differences associated with the > implementation, and ignoring one *key* difference: for integer types, > the masked implementation can handle the full numeric range of the type, > while the bit-pattern

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Eric Firing
On 06/28/2011 07:26 AM, Nathaniel Smith wrote: > On Tue, Jun 28, 2011 at 9:38 AM, Charles R Harris > wrote: >> Nathaniel, an implementation using masks will look *exactly* like an >> implementation using na-dtypes from the user's point of view. Except that >> taking a masked view of an unmasked a

Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 57, Issue 155

2011-06-28 Thread RadimRehurek
Hello, > From: santhu kumar > > After looking at the suggestions I have checked my matrix again and found > that the matrix was wrong.(Has some NAN's and other values which are wrong). > After correcting it it works fine. Not to beat a dead horse, but here's an (ancient) related numpy ticket:

Re: [Numpy-discussion] SVD does not converge

2011-06-28 Thread santhu kumar
py.linalg.linalg.LinAlgError: SVD did not converge > > I have looked in the list that it is a recurring issue but I was unable to > find any solution. Can somebody please guide me on how to fix that issue? > > Thanks > Santhosh > -- next part -- > An HT

Re: [Numpy-discussion] SVD does not converge

2011-06-28 Thread Charles R Harris
On Tue, Jun 28, 2011 at 11:36 AM, eat wrote: > Hi, > > On Tue, Jun 28, 2011 at 7:43 PM, Lou Pecora wrote: > >> >> -- >> *From:* santhu kumar >> *To:* numpy-discussion@scipy.org >> *Sent:* Tue, June 28, 2011 11:56:48 AM >> *Subject:* [Numpy-discussion] SVD does not con

Re: [Numpy-discussion] SVD does not converge

2011-06-28 Thread eat
Hi, On Tue, Jun 28, 2011 at 7:43 PM, Lou Pecora wrote: > > -- > *From:* santhu kumar > *To:* numpy-discussion@scipy.org > *Sent:* Tue, June 28, 2011 11:56:48 AM > *Subject:* [Numpy-discussion] SVD does not converge > > Hello, > > I have a 380X5 matrix and when I am c

Re: [Numpy-discussion] Missing/accumulating data

2011-06-28 Thread Charles R Harris
On Tue, Jun 28, 2011 at 10:50 AM, Joe Harrington wrote: > As with Travis, I have not had time to wade through the 150+ messages > on masked arrays, but I'd like to raise a concept I've mentioned in > the past that would enable a broader use if done slightly differently. > That is, the "masked arr

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Nathaniel Smith
On Tue, Jun 28, 2011 at 9:38 AM, Charles R Harris wrote: > Nathaniel, an implementation using masks will look *exactly* like an > implementation using na-dtypes from the user's point of view. Except that > taking a masked view of an unmasked array allows ignoring values without > destroying or cop

[Numpy-discussion] Missing/accumulating data

2011-06-28 Thread Joe Harrington
As with Travis, I have not had time to wade through the 150+ messages on masked arrays, but I'd like to raise a concept I've mentioned in the past that would enable a broader use if done slightly differently. That is, the "masked array" problem is a subset of this more-general problem. Please resp

Re: [Numpy-discussion] SVD does not converge

2011-06-28 Thread Lou Pecora
From: santhu kumar To: numpy-discussion@scipy.org Sent: Tue, June 28, 2011 11:56:48 AM Subject: [Numpy-discussion] SVD does not converge Hello, I have a 380X5 matrix and when I am calculating pseudo-inverse of the matrix using pinv(numpy.linalg) I get the fol

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Charles R Harris
On Tue, Jun 28, 2011 at 9:06 AM, Nathaniel Smith wrote: > On Mon, Jun 27, 2011 at 2:03 PM, Mark Wiebe wrote: > > On Mon, Jun 27, 2011 at 12:18 PM, Matthew Brett > > > wrote: > >> You won't get complaints, you'll just lose a group of users, who will, > >> I suspect, stick to NaNs, unsatisfactory

Re: [Numpy-discussion] SVD does not converge

2011-06-28 Thread Sebastian Berg
Hi, On Tue, 2011-06-28 at 10:56 -0500, santhu kumar wrote: > Hello, > > I have a 380X5 matrix and when I am calculating pseudo-inverse of the > matrix using pinv(numpy.linalg) I get the following error message: > > raise LinAlgError, 'SVD did not converge' > numpy.linalg.linalg.LinAlgError: SVD

Re: [Numpy-discussion] SVD does not converge

2011-06-28 Thread Charles R Harris
On Tue, Jun 28, 2011 at 9:56 AM, santhu kumar wrote: > Hello, > > I have a 380X5 matrix and when I am calculating pseudo-inverse of the > matrix using pinv(numpy.linalg) I get the following error message: > > raise LinAlgError, 'SVD did not converge' > numpy.linalg.linalg.LinAlgError: SVD did not

[Numpy-discussion] SVD does not converge

2011-06-28 Thread santhu kumar
Hello, I have a 380X5 matrix and when I am calculating pseudo-inverse of the matrix using pinv(numpy.linalg) I get the following error message: raise LinAlgError, 'SVD did not converge' numpy.linalg.linalg.LinAlgError: SVD did not converge I have looked in the list that it is a recurring issue b

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Nathaniel Smith
On Mon, Jun 27, 2011 at 2:03 PM, Mark Wiebe wrote: > On Mon, Jun 27, 2011 at 12:18 PM, Matthew Brett > wrote: >> You won't get complaints, you'll just lose a group of users, who will, >> I suspect, stick to NaNs, unsatisfactory as they are. > > This blade cuts both ways, we'd lose a group of user

[Numpy-discussion] תשובה: Recommndations for an easy GUI

2011-06-28 Thread Nadav Horesh
I tried Root’s advice and with the get_data method and GTK (without Agg) I got decent speed -- 30 fps (display speed, without the calculations overhead). The combination of matplotlib and glumpy resulted in 88 fps. I think I’ll have a solutionif glumpy lack of documentation will net get in th

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-28 Thread Lluís
Charles R Harris writes: > I think we may need some standard format for masked data on disk if we > don't go the NA value route. As I see it, the mask array is just some metadata that is attached to the dtype descriptor. I don't know how an ndarray is (un)pickled from disk, but I imagine that eac

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

2011-06-28 Thread Lluís
Mark Wiebe writes: > The design that's forming is a combination of: > * Solve the missing data problem  > * My ideas of what a good solution looks like: >    * applies to all NumPy dtypes in a fully general way >    * high-performance, low overhead where possible >    * makes the C-level implement

Re: [Numpy-discussion] missing data discussion round 2

2011-06-28 Thread Matthew Brett
Hi, On Mon, Jun 27, 2011 at 10:03 PM, Mark Wiebe wrote: > On Mon, Jun 27, 2011 at 12:18 PM, Matthew Brett ... >> That seems like a risky strategy to me, as the most likely outcome is >> that people worried about memory will avoid masked arrays because they >> know they use more memory.  The memo

Re: [Numpy-discussion] data type specification when using numpy.genfromtxt

2011-06-28 Thread Chao YUE
Thanks very much!! you are right. It's becuase the extra semicolon in the head row. I have no problems anymore. I thank you for your time. cheeers, Chao 2011/6/28 Derek Homeier > Hi Chao, > > by mistake did not reply to the list last time... > > On 27.06.2011, at 10:30PM, Chao YUE wrote: > H