Re: [Numpy-discussion] Missing data again

2012-03-15 Thread Nathaniel Smith
Hi Chuck, I think I let my frustration get the better of me, and the message below is too confrontational. I apologize. I truly would like to understand where you're coming from on this, though, so I'll try to make this more productive. My summary of points that no-one has disagreed with yet is h

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 7:39 PM, Benjamin Root wrote: > On Wed, Mar 7, 2012 at 1:26 PM, Nathaniel Smith wrote: >> When it comes to "missing data", bitpatterns can do everything that >> masks can do, are no more complicated to implement, and have better >> performance characteristics. >> > > Not tr

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 7:37 PM, Charles R Harris wrote: > > > On Wed, Mar 7, 2012 at 12:26 PM, Nathaniel Smith wrote: >> When it comes to "missing data", bitpatterns can do everything that >> masks can do, are no more complicated to implement, and have better >> performance characteristics. >> >

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Eric Firing
On 03/07/2012 11:15 AM, Pierre Haessig wrote: > Hi, > Le 07/03/2012 20:57, Eric Firing a écrit : >> In other words, good low-level support for numpy.ma functionality? > Coming back to *existing* ma support, I was just wondering whether it > was now possible to "np.save" a masked array. > (I'm using

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Pierre Haessig
Hi, Le 07/03/2012 20:57, Eric Firing a écrit : > In other words, good low-level support for numpy.ma functionality? Coming back to *existing* ma support, I was just wondering whether it was now possible to "np.save" a masked array. (I'm using numpy 1.5) In the end, this is the most annoying problem

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Eric Firing
On 03/07/2012 09:26 AM, Nathaniel Smith wrote: > On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris > wrote: >> On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig >>> Coming back to Travis proposition "bit-pattern approaches to missing >>> data (*at least* for float64 and int32) need to be implemented.

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Matthew Brett
Hi, On Wed, Mar 7, 2012 at 11:37 AM, Charles R Harris wrote: > > > On Wed, Mar 7, 2012 at 12:26 PM, Nathaniel Smith wrote: >> >> On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris >> wrote: >> > On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig >> > >> >> Coming back to Travis proposition "bit-patt

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Benjamin Root
On Wed, Mar 7, 2012 at 1:26 PM, Nathaniel Smith wrote: > On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris > wrote: > > On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig > > >> Coming back to Travis proposition "bit-pattern approaches to missing > >> data (*at least* for float64 and int32) need to

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Charles R Harris
On Wed, Mar 7, 2012 at 12:26 PM, Nathaniel Smith wrote: > On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris > wrote: > > On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig > > >> Coming back to Travis proposition "bit-pattern approaches to missing > >> data (*at least* for float64 and int32) need to

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris wrote: > On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig >> Coming back to Travis proposition "bit-pattern approaches to missing >> data (*at least* for float64 and int32) need to be implemented.", I >> wonder what is the amount of extra work to go

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Charles R Harris
On Wed, Mar 7, 2012 at 11:21 AM, Lluís wrote: > Charles R Harris writes: > [...] > > One inconvenience I have run into with the current API is that is should > be > > easier to clear the mask from an "ignored" value without taking a new > view or > > assigning known data. > > AFAIR, the inability

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Lluís
Charles R Harris writes: [...] > One inconvenience I have run into with the current API is that is should be > easier to clear the mask from an "ignored" value without taking a new view or > assigning known data. AFAIR, the inability to directly access a "mask" attribute was intentional to make bi

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Charles R Harris
On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig wrote: > Hi, > > Thanks you very much for your lights ! > > Le 06/03/2012 21:59, Nathaniel Smith a écrit : > > Right -- R has a very impoverished type system as compared to numpy. > > There's basically four types: "numeric" (meaning double precision >

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 4:35 PM, Pierre Haessig wrote: > Hi, > > Thanks you very much for your lights ! > > Le 06/03/2012 21:59, Nathaniel Smith a écrit : >> Right -- R has a very impoverished type system as compared to numpy. >> There's basically four types: "numeric" (meaning double precision >>

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Pierre Haessig
Hi, Thanks you very much for your lights ! Le 06/03/2012 21:59, Nathaniel Smith a écrit : > Right -- R has a very impoverished type system as compared to numpy. > There's basically four types: "numeric" (meaning double precision > float), "integer", "logical" (boolean), and "character" (string).

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Tue, Mar 6, 2012 at 9:14 PM, Ralf Gommers wrote: > On Tue, Mar 6, 2012 at 9:25 PM, Nathaniel Smith wrote: >> On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant >> wrote: >> > Hi all, >> >> Hi Travis, >> >> Thanks for bringing this back up. >> >> Have you looked at the summary from the last threa

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Ralf Gommers
On Tue, Mar 6, 2012 at 9:25 PM, Nathaniel Smith wrote: > On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant > wrote: > > Hi all, > > Hi Travis, > > Thanks for bringing this back up. > > Have you looked at the summary from the last thread? > https://github.com/njsmith/numpy/wiki/NA-discussion-statu

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Tue, Mar 6, 2012 at 4:38 PM, Mark Wiebe wrote: > On Tue, Mar 6, 2012 at 5:48 AM, Pierre Haessig > wrote: >> >From a potential user perspective, I feel it would be nice to have NA >> and non-NA cases look as similar as possible. Your code example is >> particularly striking : two different dtyp

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant wrote: > Hi all, Hi Travis, Thanks for bringing this back up. Have you looked at the summary from the last thread? https://github.com/njsmith/numpy/wiki/NA-discussion-status The goal was to try and at least work out what points we all *could* ag

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Mark Wiebe
Hi Pierre, On Tue, Mar 6, 2012 at 5:48 AM, Pierre Haessig wrote: > Hi Mark, > > I went through the NA NEP a few days ago, but only too quickly so that > my question is probably a rather dumb one. It's about the usability of > bitpatter-based NAs, based on your recent post : > > Le 03/03/2012 22:4

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Pierre Haessig
Hi Mark, I went through the NA NEP a few days ago, but only too quickly so that my question is probably a rather dumb one. It's about the usability of bitpatter-based NAs, based on your recent post : Le 03/03/2012 22:46, Mark Wiebe a écrit : > Also, here's a thought for the usability of NA-float6

Re: [Numpy-discussion] Missing data again

2012-03-03 Thread Skipper Seabold
On Sat, Mar 3, 2012 at 4:46 PM, Mark Wiebe wrote: > On Sat, Mar 3, 2012 at 12:30 PM, Travis Oliphant >> >>        * the reduction operations need to default to "skipna" --- this is >> the most common use case which has been re-inforced again to me today by a >> new user to Python who is using ma

Re: [Numpy-discussion] Missing data again

2012-03-03 Thread Travis Oliphant
> > Mind, Mark only had a few weeks to write code. I think the unfinished state > is a direct function of that. > > I have heard from several users that they will *not use the missing data* in > NumPy as currently implemented, and I can now see why.For better or for > worse, my approach t

Re: [Numpy-discussion] Missing data again

2012-03-03 Thread Charles R Harris
On Sat, Mar 3, 2012 at 1:30 PM, Travis Oliphant wrote: > Hi all, > > I've been thinking a lot about the masked array implementation lately. > I finally had the time to look hard at what has been done and now am of the > opinion that I do not think that 1.7 can be released with the current state >

Re: [Numpy-discussion] Missing data again

2012-03-03 Thread Ralf Gommers
On Sat, Mar 3, 2012 at 9:30 PM, Travis Oliphant wrote: > Hi all, > > I've been thinking a lot about the masked array implementation lately. > I finally had the time to look hard at what has been done and now am of the > opinion that I do not think that 1.7 can be released with the current state >

Re: [Numpy-discussion] Missing data again

2012-03-03 Thread Mark Wiebe
On Sat, Mar 3, 2012 at 12:30 PM, Travis Oliphant wrote: > > > First of all, I want to be clear that I think there is much great work > that has been done in the current missing data code. There are some nice > features in the where clause of the ufunc and the machinery for the > iterator that

[Numpy-discussion] Missing data again

2012-03-03 Thread Travis Oliphant
Hi all, I've been thinking a lot about the masked array implementation lately. I finally had the time to look hard at what has been done and now am of the opinion that I do not think that 1.7 can be released with the current state of the masked array implementation *unless* it is clearly m