Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-18 Thread Nathaniel Smith
On Sat, Feb 18, 2012 at 10:54 PM, Travis Oliphant wrote: > I'm reading very carefully any arguments against using C++ because I've > actually pushed back on Mark pretty hard as we've discussed these things over > the past months.  I am nervous about corner use-cases that will be unpleasant > fo

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-18 Thread Nathaniel Smith
On Sat, Feb 18, 2012 at 11:09 PM, David Cournapeau wrote: > On Sat, Feb 18, 2012 at 10:50 PM, Sturla Molden wrote: > >>  > In an ideal world, we would have a better language than C++ that can >> be spit out as > C for portability. >> >> What about a statically typed Python? (That is, not Cython.)

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 9:16 AM, David Cournapeau wrote: > On Sun, Feb 19, 2012 at 8:08 AM, Mark Wiebe wrote: >> Is there a specific >> target platform/compiler combination you're thinking of where we can do >> tests on this? I don't believe the compile times are as bad as many people >> suspect,

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 4:13 PM, xavier.gn...@gmail.com wrote: > I'm no sure. If you want to be able to write A=B+C+D; with decent > performances, I think you have to use a lib based on expression templates. > It would be great if C++ compilers could automatically optimize out > spurious copies in

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 7:13 PM, Mark Wiebe wrote: > On Sun, Feb 19, 2012 at 5:25 AM, Nathaniel Smith wrote: >> Precompiled headers can help some, but require complex and highly >> non-portable build-system support. (E.g., gcc's precompiled header >> constraints are h

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-19 Thread Nathaniel Smith
On Sun, Feb 19, 2012 at 1:42 PM, Neal Becker wrote: > On Fedora linux I use ccache, which is completely transparant and makes a huge > difference in build times. ccache is fabulous (and it's fabulous for C too), but it only helps when 'make' has screwed up and decided to rebuild some file that di

Re: [Numpy-discussion] Proposed Roadmap Overview

2012-02-21 Thread Nathaniel Smith
On Tue, Feb 21, 2012 at 4:04 AM, Travis Oliphant wrote: > It uses llvm-py (modified to work with LLVM 3.0) and code I wrote to do the > translation from Python byte-code to LLVM.   This LLVM can then be "JIT"ed. >   I have several applications that I would like to use this for.   It would > be pos

Re: [Numpy-discussion] np.longlong casts to int

2012-02-23 Thread Nathaniel Smith
On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted wrote: > Exactly.  I'd update this to read: > > float96    96 bits.  Only available on 32-bit (i386) platforms. > float128  128 bits.  Only available on 64-bit (AMD64) platforms. Except float96 is actually 80 bits. (Usually?) Plus some padding...

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-26 Thread Nathaniel Smith
On Sun, Feb 26, 2012 at 5:23 PM, Warren Weckesser wrote: > I haven't pushed it to the extreme, but the "big" example (in the examples/ > directory) is a 1 gig text file with 2 million rows and 50 fields in each > row.  This is read in less than 30 seconds (but that's with a solid state > drive).

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-26 Thread Nathaniel Smith
On Sun, Feb 26, 2012 at 7:16 PM, Warren Weckesser wrote: > On Sun, Feb 26, 2012 at 1:00 PM, Nathaniel Smith wrote: >> For this kind of benchmarking, you'd really rather be measuring the >> CPU time, or reading byte streams that are already in memory. If you >> can

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-26 Thread Nathaniel Smith
On Sun, Feb 26, 2012 at 7:58 PM, Warren Weckesser wrote: > Right, I got that.  Sorry if the placement of the notes about how to clear > the cache seemed to imply otherwise. OK, cool, np. >> Clearing the disk cache is very important for getting meaningful, >> repeatable benchmarks in code where y

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-27 Thread Nathaniel Smith
On Mon, Feb 27, 2012 at 2:44 PM, Erin Sheldon wrote: > What I've got is a solution for writing and reading structured arrays to > and from files, both in text files and binary files.  It is written in C > and python.  It allows reading arbitrary subsets of the data efficiently > without reading in

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-28 Thread Nathaniel Smith
[Re-adding the list to the To: field, after it got dropped accidentally] On Tue, Feb 28, 2012 at 12:28 AM, Erin Sheldon wrote: > Excerpts from Nathaniel Smith's message of Mon Feb 27 17:33:52 -0500 2012: >> On Mon, Feb 27, 2012 at 6:02 PM, Erin Sheldon wrote: >> > Excerpts from Nathaniel Smith's

Re: [Numpy-discussion] Determining the 'viewness' of an array, and flags.owndata confusion

2012-02-28 Thread Nathaniel Smith
On Tue, Feb 28, 2012 at 11:01 PM, Kurt Smith wrote: > For an arbitrary numpy array 'a', what does 'a.flags.owndata' indicate? I think what it really indicates is whether a's destructor should call free() on a's data pointer. > I originally thought that owndata is False iff 'a' is a view.  But >

Re: [Numpy-discussion] Possible roadmap addendum: building better text file readers

2012-02-29 Thread Nathaniel Smith
On Wed, Feb 29, 2012 at 3:11 PM, Erin Sheldon wrote: > Excerpts from Nathaniel Smith's message of Tue Feb 28 17:22:16 -0500 2012: >> > Even for binary, there are pathological cases, e.g. 1) reading a random >> > subset of nearly all rows.  2) reading a single column when rows are >> > small.  In c

Re: [Numpy-discussion] [Numpy] quadruple precision

2012-03-02 Thread Nathaniel Smith
On Mar 2, 2012 10:48 AM, "Paweł Biernat" wrote: > The portability is broken for numpy.float128 anyway (as I understand, > it behaves in different ways on different architectures), so adding a > new type (call it, say, quad128) that properly supports binary128 > shouldn't be a drawback. Later on, w

Re: [Numpy-discussion] verbose output when running python script?

2012-03-05 Thread Nathaniel Smith
Try the trace module in the standard library: http://docs.python.org/library/trace.html http://www.doughellmann.com/PyMOTW/trace/ - Nathaniel On Mar 5, 2012 3:27 PM, "Chao YUE" wrote: > Dear all, > > Sorry this is not the good place to ask but I think there must be someone > who has done this be

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant wrote: > Hi all, Hi Travis, Thanks for bringing this back up. Have you looked at the summary from the last thread? https://github.com/njsmith/numpy/wiki/NA-discussion-status The goal was to try and at least work out what points we all *could* ag

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Tue, Mar 6, 2012 at 4:38 PM, Mark Wiebe wrote: > On Tue, Mar 6, 2012 at 5:48 AM, Pierre Haessig > wrote: >> >From a potential user perspective, I feel it would be nice to have NA >> and non-NA cases look as similar as possible. Your code example is >> particularly striking : two different dtyp

Re: [Numpy-discussion] Missing data again

2012-03-06 Thread Nathaniel Smith
On Tue, Mar 6, 2012 at 9:14 PM, Ralf Gommers wrote: > On Tue, Mar 6, 2012 at 9:25 PM, Nathaniel Smith wrote: >> On Sat, Mar 3, 2012 at 8:30 PM, Travis Oliphant >> wrote: >> > Hi all, >> >> Hi Travis, >> >> Thanks for bringing this back up. &g

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 4:35 PM, Pierre Haessig wrote: > Hi, > > Thanks you very much for your lights ! > > Le 06/03/2012 21:59, Nathaniel Smith a écrit : >> Right -- R has a very impoverished type system as compared to numpy. >> There's basically four types: &qu

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 5:17 PM, Charles R Harris wrote: > On Wed, Mar 7, 2012 at 9:35 AM, Pierre Haessig >> Coming back to Travis proposition "bit-pattern approaches to missing >> data (*at least* for float64 and int32) need to be implemented.", I >> wonder what is the amount of extra work to go

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 7:37 PM, Charles R Harris wrote: > > > On Wed, Mar 7, 2012 at 12:26 PM, Nathaniel Smith wrote: >> When it comes to "missing data", bitpatterns can do everything that >> masks can do, are no more complicated to implement, and have better

Re: [Numpy-discussion] Missing data again

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 7:39 PM, Benjamin Root wrote: > On Wed, Mar 7, 2012 at 1:26 PM, Nathaniel Smith wrote: >> When it comes to "missing data", bitpatterns can do everything that >> masks can do, are no more complicated to implement, and have better >> performa

Re: [Numpy-discussion] use for missing (ignored) data?

2012-03-07 Thread Nathaniel Smith
On Wed, Mar 7, 2012 at 8:05 PM, Neal Becker wrote: > I'm wondering what is the use for the ignored data feature? > > I can use: > > A[valid_A_indexes] = whatever > > to process only the 'non-ignored' portions of A.  So at least some simple > cases > of ignored data are already supported without i

Re: [Numpy-discussion] draft enum NEP

2012-03-09 Thread Nathaniel Smith
On Fri, Mar 9, 2012 at 4:55 PM, Bryan Van de Ven wrote: > Hi all, > > I have started working on a NEP for adding an enumerated type to NumPy. > It is on my GitHub: > >     https://github.com/bryevdv/numpy/blob/enum/doc/neps/enum.rst > > It is still very rough, and incomplete in places. But I would

Re: [Numpy-discussion] Looking for people interested in helping with Python compiler to LLVM

2012-03-13 Thread Nathaniel Smith
On Tue, Mar 13, 2012 at 5:18 PM, Travis Oliphant wrote: > Cython and Numba certainly overlap.  However, Cython requires: > >        1) learning another language So is the goal for numba to actually handle arbitrary Python code with correct semantics, i.e., it's actually a compiled implementation

Re: [Numpy-discussion] draft enum NEP

2012-03-15 Thread Nathaniel Smith
On Wed, Mar 14, 2012 at 1:44 AM, Mark Wiebe wrote: > On Fri, Mar 9, 2012 at 8:55 AM, Bryan Van de Ven > wrote: >> >> Hi all, >> >> I have started working on a NEP for adding an enumerated type to NumPy. >> It is on my GitHub: >> >>     https://github.com/bryevdv/numpy/blob/enum/doc/neps/enum.rst

Re: [Numpy-discussion] Missing data again

2012-03-15 Thread Nathaniel Smith
: 1) Do you disagree with anything that is stated there? 2) Do you feel like that document accurately summarises your basic idea of what this feature is supposed to do (I assume under the IGNORED heading)? Thanks, -- Nathaniel On Wed, Mar 7, 2012 at 11:10 PM, Nathaniel Smith wrote: > On

Re: [Numpy-discussion] draft enum NEP

2012-03-16 Thread Nathaniel Smith
On Mar 16, 2012 1:02 AM, "Stéfan van der Walt" @ sun.ac.za > wrote: > > On Thu, Mar 15, 2012 at 4:02 PM, Nathaniel Smith @ pobox.com > wrote: > > I'm not sure what it would even mean to treat this kind of data as > > "flags", since you can't

Re: [Numpy-discussion] draft enum NEP

2012-03-17 Thread Nathaniel Smith
On Fri, Mar 16, 2012 at 4:26 PM, Bryan Van de Ven wrote: > Hi all, > > I have spent some time thinking about things, and discussing them with folks > nearby. I actually got to wondering whether we really need new dtypes for > this. It seems like enumerated values or factor levels could be cast as

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-29 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 5:13 PM, Travis Oliphant wrote: > While namespaces are a really good idea, I'm not a big fan of both "module" > namespaces and underscore namespaces.   It seems pretty redundant to me to > have pad.pad_mean. > > On the other hand, one could argue that pad.mean could be conf

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-29 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 5:38 PM, Nathaniel Smith wrote: > On Thu, Mar 29, 2012 at 5:13 PM, Travis Oliphant wrote: >> While namespaces are a really good idea, I'm not a big fan of both "module" >> namespaces and underscore namespaces.   It seems pretty redundant

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-29 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 5:45 PM, Charles R Harris wrote: > > > On Thu, Mar 29, 2012 at 10:38 AM, Nathaniel Smith wrote: >> >> On Thu, Mar 29, 2012 at 5:13 PM, Travis Oliphant >> wrote: >> > While namespaces are a really good idea, I'm not a big fan

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Fri, Mar 30, 2012 at 8:35 AM, Richard Hattersley wrote: > I like where this is going. > > Driven by a desire to avoid a million different methods on a single > class, we've done something similar in our library. > So instead of >   thing.mean() >   thing.max(...) >   etc. > we have: >   thi

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Thu, Mar 29, 2012 at 6:53 PM, Tim Cera wrote: > If instead you passed in a function: > >     def padwithzeros(vector, pad_width, iaxis, **kwargs): >         bvector = np.zeros(pad_width[0]) >         avector = np.zeros(pad_width[1]) >         return bvector, avector > >     b = pad(padwithzeros

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Fri, Mar 30, 2012 at 1:22 PM, Charles R Harris wrote: > > > On Fri, Mar 30, 2012 at 5:41 AM, Nathaniel Smith wrote: >> >> On Thu, Mar 29, 2012 at 6:53 PM, Tim Cera wrote: >> > If instead you passed in a function: >> > >> >     def

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-03-30 Thread Nathaniel Smith
On Fri, Mar 30, 2012 at 2:20 PM, Tim Cera wrote: >> My suggestion is: >> Step 1: Change the current PR so that it has only one user-exposed >> function, something like pad(..., mode="foo"), and commit that. >> Everyone seems to pretty much like that interface, implementing it >> would take <1 hour

Re: [Numpy-discussion] NumPy EIG much slower than MATLAB EIG

2012-04-02 Thread Nathaniel Smith
To see if this is an effect of numpy using C-order by default instead of Fortran-order, try measuring eig(x.T) instead of eig(x)? -n On Apr 1, 2012 2:28 PM, "Kamesh Krishnamurthy" wrote: > Hello all, > > I profiled NumPy EIG and MATLAB EIG on the same Macbook pro, and both were > linking to the

Re: [Numpy-discussion] NumPy EIG much slower than MATLAB EIG

2012-04-02 Thread Nathaniel Smith
On Mon, Apr 2, 2012 at 6:18 PM, Aronne Merrelli wrote: > On Sun, Apr 1, 2012 at 8:28 AM, Kamesh Krishnamurthy > wrote: >> Hello all, >> >> I profiled NumPy EIG and MATLAB EIG on the same Macbook pro, and both were >> linking to the Accelerate framework BLAS. NumPy turns out to be ~4x slower. >>

Re: [Numpy-discussion] Style for pad implementation in 'pad' namespace or functions under np.lib

2012-04-03 Thread Nathaniel Smith
On Mon, Apr 2, 2012 at 7:14 PM, Tim Cera wrote: >> >> I think the suggestion is pad(a, 5, mode='mean'), which would be >> consistent with common numpy signatures. The mode keyword should probably >> have a default, something commonly used. I'd suggest 'mean', Nathaniel >> suggests 'zero', I think

Re: [Numpy-discussion] Bitwise operations and unsigned types

2012-04-06 Thread Nathaniel Smith
On Fri, Apr 6, 2012 at 7:19 AM, Travis Oliphant wrote: > That is an interesting point of view.     I could see that point of view. >  But, was this discussed as a bug prior to this change occurring? > > I just heard from a very heavy user of NumPy that they are nervous about > upgrading because of

Re: [Numpy-discussion] Improving NumPy's indexing / subsetting / fancy indexing implementation

2012-04-06 Thread Nathaniel Smith
Hi Wes, I believe that Mark rewrote a bunch of the fancy-indexing-related code from scratch in the masked-NA branch. I don't know if it affects anything you're talking about here, but just as a heads up, you might want to benchmark master, since it may have a different performance profile. -- Nat

Re: [Numpy-discussion] Keyword argument support for vectorize.

2012-04-09 Thread Nathaniel Smith
On Mon, Apr 9, 2012 at 10:53 AM, Michael McNeil Forbes wrote: > It seems like functools.partial is the appropriate tool to use here > which means I will have to deal with the functools was added in Python 2.5, and so far numpy is still trying to maintain 2.4 compatibility. (Not that this is parti

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-09 Thread Nathaniel Smith
...isn't this an operation that will be performed once per compiled function? Is the overhead of the easy, robust method (calling ctypes.cast) actually measurable as compared to, you know, running an optimizing compiler? I mean, I doubt there'd be any real problem with adding this extra API to num

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 1:57 AM, Travis Oliphant wrote: > On Apr 9, 2012, at 7:21 PM, Nathaniel Smith wrote: > > ...isn't this an operation that will be performed once per compiled > function? Is the overhead of the easy, robust method (calling ctypes.cast) > actually measu

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 1:39 PM, Dag Sverre Seljebotn wrote: > On 04/10/2012 12:37 PM, Nathaniel Smith wrote: >> On Tue, Apr 10, 2012 at 1:57 AM, Travis Oliphant  wrote: >>> On Apr 9, 2012, at 7:21 PM, Nathaniel Smith wrote: >>> >>> ...isn't this an

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 2:15 PM, Dag Sverre Seljebotn wrote: > On 04/10/2012 03:10 PM, Dag Sverre Seljebotn wrote: >> On 04/10/2012 03:00 PM, Nathaniel Smith wrote: >>> On Tue, Apr 10, 2012 at 1:39 PM, Dag Sverre Seljebotn >>>   wrote: >>>> On 04/10/2012 12

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-10 Thread Nathaniel Smith
On Tue, Apr 10, 2012 at 2:38 PM, Dag Sverre Seljebotn wrote: > On 04/10/2012 03:29 PM, Nathaniel Smith wrote: >> Right, that's what I wasn't getting until you mentioned strcmp :-). >> >> That said, the core numpy dtypes are singletons. For this purpose, the >>

Re: [Numpy-discussion] Getting C-function pointers from Python to C

2012-04-12 Thread Nathaniel Smith
On Wed, Apr 11, 2012 at 10:23 PM, Travis Oliphant wrote: >>> In the mean-time, I think we could do as Robert essentially suggested and >>> just use Capsule Objects around an agreed-upon simple C-structure: >>> >>>      int   id   /* Some number that can be used as a "type-check" */ >>>      void

Re: [Numpy-discussion] 1.7 blockers

2012-04-16 Thread Nathaniel Smith
On Mon, Apr 16, 2012 at 10:09 PM, Ralf Gommers wrote: > > > On Sat, Mar 24, 2012 at 10:13 PM, Charles R Harris > wrote: >> >> Hi All, >> >> There several problems with numpy master that need to be fixed before a >> release can be considered. >> >> Datetime on windows with mingw. > > Opened http:/

Re: [Numpy-discussion] Different behaviour of python built sum and addition on ndarrays

2012-04-16 Thread Nathaniel Smith
On Mon, Apr 16, 2012 at 11:06 PM, Christopher Mutel wrote: > So, for both 1.5 and 1.6 (at least), it appears that the builtin sum > does not add ndarrays the way "+" (and operator.add) do: > > a = np.arange(10).reshape((2,5)) > b = np.arange(10, 20).reshape((2,5)) > sum(a,b) > Out[5]: > array([[15

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 6:44 AM, Travis Oliphant wrote: > Basically, there are two sets of changes as far as I understand right now: > >        1) ufunc infrastructure understands masked arrays >        2) ndarray grew attributes to represent masked arrays > > I am proposing that we keep 1) but ch

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett wrote: > Hi, > > On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant wrote: >> Mark and I will have conversations about NumPy while he is in Austin.   >> There are many other active stake-holders whose opinions and views are >> essential for major cha

[Numpy-discussion] the state of NA/masking

2012-04-22 Thread Nathaniel Smith
Hi all, Travis, Mark, and I talked on Skype this week about how to productively move forward with the NA debate, and I got picked to summarize for the list :-). There are three main things we discussed: 1) About process: We seem to agree that this discussion has been ineffective for a variety of

[Numpy-discussion] NEP mask code and the 1.7 release

2012-04-22 Thread Nathaniel Smith
We need to decide what to do with the NA masking code currently in master, vis-a-vis the 1.7 release. While this code is great at what it is, we don't actually have consensus yet that it's the best way to give our users what they want/need -- or even an appropriate way. So we need to figure out how

[Numpy-discussion] What is consensus anyway

2012-04-22 Thread Nathaniel Smith
If you hang around big FOSS projects, you'll see the word "consensus" come up a lot. For example, the glibc steering committee recently dissolved itself in favor of governance "directly by the consensus of the people active in glibc development"[1]. It's the governing rule of the IETF, which define

Re: [Numpy-discussion] What is consensus anyway

2012-04-23 Thread Nathaniel Smith
On Mon, Apr 23, 2012 at 1:04 AM, Charles R Harris wrote: > > > On Sun, Apr 22, 2012 at 4:15 PM, Nathaniel Smith wrote: >> >> If you hang around big FOSS projects, you'll see the word "consensus" >> come up a lot. For example, the glibc steering committe

Re: [Numpy-discussion] NEP mask code and the 1.7 release

2012-04-23 Thread Nathaniel Smith
On Mon, Apr 23, 2012 at 6:18 PM, Ralf Gommers wrote: > > > On Mon, Apr 23, 2012 at 12:15 AM, Nathaniel Smith wrote: >> >> We need to decide what to do with the NA masking code currently in >> master, vis-a-vis the 1.7 release. While this code is great at what it &g

Re: [Numpy-discussion] NEP mask code and the 1.7 release

2012-04-23 Thread Nathaniel Smith
On Mon, Apr 23, 2012 at 9:16 PM, Chris Barker wrote: > On Mon, Apr 23, 2012 at 12:57 PM, Nathaniel Smith wrote: >> Right, this part is specifically about ABI compatibility, not API >> compatibility -- segfaults would only occur for extension libraries >> that were compiled a

Re: [Numpy-discussion] Masked Arrays in NumPy 1.x

2012-04-23 Thread Nathaniel Smith
Hi Paul, On Wed, Apr 11, 2012 at 8:57 PM, Paul Hobson wrote: > Travis et al, > > This isn't a reply to anything specific in your email and I apologize > if there is a better thread or place to share this information. I've > been meaning to participate in the discussion for a long time and > never

Re: [Numpy-discussion] What is consensus anyway

2012-04-24 Thread Nathaniel Smith
On Tue, Apr 24, 2012 at 2:43 PM, Pierre Haessig wrote: > If the idea of having two payloads is to avoid a maximum of "skipna & > friends" extra keywords, I would like it much. My feeling with my small > experience with R is that I end up calling every function with a > different magical set of key

Re: [Numpy-discussion] What is consensus anyway

2012-04-24 Thread Nathaniel Smith
On Wed, Apr 25, 2012 at 12:49 AM, Charles R Harris wrote: > I think we adhere to these pretty well already, the problem is with the word > 'everyone'. I grew up in Massachusetts where town meetings were a tradition. > At those meetings the townsfolk voted on the budget, zoning, construction of > p

Re: [Numpy-discussion] What is consensus anyway

2012-04-24 Thread Nathaniel Smith
On Tue, Apr 24, 2012 at 2:14 PM, Charles R Harris wrote: > > > On Mon, Apr 23, 2012 at 11:35 PM, Fernando Perez > wrote: >> >> On Mon, Apr 23, 2012 at 8:49 PM, Stéfan van der Walt >> wrote: >> > If you are referring to the traditional concept of a fork, and not to >> > the type we frequently mak

Re: [Numpy-discussion] What is consensus anyway

2012-04-25 Thread Nathaniel Smith
On Wed, Apr 25, 2012 at 4:02 AM, Charles R Harris wrote: > > > On Tue, Apr 24, 2012 at 8:56 PM, Fernando Perez > wrote: >> >> On Tue, Apr 24, 2012 at 6:12 PM, Charles R Harris >> wrote: >> > I admit to a certain curiosity about your own involvement in FOSS >> > projects, >> > and I know I'm not

Re: [Numpy-discussion] What is consensus anyway

2012-04-25 Thread Nathaniel Smith
On Wed, Apr 25, 2012 at 1:03 PM, Charles R Harris wrote: > That is one of the reasons that the smaller > scikits attract people, they have more freedom to do what they want and > fewer people to answer to. Scipy also has some of that advantage because > there are a number of packages to choose fro

Re: [Numpy-discussion] A crazy masked-array thought

2012-04-27 Thread Nathaniel Smith
On Fri, Apr 27, 2012 at 11:32 AM, Richard Hattersley wrote: > I know used a somewhat jokey tone in my original posting, but fundamentally > it was a serious question concerning a live topic. So I'm curious about the > lack of response. Has this all been covered before? > > Sorry if I'm being too i

Re: [Numpy-discussion] A crazy masked-array thought

2012-04-28 Thread Nathaniel Smith
On Sat, Apr 28, 2012 at 7:38 AM, Richard Hattersley wrote: > So, assuming numpy.ndarray became a strict subclass of some new masked > array, it looks plausible that adding just a few checks to numpy.ndarray to > exclude the masked superclass would prevent much downstream code from > accidentally o

Re: [Numpy-discussion] sparse array data

2012-05-02 Thread Nathaniel Smith
On Wed, May 2, 2012 at 9:53 PM, Francesc Alted wrote: > On 5/2/12 11:16 AM, Wolfgang Kerzendorf wrote: >> Hi all, >> >> I'm currently writing a code that needs three dimensional data (for the >> physicists it's dimensions are atom, ion, level). The problem is that not >> all combinations do exis

Re: [Numpy-discussion] sparse array data

2012-05-02 Thread Nathaniel Smith
On Wed, May 2, 2012 at 11:26 PM, Francesc Alted wrote: > On 5/2/12 4:20 PM, Nathaniel Smith wrote: >> On Wed, May 2, 2012 at 9:53 PM, Francesc Alted  wrote: >>> On 5/2/12 11:16 AM, Wolfgang Kerzendorf wrote: >>>> Hi all, >>>> >>>> I'm

Re: [Numpy-discussion] sparse array data

2012-05-03 Thread Nathaniel Smith
On Thu, May 3, 2012 at 4:44 AM, Charles R Harris wrote: > > > On Wed, May 2, 2012 at 3:20 PM, Nathaniel Smith wrote: >> This coordinate format is also what's used by the MATLAB Tensor >> Toolbox. They have a paper justifying this choice and describing some >>

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Nathaniel Smith
On Wed, May 9, 2012 at 5:46 PM, Travis Oliphant wrote: > Hey all, > > Nathaniel and Mark have worked very hard on a joint document to try and > explain the current status of the missing-data debate.   I think they've > done an amazing job at providing some context, articulating their views and > s

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Nathaniel Smith
Hi Dag, On Wed, May 9, 2012 at 8:44 PM, Dag Sverre Seljebotn wrote: > I'm a heavy user of masks, which are used to make data NA in the > statistical sense. The setting is that we have to mask out the radiation > coming from the Milky Way in full-sky images of the Cosmic Microwave > Background. Th

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Nathaniel Smith
Hi Matthew, On Thu, May 10, 2012 at 12:01 AM, Matthew Brett wrote: >> The third proposal is certainly the best one from Cython's perspective; >> and I imagine for those writing C extensions against the C API too. >> Having PyType_Check fail for ndmasked is a very good way of having code >> fail t

[Numpy-discussion] Anyone have a cached copy of NA-discussion-status?

2012-05-11 Thread Nathaniel Smith
Hi all, I'm an idiot and seem to have accidentally deleted the NA-discussion-status web page. I do have a query into support@github, but, does anyone happen to have a local copy of the content, perhaps in their browser cache? Frustratedly yrs, -- Nathaniel

[Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-11 Thread Nathaniel Smith
I've been trying to sort through the changes that landed in master from the missingdata branch to figure out how to separate out changes related to NA support from those that aren't, and noticed that one of them should probably be flagged to the list. Traditionally, arr.diagonal() and np.diagonal(a

Re: [Numpy-discussion] Masking through generator arrays

2012-05-11 Thread Nathaniel Smith
On Thu, May 10, 2012 at 7:23 PM, Chris Barker wrote: > That is one of my concerns about the "bit pattern" idea -- we've then > created a new binary type that no other standard software understands > -- that looks like a a lot of work to me to deal with, or even worse, > ripe for weird, non-obvious

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-11 Thread Nathaniel Smith
On Fri, May 11, 2012 at 9:26 PM, T J wrote: > On Fri, May 11, 2012 at 1:12 PM, Mark Wiebe wrote: >> >> On Fri, May 11, 2012 at 2:18 PM, Pauli Virtanen wrote: >>> >>> 11.05.2012 17:54, Frédéric Bastien kirjoitti: >>> > In Theano we use a view, but that is not relevant as it is the >>> > compiler

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-13 Thread Nathaniel Smith
On Sun, May 13, 2012 at 3:28 AM, Travis Oliphant wrote: > Another approach would be to introduce a method: > > a.diag(copy=False) > > and leave a.diagonal() alone.  Then, a.diagonal() could be deprecated over > 2-3 releases. This would be a good idea if we didn't already have both np.diagonal(a)

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-16 Thread Nathaniel Smith
On Tue, May 15, 2012 at 2:49 PM, Frédéric Bastien wrote: > Hi, > > In fact, I would arg to never change the current behavior, but add the > flag for people that want to use it. > > Why? > > 1) There is probably >10k script that use it that will need to be > checked for correctness. There won't be

Re: [Numpy-discussion] Fancy-indexing reorders output in corner cases?

2012-05-16 Thread Nathaniel Smith
On Tue, May 15, 2012 at 5:03 AM, Travis Oliphant wrote: > So, the behavior is actually quite predictable, it's just that in some common > cases it doesn't do what you would expect --- especially if you think that > [0,1] is "the same" as :2.   When I wrote this code to begin with I should > hav

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-16 Thread Nathaniel Smith
On Wed, May 16, 2012 at 3:04 PM, Benjamin Root wrote: > > > On Wed, May 16, 2012 at 9:55 AM, Nathaniel Smith wrote: >> >> On Tue, May 15, 2012 at 2:49 PM, Frédéric Bastien wrote: >> > Hi, >> > >> > In fact, I would arg to never change the current

[Numpy-discussion] Scipy build can't find BLAS when using numpy master? (Was: Should arr.diagonal() return a copy or a view? (1.7 compatibility issue))

2012-05-16 Thread Nathaniel Smith
On Wed, May 16, 2012 at 3:15 PM, Robert Kern wrote: > On Wed, May 16, 2012 at 3:10 PM, Nathaniel Smith wrote: >> On Wed, May 16, 2012 at 3:04 PM, Benjamin Root wrote: > >>> Just as a sanity check, do the scipy tests run without producing any such >>> messages? >

Re: [Numpy-discussion] Scipy build can't find BLAS when using numpy master? (Was: Should arr.diagonal() return a copy or a view? (1.7 compatibility issue))

2012-05-16 Thread Nathaniel Smith
On Wed, May 16, 2012 at 4:24 PM, Robert Kern wrote: > On Wed, May 16, 2012 at 4:21 PM, Nathaniel Smith wrote: > >> I built some pristine python 2.7 installs from scratch (no virtualenv, >> no distro tweaks, etc.). Then I installed some version of numpy in >> each, then tr

Re: [Numpy-discussion] Scipy build can't find BLAS when using numpy master? (Was: Should arr.diagonal() return a copy or a view? (1.7 compatibility issue))

2012-05-16 Thread Nathaniel Smith
On Wed, May 16, 2012 at 4:50 PM, Robert Kern wrote: > On Wed, May 16, 2012 at 4:35 PM, Nathaniel Smith wrote: >> On Wed, May 16, 2012 at 4:24 PM, Robert Kern wrote: >>> On Wed, May 16, 2012 at 4:21 PM, Nathaniel Smith wrote: >>> >>>> I built some pristi

Re: [Numpy-discussion] Should arr.diagonal() return a copy or a view? (1.7 compatibility issue)

2012-05-16 Thread Nathaniel Smith
On Wed, May 16, 2012 at 3:04 PM, Benjamin Root wrote: > > > On Wed, May 16, 2012 at 9:55 AM, Nathaniel Smith wrote: >> >> On Tue, May 15, 2012 at 2:49 PM, Frédéric Bastien wrote: >> > Hi, >> > >> > In fact, I would arg to never change the current

Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

2012-05-17 Thread Nathaniel Smith
On Thu, May 17, 2012 at 7:50 PM, Stéfan van der Walt wrote: > On Wed, May 16, 2012 at 12:34 PM, Thouis Jones wrote: >> I wondered, however, if there were a better way to accomplish the same >> goal, preferably in pure python. > > Fabien recently posted this; not sure if it addresses your use case

Re: [Numpy-discussion] Masked Array for NumPy 1.7

2012-05-19 Thread Nathaniel Smith
On Sat, May 19, 2012 at 4:21 PM, Mark Wiebe wrote: > The motivation behind splitting the mask out into a separate ndmasked is > primarily so that pre-existing code will not silently function on NA-masked > arrays and produce incorrect results. This centres around using PyArray_DATA > to get at the

Re: [Numpy-discussion] Masked Array for NumPy 1.7

2012-05-19 Thread Nathaniel Smith
On Sat, May 19, 2012 at 5:45 PM, Charles R Harris wrote: > > > On Sat, May 19, 2012 at 10:02 AM, Charles R Harris > wrote: >> >> >> >> On Sat, May 19, 2012 at 9:21 AM, Mark Wiebe wrote: >>> >>> On Sat, May 19, 2012 at 10:00 AM, David Cournapeau >>> wrote: On Sat, May 19, 2012 at 3:17

[Numpy-discussion] Separating out the maskna code

2012-05-19 Thread Nathaniel Smith
Hi all, Since Mark's original missingdata branch made so many changes, I figured it would be a useful exercise to figure out what code in master is actually related to masked arrays, and which isn't. The easiest way seemed to be to delete the new fields, then keep removing any code that depended o

Re: [Numpy-discussion] Internationalization of numpy/scipy docstrings...

2012-05-19 Thread Nathaniel Smith
On May 19, 2012 11:04 PM, "Tim Cera" wrote: > A user would then install a language kit, maybe something like scikits and access the translated docstring with a new 'np.info'. As near as I can figure, Python 'help' command can't be replaced by something else, so 'help' would always display the Eng

Re: [Numpy-discussion] Separating out the maskna code

2012-05-20 Thread Nathaniel Smith
On Sun, May 20, 2012 at 6:08 AM, Travis Oliphant wrote: > Wow, Nathaniel.   This looks like a nice piece of tedious work. Honestly, it only took a few hours -- M-x grep is awesome. Would still have been better if it'd been separated in the first place, but so it goes. > I have not reviewed it in

Re: [Numpy-discussion] Separating out the maskna code

2012-05-20 Thread Nathaniel Smith
On Sun, May 20, 2012 at 11:35 AM, Ralf Gommers wrote: > > > On Sat, May 19, 2012 at 10:54 PM, Nathaniel Smith wrote: >> >> >> Ralf, IIUC merging this and my other outstanding PRs would leave the >> datetime issues on python3/win32 as the only outstanding blocke

Re: [Numpy-discussion] Separating out the maskna code

2012-05-20 Thread Nathaniel Smith
On Sun, May 20, 2012 at 6:59 PM, Nathaniel Smith wrote: >> I have not reviewed it in detail, but in general I would be very supportive >> of your plan to commit this to master, make a 1.7 release (without the >> ReduceWrapper) function and then work on the masked array / ndar

Re: [Numpy-discussion] Internationalization of numpy/scipy docstrings...

2012-05-21 Thread Nathaniel Smith
On Mon, May 21, 2012 at 10:44 PM, Ralf Gommers wrote: > Thinking about what languages to translate into would also make sense, since > having a bunch of partial translations lying around doesn't help anyone. > First thought: Spanish, Chinese. It's not like one can tell two translator volunteers t

[Numpy-discussion] Trivial pull request: tox support

2012-05-21 Thread Nathaniel Smith
I got tired of juggling virtualenvs, and probably everyone else would soon get tired of pointing out Python 2.4 incompatibilities I'd forgotten to test, so: https://github.com/numpy/numpy/pull/285 - N ___ NumPy-Discussion mailing list NumPy-Discussion@

[Numpy-discussion] un-silencing Numpy's deprecation warnings

2012-05-22 Thread Nathaniel Smith
So starting in Python 2.7 and 3.2, the Python developers have made DeprecationWarnings invisible by default: http://docs.python.org/whatsnew/2.7.html#the-future-for-python-2-x http://mail.python.org/pipermail/stdlib-sig/2009-November/000789.html http://bugs.python.org/issue7319 The only way t

Re: [Numpy-discussion] Separating out the maskna code

2012-05-22 Thread Nathaniel Smith
, 2012, at 1:06 PM, Nathaniel Smith wrote: > >> On Sun, May 20, 2012 at 6:59 PM, Nathaniel Smith wrote: >>>> I have not reviewed it in detail, but in general I would be very >>>> supportive of your plan to commit this to master, make a 1.7 release >>>&

Re: [Numpy-discussion] un-silencing Numpy's deprecation warnings

2012-05-22 Thread Nathaniel Smith
On Tue, May 22, 2012 at 11:06 AM, Robert Kern wrote: > On Tue, May 22, 2012 at 9:27 AM, Nathaniel Smith wrote: >> So starting in Python 2.7 and 3.2, the Python developers have made >> DeprecationWarnings invisible by default: >>  http://docs.python.org/whatsnew/2.7.html#the-

Re: [Numpy-discussion] subclassing ndarray subtleties??

2012-05-22 Thread Nathaniel Smith
On Mon, May 21, 2012 at 6:47 PM, Tom Aldcroft wrote: > Over on the scipy-user mailing list there was a question about > subclassing ndarray and I was interested to see two responses that > seemed to imply that subclassing should be avoided. > > >From Dag and Nathaniel, respectively: > > "Subclassi

  1   2   3   4   5   6   7   8   9   10   >