Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-12 Thread Chris.Barker
On 12/11/11 8:40 AM, Ralf Gommers wrote: > On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker * If we have a good, fast ascii (or unicode?) to array reader, hopefully > it could be leveraged for use in the more complex cases. So that rather > than genfromtxt() being written from sc

Re: [Numpy-discussion] Problem with using PyArray_AsCArray

2011-12-09 Thread Chris.Barker
On 12/9/11 11:25 AM, Ng, Enrico wrote: > I am trying to pass a multi-dimensional ndarray to C as a multi-dimensional C > array for the purposes of passing it to mathematica. I am using > PyArray_AsCArray but getting an error. I understand that SWIG, Boost, et. al are perhaps too heavyweight for

[Numpy-discussion] Fast Reading of ASCII files

2011-12-07 Thread Chris.Barker
Hi folks, This is a continuation of a conversation already started, but i gave it a new, more appropriate, thread and subject. On 12/6/11 2:13 PM, Wes McKinney wrote: > we should start talking > about building a *high performance* flat file loading solution with > good column type inference and

[Numpy-discussion] subclassing ndarray

2011-11-21 Thread Chris.Barker
Hi folks, I'm working on a "ragged array" class -- an array that can store and work with what can be considered tabular data, with the rows of different lengths: """ ragged_array A "ragged" array class -- build on numpy The idea is to be able to store data that is essentially 2-d, but each

Re: [Numpy-discussion] dedicated function for resize with averaging or rebin 2d arrays?

2011-11-15 Thread Chris.Barker
On 11/14/11 2:02 PM, Andrea Zonca wrote: On Mon, Nov 14, 2011 at 09:24, Chris.Barker wrote: On 11/11/11 8:28 PM, Craig Yoshioka wrote: I once wrote a generic n-dimensional binning routine in C that I could find if anyone is interested in integrating it into numpy... it didn't do

Re: [Numpy-discussion] how to use the name of a ndarray as a string

2011-11-14 Thread Chris.Barker
On 11/10/11 3:57 AM, Olivier Delalleau wrote: > In such a situation you should probably use a dictionary from the start, all good suggestions, but while we're at it: On 11/10/11 2:17 AM, Chao YUE wrote: > Does anyone know how I can quickly use the name of a ndarray as a string? This reflects a k

Re: [Numpy-discussion] dedicated function for resize with averaging or rebin 2d arrays?

2011-11-14 Thread Chris.Barker
On 11/11/11 8:28 PM, Craig Yoshioka wrote: > I once wrote a generic n-dimensional binning routine in C that I > could find if anyone is interested in integrating it into numpy... it > didn't do size increases though... and I think I implemented it so > that binning by a non-divisible factor trimmed

Re: [Numpy-discussion] Rebinning numpy array

2011-11-14 Thread Chris.Barker
On 11/13/11 9:55 AM, Olivier Delalleau wrote: > idea, since it will throw out a lot of information if you decrease the > number of bins: I agree -- I'd think about looking at a smooth interpolation -- maybe kernel density estimation? On 11/14/11 8:12 AM, Sturla Molden wrote: > Fit a poisson dist

Re: [Numpy-discussion] Cross-Compiling Numpy for NAO

2011-11-08 Thread Chris.Barker
On 11/7/11 10:16 AM, Carlos Neves wrote: > I am a robotics student and I have used numpy to develop a controller > for a humanoid robot. Now I am trying to implement the same controller > to a different robot, NAO - http://www.aldebaran-robotics.com/ > This robot has a Linux based OS with Python 2.

Re: [Numpy-discussion] in the NA discussion, what can we agree on?

2011-11-03 Thread Chris.Barker
On 11/2/11 7:16 PM, Nathaniel Smith wrote: > By R compatibility, I specifically had in mind in-memory > compatibility. The R crowd has had a big voice in this discussion, and I understand that there are some nice lessons to be learned from it with regard to the NA issues. However, I think makin

Re: [Numpy-discussion] float64 / int comparison different from float / int comparison

2011-11-01 Thread Chris.Barker
On 10/31/11 6:38 PM, Stéfan van der Walt wrote: > On Mon, Oct 31, 2011 at 6:25 PM, Matthew Brett > wrote: >> Oh, dear, I'm suffering now: >> In [12]: res> 2**31-1 >> Out[12]: array([False], dtype=bool) > I'm seeing: ... > Your result seems very strange, because the numpy scalars should > perf

Re: [Numpy-discussion] NA masks in the next numpy release?

2011-10-28 Thread Chris.Barker
On 10/28/11 11:37 AM, Matthew Brett wrote: > The main motivation for the alterNEP was our strong feeling that > separating ABSENT and IGNORE was easier to comprehend and cleaner. I don't know about easier to comprehend, or cleaner, but it is more feature-full. I see two issues here: 1) being ab

Re: [Numpy-discussion] NA masks in the next numpy release?

2011-10-28 Thread Chris.Barker
On 10/27/11 7:51 PM, Travis Oliphant wrote: > As I mentioned. I find the ability to separate an ABSENT idea from an > IGNORED idea convincing. In other words, I think distinguishing between > masks and bit-patterns is not just an implementation detail, but > provides a useful concept for multiple u

Re: [Numpy-discussion] yet another indexing question

2011-10-17 Thread Chris.Barker
On 10/14/11 5:04 AM, Neal Becker wrote: > suppose I have: > > In [10]: u > Out[10]: > array([[0, 1, 2, 3, 4], > [5, 6, 7, 8, 9]]) > > And I have a vector v: > v = np.array ((0,1,0,1,0)) > > I want to form an output vector which selects items from u where v is the > index > of the row of

Re: [Numpy-discussion] NumPy foundations

2011-10-13 Thread Chris.Barker
On 10/13/11 6:03 AM, Linus Jundén wrote: > I am about to make a NumPy presentation for my colleges in about a > week. I want to tell them something about the history of the library > and what kind of code it relies on. > Is NumPy based on some external code like e.g. BLAS, LAPACK etc or is > it co

Re: [Numpy-discussion] array.tofile() refuses to write into a StringIO

2011-09-27 Thread Chris.Barker
On 9/27/11 2:14 AM, oc-spam66 wrote: >> if you want to write to a string, why not use .tostring()? > > Because A.tostring() returns the binary data, while I would like the text > representation. > More precisely, I would like to use A.tofile(sep="\t"). I see -- I've always thought mingling bina

Re: [Numpy-discussion] array.tofile() refuses to write into a StringIO

2011-09-25 Thread Chris.Barker
> On 25.09.2011 18:23, OC wrote: >> as said in the subject, the following code produces an error. Is it normal ? >> >> ** >> A = r_[1] >> file_buffer = StringIO() >> A.tofile(file_buffer) >> >> IOError: first argument must be a string or open

Re: [Numpy-discussion] load from text files Pull Request Review

2011-09-13 Thread Chris.Barker
On 9/12/11 4:38 PM, Christopher Jordan-Squire wrote: > I did some timings to see what the advantage would be, in the simplest > case possible, of taking multiple lines from the file to process at a > time. Nice work, only a minor comment: > f6 and f7 use stripped down versions of Chris > Barker's

Re: [Numpy-discussion] load from text files Pull Request Review

2011-09-08 Thread Chris.Barker
On 9/8/11 1:43 PM, Christopher Jordan-Squire wrote: > I just ran a quick test on my machine of this idea. With > > dt = np.dtype([('x',np.float32),('y', np.int32),('z', np.float64)]) > temp = np.empty((), dtype=dt) > temp2 = np.zeros(1,dtype=dt) > > In [96]: def f(): > ...: l=[0]*3 >

Re: [Numpy-discussion] load from text files Pull Request Review

2011-09-07 Thread Chris.Barker
On 9/2/11 2:45 PM, Christopher Jordan-Squire wrote: > It doesn't have to parse the entire file to determine the dtypes. It > builds up a regular expression for what it expects to see, in terms of > dtypes. Then it just loops over the lines, only parsing if the regular > expression doesn't match. It

Re: [Numpy-discussion] load from text files Pull Request Review

2011-09-02 Thread Chris.Barker
On 9/2/11 9:16 AM, Christopher Jordan-Squire wrote: >>> I agree it would make a very nice addition, and could complement my >>> pre-allocation option for loadtxt - however there I've also been made >>> aware that this approach breaks streamed input etc., so the buffer.resize(…) >>> methods in accum

Re: [Numpy-discussion] load from text files Pull Request Review

2011-09-02 Thread Chris.Barker
On 9/2/11 8:22 AM, Derek Homeier wrote: > I agree it would make a very nice addition, and could complement my > pre-allocation option for loadtxt - however there I've also been made > aware that this approach breaks streamed input etc., so the buffer.resize(…) > methods in accumulator would be the

Re: [Numpy-discussion] Numpy performance boost

2011-08-31 Thread Chris.Barker
On 8/31/11 3:58 AM, Dieter Weber wrote: > just wanted to show an example of how python3 + numpy compares with just > python3 and many other languages and language implementations: > http://shootout.alioth.debian.org/u64q/performance.php?test=mandelbrot#about hmmm - it would be interesting to see w

Re: [Numpy-discussion] load from text files Pull Request Review

2011-08-30 Thread Chris.Barker
On 8/27/11 11:08 AM, Christopher Jordan-Squire wrote: > I've submitted a pull request for a new method for loading data from > text files into a record array/masked record array. > Click on the link for more info, but the general idea is to create a > regular expression for what entries should loo

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-26 Thread Chris.Barker
On 8/26/11 5:04 AM, Derek Homeier wrote: > Hmm, the pure Python version might be, but, I've used cPickle for a long time > and never noted any stability problems. well, here is the NEP: https://github.com/numpy/numpy/blob/master/doc/neps/npy-format.txt It addresses the why's and hows of the for

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-25 Thread Chris.Barker
On 8/24/11 9:22 AM, Anthony Scopatz wrote: > You can use Python pickling, if you do *not* have a requirement for: I can't recall why, but it seem pickling of numpy arrays has been fragile and not very performant. I like the npy / npz format, built in to numpy, if you don't need: > - acc

Re: [Numpy-discussion] numpy array serialization with JSON

2008-11-11 Thread Chris.Barker
Simon Palmer wrote: > "Does JSON have a representation for n-d arrays? In my little work with > it, it looked pretty lame for arrays of number, so I'd be surprised." > > yes it does, thet are just treated as nested lists and the square > bracket notation is used. then it looks like one of str(ar

Re: [Numpy-discussion] Using 2-D arrays in Numeric Python (numpy)

2008-10-13 Thread Chris.Barker
Linda Seltzer wrote: > I would appreciate it if someone could answer my question without > referring to subjects such as APIs and interfaces, since I am only > concerned with a mathematical application at this time. caution: this is a bit rude -- that was an excellent and informative answer to yo

Re: [Numpy-discussion] Using 2-D arrays in Numeric Python (numpy)

2008-10-13 Thread Chris.Barker
Bill Baxter wrote: >import numpy as npy Bill, for what it's worth, I *think* this group has reached a consensus to use: import numpy as np We all have different tastes for how they might want to spell it, but the more consistent we are, the easier it will be for newbies. -Chris -- Chri

Re: [Numpy-discussion] Binary ufuncs: minimum

2008-05-28 Thread Chris.Barker
Keith Goodman wrote: >> Interestingly, MATLAB (v7.5.0) takes a different approach: >> ans = >> 1271 -A >> ans = >> 127 -1 can anyone explain that? -- just curious. Charles R Harris wrote: > We could simply define the range of int8 as [-127,127], but that is > somewhat problematical

Re: [Numpy-discussion] Uncomfortable with matrix change

2008-05-11 Thread Chris.Barker
Could we add a "from __future__ import something" along with a deprecation warning? This could be used for Tim's "new matrix" class, or any other API change. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600

[Numpy-discussion] Broadcasting question

2008-05-01 Thread Chris.Barker
Hi all, I have a n X m X 3 array, and I a n X M array. I want to assign the values in the n X m to all three of the slices in the bigger array: A1 = np.zeros((5,4,3)) A2 = np.ones((5,4)) A1[:,:,0] = A2 A1[:,:,1] = A2 A1[:,:,2] = A2 However,it seems I should be able to broadcast that, so I don't

Re: [Numpy-discussion] recarray fun

2008-04-30 Thread Chris.Barker
Travis E. Oliphant wrote: > Stéfan van der Walt wrote: >> 2008/4/30 Christopher Barker <[EMAIL PROTECTED]>: >>> Since it is optional, shouldn't it be keyword argument? >>> >> Thanks, fixed in r5115. >> >> > This was too hasty. I had considered this before. > > The problem with this is t