Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Olivier Delalleau
For the hardcoded part, you can easily read the first line of your file and split it with the same delimiter to know the number of columns. It's sure it'd be best to be able to be able to skip this part, but you don't need to hardcode this number into your code at least. Something like: n_cols = le

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread gary ruben
Thanks guys - I'm happy with the solution for now. FYI, Derek's suggestion doesn't work in numpy 1.5.1 either. For any developers following this thread, I think this might be a nice use case for genfromtxt to handle in future. As a corollary of this problem, I wonder whether there's a human-readabl

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Derek Homeier
On 17.06.2011, at 11:01PM, Olivier Delalleau wrote: >> You were just overdoing it by already creating an array with the converter, >> this apparently caused genfromtxt to create a structured array from the >> input (which could be converted back to an ndarray, but that can prove >> tricky as we

Re: [Numpy-discussion] crash in multiarray

2011-06-17 Thread Benjamin Root
On Fri, Jun 17, 2011 at 3:48 PM, Pauli Virtanen wrote: > On Fri, 17 Jun 2011 15:39:21 -0500, Robert Kern wrote: > [clip] > >> File > >> > "/home/bvr/Programs/scipy/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", > >> line 378 in test_ticket_1459_arpack_crash > > > > Well, if it's testing

Re: [Numpy-discussion] code review/build & test for datetime business day API

2011-06-17 Thread Mark Wiebe
On Fri, Jun 17, 2011 at 2:46 PM, Derek Homeier < de...@astro.physik.uni-goettingen.de> wrote: > On 17.06.2011, at 8:05PM, Mark Wiebe wrote: > > > On Thu, Jun 16, 2011 at 8:18 PM, Derek Homeier < > de...@astro.physik.uni-goettingen.de> wrote: > >> On 17.06.2011, at 2:02AM, Mark Wiebe wrote: > >> >

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Olivier Delalleau
2011/6/17 Derek Homeier > Hi Gary, > > On 17.06.2011, at 5:39PM, gary ruben wrote: > > Thanks for the hints Olivier and Bruce. Based on them, the following > > is a working solution, although I still have that itchy sense that > genfromtxt > > should be able to do it directly. > > > > import nump

Re: [Numpy-discussion] numpy build: automatic fortran detection

2011-06-17 Thread Ralf Gommers
On Mon, Jun 13, 2011 at 8:53 PM, Russell E. Owen wrote: > In article , > Ralf Gommers wrote: > > > On Thu, Jun 9, 2011 at 11:46 PM, Russell E. Owen wrote: > > > > > What would it take to automatically detect which flavor of fortran to > > > use to build numpy on linux? > > > > > > > You want t

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Derek Homeier
Hi Gary, On 17.06.2011, at 5:39PM, gary ruben wrote: > Thanks for the hints Olivier and Bruce. Based on them, the following > is a working solution, although I still have that itchy sense that genfromtxt > should be able to do it directly. > > import numpy as np > from StringIO import StringIO >

Re: [Numpy-discussion] crash in multiarray

2011-06-17 Thread Pauli Virtanen
On Fri, 17 Jun 2011 15:39:21 -0500, Robert Kern wrote: [clip] >> File >> "/home/bvr/Programs/scipy/scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py", >> line 378 in test_ticket_1459_arpack_crash > > Well, if it's testing that it crashes, test passed! > > Here is the referenced ticket and the

Re: [Numpy-discussion] crash in multiarray

2011-06-17 Thread Robert Kern
On Fri, Jun 17, 2011 at 15:18, Benjamin Root wrote: > On Fri, Jun 17, 2011 at 3:07 PM, Robert Kern wrote: >> >> On Fri, Jun 17, 2011 at 15:03, Benjamin Root wrote: >> > Using the master branch, I was running the scipy tests when a crash >> > occurred.  I believe the crash originates within numpy

Re: [Numpy-discussion] crash in multiarray

2011-06-17 Thread Benjamin Root
On Fri, Jun 17, 2011 at 3:07 PM, Robert Kern wrote: > On Fri, Jun 17, 2011 at 15:03, Benjamin Root wrote: > > Using the master branch, I was running the scipy tests when a crash > > occurred. I believe the crash originates within numpy. The tests for > numpy > > do pass on my machine (F15, x86

Re: [Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Robert Kern
On Fri, Jun 17, 2011 at 14:26, Christopher Barker wrote: > I guess I'm confused about the difference between: > > np.dtype(uint16) > > and > > np.uint16 > > since I always pass in the later when a dtype is called for. Whenever a dtype= argument is present, we also accept anything that can be con

Re: [Numpy-discussion] crash in multiarray

2011-06-17 Thread Robert Kern
On Fri, Jun 17, 2011 at 15:03, Benjamin Root wrote: > Using the master branch, I was running the scipy tests when a crash > occurred.  I believe the crash originates within numpy.  The tests for numpy > do pass on my machine (F15, x86_64, python 2.7).  Below is the backtrace: Please run "scipy.te

[Numpy-discussion] crash in multiarray

2011-06-17 Thread Benjamin Root
Using the master branch, I was running the scipy tests when a crash occurred. I believe the crash originates within numpy. The tests for numpy do pass on my machine (F15, x86_64, python 2.7). Below is the backtrace: *** glibc detected *** python: double free or corruption (!prev): 0x052

Re: [Numpy-discussion] code review/build & test for datetime business day API

2011-06-17 Thread Derek Homeier
On 17.06.2011, at 8:05PM, Mark Wiebe wrote: > On Thu, Jun 16, 2011 at 8:18 PM, Derek Homeier > wrote: >> On 17.06.2011, at 2:02AM, Mark Wiebe wrote: >> >> >> ok, that was a lengthy hunt, but it's in printing the string in >> >> make_iso_8601_date: >> >> >> >>tmplen = snprintf(substr, suble

Re: [Numpy-discussion] unwrap for masked arrays?

2011-06-17 Thread Benjamin Root
On Fri, Jun 17, 2011 at 1:26 PM, Eric Firing wrote: > On 06/17/2011 06:56 AM, Benjamin Root wrote: > > It does not appear that unwrap works properly for masked arrays. First, > > it uses np.asarray() at the start of the function. However, that alone > > would not fix the problem given the natur

Re: [Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Christopher Barker
Robert Kern wrote: > On Fri, Jun 17, 2011 at 13:27, Christopher Barker > wrote: > >> Actually, I'm a bit confused about dtypes from an OO design perspective >> anyway. I note that the dtypes seem to have all (most?) of the methods >> of ndarrays (or placeholders, anyway), which I don't quite get

Re: [Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Bruce Southey
On 06/17/2011 12:38 PM, Christopher Barker wrote: > Hi all, > > I'm wondering if there is a way to get the range of values a given dtype > can hold? > > essentially, I'm looking for something like sys.maxint, for for whatever > numpy dtype I have n hand (and eps for floating point types). > > I was

Re: [Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Robert Kern
On Fri, Jun 17, 2011 at 13:27, Christopher Barker wrote: > Actually, I'm a bit confused about dtypes from an OO design perspective > anyway. I note that the dtypes seem to have all (most?) of the methods > of ndarrays (or placeholders, anyway), which I don't quite get. No, they don't. [~] |33>

Re: [Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Christopher Barker
Fabrice Silva wrote: >> I'm wondering if there is a way to get the range of values a given dtype >> can hold? > http://docs.scipy.org/doc/numpy/reference/routines.dtype.html#data-type-information yup, that does it, I hadn't found that, as I was expecting a more OO way, i.e. as a method of the d

Re: [Numpy-discussion] unwrap for masked arrays?

2011-06-17 Thread Eric Firing
On 06/17/2011 06:56 AM, Benjamin Root wrote: > It does not appear that unwrap works properly for masked arrays. First, > it uses np.asarray() at the start of the function. However, that alone > would not fix the problem given the nature of how unwrap works > (performing diff operations). I tried

Re: [Numpy-discussion] code review/build & test for datetime business day API

2011-06-17 Thread Mark Wiebe
On Thu, Jun 16, 2011 at 8:18 PM, Derek Homeier < de...@astro.physik.uni-goettingen.de> wrote: > On 17.06.2011, at 2:02AM, Mark Wiebe wrote: > > >> ok, that was a lengthy hunt, but it's in printing the string in > make_iso_8601_date: > >> > >>tmplen = snprintf(substr, sublen, "%04" NPY_INT64_FM

Re: [Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Fabrice Silva
Le vendredi 17 juin 2011 à 10:38 -0700, Christopher Barker a écrit : > Hi all, > > I'm wondering if there is a way to get the range of values a given dtype > can hold? > > essentially, I'm looking for something like sys.maxint, for for whatever > numpy dtype I have n hand (and eps for floating

[Numpy-discussion] get a dtypes' range?

2011-06-17 Thread Christopher Barker
Hi all, I'm wondering if there is a way to get the range of values a given dtype can hold? essentially, I'm looking for something like sys.maxint, for for whatever numpy dtype I have n hand (and eps for floating point types). I was hoping there would be something like a_dtype.max_value a_dtyp

[Numpy-discussion] unwrap for masked arrays?

2011-06-17 Thread Benjamin Root
It does not appear that unwrap works properly for masked arrays. First, it uses np.asarray() at the start of the function. However, that alone would not fix the problem given the nature of how unwrap works (performing diff operations). I tried a slightly modified version of unwrap, but I could n

Re: [Numpy-discussion] Unicode characters in a numpy array

2011-06-17 Thread Gökhan Sever
On Thu, Jun 16, 2011 at 8:54 PM, Charles R Harris wrote: > > > On Wed, Jun 15, 2011 at 1:30 PM, Gökhan Sever wrote: >> >> Hello, >> >> The following snippet works fine for a regular string and prints out >> the string without a problem. >> >> python >> Python 2.7 (r27:82500, Sep 16 2010, 18:02:00

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread gary ruben
Thanks for the hints Olivier and Bruce. Based on them, the following is a working solution, although I still have that itchy sense that genfromtxt should be able to do it directly. import numpy as np from StringIO import StringIO a = StringIO('''\ (-3.9700,-5.0400) (-1.1318,-2.5693) (-4.6027,-0.

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Bruce Southey
On 06/17/2011 08:51 AM, Olivier Delalleau wrote: 2011/6/17 Bruce Southey mailto:bsout...@gmail.com>> On 06/17/2011 08:22 AM, gary ruben wrote: > Thanks Olivier, > Your suggestion gets me a little closer to what I want, but doesn't > quite work. Replacing the conversion with

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Olivier Delalleau
2011/6/17 Bruce Southey > On 06/17/2011 08:22 AM, gary ruben wrote: > > Thanks Olivier, > > Your suggestion gets me a little closer to what I want, but doesn't > > quite work. Replacing the conversion with > > > > c = lambda x:np.cast[np.complex64](complex(*eval(x))) > > b = np.genfromtxt(a,conve

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Bruce Southey
On 06/17/2011 08:22 AM, gary ruben wrote: > Thanks Olivier, > Your suggestion gets me a little closer to what I want, but doesn't > quite work. Replacing the conversion with > > c = lambda x:np.cast[np.complex64](complex(*eval(x))) > b = np.genfromtxt(a,converters={0:c, 1:c, 2:c, > 3:c},dtype=None,

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread gary ruben
Thanks Olivier, Your suggestion gets me a little closer to what I want, but doesn't quite work. Replacing the conversion with c = lambda x:np.cast[np.complex64](complex(*eval(x))) b = np.genfromtxt(a,converters={0:c, 1:c, 2:c, 3:c},dtype=None,delimiter=18,usecols=range(4)) produces [[(-3.970

Re: [Numpy-discussion] Using multiprocessing (shared memory) with numpy array multiplication

2011-06-17 Thread Robin
I didn't have time yesterday but the attached illustrates what I mean about putting the shared data in a module (it should work with the previous myutil). I don't get a big speed up but at least it is faster using multiple subprocesses: Not threaded: 0.450406074524 Using 8 processes: 0.282383 I

Re: [Numpy-discussion] genfromtxt converter question

2011-06-17 Thread Olivier Delalleau
If I understand correctly, your error is that you convert only the second column, because your converters dictionary contains a single key (1). If you have it contain keys from 0 to 3 associated to the same function, it should work. -=- Olivier 2011/6/17 gary ruben > I'm trying to read a file c

Re: [Numpy-discussion] 3d ODR

2011-06-17 Thread Christian K .
Robert Kern gmail.com> writes: > > On Thu, Jun 16, 2011 at 06:28, Christian K. hoc.net> wrote: > > Hi, > > > > I need to do fit a 3d surface to a point cloud. This sounds like a job for 3d > > orthogonal distance regression. Does anybody know of an implementation? > > As eat points out, scipy