[Numpy-discussion] stderr

2009-06-26 Thread Charles R Harris
There are a few spots where messages are printed to stderr. Some of these look almost like debugging stuff, for instance NPY_NO_EXPORT void form...@name@(char *buf, size_t buflen, @name@ val, unsigned int prec) { /* XXX: Find a correct size here for format string */ char format[64], *res;

Re: [Numpy-discussion] Import Numpy on Windows Vista x64 AMD

2009-06-26 Thread David Cournapeau
On Sat, Jun 27, 2009 at 6:42 AM, Dinesh B Vadhia wrote: > Ticket# 1084 > (http://projects.scipy.org/numpy/timeline?from=2009-06-09T03%3A01%3A59-0500&precision=second) > says that the numpy import on Windows Vista x64 AMD systems works now. I mistakenly closed it as fixed, but it is just a duplicat

[Numpy-discussion] fromfile and ticket #1152

2009-06-26 Thread Charles R Harris
The question is: what should happen when fewer items are read than requested. The current behaviour is 1) Error message written to stderr (needs to be fixed) 2) If 0 items are read then nomemory error is raised ;) So, should a warning be raised and an array returned with however many items were r

[Numpy-discussion] Import Numpy on Windows Vista x64 AMD

2009-06-26 Thread Dinesh B Vadhia
Ticket# 1084 (http://projects.scipy.org/numpy/timeline?from=2009-06-09T03%3A01%3A59-0500&precision=second) says that the numpy import on Windows Vista x64 AMD systems works now. Is this for Numpy 1.3 or 1.4 and if 1.3 has anyone tried it successfully? Thanks. Dinesh _

Re: [Numpy-discussion] switching to float32

2009-06-26 Thread Christian K.
Robert Kern schrieb: > On Fri, Jun 26, 2009 at 03:39, Christian K. wrote: >> John Schulman caltech.edu> writes: >> >>> >>> I'm trying to reduce the memory used in a calculation, so I'd like to >>> switch my program to float32 instead of float64. Is it possible to >>> change the numpy default float

Re: [Numpy-discussion] Rec array: numpy.rec vs numpy.array with complex dtype

2009-06-26 Thread Pierre GM
On Jun 26, 2009, at 3:59 PM, Dan Yamins wrote: > > Short answer: > a np.recarray is a subclass of ndarray with structured dtype, where > fields can be accessed has attributes (as in 'yourarray.yourfield') > instead of as items (as in yourarray['yourfield']). > > Is this the only substantial thing

Re: [Numpy-discussion] Rec array: numpy.rec vs numpy.array with complex dtype

2009-06-26 Thread Dan Yamins
Pierre, thanks for your response. I have some follow up questions. Short answer: > a np.recarray is a subclass of ndarray with structured dtype, where > fields can be accessed has attributes (as in 'yourarray.yourfield') > instead of as items (as in yourarray['yourfield']). Is this the only su

Re: [Numpy-discussion] switching to float32

2009-06-26 Thread Robert Kern
On Fri, Jun 26, 2009 at 03:39, Christian K. wrote: > John Schulman caltech.edu> writes: > >> >> I'm trying to reduce the memory used in a calculation, so I'd like to >> switch my program to float32 instead of float64. Is it possible to >> change the numpy default float size, so I don't have to exp

Re: [Numpy-discussion] Rec array: numpy.rec vs numpy.array with complex dtype

2009-06-26 Thread Pierre GM
On Jun 26, 2009, at 2:51 PM, Dan Yamins wrote: > > We've been using the numpy.rec classes to make record array objects. > > We've noticed that in more recent versions of numpy, record-array > like objects can be made directly with the numpy.ndarray class, by > passing a complex data type. Has

[Numpy-discussion] Rec array: numpy.rec vs numpy.array with complex dtype

2009-06-26 Thread Dan Yamins
Dear Numpy list: We've been using the numpy.rec classes to make record array objects. We've noticed that in more recent versions of numpy, record-array like objects can be made directly with the numpy.ndarray class, by passing a complex data type. However, it looks like the numpy.rec class is st

Re: [Numpy-discussion] loading data

2009-06-26 Thread Francesc Alted
A Friday 26 June 2009 13:46:13 Mag Gam escrigué: > Yes, you are correct! > > I think this is the best path. > > However, I need to learn how to append a hdf5 dataset . I looked at > this, http://code.google.com/p/h5py/wiki/FAQ#Appending_data_to_a_dataset > but was not able to do so. Do you happen t

Re: [Numpy-discussion] loading data

2009-06-26 Thread Mag Gam
Yes, you are correct! I think this is the best path. However, I need to learn how to append a hdf5 dataset . I looked at this, http://code.google.com/p/h5py/wiki/FAQ#Appending_data_to_a_dataset but was not able to do so. Do you happen to have any sample code for this, if you used hdf5. On Fri

Re: [Numpy-discussion] loading data

2009-06-26 Thread Francesc Alted
A Friday 26 June 2009 13:09:13 Mag Gam escrigué: > I really like the slice by slice idea! Hmm, after looking at the np.loadtxt() docstrings it seems it works by loading the complete file at once, so you shouldn't use this directly (unless you split your big file before, but this will take time t

Re: [Numpy-discussion] loading data

2009-06-26 Thread Mag Gam
I really like the slice by slice idea! But, I don't know how to implement the code. Do you have any sample code? I suspect its the writing portion thats taking the lonest. I did a simple decompress test and its fast. On Fri, Jun 26, 2009 at 7:05 AM, Francesc Alted wrote: > A Friday 26 June 20

Re: [Numpy-discussion] loading data

2009-06-26 Thread Francesc Alted
A Friday 26 June 2009 12:38:11 Mag Gam escrigué: > Thanks everyone for the great and well thought out responses! > > To make matters worse, this is actually a 50gb compressed csv file. So > it looks like this, 2009.06.01.plasmasub.csv.gz > We get this data from another lab from the Westcoast every

Re: [Numpy-discussion] loading data

2009-06-26 Thread Mag Gam
Thanks everyone for the great and well thought out responses! To make matters worse, this is actually a 50gb compressed csv file. So it looks like this, 2009.06.01.plasmasub.csv.gz We get this data from another lab from the Westcoast every night therefore I don't have the option to have this file

Re: [Numpy-discussion] switching to float32

2009-06-26 Thread Christian K .
John Schulman caltech.edu> writes: > > I'm trying to reduce the memory used in a calculation, so I'd like to > switch my program to float32 instead of float64. Is it possible to > change the numpy default float size, so I don't have to explicitly > state dtype=np.float32 everywhere? Possibly no