Re: [Numpy-discussion] saving incrementally numpy arrays

2009-08-11 Thread Citi, Luca
You can do something a bit tricky but possibly working. I made the assumption of a C-ordered 1d vector. import numpy as np import numpy.lib.format as fmt # example of chunks chunks = [np.arange(l) for l in range(5,10)] # at the beginning fp = open('myfile.npy', 'wb') d = dict( desc

Re: [Numpy-discussion] saving incrementally numpy arrays

2009-08-11 Thread Juan Fiol
Hi, again, I can confirm that you have to load multiple times. Also I do not see differences if using or not the f.seek line The following snippet gives the expected result. The problem is that that way I have to load as many times as I wrote. Besides that, it works. Thanks, Juan #---

Re: [Numpy-discussion] saving incrementally numpy arrays

2009-08-11 Thread Juan Fiol
Hi, thanks for all the answers. I am checking how to use pytables now, though I probably prefer to do it without further dependencies. I tried opening the file as 'append' and then pickle the array (because looking to the numpy.save it looked like what they did), but to retrieve the data then I

Re: [Numpy-discussion] saving incrementally numpy arrays

2009-08-11 Thread Keith Goodman
On Tue, Aug 11, 2009 at 11:05 AM, Robert Kern wrote: > On Mon, Aug 10, 2009 at 22:29, Juan Fiol wrote: >> Hi, I am creating numpy arrays in chunks and I want to save the chunks while >> my program creates them. I tried to use numpy.save but it failed (because it >> is not intended to append data)

Re: [Numpy-discussion] saving incrementally numpy arrays

2009-08-11 Thread Robert Kern
On Mon, Aug 10, 2009 at 22:29, Juan Fiol wrote: > Hi, I am creating numpy arrays in chunks and I want to save the chunks while > my program creates them. I tried to use numpy.save but it failed (because it > is not intended to append data). I'd like to know what is, in your opinion, > the best w