Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-26 Thread Robert Kern
On Fri, Aug 26, 2011 at 07:04, Derek Homeier wrote: > On 25.08.2011, at 8:42PM, Chris.Barker wrote: > >> On 8/24/11 9:22 AM, Anthony Scopatz wrote: >>>    You can use Python pickling, if you do *not* have a requirement for: >> >> I can't recall why, but it seem pickling of numpy arrays has been >>

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-26 Thread Chris.Barker
On 8/26/11 5:04 AM, Derek Homeier wrote: > Hmm, the pure Python version might be, but, I've used cPickle for a long time > and never noted any stability problems. well, here is the NEP: https://github.com/numpy/numpy/blob/master/doc/neps/npy-format.txt It addresses the why's and hows of the for

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-26 Thread Derek Homeier
On 25.08.2011, at 8:42PM, Chris.Barker wrote: > On 8/24/11 9:22 AM, Anthony Scopatz wrote: >>You can use Python pickling, if you do *not* have a requirement for: > > I can't recall why, but it seem pickling of numpy arrays has been > fragile and not very performant. > Hmm, the pure Python v

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-26 Thread Paul Anton Letnes
On 25. aug. 2011, at 23.49, David Warde-Farley wrote: > On 2011-08-25, at 2:42 PM, Chris.Barker wrote: > >> On 8/24/11 9:22 AM, Anthony Scopatz wrote: >>> You can use Python pickling, if you do *not* have a requirement for: >> >> I can't recall why, but it seem pickling of numpy arrays has be

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-25 Thread David Warde-Farley
On 2011-08-25, at 2:42 PM, Chris.Barker wrote: > On 8/24/11 9:22 AM, Anthony Scopatz wrote: >>You can use Python pickling, if you do *not* have a requirement for: > > I can't recall why, but it seem pickling of numpy arrays has been > fragile and not very performant. > > I like the npy / np

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-25 Thread Chris.Barker
On 8/24/11 9:22 AM, Anthony Scopatz wrote: > You can use Python pickling, if you do *not* have a requirement for: I can't recall why, but it seem pickling of numpy arrays has been fragile and not very performant. I like the npy / npz format, built in to numpy, if you don't need: > - acc

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-24 Thread Anthony Scopatz
On Sun, Aug 21, 2011 at 7:24 AM, Pauli Virtanen wrote: > On Sat, 20 Aug 2011 16:18:55 -0700, Chris Withers wrote: > > I've got a tree of nested dicts that at their leaves end in numpy arrays > > of identical sizes. > > > > What's the easiest way to persist these to disk so that I can pick up > >

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-21 Thread Pauli Virtanen
On Sat, 20 Aug 2011 16:18:55 -0700, Chris Withers wrote: > I've got a tree of nested dicts that at their leaves end in numpy arrays > of identical sizes. > > What's the easiest way to persist these to disk so that I can pick up > with them where I left off? Depends on your requirements. You can

Re: [Numpy-discussion] saving groups of numpy arrays to disk

2011-08-21 Thread Paul Anton Letnes
Hi! On 21. aug. 2011, at 00.18, Chris Withers wrote: > Hi All, > > I've got a tree of nested dicts that at their leaves end in numpy arrays > of identical sizes. > > What's the easiest way to persist these to disk so that I can pick up > with them where I left off? Probably giving them names

[Numpy-discussion] saving groups of numpy arrays to disk

2011-08-20 Thread Chris Withers
Hi All, I've got a tree of nested dicts that at their leaves end in numpy arrays of identical sizes. What's the easiest way to persist these to disk so that I can pick up with them where I left off? What's the most "correct" way to do so? I'm using IPython if that makes things easier... I ha