Re: [Numpy-discussion] checksum on numpy float array

2008-12-08 Thread Francesc Alted
A Sunday 07 December 2008, Brennan Williams escrigué: > OK so maybe I should > > (1) not add some sort of checksum type functionality to my read/write > methods > > these read/write methods simply read/write numpy arrays to a > binary file which contains one or more numpy arrays (and noth

Re: [Numpy-discussion] checksum on numpy float array

2008-12-06 Thread Brennan Williams
OK so maybe I should (1) not add some sort of checksum type functionality to my read/write methods these read/write methods simply read/write numpy arrays to a binary file which contains one or more numpy arrays (and nothing else). (2) replace my binary files iwith either HDF5 or PyT

Re: [Numpy-discussion] checksum on numpy float array

2008-12-06 Thread Francesc Alted
A Friday 05 December 2008, Andrew Collette escrigué: > > Another possibility would be to use HDF5 as a data container. It > > supports the fletcher32 filter [1] which basically computes a > > chuksum for evey data chunk written to disk and then always check > > that the data read satifies the chec

Re: [Numpy-discussion] checksum on numpy float array

2008-12-05 Thread Andrew Collette
> Another possibility would be to use HDF5 as a data container. It > supports the fletcher32 filter [1] which basically computes a chuksum > for evey data chunk written to disk and then always check that the data > read satifies the checksum kept on-disk. So, if the HDF5 layer doesn't > comp

Re: [Numpy-discussion] checksum on numpy float array

2008-12-05 Thread Francesc Alted
A Friday 05 December 2008, Brennan Williams escrigué: > Robert Kern wrote: > > On Thu, Dec 4, 2008 at 18:54, Brennan Williams > > > > <[EMAIL PROTECTED]> wrote: > >> Thanks > >> > >> [EMAIL PROTECTED] wrote: > >>> I didn't check what this does behind the scenes, but try this > >> > >> import hashli

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Brennan Williams
Robert Kern wrote: > On Thu, Dec 4, 2008 at 18:54, Brennan Williams > <[EMAIL PROTECTED]> wrote: > >> Thanks >> >> [EMAIL PROTECTED] wrote: >> >>> I didn't check what this does behind the scenes, but try this >>> >>> >>> >> import hashlib #standard python library >> import numpy as n

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Robert Kern
On Thu, Dec 4, 2008 at 18:54, Brennan Williams <[EMAIL PROTECTED]> wrote: > Thanks > > [EMAIL PROTECTED] wrote: >> I didn't check what this does behind the scenes, but try this >> >> > import hashlib #standard python library > import numpy as np >> m = hashlib.md5() >> m.update(np.array(range(100))

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Brennan Williams
Thanks [EMAIL PROTECTED] wrote: > I didn't check what this does behind the scenes, but try this > > import hashlib #standard python library import numpy as np > m = hashlib.md5() > m.update(np.array(range(100))) > m.update(np.array(range(200))) > > m2 = hashlib.md5() > m2.update(np.array(range(

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread josef . pktd
On Thu, Dec 4, 2008 at 6:57 PM, <[EMAIL PROTECTED]> wrote: > I didn't check what this does behind the scenes, but try this > I forgot to paste: import hashlib #standard python library Josef ___ Numpy-discussion mailing list Numpy-discussion@scipy.org

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread josef . pktd
I didn't check what this does behind the scenes, but try this m = hashlib.md5() m.update(np.array(range(100))) m.update(np.array(range(200))) m2 = hashlib.md5() m2.update(np.array(range(100))) m2.update(np.array(range(200))) print m.hexdigest() print m2.hexdigest() assert m.hexdigest() == m2.h

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Robert Kern
On Thu, Dec 4, 2008 at 17:43, Brennan Williams <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: >> On Thu, Dec 4, 2008 at 6:17 PM, Brennan Williams >> <[EMAIL PROTECTED]> wrote: >> >>> My app reads in one or more float arrays from a binary file. >>> >>> Sometimes due to network timeouts etc th

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Brennan Williams
[EMAIL PROTECTED] wrote: > On Thu, Dec 4, 2008 at 6:17 PM, Brennan Williams > <[EMAIL PROTECTED]> wrote: > >> My app reads in one or more float arrays from a binary file. >> >> Sometimes due to network timeouts etc the array is not read correctly. >> >> What would be the best way of checking the

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread josef . pktd
On Thu, Dec 4, 2008 at 6:17 PM, Brennan Williams <[EMAIL PROTECTED]> wrote: > My app reads in one or more float arrays from a binary file. > > Sometimes due to network timeouts etc the array is not read correctly. > > What would be the best way of checking the validity of the data? > > Would some s

Re: [Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Robert Kern
On Thu, Dec 4, 2008 at 17:17, Brennan Williams <[EMAIL PROTECTED]> wrote: > My app reads in one or more float arrays from a binary file. > > Sometimes due to network timeouts etc the array is not read correctly. > > What would be the best way of checking the validity of the data? > > Would some sor

[Numpy-discussion] checksum on numpy float array

2008-12-04 Thread Brennan Williams
My app reads in one or more float arrays from a binary file. Sometimes due to network timeouts etc the array is not read correctly. What would be the best way of checking the validity of the data? Would some sort of checksum approach be a good idea? Would that work with an array of floating poin