Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Is it zlib or zipfile? On Fri, May 14, 2021 at 11:38 AM Benjamin Root wrote: > Isaac, > > What I mean is that your bug might be similar to the savemat() bug that > was fixed in scipy in 2019. Completely different functions, but both > functions need to properly interact with zlib in order to wor

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

If changing the on-disk format is an option, I would suggest h5py which allows to save numpy arrays in HDF5 format. Stefano On 14 May 2021, at 16:22, numpy-discussion-requ...@python.org wrote: Aside from writing m

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Isaac, What I mean is that your bug might be similar to the savemat() bug that was fixed in scipy in 2019. Completely different functions, but both functions need to properly interact with zlib in order to work properly. On Fri, May 14, 2021 at 10:22 AM Isaac Gerg wrote: > Hi Ben, I am not sur

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

, May 14, 2021 3:22 PMTo: Discussion of Numerical PythonSubject: Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though! Hi Ben,  I am not sure.  However, in looking at the dates, it looks like that was fixed in scipy as of 2019.   Would you recommend using the scipy save

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Hi Ben, I am not sure. However, in looking at the dates, it looks like that was fixed in scipy as of 2019. Would you recommend using the scipy save interface as opposed to the numpy one? On Fri, May 14, 2021 at 10:16 AM Benjamin Root wrote: > Perhaps it is a similar bug as this one? > https:/

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Perhaps it is a similar bug as this one? https://github.com/scipy/scipy/issues/6999 Basically, it turned out that the CRC was getting computed on an unflushed buffer, or something like that. On Fri, May 14, 2021 at 10:05 AM Isaac Gerg wrote: > I am using 1.19.5 on Windows 10 using Python 3.8.6

[Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

I am using 1.19.5 on Windows 10 using Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)]. I have two python processes running (i.e. no threads) which do independent processing jobs and NOT writing to the same directories. Each process runs for 5-10 hours and the