On 26/10/14 09:46, Saullo Castro wrote: > I would like to start working on a memory efficient alternative for > np.loadtxt and np.genfromtxt that uses arrays instead of lists to store > the data while the file iterator is exhausted.
... > I would be glad if you could share your experience on this matter. I'm of the opinion that if your workflow requires you to regularly load large arrays from text files, something else needs to be fixed rather than the numpy speed and memory usage in reading data from text files. There are a number of data formats that are interoperable and allow to store data much more efficiently. hdf5 is one natural choice, maybe with the blosc compressor. Cheers, Daniele _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
