Re: [Numpy-discussion] Pickle, pytables, and sqlite - loading and saving recarray's

2007-07-19 Thread Gael Varoquaux
On Thu, Jul 19, 2007 at 09:42:42PM -0500, Vincent Nijs wrote: >I'd luv to hear from people using sqlite, pytables, and cPickle about >their experiences. I was about to point you to this discussion: http://projects.scipy.org/pipermail/scipy-user/2007-April/011724.html but I see that you pa

Re: [Numpy-discussion] Wrong lapack version detection (32/64bits)

2007-07-19 Thread David Cournapeau
Xavier Gnata wrote: > Xavier Gnata wrote: > >> Hi, >> >> I'm trying to update numpy by compiling the up to date svn: >> >> I get this error : >> gcc: numpy/linalg/lapack_litemodule.c >> gcc -pthread -shared >> build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o >> -lmkl_lapack32 -lmkl_l

[Numpy-discussion] Pickle, pytables, and sqlite - loading and saving recarray's

2007-07-19 Thread Vincent Nijs
I am interesting in using sqlite (or pytables) to store data for scientific research. I wrote the attached test program to save and load a simulated 11x500,000 recarray. Average save and load times are given below (timeit with 20 repetitions). The save time for sqlite is not really fair because I h

Re: [Numpy-discussion] Wrong lapack version detection (32/64bits)

2007-07-19 Thread Xavier Gnata
Xavier Gnata wrote: > Hi, > > I'm trying to update numpy by compiling the up to date svn: > > I get this error : > gcc: numpy/linalg/lapack_litemodule.c > gcc -pthread -shared > build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o > -lmkl_lapack32 -lmkl_lapack64 -lmkl -lvml -lguide -lpthrea

[Numpy-discussion] Wrong lapack version detection (32/64bits)

2007-07-19 Thread Xavier Gnata
Hi, I'm trying to update numpy by compiling the up to date svn: I get this error : gcc: numpy/linalg/lapack_litemodule.c gcc -pthread -shared build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o -lmkl_lapack32 -lmkl_lapack64 -lmkl -lvml -lguide -lpthread -o build/lib.linux-i686-2.4/numpy

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-19 Thread Torgil Svensson
Hi again, On 7/19/07, Torgil Svensson <[EMAIL PROTECTED]> wrote: If memory really is an issue, you have the nice "load_spec" version and can always convert the files once by iterating over the file twice like the attached script does. I discovered that my script was broken and too complex. Th

Re: [Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

2007-07-19 Thread Torgil Svensson
Hi, 1. Your code is fast due to that you convert whole at once columns in numpy. The first step with the lists is also very fast (python implements lists as arrays). I like your version, I think it's as fast as it gets in pure python and has to keep only two versions of the data at once in memory