<[EMAIL PROTECTED]> wrote:
> Vincent,
>
> A Divendres 20 Juliol 2007 15:35, Vincent Nijs escrigué:
>> Still curious however ... does no one on this list use (and like) sqlite?
>
> First of all, while I'm not a heavy user of relational databases, I've used
>
, "Francesc Altet" <[EMAIL PROTECTED]> wrote:
> A Divendres 20 Juliol 2007 04:42, Vincent Nijs escrigué:
>> I am interesting in using sqlite (or pytables) to store data for scientific
>> research. I wrote the attached test program to save and load a simulated
>> 11x50
quaux" <[EMAIL PROTECTED]> wrote:
> On Thu, Jul 19, 2007 at 09:42:42PM -0500, Vincent Nijs wrote:
>>I'd luv to hear from people using sqlite, pytables, and cPickle about
>>their experiences.
>
> I was about to point you to this discussion:
> http:/
I am interesting in using sqlite (or pytables) to store data for scientific
research. I wrote the attached test program to save and load a simulated
11x500,000 recarray. Average save and load times are given below (timeit
with 20 repetitions). The save time for sqlite is not really fair because I
h
Hi,
I am trying to write a couple of simple functions to (1) save recarray's to
an sqlite database and (2) load a recarray from an sqllite database. I am
stuck on 2 points and hope there are some people on this list that use
sqlite for numpy stuff.
1. How to detect the variable names and types fr
> Isn't this the same as "types = zip(varnm, types2)" ?
>
> 4. return N.fromiter(iter(reader),dtype = types)
> Isn't "reader" an iterator already? What does the "iter()" operator do
> in this case?
>
> Best regards,
>
> //Torgi
I combined some of the very useful comments/code from Tim and Torgil and
came-up with the attached program to read csv files and convert the data
into a recarray. I couldn¹t use all of their suggestions because, frankly, I
didn¹t understand all of them :)
The program use variable names if provided
Cool! Thanks Tim.
Vincent
On 7/8/07 10:25 PM, "Timothy Hochberg" <[EMAIL PROTECTED]> wrote:
>
>
> On 7/8/07, Vincent Nijs <[EMAIL PROTECTED]> wrote:
>> Thanks for looking into this Torgil! I agree that this is a much more
>> complicated setup. I&#x
in a column,
> it'll cost double memory. Also if many columns change format at
> different places it has to re-convert every time.
>
> I don't recommend this because of the drawbacks and extra complexity.
> I think it is best to convert your files (or file generation) so
create a
new array from the old one with an updated dtype list.
Thanks,
Vincent
On 7/8/07 4:51 PM, "Timothy Hochberg" <[EMAIL PROTECTED]> wrote:
>
>
> On 7/8/07, Vincent Nijs <[EMAIL PROTECTED]> wrote:
>> Torgil,
>>
>> The function seems to wo
tumble on these types of problems from time to time so I'm
> interested in efficient solutions myself.
>
> Do you have a column which starts with something suitable for int on
> the first row (without decimal separator) but has decimals further
> down?
>
> This will be l
ould
> be done by trying different approaches to the "convert_row" function,
> for example using "zip" or "enumerate" instead of "izip".
>
> Best Regards,
>
> //Torgil
>
>
> On 7/8/07, Vincent Nijs <[EMAIL PROTECTED]> wrote:
>
appropriate.
Best,
Vincent
On 7/6/07 8:53 PM, "John Hunter" <[EMAIL PROTECTED]> wrote:
> On 7/6/07, Vincent Nijs <[EMAIL PROTECTED]> wrote:
>> I wrote the attached (small) program to read in a text/csv file with
>> different data types and convert it into a recarray
I wrote the attached (small) program to read in a text/csv file with
different data types and convert it into a recarray without having to
pre-specify the dtypes or variables names. I am just too lazy to type-in
stuff like that :) The supported types are int, float, dates, and strings.
I works pre
It seems to be a lot slower than pylab.load for large arrays. Also, it
doesn't handle dates.
Vincent
On 4/22/07 10:33 AM, "Markus Rosenstihl"
<[EMAIL PROTECTED]> wrote:
> Whats wrong with scipy.io.read_array?
>
> Am 19.04.2007 um 15:50 schrieb Lisandro Dalcin:
>
>> On 4/19/07, Travis Oliphant
I think it would be a great idea to have pylab.load in numpy. It also seems
to be a lot faster than scipy.io.
One thing that is very nice about pylab.load is that it can read-in dates.
However, it can't, as far a I know, handle other non-float data.
I played around with python's csv module and py
I think that error went away when I used the latest developer tools from
Apple, and made sure I was using gcc4.
Take a look at:
http://www.scipy.org/Installing_SciPy/Mac_OS_X
Vincent
On 4/12/07 1:19 AM, "Anton Sherwood" <[EMAIL PROTECTED]> wrote:
> When I try to build numpy (or PIL) on MacOS,
Mac OS X 10.4.8
G5 PPC
Numpy svn: 2602
Target: powerpc-apple-darwin8
Configured with: /private/var/tmp/gcc/gcc-5367.obj~1/src/configure
--disable-checking -enable-werror --prefix=/usr --mandir=/share/man
--enable-languages=c,objc,c++,obj-c++
--program-transform-name=/^[cg][^.-]*$/s/$/-4.0/
--with
.
Vincent
On 1/5/07 7:00 PM, "Robert Kern" <[EMAIL PROTECTED]> wrote:
> Vincent Nijs wrote:
>> Say I use a function that expects a boolean array called sel to be passed as
>> an argument:
>>
>> def foo(x,sel = None):
>> return x[sel]
>>
Say I use a function that expects a boolean array called sel to be passed as
an argument:
def foo(x,sel = None):
return x[sel]
If x is a 1-d array and sel is a (1-d) boolean array, x.shape will give (n,)
where n is len(x).
However, if the default value None is used (i.e., when no boolean arr
--b,
The only data type in python that has a keys() method is a dictionary.
Unless it is a record array (http://www.scipy.org/RecordArrays) there is no
information on variable names contained in the object. However, even record
arrays don't have a keys() method.
Vincent
On 1/4/07 1:18 PM, "b
Sorry for the extra post. There were are few errors in the previous
attachment.
Vincent
On 12/28/06 5:39 PM, "Vincent Nijs" <[EMAIL PROTECTED]> wrote:
> Based on Eike's input the dbase class can now also load and dump (simple)
> csv and pickle files. See the tests at
kbook/InputOutput. I
couldn't figure out how to do it.
Eike:
I think I can figure out how to add a plot method. However, if you have some
more suggestions on how to implement the getAtTime, extract, and set methods
you mentioned that would be great.
Vincent
On 12/28/06 1:40 PM, "Vincen
Thanks for the input Eike.
I will add load and store methods to Pickle/UnPickle the object. I have got
to get the data into the class first however from an ascii file (txt or
csv).
I'd like to read the data and variable names directly from a single csv
file. I tried this through the python csv mo
n't need this flexibility when calling diff() since, at least
in TS applications, I only ever need diff(a,1) or diff(a,2).
Thanks,
Vincent
On 12/27/06 6:17 PM, "Sven Schreiber" <[EMAIL PROTECTED]> wrote:
> Vincent Nijs schrieb:
>> I am tryin to convert some of m
I am tryin to convert some of my time-series code written in Ox to
scipy/numpy (e.g., unit root tests, IRFs, cointegration, etc). Two key
functions I need for this are 'lag' and 'diff'. 'diff' is available but
'lag' is apparently not.
Below is my attempt at a lag function. I tried to be somewhat
26 matches
Mail list logo
