Re: [PHP] CSV speed

2008-03-11 Thread Ray Hauge
Wolf wrote: Danny Brow wrote: I have about 10 csv files I need to open to access data. It takes a lot of time to search each file for the values I need. Would it be best to just dump all the cvs files to an SQL db and then just grab what I need from there? I'm starting to think it would make a l

Re: [PHP] CSV speed

2008-03-11 Thread Arvids Godjuks
I should say that running 7 quueries against 200MB table with ~600 000 rows for 4 seconds looks to me as if there was no indexes and very poor database design (i mean that CVS file is dumped into database incorrectly). Or you'r server is Pentium 100 :) P.S. Having myself a 1.5GB database with some

Re: [PHP] CSV speed

2008-03-10 Thread Wolf
Greg Donald wrote: On Mon, Mar 10, 2008 at 8:09 PM, Danny Brow <[EMAIL PROTECTED]> wrote: I have about 10 csv files I need to open to access data. It takes a lot of time to search each file for the values I need. Would it be best to just dump all the cvs files to an SQL db and then just grab w

Re: [PHP] CSV speed

2008-03-10 Thread Wolf
Danny Brow wrote: On Mon, 2008-03-10 at 22:36 -0400, Wolf wrote: Danny Brow wrote: I have about 10 csv files I need to open to access data. It takes a lot of time to search each file for the values I need. Would it be best to just dump all the cvs files to an SQL db and then just grab what I ne

Re: [PHP] CSV speed

2008-03-10 Thread Greg Donald
On Mon, Mar 10, 2008 at 8:09 PM, Danny Brow <[EMAIL PROTECTED]> wrote: > I have about 10 csv files I need to open to access data. It takes a lot > of time to search each file for the values I need. Would it be best to > just dump all the cvs files to an SQL db and then just grab what I need > fr

Re: [PHP] CSV speed

2008-03-10 Thread Larry Garfield
You may want to consider an SQLite database. Any PHP 5 install is likely to support it, and it should be very fast for reads. (Not so much for writes, but if you're dealing with something that can use CSV, it will probably work in SQLite, too.) On Monday 10 March 2008, TG wrote: > If you can'

Re: [PHP] CSV speed

2008-03-10 Thread Danny Brow
On Mon, 2008-03-10 at 22:36 -0400, Wolf wrote: > Danny Brow wrote: > > I have about 10 csv files I need to open to access data. It takes a lot > > of time to search each file for the values I need. Would it be best to > > just dump all the cvs files to an SQL db and then just grab what I need > > f

Re: [PHP] CSV speed

2008-03-10 Thread Wolf
Danny Brow wrote: I have about 10 csv files I need to open to access data. It takes a lot of time to search each file for the values I need. Would it be best to just dump all the cvs files to an SQL db and then just grab what I need from there? I'm starting to think it would make a lot of sense.

Re: [PHP] CSV speed

2008-03-10 Thread TG
If you can't store the values in a real DB, then that might be an option. You might consider using a temporary table if it's available in the database you're using, that way you don't have to worry about implicitly creating then destroying the table. The only problem is how temporary the temp