Re: [Tutor] Getting total counts (Steven D'Aprano)

2010-10-02 Thread Steven D'Aprano
On Sun, 3 Oct 2010 08:29:29 am aenea...@priest.com wrote: > Thanks very much for the extensive comments, Steve. I can get the > code you wrote to work on my toy data, but my real input data is > actually contained in 10 files that are about 1.5 GB each--when I try > to run the code on one of those

Re: [Tutor] Getting total counts (Steven D'Aprano)

2010-10-02 Thread Alan Gauld
wrote I can get the code you wrote to work on my toy data, but my real input data is actually contained in 10 files that are about 1.5 GB each--when I try to run the code on one of those files, everything freezes. Fot those kind of volumes I'd go for a SQL database every time! (SQLlite might

Re: [Tutor] Getting total counts (Steven D'Aprano)

2010-10-02 Thread aeneas24
the ocation - thats what databases do for you under the covers. > Funny you should mention sqlite: I was just considering it yesterday. Gosh, Python has so much interesting stuff to offer! Sqlite operating in-memory would be a good solution for you I think. You can get a basic tutorial on Sqll

Re: [Tutor] Getting total counts

2010-10-01 Thread Steven D'Aprano
On Sat, 2 Oct 2010 06:31:42 am aenea...@priest.com wrote: > Hi, > > I have created a csv file that lists how often each word in the > Internet Movie Database occurs with different star-ratings and in > different genres. I would have thought that IMDB would probably have already made that informat

[Tutor] Getting total counts

2010-10-01 Thread aeneas24
Hi, I have created a csv file that lists how often each word in the Internet Movie Database occurs with different star-ratings and in different genres. The input file looks something like this--since movies can have multiple genres, there are three genre rows. (This is fake, simplified data.)