Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-21 Thread Emile van Sebille
On 11/21/2010 8:15 AM Josep M. Fontana said... return sorted(word_table.items(), key=lambda item: item[1], reverse=True) What I don't understand is the syntax of "item : item[1]". Key defines a lambda function that accepts as a single passed parameter named item and returns the element i

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-21 Thread python
> it is difficult for me not to be profuse in my thanks because you guys really > go beyond the call of duty. I love this list. The responses in this list most > of the times don't just address the problem at hand but are also useful in a > more general sense and help people become better progra

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-21 Thread Josep M. Fontana
Martin, Alan, col speed and everybody that helped: I think I'm going to stop because I'm repeating myself but it is difficult for me not to be profuse in my thanks because you guys really go beyond the call of duty. I love this list. The responses in this list most of the times don't just address t

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread col speed
--> snip >However, even with countWords2, which is supposed to overcome this >problem, it feels as if I've entered an infinite loop. >Josep M. Just my twopenneth, I'm a noob and I'm not going to try such a big file on my old machine, but: 1. Maybe create a *set* from the wordlist, loop through

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread Alan Gauld
"Martin A. Brown" wrote * Somebody will be certain to point out a language or languages that provide some sort of facility to abstract the use of multiple processors without the explicit use of threads. ISTR Occam did that? Occam being the purpose designed language for the transputer,

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread Martin A. Brown
Good evening, : It turns out that matters of efficiency appear to be VERY : important in this case. The example in my message was a very : short string but the file that I'm trying to process is pretty : big (20MB of text). Efficiency is best addressed first and foremost, not by hardware,

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread ALAN GAULD
If the file is big use Peter's method, but 45 minutes still seems very long so it may be theres a hidden bug in there somehwew. However... > When I look at the current processes running on my computer, I see the > Python process taking 100% of the CPU. Since my computer has a > multi-core pr

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread Josep M. Fontana
Thanks Alan, Peter and Steve, Instead of answering each one of you independently let me try to use my response to Steve's message as the basis for an answer to all of you. It turns out that matters of efficiency appear to be VERY important in this case. The example in my message was a very short

Re: [Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread Steven D'Aprano
Josep M. Fontana wrote: def countWords(a_list): words = {} for i in range(len(a_list)): item = a_list[i] count = a_list.count(item) words[item] = count return sorted(words.items(), key=lambda item: item[1], reverse=True) with open('output.txt', 'a') as token_f

[Tutor] Simple counter to determine frequencies of words in a document

2010-11-20 Thread Josep M. Fontana
Hi, I'm trying to do something that should be very simple. I want to generate a list of the words that appear in a document according to their frequencies. So, the list generated by the script should be something like this: the : 3 book: 2 was : 2 read: 1 by: 1 [...] This would be obtained from