Re: [Tutor] Most common words in a text file
On 30/09/2017 18:12, Sri G. wrote: I'm learning programming with Python. I’ve written the code below for finding the most common words in a text file that has about 1.1 million words. It's working fine, but I believe there is always room for improvement. When run, the function in the script gets a text file from the command-line argument sys.argv[1], opens the file in read mode, converts the text to lowercase, makes a list of words from the text after removing any whitespaces or empty strings, and stores the list elements as dictionary keys and values in a collections.Counter object. Finally, it returns a dictionary of the most common words and their counts. The words.most_common() method gets its argument from the optional top parameter. import sysimport collections def find_most_common_words(textfile, top=10): ''' Returns the most common words in the textfile.''' textfile = open(textfile) text = textfile.read().lower() textfile.close() The modern Pythonic way is:- with open(textfile) as textfile: text = textfile.read().lower() The file close is handled automatically for you. For those who don't know this construct using the "with" keyword is called a context manager, here's an article about them https://jeffknupp.com/blog/2016/03/07/python-with-context-managers/ words = collections.Counter(text.split()) # how often each word appears return dict(words.most_common(top)) filename = sys.argv[1] How about some error handling if the user forgets the filename? The Pythonic way is to use a try/except looking for an IndexError, but there's nothing wrong with checking the length of sys.argv. top_five_words = find_most_common_words(filename, 5) I need your comments please. Sri Pretty good all in all :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email has been checked for viruses by AVG. http://www.avg.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] logging to cmd.exe
On 26/09/2017 12:22, Albert-Jan Roskam wrote: PS: sorry about the missing quote (>>) markers. Hotmail can't do this. Is Gmail better? > Get a decent email client and it'll do the work for you. I use Thunderbird on Windows with hotmail, gmail and yahoo addresses and never have a problem. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email has been checked for viruses by AVG. http://www.avg.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Beginner's guessing game
Can u please tell me why this program does not work in line 28? That is guessesTaken. It reads 0 when it should be a larger number. I am a beginner having another try to get it! Thank you, Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beginner's guessing game
On Sun, Oct 1, 2017 at 4:38 AM, Steve Lett wrote: > Can u please tell me why this program does not work in line 28? That is > guessesTaken. It reads 0 when it should be a larger number. > > I am a beginner having another try to get it! > > Thank you, Steve > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > Welcome Steve Please copy and paste your code into your email body. Attachments don't come through this list -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Am I missing something obvious about "FizzBuzz"?
On 01/10/17 06:56, boB Stepp wrote: > I definitely was *not* looking for a pat on the back. I just could > not believe that "FizzBuzz" (Or similar questions.) would ever be > needed in a job interview for programming/software engineering. The fizzbuzz one is definitely a bit too simplistic, but the one cited by McConnel (reverse a linked list in C) is typical of the kind of question we used. And yes, most candidates failed. Some of that is interview nerves so we would give them some hints and see if they could find the errors themselves. But some people literally couldn't even start! Another common ploy we used was to ask the candidate to find say 2 points of failure in a short function - say 4 or 5 lines long. And tell me how they'd fix it. Even fewer candidates can pass that one. > I truly hope that the above article does not reflect reality! I think it exaggerates (or maybe things have gotten much worse in the last 10 years!) but I do think more than half the applicants we got couldn't really program. (It doesn't help that many people today think that writing HTML/CSS is "programming"!) A final note. The jobs I was interviewing for were all internal vacancies advertised only within our own IT department. So all the candidates were already employed by us as IT people If you advertised externally it probably would be much worse. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beginner's guessing game
On 01/10/17 09:38, Steve Lett wrote: > Can u please tell me why this program does not work in line 28? That is > guessesTaken. It reads 0 when it should be a larger number. > > I am a beginner having another try to get it! > > Thank you, Steve Welcome Steve, but I can't see any program? Did you send it as an attachment? If so the server probably stripped it off for security reasons. Usually you can just paste the text into the email message body. (use plain text format to preserve layout) Also please include any error messages you get in their entirety and tell us your OS and Python version. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Most common words in a text file
On 30/09/17 18:12, Sri G. wrote: > import sysimport collections I assume that should be two lines? But you can also import multiple modules on a single line. import sys, collections Although some folks don't like that style. > def find_most_common_words(textfile, top=10): > ''' Returns the most common words in the textfile.''' The comment is slightly inaccurate since you really return a dict of the most common words *with the counts* added. It is good practice to specify the precise return type (list, tuple, dict etc) since that tells the user what they can do with it once they have it. Also by using the parameter textfile it is not clear whether I should pass a file object or a file name. Again it helps users if the comment is as specific as possible. > textfile = open(textfile) > text = textfile.read().lower() potential memory hog, others have already suggested reading line by line > textfile.close() > words = collections.Counter(text.split()) # how often each word appears > > return dict(words.most_common(top)) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Most common words in a text file
On 09/30/2017 11:12 AM, Sri G. wrote: > I'm learning programming with Python. > > I’ve written the code below for finding the most common words in a text > file that has about 1.1 million words. It's working fine, but I believe > there is always room for improvement. > > When run, the function in the script gets a text file from the command-line > argument sys.argv[1], opens the file in read mode, converts the text to > lowercase, makes a list of words from the text after removing any > whitespaces or empty strings, and stores the list elements as dictionary > keys and values in a collections.Counter object. Finally, it returns a > dictionary of the most common words and their counts. The > words.most_common() method gets its argument from the optional top > parameter. > > import sysimport collections > def find_most_common_words(textfile, top=10): > ''' Returns the most common words in the textfile.''' > > textfile = open(textfile) > text = textfile.read().lower() > textfile.close() > words = collections.Counter(text.split()) # how often each word appears > > return dict(words.most_common(top)) > > filename = sys.argv[1] > top_five_words = find_most_common_words(filename, 5) > > I need your comments please. Others have made some pertinent comments already. How much you spend time to "improve" a bit of code depends on what you're going to do with it. If you've solved your problem, and it's a one-shot: don't worry much about it. Nothing wrong with a bit of throwaway code (although things do sometimes take on a life much longer than you intended, I can say from experience!!!) I'd ask a question or two to think about: first off, if you know the intended use of this function always will be to get a "top 10 list" - then why convert it back to a dict (unsorted) to return after Counter.most_common() has just given you a sorted list? You're most likely going to have to take your top_five_words dictionary and turn it back into something sorted to report back out on the counts. Second, if your function is likely to be called from many places in your code, is it too specialized? Can you design it as a more general API that could be used for other things than this specific purpose? For example, perhaps you just want to return the Counter instance without the further processing of "most_common", and let the client decide what it wants to do with that object. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Am I missing something obvious about "FizzBuzz"?
On Sun, Oct 01, 2017 at 12:04:13PM +0100, Alan Gauld via Tutor wrote: > The fizzbuzz one is definitely a bit too simplistic, but the one > cited by McConnel (reverse a linked list in C) is typical of > the kind of question we used. And yes, most candidates failed. > > Some of that is interview nerves so we would give them some > hints and see if they could find the errors themselves. But > some people literally couldn't even start! I should think not! It's been about 30 years since I've last needed to reverse a linked list (in Pascal, not C, but the principle is the same). Who does that these days? I would have *no idea* how to traverse a singly-linked list in reverse without making a copy of it first. Okay, if you're specifically looking for somebody to write low-level algorithmic code, that's one thing. But 95% of programmers spend 95% of their time using library calls. And the remaining time, they certainly don't have to come up with a "reverse this linked list" algorithm from scratch. Google it, or look it up in a book. -- Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Am I missing something obvious about "FizzBuzz"?
On 10/01/2017 09:09 AM, Steven D'Aprano wrote: > On Sun, Oct 01, 2017 at 12:04:13PM +0100, Alan Gauld via Tutor wrote: > >> The fizzbuzz one is definitely a bit too simplistic, but the one >> cited by McConnel (reverse a linked list in C) is typical of >> the kind of question we used. And yes, most candidates failed. >> >> Some of that is interview nerves so we would give them some >> hints and see if they could find the errors themselves. But >> some people literally couldn't even start! > > I should think not! It's been about 30 years since I've last needed to > reverse a linked list (in Pascal, not C, but the principle is the same). > Who does that these days? > > I would have *no idea* how to traverse a singly-linked list in reverse > without making a copy of it first. > > Okay, if you're specifically looking for somebody to write low-level > algorithmic code, that's one thing. But 95% of programmers spend 95% of > their time using library calls. And the remaining time, they certainly > don't have to come up with a "reverse this linked list" algorithm from > scratch. Google it, or look it up in a book. My problem with programming tests too. There's only so much stuff you can keep in easily accessible memory, and the ability to find snippets and other answers and make use of them in order to help develop code efficiently is a more valuable programming skill than memorization of algorithms! - we're trying not to reinvent the wheel unless we absolutely have to. (We could go off on a tangent and argue about software patents here but I'll spare you all that). Many years ago I had a form of this discussion with my Physicist father, before computers were a big factor in his field: you needed to understand how you would arrive at an answer mainly in order to have a sense of whether something you were going to use was reasonable or off by orders of magnitude, because in real life you looked things up in the Rubber Book (https://en.wikipedia.org/wiki/CRC_Handbook_of_Chemistry_and_Physics) and other resources. We use google today in much the same way: why repeat work someone's already done? So employment-type programming tests often test for something that's not really part of your day-to-day routine as a programmer. Probably the best programming test there is look at code that's already been developed, but it's unfair to assume (as some companies do today) that a programmer's life's work will be sitting out in the open on github - that's just not true for everyone/everything, and not nearly all of the important skills of the professional programmer are in code either. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: Python programming for the absolute beginner
-- Forwarded message -- From: Howard B Date: Sun, Oct 1, 2017 at 2:20 PM Subject: Re: [Tutor] Python programming for the absolute beginner To: boB Stepp The 2010 copyright version, ISBN 978-1-4354-5500-9, discusses installing Python 3.1 (main text on page 5, and Appendix A on page 405). You will be able to follow the book using any Python 3.x. Consider installing Python 3.6 (the latest version) from Anaconda -- available completely free and for all operating systems. This is a well supported and commonly used package. https://www.anaconda.com/download/ The installation also includes the Spyder and Jupyter development environments, which you may find useful. On Sat, Sep 30, 2017 at 7:59 AM, boB Stepp wrote: > On Fri, Sep 29, 2017 at 1:20 PM, Alan Gauld via Tutor > wrote: > > On 29/09/17 08:51, Peter Collidge wrote: > >> I have borrowed the above book from my local library but I believe it > was > >> written in 2010 and as a result I am having difficulty in deciding which > >> version of Python to download. > >> Can anyone help? > > > > If you want to follow the book use the version the book > > uses - probably 2.6 or something close? > > I no longer have a copy of this book, but I am fairly certain it uses > an early version of Python 3. The main problem, though, is it uses a > customized version of Pygame for its later gaming code called > Livewires, so that would have to be Python-version-compatible. IIRC, > the author had a download site where you could get the whole package > together: correct Python version used in book, correct Livewires > version, etc., and that this was addressed in the book (Maybe an > appendix? Or the first get things setup chapter? Can't remember for > sure.) > > If the OP goes on to use the sequel to this book, named "More Python > Programming for the Absolute Beginner" (By a different author.), > *that* book uses straight-up Pygame, again with an early version of > Python 3. Again, I don't think there will be any problems with the > exact version of Python 3 used as long as the version of Pygame is > compatible with the chosen Python. > > > > -- > boB > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Am I missing something obvious about "FizzBuzz"?
On 01/10/17 16:09, Steven D'Aprano wrote:
>> The fizzbuzz one is definitely a bit too simplistic, but the one
>> cited by McConnel (reverse a linked list in C) is typical of
>> the kind of question we used. And yes, most candidates failed.
>
> I would have *no idea* how to traverse a singly-linked list in reverse
> without making a copy of it first.
The point is you don't need to traverse it in reverse
you simply iterate over it from the first element and
prepend each node. It really is trivially simple - or
should be for anyone purporting to be a C programmer.
And its not about memorizing algorithms - I never would
expect that - its about looking at the problem and
thinking about the solution.
And I should add that I was last interviewing programmers
in the 1990's (for the millennium bug the very last time,
and the code was in COBOL!).
But to address another issue raised by Mats:
> Probably the best programming test there is look at code
> that's already been developed,
Very true and in an ideal world what you would do, but...
It is what we did with the bug finding test. But, as alluded
to by Steve, it's really hard to do this for anything
significant in an interview, since you would need to be
familiar with the system's framework and libraries since
most code consists of calls to libraries. And the candidates
will probably not all be familiar or even aware of such,
so, to level the playing field, you give them a simple but
generic problem, like the fizzbiz or linked list tests.
> ...not nearly all of the important skills of the
> professional programmer are in code
Absolutely, or at least for the software engineer.
There is I believe a big cultural difference between,
say the UK, and the USA in what a programming job
looks like. (I'm less familiar with other countries,
even in Europe)
In the UK a "programmer" is a code monkey who takes
a code spec (flow chart or pseudo code) and turns
it into industrial strength code. It's a low paid
(relatively) position and the programmer probably
does not have any computer related degree, maybe
a craft school diploma. I believe programmer salaries
are currently around £22-25K per annum.
A software engineer can program but is also supposed
to be able to capture and analyze client requirements
produce a top level design/architecture, develop low
level design/spec (possibly for handing to a "programmer")
write the code, test it completely ("programmers" only do
unit tests, not system tests) and write appropriate
documentation. Salaries here range from about £30-60K
per annum depending on experience etc.
When I was interviewing it was always for software
engineers - we only employed programmers in our mainframe
systems. Everyone else was an "engineer".
So I guess expectations for the role might have a
big bearing too.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Am I missing something obvious about "FizzBuzz"?
On Sun, Oct 1, 2017 at 12:48 PM, Alan Gauld via Tutor wrote: > But to address another issue raised by Mats: > > Probably the best programming test there is look at code > > that's already been developed, > > Very true and in an ideal world what you would do, but... > > It is what we did with the bug finding test. But, as alluded > to by Steve, it's really hard to do this for anything > significant in an interview, since you would need to be > familiar with the system's framework and libraries since > most code consists of calls to libraries. First off, a confession: I'm an English major, not CS or IT. But this conversation reminded me of my interview for (what turned out to be) my first-ever programming job. I'd been working as BOFH (operating a System/360) for a few months, and heard about a better job in an HP shop closer to where I lived. So I applied, and during the course of the interview I discovered that it was a blended position: programming during the day, operator in the evening. I kept a poker face, and when I was asked whether I'd written any COBOL I lied and said yes. (I knew the name, but had never seen a line of code - only BASIC and Pascal up to that time.) My test was, fortunately for me, not a start-from-scratch FizzBuzz problem, but debugging a report module that was on the department's "to-fix" list. It took me about twenty-five minutes to find the problem (it was an off-by-one bug, as I recall), most of which was taken up translating COBOL to Pascal in my head... I got the job, and went straight to the local university bookstore and bought every book I could find on COBOL. Good times, good times... ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] closing the window through code
Hi there i am preparing for a mock exam using python and was wondering if you can help, my task is to create apassword checker and one of the criteria is to have a quit option. i have looked and looked but can't seem to find a bit of code allowing to do this! if there is any way you coild help me it woild be much appreciated yours sincerely Max Patient Sent from my iPhone ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Am I missing something obvious about "FizzBuzz"?
On 01/10/17 22:39, Marc Tompkins wrote: >>> Probably the best programming test there is look at code >>> that's already been developed, >> >> It is what we did with the bug finding test. > My test was, fortunately for me, not a start-from-scratch FizzBuzz problem, > but debugging a report module that was on the department's "to-fix" list. I think that's the point, its much easier to use real code in a debug situation than in a write-something-new type test. If the code is halfway decent you can figure out what it does even if you don;t know the libraries - or even, as in your case, the language! But if you had been asked to write a new function in that reporting system (assuming you knew COBOL) that would have been much harder without knowing the database schema or the query libraries that existed. > It took me about twenty-five minutes to find the problem But here is another problem, when we interviewed folks we got an hour in total per candidate so if a single exercise took 25 minutes we would never have had a chance to explore the other skills they needed. It had to be 10-15 mins max. The reality is that interviewers need some kind of filter and there is no perfect one. I've come to the conclusion that the hire process should work on the basis that you take someone on for a 3 month trial, its the only real way to tell if they are any good. But legal and corporate requirements mean that's unlikely to ever happen. In the meantime interviewees and interviewers alike struggle to find a meaningful way to select real talent in a market where far more unsuitable candidates present themselves than suitable. I fact often we could not even get enough good candidates to fill the jobs available. It typically went like this: 10 jobs 1000 applicants (if it was advertised externally) 100 interviews (run by the HR team) 40 tested (this is where I came in) 8 selected + 2 temporary contractors hired Over the years we tried lots of different formats but the end results were always about the same. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] closing the window through code
On 01/10/17 19:36, Max Patient wrote: > my task is to create a password checker and one of the > criteria is to have a quit option. What does the quit do? Quit the login? or quit the application? Based on your subject line I'm assuming just quit the login - but then how would you do anything in the app? You don't say what GUI framework you are using (and from the subject I assume it is a GUI?) but most have some kind of quit function. For example in Tkinter there is a quit() method in the top level widget and inherited by all the lower level ones. If you do just want to close a window/dialog the normal approach is just to make it invisible. In Tkinter that's done using the withdraw() method of a window. (restore it with deiconify()) Other frameworks will have their equivalent methods/functions. In some GUIs it's the window's state value that controls visibility. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
