Re: [Tutor] reading lines from a list of files
On 14/05/15 06:27, Alex Kleider wrote: The following seems to work- for f_name in list_of_file_names: for line in open(f_name, 'r'): process(line) but should I be worried that the file doesn't get explicitly closed? If you are only ever reading from the file then no, I'd not worry too much. But in the general case, where you might be making changes then yes, you should worry. It will work 99% of the time but if things go wrong there's always a chance that a file has not been closed yet and your changes have not been written to the file. But if you are not changing the file it doesn't matter too much. The only other consideration is that some OS might put a lock on the file even if it's only read access, so in those cases closing will release the lock sooner. But I don't think any of the popular OS do that any more. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading lines from a list of files
On 2015-05-14 00:15, Laura Creighton wrote: In a message of Wed, 13 May 2015 22:27:11 -0700, Alex Kleider writes: As a follow up question: The following seems to work- for f_name in list_of_file_names: for line in open(f_name, 'r'): process(line) but should I be worried that the file doesn't get explicitly closed? Alex If you use the with statement you will guarantee that the file closes as soon as you are done with it. It will also handle exceptions nicely for you. See: https://www.python.org/dev/peps/pep-0343/ In practice, Cpython's ref counting semantics means that running out of file descriptors doesn't happen (unless you put that code in a loop that gets called a whole lot). But the gc used by a Python version is not part of the language specification, but is a language implementation detail. If you are writing for PyPy or Jython you will need to use the with statement or close your files explicitly, so the gc knows you are done with them. Relying on 'the last reference to them went away' to close your file won't work if the gc isn't counting references. See: http://pypy.org/compat.html or for more detail: http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies Laura Thanks, Laura, for your analysis. I'll happily include the one extra line and let 'with' do its magic rather than depend on implementation details (or worry about their shortcomings:-) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading lines from a list of files
On 2015-05-14 00:01, Alan Gauld wrote: On 14/05/15 06:27, Alex Kleider wrote: The following seems to work- for f_name in list_of_file_names: for line in open(f_name, 'r'): process(line) but should I be worried that the file doesn't get explicitly closed? If you are only ever reading from the file then no, I'd not worry too much. But in the general case, where you might be making changes then yes, you should worry. It will work 99% of the time but if things go wrong there's always a chance that a file has not been closed yet and your changes have not been written to the file. But if you are not changing the file it doesn't matter too much. The only other consideration is that some OS might put a lock on the file even if it's only read access, so in those cases closing will release the lock sooner. But I don't think any of the popular OS do that any more. Thank you, Alan; I hadn't appreciated the important difference in risk with write vs read. It's clear that 'best practice' would be to use 'with' so as to 'cover all bases.' ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading lines from a list of files
On 2015-05-13 23:24, Danny Yoo wrote: As a follow up question: The following seems to work- for f_name in list_of_file_names: for line in open(f_name, 'r'): process(line) but should I be worried that the file doesn't get explicitly closed? It depends on context. Personally, I'd write it with the 'with' to make it very clear that the loop will manage its resource. That being said, it sounds like there might be concerned about the nesting. We're nesting three or four levels deep, at the very least! I'd agree with that. Because of this, it might be worthwile to consider refactoring the processing of the file in a separate function, something like this: ### def processFile(f): for line in f: ... for f_name in list_of_file_names: with open(f_name, 'r') as f: processFile(f) ### The primary reason is to reduce the nesting. But there's also a potential side benefit: processFile() has a better chance of being unit-testable, since we can pass in instances of other file-like objects, such as io.StringIO(), to spot-check the behavior of the process. Thanks, Danny. This is particularly germane since the issue has come up in the midst of my first attempt to do test driven development. I'll try refactoring per your suggestion. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading lines from a list of files
In a message of Wed, 13 May 2015 22:27:11 -0700, Alex Kleider writes: >As a follow up question: >The following seems to work- > > for f_name in list_of_file_names: > for line in open(f_name, 'r'): > process(line) > >but should I be worried that the file doesn't get explicitly closed? > >Alex If you use the with statement you will guarantee that the file closes as soon as you are done with it. It will also handle exceptions nicely for you. See: https://www.python.org/dev/peps/pep-0343/ In practice, Cpython's ref counting semantics means that running out of file descriptors doesn't happen (unless you put that code in a loop that gets called a whole lot). But the gc used by a Python version is not part of the language specification, but is a language implementation detail. If you are writing for PyPy or Jython you will need to use the with statement or close your files explicitly, so the gc knows you are done with them. Relying on 'the last reference to them went away' to close your file won't work if the gc isn't counting references. See: http://pypy.org/compat.html or for more detail: http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies Laura ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Terminology question
I noticed that if I call a function that throws an error, I can catch it from the caller, instead of catching it in the function. Is this is what is known as "errors bubbling up?" Also, is this how you're supposed to do it? *** Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)] on win32. *** '''Print a table of random numbers, given min, max, rows, and columns''' from random import randint def make_table(minimum, maximum, rows, columns): for row in range(rows): print() for column in range(columns): print(randint(minimum, maximum), end = ' ') def get_input(): inp = input("Enter minimum, maximum, rows, and columns, separated by commas: ") inps = inp.split(',') try: minimum = int(inps[0]) maximum = int(inps[1]) rows = int(inps[2]) columns = int(inps[3]) return minimum, maximum, rows, columns except ValueError as err: print("non-integer entered", err) return None except IndexError as err: print("You didn't enter enough values.", err) return None vals = get_input() if vals: minimum, maximum, rows, columns = vals try: make_table(minimum, maximum, rows, columns) except ValueError as err: # CATCH FUNCTION ERROR HERE INSTEAD OF IN FUNCTION print("Enter min before max.") else: print('Nothing to do.') -- Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Terminology question
On 14/05/15 23:43, Jim Mooney Py3.4.3winXP wrote: I noticed that if I call a function that throws an error, I can catch it from the caller, instead of catching it in the function. Is this is what is known as "errors bubbling up?" Yes. They can bubble up naturally because there are no handlers lower down or you can partially handle it in the function (eg by adding some error data to the exception) then call raise to push it up to the next level. Also, is this how you're supposed to do it? More or less, except you should aim to deal with errors at the earliest opportunity. But bvery often you don;t know the best way to deal with an error at the point where it occurs, you need a wider context. So in that case bubbling up to a higher level, with a view of the context is the right thing to do. def get_input(): ... try: minimum = int(inps[0]) maximum = int(inps[1]) rows = int(inps[2]) columns = int(inps[3]) return minimum, maximum, rows, columns except ValueError as err: print("non-integer entered", err) return None except IndexError as err: print("You didn't enter enough values.", err) return None Rather than printing the messages you could re-raise the error but with the error message attached as a string. Or, more usefully in a function called get_input() force it to go round a while loop until they provide valid input. vals = get_input() if vals: minimum, maximum, rows, columns = vals try: make_table(minimum, maximum, rows, columns) make_table(*vals) would be neater But it would be better to catch bad input data in the get_input() function rather than wait till you use it. It will make debugging easier and potentially improve the reliability of the get_input() in terms of reuse opportunities. After all it's the get_input() function's job to get input, so it should check that the inputs are sane. So for example you could do some basic checks like ensuring min <= max, and that rows and cols are non-zero. If not then either get more inputs or throw a ValueError at that point. Your try/except then goes round get_input() not the function that uses the inputs. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Terminology question
"Jim Mooney Py3.4.3winXP" writes: > I noticed that if I call a function that throws an error (Known in Python as “raise an exception”. I understood you, but it's better to use the terminology that matches what the Python docs say.) > I can catch it from the caller, instead of catching it in the > function. Is this is what is known as "errors bubbling up?" Also, is > this how you're supposed to do it? Yes, the “call stack” (the stack of function calls waiting for execution to return to them) can be thought of as a vertical stack through which an exception will “bubble up”. You should allow exceptions to bubble up to a point where they can be handled sensibly. Conversely, you should only ever catch a (type of) excption that you will be able to handle sensibly at that point, otherwise allow it to continue up the stack. -- \“It's all in the mind, you know.” —The Goon Show | `\ | _o__) | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor