Re: [Tutor] reading lines from a list of files

2015-05-14 Thread Alan Gauld

On 14/05/15 06:27, Alex Kleider wrote:


The following seems to work-

 for f_name in list_of_file_names:
 for line in open(f_name, 'r'):
 process(line)

but should I be worried that the file doesn't get explicitly closed?


If you are only ever reading from the file then no, I'd not worry
too much. But in the general case, where you might be making
changes then yes, you should worry.

It will work 99% of the time but if things go wrong there's always
a chance that a file has not been closed yet and your changes have
not been written to the file. But if you are not changing the file
it doesn't matter too much.

The only other consideration is that some OS might put a lock
on the file even if it's only read access, so in those cases
closing will release the lock sooner. But I don't think any of
the popular OS do that any more.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] reading lines from a list of files

2015-05-14 Thread Alex Kleider

On 2015-05-14 00:15, Laura Creighton wrote:

In a message of Wed, 13 May 2015 22:27:11 -0700, Alex Kleider writes:

As a follow up question:
The following seems to work-

for f_name in list_of_file_names:
for line in open(f_name, 'r'):
process(line)

but should I be worried that the file doesn't get explicitly closed?

Alex


If you use the with statement you will guarantee that the file closes
as soon as you are done with it. It will also handle exceptions nicely 
for you.

See: https://www.python.org/dev/peps/pep-0343/

In practice, Cpython's ref counting semantics means that running out
of file descriptors doesn't happen (unless you put that code in a
loop that gets called a whole lot).  But the gc used by a Python
version is not part of the language specification, but is a
language implementation detail.  If you are writing for PyPy or
Jython you will need to use the with statement or close your files
explicitly, so the gc knows you are done with them.  Relying on
'the last reference to them went away' to close your file won't
work if the gc isn't counting references.

See: http://pypy.org/compat.html
or for more detail:
http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies

Laura


Thanks, Laura, for your analysis.  I'll happily include the one extra 
line and
let 'with' do its magic rather than depend on implementation details (or 
 worry

about their shortcomings:-)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] reading lines from a list of files

2015-05-14 Thread Alex Kleider

On 2015-05-14 00:01, Alan Gauld wrote:

On 14/05/15 06:27, Alex Kleider wrote:


The following seems to work-

 for f_name in list_of_file_names:
 for line in open(f_name, 'r'):
 process(line)

but should I be worried that the file doesn't get explicitly closed?


If you are only ever reading from the file then no, I'd not worry
too much. But in the general case, where you might be making
changes then yes, you should worry.

It will work 99% of the time but if things go wrong there's always
a chance that a file has not been closed yet and your changes have
not been written to the file. But if you are not changing the file
it doesn't matter too much.

The only other consideration is that some OS might put a lock
on the file even if it's only read access, so in those cases
closing will release the lock sooner. But I don't think any of
the popular OS do that any more.


Thank you, Alan; I hadn't appreciated the important difference in risk
with write vs read.  It's clear that 'best practice' would be to use
'with' so as to 'cover all bases.'
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] reading lines from a list of files

2015-05-14 Thread Alex Kleider

On 2015-05-13 23:24, Danny Yoo wrote:

As a follow up question:
The following seems to work-

for f_name in list_of_file_names:
for line in open(f_name, 'r'):
process(line)

but should I be worried that the file doesn't get explicitly closed?



It depends on context.  Personally, I'd write it with the 'with' to
make it very clear that the loop will manage its resource.  That being
said, it sounds like there might be concerned about the nesting.
We're nesting three or four levels deep, at the very least!

I'd agree with that.  Because of this, it might be worthwile to
consider refactoring the processing of the file in a separate
function, something like this:

###
def processFile(f):
for line in f:
...

for f_name in list_of_file_names:
 with open(f_name, 'r') as f:
 processFile(f)
###

The primary reason is to reduce the nesting.  But there's also a
potential side benefit: processFile() has a better chance of being
unit-testable, since we can pass in instances of other file-like
objects, such as io.StringIO(), to spot-check the behavior of the
process.


Thanks, Danny.  This is particularly germane since the issue has come up 
in the
midst of my first attempt to do test driven development.  I'll try 
refactoring per

your suggestion.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] reading lines from a list of files

2015-05-14 Thread Laura Creighton
In a message of Wed, 13 May 2015 22:27:11 -0700, Alex Kleider writes:
>As a follow up question:
>The following seems to work-
>
> for f_name in list_of_file_names:
> for line in open(f_name, 'r'):
> process(line)
>
>but should I be worried that the file doesn't get explicitly closed?
>
>Alex

If you use the with statement you will guarantee that the file closes
as soon as you are done with it. It will also handle exceptions nicely for you.
See: https://www.python.org/dev/peps/pep-0343/

In practice, Cpython's ref counting semantics means that running out
of file descriptors doesn't happen (unless you put that code in a
loop that gets called a whole lot).  But the gc used by a Python
version is not part of the language specification, but is a
language implementation detail.  If you are writing for PyPy or
Jython you will need to use the with statement or close your files
explicitly, so the gc knows you are done with them.  Relying on
'the last reference to them went away' to close your file won't
work if the gc isn't counting references.

See: http://pypy.org/compat.html
or for more detail: 
http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies

Laura



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


[Tutor] Terminology question

2015-05-14 Thread Jim Mooney Py3.4.3winXP
I noticed that if I call a function that throws an error, I can catch it
from the caller, instead of catching it in the function. Is this is what is
known as "errors bubbling up?" Also, is this how you're supposed to do it?

*** Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600
32 bit (Intel)] on win32. ***


'''Print a table of random numbers, given min, max, rows, and columns'''
from random import randint


def make_table(minimum, maximum, rows, columns):
for row in range(rows):
print()
for column in range(columns):
print(randint(minimum, maximum), end = ' ')


def get_input():
inp = input("Enter minimum, maximum, rows, and columns, separated by
commas: ")
inps = inp.split(',')

try:
minimum = int(inps[0])
maximum = int(inps[1])
rows = int(inps[2])
columns = int(inps[3])
return minimum, maximum, rows, columns
except ValueError as err:
print("non-integer entered", err)
return None
except IndexError as err:
print("You didn't enter enough values.", err)
return None

vals = get_input()
if vals:
minimum, maximum, rows, columns = vals
try:
make_table(minimum, maximum, rows, columns)
except ValueError as err: # CATCH FUNCTION ERROR HERE INSTEAD OF IN
FUNCTION
print("Enter min before max.")
else:
print('Nothing to do.')

-- 
Jim
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Terminology question

2015-05-14 Thread Alan Gauld

On 14/05/15 23:43, Jim Mooney Py3.4.3winXP wrote:

I noticed that if I call a function that throws an error, I can catch it
from the caller, instead of catching it in the function. Is this is what is
known as "errors bubbling up?"


Yes. They can bubble up naturally because there are no handlers lower 
down or you can partially handle it in the function (eg by adding some 
error data to the exception) then call raise to push it up to the next 
level.



Also, is this how you're supposed to do it?


More or less, except you should aim to deal with errors at the earliest 
opportunity. But bvery often you don;t know the best way to deal with an 
error at the point where it occurs, you need a wider context. So in that 
case bubbling up to a higher level, with a view of the context is the 
right thing to do.




def get_input():

...

 try:
 minimum = int(inps[0])
 maximum = int(inps[1])
 rows = int(inps[2])
 columns = int(inps[3])
 return minimum, maximum, rows, columns
 except ValueError as err:
 print("non-integer entered", err)
 return None
 except IndexError as err:
 print("You didn't enter enough values.", err)
 return None


Rather than printing the messages you could re-raise
the error but with the error message attached as a string.
Or, more usefully in a function called get_input() force
it to go round a while loop until they provide valid
input.


vals = get_input()
if vals:
 minimum, maximum, rows, columns = vals
 try:
 make_table(minimum, maximum, rows, columns)


   make_table(*vals)

would be neater

But it would be better to catch bad input data in the get_input() 
function rather than wait till you use it. It will make debugging easier 
and potentially improve the reliability of the get_input()
in terms of reuse opportunities. After all it's the get_input() 
function's job to get input, so it should check that the inputs

are sane.

So for example you could do some basic checks like ensuring
min <= max, and that rows and cols are non-zero. If not then
either get more inputs or throw a ValueError at that point.
Your try/except then goes round get_input() not the function
that uses the inputs.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Terminology question

2015-05-14 Thread Ben Finney
"Jim Mooney Py3.4.3winXP"  writes:

> I noticed that if I call a function that throws an error

(Known in Python as “raise an exception”. I understood you, but it's
better to use the terminology that matches what the Python docs say.)

> I can catch it from the caller, instead of catching it in the
> function. Is this is what is known as "errors bubbling up?" Also, is
> this how you're supposed to do it?

Yes, the “call stack” (the stack of function calls waiting for execution
to return to them) can be thought of as a vertical stack through which
an exception will “bubble up”.

You should allow exceptions to bubble up to a point where they can be
handled sensibly.

Conversely, you should only ever catch a (type of) excption that you
will be able to handle sensibly at that point, otherwise allow it to
continue up the stack.

-- 
 \“It's all in the mind, you know.” —The Goon Show |
  `\   |
_o__)  |
Ben Finney

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor