Re: [Tutor] (no subject)
Matthew Herzog wrote:
> I have a Python3 script that reads the first eight characters of every
> filename in a directory in order to determine whether the file was created
> before or after 180 days ago based on each file's name. The file names all
> begin with MMDD or erased_MMDD_etc.xls. I can collect all these
> filenames already.
> I need to tell my script to ignore any filename that does not conform to
> the standard eight leading numerical characters, example: 20180922 or
> erased_20171207_1oIkZf.so.
> Here is my code.
>
> if name.startswith('scrubbed_'):
> fileDate = datetime.strptime(name[9:17], DATEFMT).date()
> else:
> fileDate = datetime.strptime(name[0:8], DATEFMT).date()
>
> I need logic to prevent the script from 'choking' on files that don't fit
> these patterns. The script needs to carry on with its work and forget
> about non-conformant filenames. Do I need to add code that causes an
> exception or just add an elif block?
Personally I would use a try...except clause because with that you can
handle invalid dates like _etc.xls gracefully. So
ERASED = "erased_"
def strip_prefix(name):
if name.startswith(ERASED):
name = name[len(ERASED):]
return name
def extract_date(name):
datestr = strip_prefix(name)[:8]
return datetime.datetime.strptime(datestr, DATEFMT).date()
for name in ...:
try:
file_date = extract_date(name)
except ValueError:
pass
else:
print(file_date)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] properly propagate problems
One thing I often struggle with is how to deal with exceptions, especially when I have a chain of functions that use each others output and/or long running processes. As the answer will probably be "it depends" take for example this program flow: open a file and read into BytesIO buffer get a FTP connection from pool send buffer to plantuml.jar in memory FTP server render file to image get image from FTP server push the image onto CherryPy bus push (SSE) the image to web browser def read_file(input_file): try: with open(input_file, 'rb') as f: buffer = io.BytesIO(f.read()) except FileNotFoundError as e: print(e) return buffer assume the file is not found, I cannot just kill the whole process. Catching the exception is one thing, but how to deal with it properly, I have to inform the client somehow what went wrong. In this case I could push the error message into the returned buffer and just go from there and the image will show the message. I could also bypass the whole process somehow and push the error message directly on the CherryPy bus. What is wisdom, are there some general rules to follow in such cases ingo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] After virtualenv, how to use ide
I had installed Python on Ubuntu 14.04 using Anaconda package long back after failing to install independently for a long time. I was quietly using it's packaged ide Spyder and had no troubles, in fact I love spider more then atom. I recently learned how to setup a virtual environment though, which recommends to never use system wide Python install. But even after activating the virtual environment, I don't know how can i ask the spyder ide to use the new Python directory. Can anyone help me change the setting of Spyder so that it uses different versions? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] After virtualenv, how to use ide
On Sat, Mar 23, 2019 at 12:50 PM anand warik wrote: > > I had installed Python on Ubuntu 14.04 using Anaconda package long back > after failing to install independently for a long time. I was quietly using > it's packaged ide Spyder and had no troubles, in fact I love spider more > then atom. I recently learned how to setup a virtual environment though, > which recommends to never use system wide Python install. But even after > activating the virtual environment, I don't know how can i ask the spyder > ide to use the new Python directory. Can anyone help me change the setting > of Spyder so that it uses different versions? A quick search yields this Stack Overflow thread with what appears to be several useful links embedded: https://stackoverflow.com/questions/30170468/how-to-run-spyder-in-virtual-environment -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] (no subject)
On 3/23/19 3:16 AM, Peter Otten wrote: > Personally I would use a try...except clause because with that you can > handle invalid dates like _etc.xls gracefully. So > > ERASED = "erased_" > > > def strip_prefix(name): > if name.startswith(ERASED): > name = name[len(ERASED):] > return name > > > def extract_date(name): > datestr = strip_prefix(name)[:8] > return datetime.datetime.strptime(datestr, DATEFMT).date() > > > for name in ...: > try: > file_date = extract_date(name) > except ValueError: > pass > else: > print(file_date) I'd endorse this approach as well.. with a DATEFMT of "%Y%m%d", you will get a ValueError for every date string that is not a valid date... which you then just ignore (the "pass"), since that's what you wanted to do. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] properly propagate problems
On 23Mar2019 11:04, ingo janssen wrote:
One thing I often struggle with is how to deal with exceptions,
especially when I have a chain of functions that use each others
output and/or long running processes. As the answer will probably be
"it depends"
Oh yes!
The core rule of thumb is "don't catch an exception which you don't know
how to handle", but that is for truly unexpected errors not envisaged by
the programmer. Then your programme aborts with a debugging stack trace.
Your situation below is more nuanced. Discussion below.
take for example this program flow:
open a file and read into BytesIO buffer
get a FTP connection from pool
send buffer to plantuml.jar in memory FTP server
render file to image
get image from FTP server
push the image onto CherryPy bus
push (SSE) the image to web browser
def read_file(input_file):
try:
with open(input_file, 'rb') as f:
buffer = io.BytesIO(f.read())
except FileNotFoundError as e:
print(e)
return buffer
assume the file is not found, I cannot just kill the whole process.
Catching the exception is one thing, but how to deal with it properly,
I have to inform the client somehow what went wrong.
Given a function like that, I would be inclined to do one of 2 things:
A) don't make a policy decision (catching the exception) this close to
the failure, instead let the exception out and let the caller handle it:
def read_file(input_file):
with open(input_file, 'rb') as f:
return io.BytesIO(f.read())
filename = "foo"
try:
buffer = read_file(filename)
except OSError as e:
error("could not load %r: %s", filename, e)
... failure action, maybe return from the function ...
... proceed with buffer ...
This leaves the policy decision with the calling code, which may have a
better idea about what is suitable. For example, you might pass some
useful response to your web client here. The low level function
read_file() doesn't know that it is part of a web service.
The handy thing about exceptions is that you can push that policy
decision quite a long way out. Provided the outer layer where you decide
to catch the exception knows that this involved accessing a file you can
put that try/except quite a long way out and still produce a sensible
looking error response.
Also, the further out the policy try/except lives, the simpler the inner
functions can be because they don't need to handle failure - they can be
written for success provided that failures raise exceptions, making them
_much_ simpler and easier to maintain. And with far fewer policy
decisions!
The flip side to this is that there is a limit to how far out in the
call chain this try/except can sensibly happen: if you're far enough out
that the catching code _doesn't_ know that there was a file read
involved, the error message becomes more vague (although you still have
the exception instance itself with the low level detail).
B) to return None on failure:
def read_file(input_file):
try:
with open(input_file, 'rb') as f:
return io.BytesIO(f.read())
except OSError as e:
error(
"read_file(%r): could not read input file: %s",
input_file, e)
return None
None is a useful sentinel value for failure. Note that sometimes you
will want something else if None is meaningful return value in ordinary
circumstances. Then your calling code can handle this without
exceptions:
buffer = read_file("foo")
if buffer is None:
... return nice message to web client ...
else:
... process the image ...
However, it does mean that this handling has to happen right at the call
to read_file. That can be fine, but might be inconvenient.
Finally, some related points:
I find it useful to distinguish "mechanism" and "policy". In my ideal
world a programme is at least 90% mechanism with a thin layer of policy
outside it. Here "policy" is what might be termed "business logic" or
"application logic" in some circumstances: what to do to achieve the
high level goal. The high level is where you decide how to behave in
various circumstances.
This has a few advantages: almost all low level code is mechanism: it
has a well defined, usually simple, purpose. By having almost all
failures raise an exception you can make the low level functions very
simple: do A then B then C until success, where you return the result;
raise exceptions when things go wrong (failure to open files, invalid
input parameters, what have you). This produces what I tend to call
"white list" code: code which only returns a result when all the
required operations succeed.
This is option (A) above, and makes for very simple inner functions.
For option (B) "return None on failure", this is where we decide that
specific failures are in fact valid execution paths, and None is a valid
function return, indicating
[Tutor] How to avoid "UnboundLocalError: local variable 'goal_year' referenced before assignment"?
I have just written a small program earlier today to allow the user
(me) to enter a date by which I wish to finish reading a book (=
massive programming-related book) and calculate how many pages I need
to read each day (starting today) in order to finish the book by the
target date. Looking over my code I found that I was repeating a
fairly regular pattern in collecting the user input, so I thought I
would *improve* my working program by removing the duplicated code
into a single function. So I came up with:
from datetime import date
def get_input(greeting_msg, identifier, input_prompt, date_value_err_ck,
err_msg, conditions):
""" ??? """
if greeting_msg:
print(greeting_msg)
while True:
try:
identifier = int(input(input_prompt))
if date_value_err_ck:
date(*date_value_err_ck)
except ValueError:
print(err_msg)
continue
for (condition, condition_msg) in conditions:
if condition:
print(condition_msg)
break
else:
return identifier
When I attempt to use this function with:
goal_year_params = {
'greeting_msg': "Please enter the date by which you wish to attain"
" your goal.\n",
'identifier': 'goal_year',
'input_prompt': "Enter year of your goal as an integer: ",
'date_value_err_ck': (goal_year, 1, 1),
'err_msg': "That is not a valid year. Please try again.",
'conditions': [
('goal_year < date.today().year',
"Have you invented a time machine? If not, please enter a"
" year that makes more sense!"),
('goal_year >= date.today().year + 100',
"Have you discovered the secret to eternal life? And how"
" long is this book anyway? Please enter a year that"
" makes more sense.")]}
goal_year = get_input(**goal_year_params)
I get the following traceback:
Traceback (most recent call last):
File "pages_per_day.py", line 250, in
start_pgm()
File "pages_per_day.py", line 246, in start_pgm
goal_date_obj, pages_read, total_pages_to_read = get_inputs()
File "pages_per_day.py", line 63, in get_inputs
'date_value_err_ck': (goal_year, 1, 1),
UnboundLocalError: local variable 'goal_year' referenced before assignment
I understand this result, but cannot come up with a way to implement
my desired DRY strategy as I am stuck on how to get around this "local
variable ... referenced before assignment" issue. On subsequent
passes "goal_year" will become "goal_month" and then "goal_day" as the
user needs to input all three of these numbers. Is there a way to
accomplish my goal or am I attempting to be too clever?
--
boB
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How to avoid "UnboundLocalError: local variable 'goal_year' referenced before assignment"?
Discussion below your post here, since I feel I should quote it all:
On 23Mar2019 22:15, boB Stepp wrote:
I have just written a small program earlier today to allow the user
(me) to enter a date by which I wish to finish reading a book (=
massive programming-related book) and calculate how many pages I need
to read each day (starting today) in order to finish the book by the
target date. Looking over my code I found that I was repeating a
fairly regular pattern in collecting the user input, so I thought I
would *improve* my working program by removing the duplicated code
into a single function. So I came up with:
from datetime import date
def get_input(greeting_msg, identifier, input_prompt, date_value_err_ck,
err_msg, conditions):
""" ??? """
if greeting_msg:
print(greeting_msg)
while True:
try:
identifier = int(input(input_prompt))
if date_value_err_ck:
date(*date_value_err_ck)
except ValueError:
print(err_msg)
continue
for (condition, condition_msg) in conditions:
if condition:
print(condition_msg)
break
else:
return identifier
When I attempt to use this function with:
goal_year_params = {
'greeting_msg': "Please enter the date by which you wish to attain"
" your goal.\n",
'identifier': 'goal_year',
'input_prompt': "Enter year of your goal as an integer: ",
'date_value_err_ck': (goal_year, 1, 1),
'err_msg': "That is not a valid year. Please try again.",
'conditions': [
('goal_year < date.today().year',
"Have you invented a time machine? If not, please enter a"
" year that makes more sense!"),
('goal_year >= date.today().year + 100',
"Have you discovered the secret to eternal life? And how"
" long is this book anyway? Please enter a year that"
" makes more sense.")]}
goal_year = get_input(**goal_year_params)
I get the following traceback:
Traceback (most recent call last):
File "pages_per_day.py", line 250, in
start_pgm()
File "pages_per_day.py", line 246, in start_pgm
goal_date_obj, pages_read, total_pages_to_read = get_inputs()
File "pages_per_day.py", line 63, in get_inputs
'date_value_err_ck': (goal_year, 1, 1),
UnboundLocalError: local variable 'goal_year' referenced before assignment
I understand this result, but cannot come up with a way to implement
my desired DRY strategy as I am stuck on how to get around this "local
variable ... referenced before assignment" issue. On subsequent
passes "goal_year" will become "goal_month" and then "goal_day" as the
user needs to input all three of these numbers. Is there a way to
accomplish my goal or am I attempting to be too clever?
You're not attemnpting to be too clever, but you are making some basic
errors. For example, you can't just run a string like 'goal_year <
date.today).year' and you shouldn't be trying i.e. do _not_ reach for
the eval() function.
Let's look at your get_input() function. It basicly loops until you get
a value you're happy with, and returns that value. Or it would if it
were correct. Let's examine the main loop body:
try:
identifier = int(input(input_prompt))
if date_value_err_ck:
date(*date_value_err_ck)
except ValueError:
print(err_msg)
continue
for (condition, condition_msg) in conditions:
if condition:
print(condition_msg)
break
else:
return identifier
To start with, you have confusion in the code bwteen the name you're
intending to use for the input value (the "identifier" parameter) and
the value you're reading from the user. You go:
identifier = int(input(input_prompt))
That immediatey destroys the name you passed in as a parameter. Instead,
use a distinct variable for the input value. Let's be imaginitive and
call it "value":
value = int(input(input_prompt))
and at the bottom of the get_input() function you should:
return value
Then you try to create a date from that value (though you don't save it
anywhere). I presume you want to use the datetime.date() constructor.
So:
# at the start of your programme
import datetime
then in the function:
date = datetime.date(*date_value_err_ck)
I think your plan is that datetime.date() will also raise a ValueError
for a bad year number in "value". So you want, in fact, to call:
date = datetime.date(goal_year, 1, 1)
And the construction of that will be different on your subsequent calls
where you have goal_year and are inputting goal_month, and so on.
Your date_value_err_ck comes from your dict at the bottom of the
programme. However, it is in the natural of things that you're computing
the content of that dict _before_ you call get_input().
Let
Re: [Tutor] How to avoid "UnboundLocalError: local variable 'goal_year' referenced before assignment"?
Ah! After another long break from coding, I resume a pattern of one
step forward, who knows how many steps back, as I forget what I once
remembered and understood.
It is the wee morning hours where I am at, so I don't think I will
make it through everything, but I will make a start...
On Sun, Mar 24, 2019 at 12:22 AM Cameron Simpson wrote:
>
> Discussion below your post here, since I feel I should quote it all:
>
> On 23Mar2019 22:15, boB Stepp wrote:
> >Traceback (most recent call last):
> > File "pages_per_day.py", line 250, in
> >start_pgm()
> > File "pages_per_day.py", line 246, in start_pgm
> >goal_date_obj, pages_read, total_pages_to_read = get_inputs()
> > File "pages_per_day.py", line 63, in get_inputs
> >'date_value_err_ck': (goal_year, 1, 1),
> >UnboundLocalError: local variable 'goal_year' referenced before assignment
> >
> >I understand this result, but cannot come up with a way to implement
> >my desired DRY strategy as I am stuck on how to get around this "local
> >variable ... referenced before assignment" issue. On subsequent
> >passes "goal_year" will become "goal_month" and then "goal_day" as the
> >user needs to input all three of these numbers. Is there a way to
> >accomplish my goal or am I attempting to be too clever?
>
> You're not attemnpting to be too clever, but you are making some basic
> errors. For example, you can't just run a string like 'goal_year <
> date.today).year' and you shouldn't be trying i.e. do _not_ reach for
> the eval() function.
Too late! About the time your message hit my inbox, I just finished a
working replacement after pulling the the eval() trigger. I found a
non-eval() workaround for dealing with date(*date_value_err_ck), but
could not come up with anything better than using
eval(condition.format(user_input)) to replace the "if condition:",
where the embedded "{0}" in "condition" from the calling code is being
used to pass in the actual user input value. I imagine a clever user
(other than myself) could now wreak all sorts of havoc! I totally got
rid of "identifier" as an argument.
> Let's look at your get_input() function. It basicly loops until you get
> a value you're happy with, and returns that value. Or it would if it
> were correct. Let's examine the main loop body:
>
> try:
> identifier = int(input(input_prompt))
> if date_value_err_ck:
> date(*date_value_err_ck)
> except ValueError:
> print(err_msg)
> continue
> for (condition, condition_msg) in conditions:
> if condition:
> print(condition_msg)
> break
> else:
> return identifier
>
> To start with, you have confusion in the code bwteen the name you're
> intending to use for the input value (the "identifier" parameter) and
> the value you're reading from the user. You go:
It wasn't really confusion on my part. What I *wanted* to do was to
substitute a more specific identifier from the calling code for the
generic "identifier" in the get_input() function, and use it both for
the actual user input and to fill in that value wherever I need it.
But I could not find a way to accomplish this and just left my
question at the last error state. But I guess this is not a doable
thing.
> identifier = int(input(input_prompt))
>
> That immediatey destroys the name you passed in as a parameter. Instead,
> use a distinct variable for the input value. Let's be imaginitive and
> call it "value":
Or "user_input", which I finally wound up with.
> Then you try to create a date from that value (though you don't save it
> anywhere). I presume you want to use the datetime.date() constructor.
> So:
>
> # at the start of your programme
> import datetime
>
> then in the function:
>
> date = datetime.date(*date_value_err_ck)
>
> I think your plan is that datetime.date() will also raise a ValueError
> for a bad year number in "value". So you want, in fact, to call:
That is, in fact, the plan. Since I only want to check for a bad
year, then a bad month, and finally, a bad day of the month, I saw no
point in binding a name to the date object. In the part of the code I
did not include, I create a date object from date(goal_year,
goal_month, goal_day) to use in the rest of the program.
> Let me introduce you to the lambda.
Or, in my case, "reintroduce" me to the lambda. It has been probably
at least a couple of years since I had need of this and I had totally
forgotten about it! There are those backward steps again!
I guess I will have to head for bed now and look more closely at the
rest later today. But I can see that this totally eliminates any need
for eval() -- which I did not want to use anyway -- and solves what I
was struggling so much with.
As for closures, I will have to read that portion *most* carefully
later. I know I have asked some questions in the past where these
were brought up as answers, but this did not stick in my brain at all.
Thanks, Cameron,
