[Tutor] try/exception - error block
Hi.
I have a long running process, it generates calls to a separate py
app. The py app appears to generate errors, as indicated in the
/var/log/messages file for the abrtd daemon.. The errors are
intermittent.
So, to quickly capture all possible exceptions/errors, I decided to
wrap the entire "main" block of the test py func in a try/exception
block.
This didn't work, as I'm not getting any output in the err file
generated in the exception block.
I'm posting the test code I'm using. Pointers/comments would be helpful/useful.
the if that gets run is the fac1 logic which operates on the input
packet/data..
elif (level=='collegeFaculty1'):
#getClasses(url, college, termVal,termName,deptName,deptAbbrv)
ret=getParseCollegeFacultyList1(url,content)
Thanks.
if __name__ == "__main__":
# main app
try:
#college="asu"
#url="https://webapp4.asu.edu/catalog";
#termurl="https://webapp4.asu.edu/catalog/TooltipTerms.ext";
#termVal=2141
#
# get the input struct, parse it, determine the level
#
#cmd='cat /apps/parseapp2/asuclass1.dat'
#print "cmd= "+cmd
#proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
#content=proc.communicate()[0].strip()
#print content
#sys.exit()
#s=getClasses(content)
#print "arg1 =",sys.argv[0]
if(len(sys.argv)<2):
print "error\n"
sys.exit()
a=sys.argv[1]
aaa=a
#
# data is coming from the parentApp.php
#data has been rawurlencode(json_encode(t))
#-reverse/split the data..
#-do the fetch,
#-save the fetched page/content if any
#-create the returned struct
#-echo/print/return the struct to the
# calling parent/call
#
##print urllib.unquote_plus(a).decode('utf8')
#print "\n"
#print simplejson.loads(urllib.unquote_plus(a))
z=simplejson.loads(urllib.unquote_plus(a))
##z=simplejson.loads(urllib.unquote(a).decode('utf8'))
#z=simplejson.loads(urllib2.unquote(a).decode('utf8'))
#print "aa \n"
print z
#print "\n bb \n"
#
#-passed in
#
url=str(z['currentURL'])
level=str(z['level'])
cname=str(z['parseContentFileName'])
#
# need to check the contentFname
# -should have been checked in the parentApp
# -check it anyway, return err if required
# -if valid, get/import the content into
# the "content" var for the function/parsing
#
##cmd='echo ${yolo_clientFetchOutputDir}/'
cmd='echo ${yolo_clientParseInputDir}/'
#print "cmd= "+cmd
proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
cpath=proc.communicate()[0].strip()
cname=cpath+cname
#print "cn = "+cname+"\n"
#sys.exit()
cmd='test -e '+cname+' && echo 1'
#print "cmd= "+cmd
proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
c1=proc.communicate()[0].strip()
if(not c1):
#got an error - process it, return
print "error in parse"
#
# we're here, no err.. got content
#
#fff= "sdsu2.dat"
with open(cname,"r") as myfile:
content=myfile.read()
myfile.close()
#-passed in
#college="louisville"
#url="http://htmlaccess.louisville.edu/classSchedule/";
#termVal="4138"
#print "term = "+str(termVal)+"\n"
#print "url = "+url+"\n"
#jtest()
#sys.exit()
#getTerm(url,college,termVal)
ret={} # null it out to start
if (level=='rState'):
#ret=getTerm(content,termVal)
ret=getParseStates(content)
elif (level=='stateCollegeList'):
#getDepts(url,college, termValue,termName)
ret=getParseStateCollegeList(url,content)
elif (level=='collegeFaculty1'):
#getClasses(url, college, termVal,termName,deptName,deptAbbrv)
ret=getParseCollegeFacultyList1(url,content)
elif (level=='collegeFaculty2'):
#getClasses(url, college, termVal,termName,deptName,deptAbbrv)
ret=getParseCollegeFacultyList2(content)
#
# the idea of this section.. we have the resulting
# fetched content/page...
#
a={}
status=False
if(ret['status']==True):
s=ascii_strip(ret['data'])
if(((s.find("-1) or (s.find("-1)) and
((s.find("-1) or (s.find("-1)) and
level=='classSectionDay'):
status=True
#print "herh"
#sys.exit()
#
# build the returned struct
#
#
a['Status']=True
a['recCount']=ret['count']
a['data']=ret['data']
a['nextLevel']=''
a['timestamp']=''
a['macAddress']=''
elif(ret['status']==False):
a['Status']=False
a['recCount']=0
a['data']=''
a['nextLevel']=''
a['timestamp']=''
a['macAddress']=''
res=urllib.quote(simplejson.dumps(a))
##print res
name=subprocess.Popen('uuidgen -t', shell=True,stdout=subprocess.PIPE)
name=name.communicat
Re: [Tutor] try/exception - error block
On Sun, Aug 03, 2014 at 10:29:23AM -0400, bruce wrote:
> Hi.
>
> I have a long running process, it generates calls to a separate py
> app. The py app appears to generate errors, as indicated in the
> /var/log/messages file for the abrtd daemon.. The errors are
> intermittent.
Well, what do the errors say?
> So, to quickly capture all possible exceptions/errors, I decided to
> wrap the entire "main" block of the test py func in a try/exception
> block.
I don't understand what you mean here. You're using abbreviations I
don't understand -- are you just being a lazy typist, or do they have
some specific meaning I'm missing? What's a "test py func"? How does it
relate to the long running process (written in what language?) and the
"py app" that is generating errors? Where does the "abrtd daemon" fit
into this?
I'm sure your code is really familiar to you, but it isn't familiar to
us, so taking shortcuts and abbreviations when describing it will just
lead to confusion. We don't know your
code, and we don't know your level of knowledge. Since this is a
mailing list for beginners, it's natural to assume that you must be a
beginner too, so when you call something a "daemon", does that mean it
actually is a Unix daemon, or are you just using the wrong word?
> This didn't work, as I'm not getting any output in the err file
> generated in the exception block.
I see your except block tries printing some things. Do they appear
anywhere? I wouldn't expect so, if it actually is running as a daemon,
since daemons have no access to stdout and stderr.
> I'm posting the test code I'm using. Pointers/comments would be
> helpful/useful.
For starters, cut out all the old, long dead code. We don't need, or
want, to see the fossilised remains of your code as it was six versions
back, we want to see how it is *now*. That includes comments which no
longer have any relevance to your code.
Secondly, you should try to reduce the problem to the smallest possible
amount of code. Simplify, simplify, simplify: cut out everything which
is not relevant to the actual issue at hand.
>
> the if that gets run is the fac1 logic which operates on the input
> packet/data..
> elif (level=='collegeFaculty1'):
> #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
> ret=getParseCollegeFacultyList1(url,content)
>
I'm afraid I don't have the foggiest idea of what you're trying to say
here.
> if __name__ == "__main__":
> # main app
>
> try:
[deleting lots of commented out code]
> if(len(sys.argv)<2):
> print "error\n"
> sys.exit()
You really should raise an exception on errors, but if you insist on
doing things this way, you should print to stderr, not stdout, and you
should exit with a non-zero status:
print >>sys.stdout, "descriptive error messages are better\n"
sys.exit(101)
> a=sys.argv[1]
> aaa=a
A minor stylistic thing: you can write this as:
a = aaa = sys.argv[1]
but of course you really ought to use descriptive variable names rather
than cryptic "a" and "aaa" and "z" and other meaningless names.
[deleting more fossil code]
> z=simplejson.loads(urllib.unquote_plus(a))
> print z
> url=str(z['currentURL'])
> level=str(z['level'])
> cname=str(z['parseContentFileName'])
> cmd='echo ${yolo_clientParseInputDir}/'
> proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
> cpath=proc.communicate()[0].strip()
Hmmm. Are you trying to read the value of an environment variable using
subprocess? If so, then try this instead:
cpath = os.getenv('yolo_clientParseInputDir')
If not, then sorry for the noise. Perhaps you could explain what your
call to echo in the shell is meant to do?
> cname=cpath+cname
> cmd='test -e '+cname+' && echo 1'
> proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
> c1=proc.communicate()[0].strip()
And here I think you're trying to test whether a file exists?
os.path.exists(cname)
> if(not c1):
> #got an error - process it, return
> print "error in parse"
Um, surely not? Surely the error is that the file doesn't exist, not
that it is a parsing error?
> with open(cname,"r") as myfile:
> content=myfile.read()
> myfile.close()
If you use the "with open" form, there is no need to manually close the
file, it will be automatically closed for you.
with open(cname,"r") as myfile:
content = myfile.read()
is all you need (assuming you have permission to open the file, and that
it still exists).
[lots more fossils deleted]
> ret={} # null it out to start
> if (level=='rState'):
> ret=getParseStates(content)
> elif (level=='stateCollegeList'):
> ret=getParseStateCollegeList(url,content)
> elif (level=='collegeFaculty1'):
> ret=getParseCollegeFacultyList1(url,content)
> elif (level=='collegeFaculty2'):
> ret=getPars
Re: [Tutor] try/exception - error block
chris.. my bad.. I wasnt intending to mail you personally.
Or I wouldn't have inserted the "thanks guys"!
> thanks guys...
>
> but in all that.. no one could tell me .. why i'm not getting any
> errs/exceptions in the err file which gets created on the exception!!!
>
> but thanks for the information on posting test code!
Don't email me privately - respond to the list :)
Also, please don't top-post.
ChrisA
On Sun, Aug 3, 2014 at 10:29 AM, bruce wrote:
> Hi.
>
> I have a long running process, it generates calls to a separate py
> app. The py app appears to generate errors, as indicated in the
> /var/log/messages file for the abrtd daemon.. The errors are
> intermittent.
>
> So, to quickly capture all possible exceptions/errors, I decided to
> wrap the entire "main" block of the test py func in a try/exception
> block.
>
> This didn't work, as I'm not getting any output in the err file
> generated in the exception block.
>
> I'm posting the test code I'm using. Pointers/comments would be
> helpful/useful.
>
>
> the if that gets run is the fac1 logic which operates on the input
> packet/data..
> elif (level=='collegeFaculty1'):
> #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
> ret=getParseCollegeFacultyList1(url,content)
>
>
> Thanks.
>
> if __name__ == "__main__":
> # main app
>
> try:
> #college="asu"
> #url="https://webapp4.asu.edu/catalog";
> #termurl="https://webapp4.asu.edu/catalog/TooltipTerms.ext";
>
>
> #termVal=2141
> #
> # get the input struct, parse it, determine the level
> #
>
> #cmd='cat /apps/parseapp2/asuclass1.dat'
> #print "cmd= "+cmd
> #proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
> #content=proc.communicate()[0].strip()
> #print content
> #sys.exit()
>
> #s=getClasses(content)
>
> #print "arg1 =",sys.argv[0]
> if(len(sys.argv)<2):
> print "error\n"
> sys.exit()
>
> a=sys.argv[1]
> aaa=a
>
> #
> # data is coming from the parentApp.php
> #data has been rawurlencode(json_encode(t))
> #-reverse/split the data..
> #-do the fetch,
> #-save the fetched page/content if any
> #-create the returned struct
> #-echo/print/return the struct to the
> # calling parent/call
> #
>
> ##print urllib.unquote_plus(a).decode('utf8')
> #print "\n"
> #print simplejson.loads(urllib.unquote_plus(a))
> z=simplejson.loads(urllib.unquote_plus(a))
> ##z=simplejson.loads(urllib.unquote(a).decode('utf8'))
> #z=simplejson.loads(urllib2.unquote(a).decode('utf8'))
>
> #print "aa \n"
> print z
> #print "\n bb \n"
>
> #
> #-passed in
> #
> url=str(z['currentURL'])
> level=str(z['level'])
> cname=str(z['parseContentFileName'])
>
>
> #
> # need to check the contentFname
> # -should have been checked in the parentApp
> # -check it anyway, return err if required
> # -if valid, get/import the content into
> # the "content" var for the function/parsing
> #
>
> ##cmd='echo ${yolo_clientFetchOutputDir}/'
> cmd='echo ${yolo_clientParseInputDir}/'
> #print "cmd= "+cmd
> proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
> cpath=proc.communicate()[0].strip()
>
> cname=cpath+cname
> #print "cn = "+cname+"\n"
> #sys.exit()
>
>
> cmd='test -e '+cname+' && echo 1'
> #print "cmd= "+cmd
> proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
> c1=proc.communicate()[0].strip()
>
> if(not c1):
> #got an error - process it, return
> print "error in parse"
>
> #
> # we're here, no err.. got content
> #
>
> #fff= "sdsu2.dat"
> with open(cname,"r") as myfile:
> content=myfile.read()
> myfile.close()
>
>
> #-passed in
> #college="louisville"
> #url="http://htmlaccess.louisville.edu/classSchedule/";
> #termVal="4138"
>
>
> #print "term = "+str(termVal)+"\n"
> #print "url = "+url+"\n"
>
> #jtest()
> #sys.exit()
>
> #getTerm(url,college,termVal)
>
>
> ret={} # null it out to start
> if (level=='rState'):
> #ret=getTerm(content,termVal)
> ret=getParseStates(content)
>
> elif (level=='stateCollegeList'):
> #getDepts(url,college, termValue,termName)
> ret=getParseStateCollegeList(url,content)
>
> elif (level=='collegeFaculty1'):
> #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
> ret=getParseCollegeFacultyList1(url,content)
>
> elif (level=='collegeFaculty2'):
> #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
> ret=getParseCollegeFacultyList2(content)
>
>
>
> #
> # the idea of this section.. we have the resulting
> # fetched content/page...
> #
>
> a={}
> status=False
> if(ret['status']==True):
>
>
Re: [Tutor] try/exception - error block
Hi Alan.
Yep, the err file in the exception block gets created. and the weird
thing is it matches the time of the abrtd information in the
/var/log/messages log..
Just nothing in the file!
On Sun, Aug 3, 2014 at 4:01 PM, Alan Gauld wrote:
> On 03/08/14 18:52, bruce wrote:
>
>>> but in all that.. no one could tell me .. why i'm not getting any
>>> errs/exceptions in the err file which gets created on the exception!!!
>
>
> Does the file actually get created?
> Do you see the print statement output - are they what you expect?
>
> Did you try the things Steven suggested.
>
>
>>>except Exception, e:
>>> print e
>>> print "pycolFac1 - error!! \n";
>>> name=subprocess.Popen('uuidgen -t',
>>> shell=True,stdout=subprocess.PIPE)
>>> name=name.communicate()[0].strip()
>>> name=name.replace("-","_")
>
>
> This is usually a bad idea. You are using name for the process and its
> output. Use more names...
> What about:
>
> uuid=subprocess.Popen('uuidgen -t',shell=True,stdout=subprocess.PIPE)
> output=uuid.communicate()[0].strip()
> name=output.replace("-","_")
>
>>> name2="/home/ihubuser/parseErrTest/pp_"+name+".dat"
>
>
> This would be a good place to insert a print
>
> print name2
>
>>> ofile1=open(name2,"w+")
>
>
> Why are you using w+ mode? You are only writing.
> Keep life as simple as possible.
>
>>> ofile1.write(e)
>
>
> e is quite likely to be empty
>
>>> ofile1.write(aaa)
>
>
> Are you sure aaa exists at this point? Remember you are catching all errors
> so if an error happens prior to aaa being created this will
> fail.
>
>>> ofile1.close()
>
>
> You used the with form earlier, why not here too.
> It's considered better style...
>
> Some final comments.
> 1) You call sys.exit() several times inside
> the try block. sys.exit will not be caught by your except block,
> is that what you expect?.
>
> 2) The combination of confusing naming of variables,
> reuse of names and poor code layout and excessive commented
> code makes it very difficult to read your code.
> That makes it hard to figure out what might be going on.
> - Use sensible variable names not a,aaa,z, etc
> - use 3 or 4 level indentation not 2
> - use a version control system (RCS,CVS, SVN,...) instead
> of commenting out big blocks
> - use consistent code style
> eg with f as ... or open(f)/close(f) but not both
> - use the os module (and friends) instead of subprocess if possible
>
> 3) Have you tried deleting all the files in the
> /home/ihubuser/parseErrTest/ folder and starting again,
> just to be sure that your current code is actually
> producing the empty files?
>
> 4) You use tmpParseDir in a couple of places but I don't
> see it being set anywhere?
>
>
> That's about the best I can offer based on the
> information available.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.flickr.com/photos/alangauldphotos
>
> --
> https://mail.python.org/mailman/listinfo/python-list
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
