Re: [Tutor] python logging module: two handlers writing to the same file - okay?
Ricardo Aráoz wrote: Kent Johnson wrote: I don't know the answer, but it has nothing to do with the logging module. The question is, can the same file reliably be opened twice for writing in the same module. Well, the question would actually be if the logging module is smart enough to find out that both your filehandlers are referring to the same file and open it only once. A quick glance at the logging module shows that it is not that smart. It just opens the file twice; hence my original question. To summarise the situation: the logging module is not clever enough to detect that I am opening the same file twice. If I use 'w' as the opening mode, then I know (and have an example program for this) that the logging 'fails': I suppose this can be tracked to the first part of the log file being overriden when the second filehandler attempts to log its first message, and thus opens the file (again) in 'w' mode. Using 'a' (append) or 'a+' (?) *seems* to solve this problem but we are still short of knowing as to whether this is guaranteed to work (on all platforms...) or whether it just happened to be okay on Mac OS X and (Debian) Linux (on which I have tested this). I think my example raises another question (in particular in context with openening the files in 'w' mode): would it be worth contacting the developers of the logging module to suggest that it - tracks which files it writes to and - warns if the same file is opened twice (or more ofter) using 'w'? Alternatively, one could enhance the internal logic of the logging module so that it is aware of the same file being used twice (or more times) so that it only opens the file once, but I suppose this raises some other questions (if the two filehandlers have been given different opening modes, say 'a' and 'w', which one should it take when opening the file only once?). Therefore, just issueing a warning may be the better solution: should be easy to implement, doesn't change the interface, and prevents (naive) users (like myself) from wasting time trying to track down the problem of the beginning of the log file missing. As far as I can see, the only reason in your example program to open the same file twice is to use two different formatters (i.e. two different type of lines) in the same log, Absolutely right. if you'd drop the requirement for two different format of line in the same log (which is itself somewhat questionable) then you could use the same handler for both loggers, thus opening only one file. True. FYI: The reason for wanting to use two different file formats is this: we have a somewhat larger project (http://nmag.soton.ac.uk) where we combine high-level Python code with low-level Objective Caml code (and a number of libraries). We would like to use the Python-logging module to log all sorts of messages coming from various parts of the code. In particular, we find having the line number (and function name if the python version provides this) useful information in the log messages (which can bedone for Python code issueing log messages). However, this information cannot be obtained from the OCaml code; thus we do not want to waste space in the log messages, and therefore we'd like to use different formatters. I'll reply to Ken's email in a minute, and I think that closes my query. Best wishes, Hans ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python logging module: two handlers writing to the same file - okay?
Ricardo Aráoz wrote: Why should it be all or nothing. Couldn't the programmer indicate that both handlers use the same file? It would be very easy to do this using StreamHandlers instead of FileHandlers. It would also be easy to make a FileHandler subclass that keeps a map from file name to file objects so if the same file is given to two instances it reuses the first one. This could be specified as the handler in the config file even. This is probably just the right suggestion (subclassing FileHandler, including a map of files and overriding the open() method). I realise that our requirements are somewhat special (needing different formatters for one file), and that it maybe unwise trying to 'protect' the programmer from himself. On the positive side: while the logging module is not clever enough to provide the discussed functionality, it is easy to understand (and extend). Python has 'batteries included' but sometimes you have to add some wiring ;-) Thanks to everybody who contributed to this; I think we have found a good solution. Regards, Hans Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python logging module: two handlers writing to the same file - okay?
Hans Fangohr wrote: Ricardo Aráoz wrote: > Kent Johnson wrote: >> I don't know the answer, but it has nothing to do with the logging >> module. The question is, can the same file reliably be opened >> twice for >> writing in the same module. > > Well, the question would actually be if the logging module is smart > enough to find out that both your filehandlers are referring to the > same > file and open it only once. A quick glance at the logging module shows that it is not that smart. It just opens the file twice; hence my original question. >>> >>> To summarise the situation: the logging module is not clever enough to >>> detect that I am opening the same file twice. If I use 'w' as the >>> opening mode, then I know (and have an example program for this) that >>> the logging 'fails': I suppose this can be tracked to the first part of >>> the log file being overriden when the second filehandler attempts to log >>> its first message, and thus opens the file (again) in 'w' mode. >>> >>> Using 'a' (append) or 'a+' (?) *seems* to solve this problem but we are >>> still short of knowing as to whether this is guaranteed to work (on all >>> platforms...) or whether it just happened to be okay on Mac OS X and >>> (Debian) Linux (on which I have tested this). >>> >>> I think my example raises another question (in particular in context >>> with openening the files in 'w' mode): would it be worth contacting the >>> developers of the logging module to suggest that it >>> - tracks which files it writes to and >>> - warns if the same file is opened twice (or more ofter) using 'w'? >>> >>> Alternatively, one could enhance the internal logic of the logging >>> module so that it is aware of the same file being used twice (or more >>> times) so that it only opens the file once, but I suppose this raises >>> some other questions (if the two filehandlers have been given different >>> opening modes, say 'a' and 'w', which one should it take when opening >>> the file only once?). Therefore, just issueing a warning may be the >>> better solution: should be easy to implement, doesn't change the >>> interface, and prevents (naive) users (like myself) from wasting time >>> trying to track down the problem of the beginning of the log file >>> missing. >>> >> >> >> As far as I can see, the only reason in your example program to open the >> same file twice is to use two different formatters (i.e. two different >> type of lines) in the same log, > Absolutely right. > >> if you'd drop the requirement for two >> different format of line in the same log (which is itself somewhat >> questionable) then you could use the same handler for both loggers, thus >> opening only one file. > > True. > > FYI: The reason for wanting to use two different file formats is this: > we have a somewhat larger project (http://nmag.soton.ac.uk) where we > combine high-level Python code with low-level Objective Caml code (and > a number of libraries). We would like to use the Python-logging module > to log all sorts of messages coming from various parts of the code. In > particular, we find having the line number (and function name if the > python version provides this) useful information in the log messages > (which can bedone for Python code issueing log messages). However, > this information cannot be obtained from the OCaml code; thus we do > not want to waste space in the log messages, and therefore we'd like > to use different formatters. > > I'll reply to Ken's email in a minute, and I think that closes my query. > > Best wishes, > > Hans > One last thing Hans. If you are concerned about the space the log files will use maybe it would be interesting to check out the zlib module which could be used to compress your log strings before sending them to the logging module, then a utility to read the modules would close the cycle (caveat: I've never done this, just a crazy idea, but... what if it works?). Best of luck. Ricardo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python logging module: two handlers writing to the same file - okay?
HI Riccardo, >>> >>> As far as I can see, the only reason in your example program to >>> open the >>> same file twice is to use two different formatters (i.e. two >>> different >>> type of lines) in the same log, >> Absolutely right. >> >>> if you'd drop the requirement for two >>> different format of line in the same log (which is itself somewhat >>> questionable) then you could use the same handler for both >>> loggers, thus >>> opening only one file. >> >> True. >> >> FYI: The reason for wanting to use two different file formats is >> this: >> we have a somewhat larger project (http://nmag.soton.ac.uk) where we >> combine high-level Python code with low-level Objective Caml code >> (and >> a number of libraries). We would like to use the Python-logging >> module >> to log all sorts of messages coming from various parts of the >> code. In >> particular, we find having the line number (and function name if the >> python version provides this) useful information in the log messages >> (which can bedone for Python code issueing log messages). However, >> this information cannot be obtained from the OCaml code; thus we do >> not want to waste space in the log messages, and therefore we'd like >> to use different formatters. >> >> I'll reply to Ken's email in a minute, and I think that closes my >> query. >> >> Best wishes, >> >> Hans >> > > One last thing Hans. If you are concerned about the space the log > files > will use maybe it would be interesting to check out the zlib module > which could be used to compress your log strings before sending > them to > the logging module, then a utility to read the modules would close the > cycle (caveat: I've never done this, just a crazy idea, but... what if > it works?). Apologies, I wasn't clear: just wanted to be nice to the user and avoid getting "(unknown)" entries in the log messages for the line number and the function name which pushes the useful message further to the right. If we are talking compression, I guess you could inherit from the RotatingFileHandler and modify such that you gzip the rotated log files. > Best of luck. > Many thanks, Hans ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python logging module: two handlers writing to the same file - okay?
Hans Fangohr wrote: > FYI: The reason for wanting to use two different file formats is this: > we have a somewhat larger project (http://nmag.soton.ac.uk) where we > combine high-level Python code with low-level Objective Caml code (and > a number of libraries). We would like to use the Python-logging module > to log all sorts of messages coming from various parts of the code. In > particular, we find having the line number (and function name if the > python version provides this) useful information in the log messages > (which can bedone for Python code issueing log messages). However, > this information cannot be obtained from the OCaml code; thus we do > not want to waste space in the log messages, and therefore we'd like > to use different formatters. Perhaps you could write a formatter that holds two different format strings and selects between them based on whether the line number is available. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Bound To Be A Typo
Okay I copied this code directly from a book (author Michael Dawson) and it's
not working. I'm sure I've missed something obvious like the spacing or
something but I've been staring at it for 10 minutes and I can't see it. I'll
put the code and error message below. Can someone else spot the problem?
class Critter(object):
"""A virtual pet"""
def ___init___(self, name):
print "A new critter has been born!"
self.name = name
def __str__(self):
rep = "Critter object\n"
rep += "name: " + self.name + "\n"
return rep
def talk(self):
print "Hi, I'm", self.name, "\n"
#main
crit1 = Critter("Poochie")
crit1.talk()
Here's the error message:
Traceback (most recent call last):
File "C:/Python25/attributecrit.py", line 15, in
crit1 = Critter("Poochie")
TypeError: default __new__ takes no parameters
-
Looking for last minute shopping deals? Find them fast with Yahoo! Search.___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Bound To Be A Typo
earlylight publishing wrote: > class Critter(object): > """A virtual pet""" > def ___init___(self, name): Too many underscores, just use two before and two behind, i.e. __init__ Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Bound To Be A Typo
You've inadvertently used three underscores around __init__ rather
than two, and therefore you are not really defining __init__ but
instead are relying upon the inherited one from object (which takes no
parameters).
With regard,
Michael
On Monday December 17, 2007, earlylight publishing wrote:
>Okay I copied this code directly from a book (author Michael Dawson) and
> it's not working. I'm sure I've missed something obvious like the spacing or
> something but I've been staring at it for 10 minutes and I can't see it.
> I'll put the code and error message below. Can someone else spot the problem?
>
> class Critter(object):
>"""A virtual pet"""
>def ___init___(self, name):
>print "A new critter has been born!"
>self.name = name
>
>def __str__(self):
>rep = "Critter object\n"
>rep += "name: " + self.name + "\n"
>return rep
>
>def talk(self):
>print "Hi, I'm", self.name, "\n"
> #main
>crit1 = Critter("Poochie")
>crit1.talk()
>
> Here's the error message:
>
> Traceback (most recent call last):
> File "C:/Python25/attributecrit.py", line 15, in
>crit1 = Critter("Poochie")
>TypeError: default __new__ takes no parameters
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Bound To Be A Typo Thank You
I have no idea why I did that either. I know perfectly well it's supposed to
be 2 underscores! Thanks to everyone who spotted the problem.
"Michael H. Goldwasser" <[EMAIL PROTECTED]> wrote:
You've inadvertently used three underscores around __init__ rather
than two, and therefore you are not really defining __init__ but
instead are relying upon the inherited one from object (which takes no
parameters).
With regard,
Michael
On Monday December 17, 2007, earlylight publishing wrote:
> Okay I copied this code directly from a book (author Michael Dawson) and it's
> not working. I'm sure I've missed something obvious like the spacing or
> something but I've been staring at it for 10 minutes and I can't see it. I'll
> put the code and error message below. Can someone else spot the problem?
>
> class Critter(object):
> """A virtual pet"""
> def ___init___(self, name):
> print "A new critter has been born!"
> self.name = name
>
> def __str__(self):
> rep = "Critter object\n"
> rep += "name: " + self.name + "\n"
> return rep
>
> def talk(self):
> print "Hi, I'm", self.name, "\n"
> #main
> crit1 = Critter("Poochie")
> crit1.talk()
>
> Here's the error message:
>
> Traceback (most recent call last):
> File "C:/Python25/attributecrit.py", line 15, in
> crit1 = Critter("Poochie")
> TypeError: default __new__ takes no parameters
-
Never miss a thing. Make Yahoo your homepage.___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Bound To Be A Typo
On Dec 17, 2007 12:16 PM, earlylight publishing < [EMAIL PROTECTED]> wrote: > > class Critter(object): > """A virtual pet""" > def ___init___(self, name): > print "A new critter has been born!" > > You're using 3 underscores before and after 'init'. The constructor for Python classes is '__init__' not '___init___' -- - http://stderr.ws/ "Insert pseudo-insightful quote here." - Some Guy ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Bound To Be A Typo
Your init method has 3 underscores each side, it should be 2.
(As all the magic methods in Python do)
The result is the init is not recognised as an initialiser and
the default object.new method is called. as the messaage
says it takes no parameters...
HTH,
Alan G.
"earlylight publishing" <[EMAIL PROTECTED]> wrote in
message news:[EMAIL PROTECTED]
> Okay I copied this code directly from a book (author Michael Dawson)
> and it's not working. I'm sure I've missed something obvious like
> the spacing or something but I've been staring at it for 10 minutes
> and I can't see it. I'll put the code and error message below. Can
> someone else spot the problem?
>
> class Critter(object):
>"""A virtual pet"""
>def ___init___(self, name):
>print "A new critter has been born!"
>self.name = name
>
>def __str__(self):
>rep = "Critter object\n"
>rep += "name: " + self.name + "\n"
>return rep
>
>def talk(self):
>print "Hi, I'm", self.name, "\n"
> #main
> crit1 = Critter("Poochie")
> crit1.talk()
>
> Here's the error message:
>
> Traceback (most recent call last):
> File "C:/Python25/attributecrit.py", line 15, in
>crit1 = Critter("Poochie")
> TypeError: default __new__ takes no parameters
>
>
>
> -
> Looking for last minute shopping deals? Find them fast with Yahoo!
> Search.
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Bound To Be A Typo
Its "under under", not "under under under" before and after init
--Michael
On 12/17/07, earlylight publishing <[EMAIL PROTECTED]> wrote:
> Okay I copied this code directly from a book (author Michael Dawson) and
> it's not working. I'm sure I've missed something obvious like the spacing
> or something but I've been staring at it for 10 minutes and I can't see it.
> I'll put the code and error message below. Can someone else spot the
> problem?
>
> class Critter(object):
> """A virtual pet"""
> def ___init___(self, name):
> print "A new critter has been born!"
> self.name = name
>
> def __str__(self):
> rep = "Critter object\n"
> rep += "name: " + self.name + "\n"
> return rep
>
> def talk(self):
> print "Hi, I'm", self.name, "\n"
> #main
> crit1 = Critter("Poochie")
> crit1.talk()
>
> Here's the error message:
>
> Traceback (most recent call last):
> File "C:/Python25/attributecrit.py", line 15, in
> crit1 = Critter("Poochie")
> TypeError: default __new__ takes no parameters
>
>
>
> Looking for last minute shopping deals? Find them fast with Yahoo! Search.
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
--
Michael Langford
Phone: 404-386-0495
Consulting: http://www.RowdyLabs.com
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Something I don't understand
Below, "student_seats" is a list of the class "student". Why does this code set every student.row to zero when there is only one student in the list with row > 5? It still sets them all to zero if I change the test to ">200" when there are no student.rows > 200. But if I change the test to "<1" then nothing gets set to zero. Jim class student: def __init__ (self, name, row, column): self.name = name self.row = row self.column = column for student in student_seats: print student.name, "row = ", student.row, "column = ", student.column if student.row > 5: student.row = 0 print student.name, "row = ", student.row, "column = ", student.column ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Oops:
I solved my last problem. The data was string data and of course '1' is > 5. Now, if I take int(string) the code will work, except it crashes out when the data is null. student.row = int(student.row) ValueError: invalid literal for int() with base 10: '' What is the easiest and recomended way of turning the strings into numerics? Jim ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Oops:
On 18/12/2007, Jim Morcombe <[EMAIL PROTECTED]> wrote: > Now, if I take int(string) the code will work, except it crashes out when > the data is null. > > student.row = int(student.row) > ValueError: invalid literal for int() with base 10: '' > > What is the easiest and recomended way of turning the strings into numerics? It doesn't crash -- it just throws an exception. Exceptions are a big part of python; you should learn to love them :-) As an example, you could do this: try: student.row = int(student.row) except ValueError: student.row = None (assuming that setting student.row to None is the appropriate thing to do when it's blank) -- John. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
