On Oct 11, 7:57 pm, Hanne Moa wrote:
> On Sun, Oct 11, 2009 at 04:42, Simon Willison wrote:
> > I'm keen to receive as much feedback on the implementation as possible.
>
> It is perfectly possible to set up logging in such a way that all the
> logs for several programs wind up in the same file,
On Sun, Oct 11, 2009 at 04:42, Simon Willison wrote:
> I'm keen to receive as much feedback on the implementation as possible.
It is perfectly possible to set up logging in such a way that all the
logs for several programs wind up in the same file, or there are
duplicate entries. The first happe
I'm now tracking the logging implementation in ticket #12012:
http://code.djangoproject.com/ticket/12012
I've started posting patches from my branch there. Again, this is just
exploratory code at the moment but I'm keen to receive as much
feedback on the implementation as possible.
--~--~---
On Sat, Oct 10, 2009 at 9:11 AM, Vinay Sajip wrote:
>
>
>
> On Oct 10, 1:56 pm, Simon Willison wrote:
>> I just hacked in a logger that increments a counter for every message
>> - it indicates that there are 158,527 SQL statements executed by the
>> test suite, which means we should expect to ad
On Oct 10, 1:56 pm, Simon Willison wrote:
> I just hacked in a logger that increments a counter for every message
> - it indicates that there are 158,527 SQL statements executed by the
> test suite, which means we should expect to add slightly over a second
> to the time taken for the overall u
On Oct 10, 1:41 pm, Simon Willison wrote:
> Oops, yes you're right - I misread the numbers. A 2.5 second different
> would correspond to around 350,000 log messages (assuming the rate of
> 130,000/second from my microbenchmark) - I have no idea how many
> statements the unit tests execute in tota
On Oct 10, 12:55 pm, Luke Plant wrote:
> > master: 314.442s
> > logging: 317.096s
>
> > Since there's nothing at all in my logging code that will speed
> > anything up, I'm putting that down to chance (I did run the suite a
> > few times first to "warm up" the VM I used, but like I said, this
> >
On Saturday 10 October 2009 07:31:31 Simon Willison wrote:
> master: 314.442s
> logging: 317.096s
>
> Since there's nothing at all in my logging code that will speed
> anything up, I'm putting that down to chance (I did run the suite a
> few times first to "warm up" the VM I used, but like I sai
On Oct 9, 4:54 pm, Jacob Kaplan-Moss wrote:
> One big question I have is about performance: have you done any a/b
> testing against trunk to see how much adding logging -- especially
> upon every SQL query -- impacts performance?
>
> For me, at least, performance is going to be the critical facto
On Fri, Oct 9, 2009 at 4:33 AM, Simon Willison wrote:
> For anyone interested in following this, I've just started a
> (currently experimental) branch on GitHub:
>
> http://github.com/simonw/django/tree/logging
One big question I have is about performance: have you done any a/b
testing against t
On Fri, Oct 9, 2009 at 6:12 AM, Luke Plant wrote:
>
> On Friday 09 October 2009 10:33:28 Simon Willison wrote:
>> For anyone interested in following this, I've just started a
>> (currently experimental) branch on GitHub:
>>
>> http://github.com/simonw/django/tree/logging
>
> Is there some easy wa
On Oct 9, 11:12 am, Luke Plant wrote:
> >http://github.com/simonw/django/tree/logging
>
> Is there some easy way to see the patch? Preferably a link which will
> just show the diff between the latest of your branch and trunk, but
> failing that, a recipe of commands using git (for those of us wh
On Friday 09 October 2009 10:33:28 Simon Willison wrote:
> For anyone interested in following this, I've just started a
> (currently experimental) branch on GitHub:
>
> http://github.com/simonw/django/tree/logging
Is there some easy way to see the patch? Preferably a link which will
just show
For anyone interested in following this, I've just started a
(currently experimental) branch on GitHub:
http://github.com/simonw/django/tree/logging
So far I've added a 'django' logger which is silent by default, and
hooked up a new CursorLoggingWrapper to log all SQL statements from
the ORM to
On Sep 29, 2:25 pm, Ivan Sagalaev wrote:
> LOGGING = {
> 'errors': {
> 'handler': 'django.logging.FileLogger', # WatchedFileLogger copy
> 'filename': '...',
> 'level': 'debug',
> },
> 'maintenance': {
> 'handler': 'logging.handlers.HTTPHandler',
On Oct 4, 9:22 pm, Ivan Sagalaev wrote:
> I see one problem with explicitly calling a function from settings.py.
> This will execute instantiation of handlers and all the user staff very
> early when settings aren't fully configured yet. This prevents a user to
> have a handler that will use Dj
On Oct 4, 8:55 pm, Ivan Sagalaev wrote:
> Uhm... No this is not about hand-holding or something like that. I
> propose a format that is shorter and more convenient than the
> ConfigParser's. This is the point.
>
> > My reservation with Django's own take on it is simply
> > that it goes against
Vinay Sajip wrote:
> We should provide a configure_logging(dict) utility function which
> takes a dict in an agreed format (such as Ivan's suggestion). Then in
> settings.py users can invoke configure_logging(LOGGING) after
> defining the configuration right there, in settings.py itself, or
> obt
Sorry for taking a bit long to respond. Looks like we aren't disagree
much anyway but I had to get some thoughts off my chest :-).
Vinay Sajip wrote:
> It's similar to Django's support for, say, simplejson. I think it's
> reasonable for Django to alias WatchedFileHandler so that it's
> available
On Sep 30, 1:20 pm, Russell Keith-Magee
wrote:
>
> I was thinking that a calming, motherly "there there, it's all right,
> the boogeymonster isn't real" would do the trick :-)
I'll see what I can do...
>
> I suppose this is a big part of the problem. The logging module isn't
> a trivial API,
On Tue, Sep 29, 2009 at 10:59 PM, Vinay Sajip wrote:
>
> On Sep 29, 1:53 pm, Russell Keith-Magee
> wrote:
>>
>> However, since you're interested in feedback, my suggestion would be
>> to look at every defense you've made of logging in this thread (and
>> any other threads where you've had simila
On Sep 29, 11:29 pm, Simon Willison wrote:
>
> Here's my understanding of what we need to do for Django.
>
> 1. Create a 'django' logger somewhere in the framework that will
> always be executed if part of Django has been imported - would the
> django/__init__.py file work? Configure that with
On Tue, Sep 29, 2009 at 6:29 PM, Simon Willison wrote:
>
[snip]
>
> By default, runserver would display no log messages at all.
Runserver currently outputs each request to the commandline. Any
reason that couldn't be handled by a logger? I seem to recall people
complaining that they couldn't red
So I've read through the Python Logging 101 (really useful) and the
logging docs:
http://plumberjack.blogspot.com/2009/09/python-logging-101.html
http://docs.python.org/library/logging.html
Here's my understanding of what we need to do for Django.
1. Create a 'django' logger somewhere in the fr
On Sep 29, 6:07 pm, Simon Willison wrote:
> Thanks for dropping by. Firstly, I'm sorry for disparaging of your
> work. I mis-spoke when I said the logging module wasn't "nicely
> designed" - I obviously don't believe that, since I'm very keen on
> including it in Django. I should have picked my
On Sep 29, 2:00 pm, Waylan Limberg wrote:
> The hard part is that basicConfig only works like that back to Python
> 2.4 yet Django supports 2.3. When I added logging to Python-Markdown,
> this was the hardest part. Figuring out how to configure logging so
> that it works in 2.3 as well. The docu
On Sep 18, 5:21 pm, Eric Holscher wrote:
> I have looked into Logging before for another project, and I found that
> SQLAlchemy's support seemed to be a pretty good model to follow. They define
> all of their loggers under the sqlalchemy namespace, and then you can
> configure different handlers
On Sep 29, 9:36 am, Vinay Sajip wrote:
> Simon, I'm the author of Python's logging package. Sorry for the delay
> in replying, I've been away from this list awhile. I think the "Java
> heritage shines through" is just FUD.
Hi Vinjay,
Thanks for dropping by. Firstly, I'm sorry for disparaging of
On Sep 29, 1:53 pm, Russell Keith-Magee
wrote:
>
> First off - let me reinforce that I'm in your camp here - I like
> Python's logger, and I think we should be adding logging to Django.
> Any hesitation I have expressed is mostly a function of institutional
> inertia, especially with regards to
On Sep 29, 2:35 pm, Ivan Sagalaev wrote:
> Forgot one thing. As it stands now Django has this "historic" behavior
> when it imports and executes settings module twice. This results in
> breakage when you setup loggers and handlers by hand. We now circumvent
> this by having a helper function th
On Sep 29, 2:25 pm, Ivan Sagalaev wrote:
>
> Using WatchedFileHandler is a safe default because it works as
> FileHandler, just doesn't break with logrotate. I don't know of any
> disadvantages of WatchedFileHandler before the old FileHandler. So I
> don't think there's much value in giving peop
Ivan Sagalaev wrote:
> Standard Python logging configuration has
> a noticable gap between very simplistic basicConfig which configures
> only a root channel and a verbose imperative definition of handler
> objects, formatter objects and logger objects.
Forgot one thing. As it stands now Djang
Vinay Sajip wrote:
> Actually you don't need much in settings.py, and Django doesn't need
> to grow any code of its own to "wrap" logging. You can either
> configure logging programmatically (for which I use basicConfig, in
> simple setups) or using a configuration file (ConfigParser-based,
> full
On Tue, Sep 29, 2009 at 9:00 PM, Waylan Limberg wrote:
>
> On Tue, Sep 29, 2009 at 4:36 AM, Vinay Sajip wrote:
>>
>>
>>
>> On Sep 17, 9:25 am, Simon Willison wrote:
>>> Problems and challenges
>>> ===
>>>
>>> 1. The Python logging module isn't very nicely designed - its Java
On Tue, Sep 29, 2009 at 4:36 AM, Vinay Sajip wrote:
>
>
>
> On Sep 17, 9:25 am, Simon Willison wrote:
>> Problems and challenges
>> ===
>>
>> 1. The Python logging module isn't very nicely designed - its Java
>> heritage shines through, and things like logging.basicConfig beh
On Tue, Sep 29, 2009 at 5:09 PM, Vinay Sajip wrote:
>
> On Sep 17, 3:04 pm, Russell Keith-Magee
> wrote:
>> In the absence of specifics, this makes me a little bit nervous. The
>> Python logging interface may be very Java-heavy and complex, but it is
>> a thoroughly known quantity, and it houses
On Sep 17, 9:41 pm, Simon Willison wrote:
> I should clarify - by "lightweight wrapper" I basically mean a pre-
> configured log setup and a standard place to import the logger from -
There's no "the logger". Each module should have its own logger, this
allows you to control the verbosity of l
On Sep 17, 3:04 pm, Russell Keith-Magee
wrote:
> To clarify - I think that documentation is the very least we should
> do. As your comments indicate, there are a lot of things you need to
> do in order to get logging right, so we should at the very least
> provide some documentation on how to d
On Sep 17, 3:04 pm, Russell Keith-Magee
wrote:
> In the absence of specifics, this makes me a little bit nervous. The
> Python logging interface may be very Java-heavy and complex, but it is
> a thoroughly known quantity, and it houses a lot of features.
See my comment about Java-heavy being F
On Sep 17, 11:53 am, Ivan Sagalaev wrote:
> Talking about predictable and standard way I want to be sure that we
> don't break existing logging.
>
> I.e. we at Yandex now have many reusable Django apps that don't setup
> loggers themselves but just log things into named loggers and expect
> the
On Sep 17, 10:37 am, Mat Clayton wrote:
> +1 for this, another random thought which doesn't do your long post justice.
> But what are everyone's thoughts about log aggregation, taking logs from X
> app servers and combining them into a single location, something like
> Facebook's Scribe. I assu
On Sep 17, 9:25 am, Simon Willison wrote:
> Problems and challenges
> ===
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the sec
On Sep 18, 6:21 pm, Eric Holscher wrote:
> I have looked into Logging before for another project, and I found that
> SQLAlchemy's support seemed to be a pretty good model to follow. They define
> all of their loggers under the sqlalchemy namespace, and then you can
> configure different handlers
I have looked into Logging before for another project, and I found that
SQLAlchemy's support seemed to be a pretty good model to follow. They define
all of their loggers under the sqlalchemy namespace, and then you can
configure different handlers for different things[1]:
import logging
logging.b
On Sep 17, 6:25 pm, Simon Willison wrote:
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['w
On Sep 17, 4:04 pm, Russell Keith-Magee
wrote:
> I've seen several attempts to wrap Java loggers in a "nicer"
> interface, and every one of them ended up hobbling some of the power
> features of the logger. There is also the issue of our wrapper playing
> nicely with the loggers already being us
On Sep 17, 10:25 am, Simon Willison wrote:
> That's 0.0006 of a second overhead for a page logging 100 SQL
> statements. The performance overhead will go up if you attach a
> handler, but that's fine - the whole point of a framework like
> 'logging' is that you can log as much as you like but onl
+1 on the logging proposal. The stock python logging module is
definitely a bit of a finicky and confusing creature, especially for
people coming to Python for the first time with Django.
-0 On the signals based approach. I would be wary of the potential
performance overhead of replacing loggin
On Sep 17, 1:25 am, Simon Willison wrote:
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
I'm a big fan of this proposal, for exactly this reason.
+1
Thanks,
Eric Florenzano
--~--~-~--~~
On Thu, Sep 17, 2009 at 10:04 AM, Russell Keith-Magee
wrote:
> As for likely roadblocks: I've been led to believe that Adrian has
> objections to framework-level logging. I have no idea as to the nature
> of his objection, but ticket #5415 indicates that he is (or has been,
> historically) in fa
On Thu, Sep 17, 2009 at 4:25 PM, Simon Willison wrote:
>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler a
On Thu, Sep 17, 2009 at 6:53 PM, Ivan Sagalaev
wrote:
>
> Hi Simon,
>
> Simon Willison wrote:
>> 1. We'll be able to de-emphasise the current default "e-mail all
>> errors to someone" behaviour, which doesn't scale at all well.
>
> In a recent thread[1] on a similar topic Russel has also emphasiz
Hi Simon,
Simon Willison wrote:
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
In a recent thread[1] on a similar topic Russel has also emphasized that
we should improve documentation about doing logging.
> 3.
Definitely a +1 from me.
-- Horst
On Thu, Sep 17, 2009 at 10:25 AM, Simon Willison
wrote:
>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
+1 for this, another random thought which doesn't do your long post justice.
But what are everyone's thoughts about log aggregation, taking logs from X
app servers and combining them into a single location, something like
Facebook's Scribe. I assume this could be built in as a separate log
handler,
I think we should add logging to Django in version 1.2, implemented as
a light-weight wrapper around the Python logging module
(django.core.log maybe?) plus code to write errors to the Apache error
log under the mod_python handler and environ['wsgi.errors'] under WSGI
(meaning mod_wsgi will write
56 matches
Mail list logo
