admin - form instantiation add_view, change_view
Wouldn't it be handy to add get_form_instance function to ModelAdmin? Currently there is function get_form, which actually returns FormClass. This is then getting instantiated on several places. Sometimes its important to have possibility to add some attribute to a form / override some field choices, etc... There are two ways how to do it now: 1.) copy / override both add_view, change_view - too much code inside, not very handy, i think they should be split anyway. 2.) FakeForm class instanciated on __call__ (hacky).. I think function get_form_instance(*args, **kwargs) returning the instance of form called from add_view / change_view will be nice solution. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: FastCGI + Lighttpd documentation
Jyrki Pulliainen wrote: > Hi, > > is there a particular reason why Lighttpd documentation advices user > to create a separate script (for example, an init script) to create > the socket with manage.py? Why not let the Lighttpd take care of > spawning processes as needed? > > If no-one is against it, I could file a ticket about this and write > the new documentation > > - Jyrki > We use Lighttpd to serve a number of Django based projects, and I think we tend to use a restart script to spawn sprockets for development and staging sites, but let Lighttpd spawn sockets for live sites. We found that we needed to kill processes of non-production sites in order to see the effect of code changes. A server restart wasn't an option at the time as we were mixing live and staging sites on the same box (though we've moved away from this!) Another point to consider is whether a site will start automatically when the server starts if it relies on a user created script. - Stephen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: admin - form instantiation add_view, change_view
On Tue, Sep 29, 2009 at 3:51 PM, Peter Cicman wrote: > > Wouldn't it be handy to add get_form_instance function to ModelAdmin? > Currently there is function get_form, which actually returns > FormClass. This is then getting instantiated on several places. > Sometimes its important to have possibility to add some attribute to a > form / override some field choices, etc... > > There are two ways how to do it now: > 1.) copy / override both add_view, change_view - too much code > inside, not very handy, i think they should be split anyway. > 2.) FakeForm class instanciated on __call__ (hacky).. You've missed the third option: define your own form class. If you create your own ModelForm subclass based on the model you want to display in the admin, you can override or set whatever attributes you want, and then tell the ModelAdmin to use your custom form. No runtime modification of the form instance is required. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Model.objects.raw() (#11863)
Russ, On 29 Sep 2009, at 03:25, Russell Keith-Magee wrote: > (1) know about the trick of instantiating an object with the > unrolled list version of a cursor, and Any chance you could expand upon this? -- David Reynolds da...@reynoldsfamily.org.uk --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 17, 9:25 am, Simon Willison wrote:
> Problems and challenges
> ===
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it in our
> own higher level interface.
Simon, I'm the author of Python's logging package. Sorry for the delay
in replying, I've been away from this list awhile. I think the "Java
heritage shines through" is just FUD. basicConfig's behaviour is fully
documented here:
http://docs.python.org/library/logging.html#logging.basicConfig
Including the fact that it sometimes (by design) has no effect.
There are a lot of people for whom logging just means writing to a
file, and that's why they have difficulty understanding why logging is
designed as it is. I would suggest you take a quick look at
http://plumberjack.blogspot.com/2009/09/python-logging-101.html
and then tell me why you think Python logging isn't well designed for
its purpose. You can do basic logging with two lines of setup (one
line if you ignore the import):
import logging
logging.basicConfig(level=logging.DEBUG,filename='/path/to/my/log',
format='%(asctime)s %(message)s')
and then
logging.getLogger(__name__).debug("Just checking this works")
Not too sure where the Java heritage is there, or where the hard part
is.
>
> 2. There may be some performance overhead, especially if we replace
> mechanisms like django.connection.queries with logging. This should be
> negligble: here's a simple benchmark:
>
> # ("hello " * 100) gives a 600 char string, long enough for a SQL
> statement>>> import timeit, logging
> >>> t = timeit.Timer('logging.info("hello " * 100)', 'import logging')
> >>> t.timeit(number=100) # one hundred statements
>
> 0.00061702728271484375>>> t.timeit(number=100) # one million statements
>
> 6.458014965057373
>
> That's 0.0006 of a second overhead for a page logging 100 SQL
> statements. The performance overhead will go up if you attach a
> handler, but that's fine - the whole point of a framework like
> 'logging' is that you can log as much as you like but only act on
> messages above a certain logging level.
A quick-and-dirty measurement showed me that a logging call (which
writes to file) takes on the order of 57 microseconds, which can be
reduced to around 50 microseconds if you forego collecting stack
frame, thread and process informaion. Not too shabby, though perhaps
not appropriate for extremely high-performance use cases. I hasten to
add, it's not a scientific benchmark.
>
> 3. We risk people using logging where signals would be more
> appropriate.
>
They're for entirely different purposes so I can't imagine this will
happen too often.
> 4. We might go too far, and make Django a "noisy" piece of software
> which logs almost everything that happens within it. Let's be tasteful
> about this.
>
One thing about the logging design (which perhaps makes it appear
complicated) is that developers can shape the logging in such a way
that the verbosity in different parts can be turned on and off pretty
much at will, and even without restarting the server in some cases.
So, with a little care in how things are arranged, this needn't
happen.
> 5. People might leave logging on, then find their server disk has
> filled up with log files and caused their site to crash.
>
> 6. Logging levels are confusing - what exactly is the difference
> between warn, info, error, debug, critical and fatal? We would need to
> document this and make decisions on which ones get used for what
> within the framework.
DEBUG: Detailed information, of no interest when everything is working
well but invaluable when diagnosing problems.
INFO: Affirmations that things are working as expected, e.g. "service
has started" or "indexing run complete". Often ignored.
WARNING:There may be a problem in the near future, and this gives
advance warning of it. But the application is able to proceed
normally.
ERROR: The application has been unable to proceed as expected, due to
the problem being logged.
CRITICAL: This is a serious error, and some kind of application
meltdown might be imminent.
I'll be happy to clarify further if needed.
> What would it look like?
>
>
> Here's what I'm thinking at the moment (having given the API very
> little thought). In your application code:
>
> from django.core import log
I'm not sure it's a good idea to have a wrapper, as I don't believe
it's needed and may restrict the level of control you typically need
to have over logging. You'll not be convinced by my just saying so -
therefore, I'll be happy to work with you to understand what (in
specific areas, including at the module level) you're trying to
achieve, and explaining the best way to achieve it.
> # Log to the default channel:
> log.debug('Retrieving RS
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 17, 10:37 am, Mat Clayton wrote: > +1 for this, another random thought which doesn't do your long post justice. > But what are everyone's thoughts about log aggregation, taking logs from X > app servers and combining them into a single location, something like > Facebook's Scribe. I assume this could be built in as a separate log > handler, but it would be nice for the guys who need this functionality to be > able to achieve it easily, probably not right for core though. Mat, logging's design already allows you to do this aggregation - you can have logging events collected in multiple locations. See the Python documentation here for pointers: http://docs.python.org/library/logging.html#sending-and-receiving-logging-events-across-a-network Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 17, 11:53 am, Ivan Sagalaev wrote:
> Talking about predictable and standard way I want to be sure that we
> don't break existing logging.
>
> I.e. we at Yandex now have many reusable Django apps that don't setup
> loggers themselves but just log things into named loggers and expect
> them to be setup by a project that uses them.
That's normal. The pattern I use is:
In each module which needs to use logging, instantiate a module-global
logger using
logger = logging.getLogger(__name__)
and log to it in the module's code. The configuration happens in
settings.py.
> What I gather from your proposal is that you want the same model ("an
> app logs, a project setups") plus a nice declarative syntax in
> settings.py instead of boring creation of handlers, formatters and
> loggers. Right?
>
Actually you don't need much in settings.py, and Django doesn't need
to grow any code of its own to "wrap" logging. You can either
configure logging programmatically (for which I use basicConfig, in
simple setups) or using a configuration file (ConfigParser-based,
fully documented in Python docs) for more complex setups.
> > - We could replace (or complement) django.connection.queries with a
> > log of executed SQL. This would make the answer to the common question
> > "how do I see what SQL is being executed" much more obvious.
>
Yes, I do this with a patched CursorDebugWrapper. You can direct the
SQL to a separate file which contains only the SQL events and not
other logging events.
>
> We had this problem with standard logging. Then we switched to a
> RotatingFileHandler which wasn't very good however because its behavior
> is simplistic and is not controllable by admins with an interface that
> they know, namely logrotate. Setting up logrotate also wasn't without
> problems. When it rotates a file it should let an app know about it and
> it uses SIG_HUP for that. However this instantly terminates Django's
> flup-based FastCGI server which we use.
>
> Now we've settled on a WatchedFileHandler ported from Python 2.6 logging
> module. It watches for file descriptor change and doesn't require
> SIG_HUP to pick up a new file. May be we should port it to Django and
> use it as a default handler for logging to file system.
Why "port it to Django"? Do you mean, copy it into Django? I'm not
sure it should be the default - not everybody uses logrotate. I'd
leave this sort of decision for code in settings.py.
Regards,
Vinay Sajip
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: admin - form instantiation add_view, change_view
Your re right, i we missed the third option. It is just happening, if you wanna do something advanced, and pass some variable from admin to form. I have to say i have to make form instance modifications twice a month, mostly because of choices/queryset for a field. It is not always possible to fake this. On Sep 29, 10:12 am, Russell Keith-Magee wrote: > On Tue, Sep 29, 2009 at 3:51 PM, Peter Cicman wrote: > > > Wouldn't it be handy to add get_form_instance function to ModelAdmin? > > Currently there is function get_form, which actually returns > > FormClass. This is then getting instantiated on several places. > > Sometimes its important to have possibility to add some attribute to a > > form / override some field choices, etc... > > > There are two ways how to do it now: > > 1.) copy / override both add_view, change_view - too much code > > inside, not very handy, i think they should be split anyway. > > 2.) FakeForm class instanciated on __call__ (hacky).. > > You've missed the third option: define your own form class. If you > create your own ModelForm subclass based on the model you want to > display in the admin, you can override or set whatever attributes you > want, and then tell the ModelAdmin to use your custom form. No runtime > modification of the form instance is required. > > Yours, > Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Model.objects.raw() (#11863)
On Sep 29, 2:24 am, SeanOC wrote: > During the Djangocon sprints I started to work on a patch which would > add a nicer interface for dealing with raw SQL queries. While there I > talked to RKM about where it should fit into the ORM API and where in > the code base should live. I've finally gotten around to finishing > the code I wrote during the sprint and have posted a patch to the > related ticket (http://code.djangoproject.com/ticket/11863). You can > also get the code fromhttp://github.com/SeanOC/django. I'm a big fan of adding this feature - I firmly believe we should be encouraging people to roll their own SQL where necessary (it's the ultimate answer to complaints about missing features in the ORM), and I don't think just telling them to instantiate their own cursor is a good enough answer. It's worth talking to Jacob about this - last year he had something like this in a private branch, but I don't think he ever released it. Cheers, Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 17, 3:04 pm, Russell Keith-Magee wrote: > In the absence of specifics, this makes me a little bit nervous. The > Python logging interface may be very Java-heavy and complex, but it is > a thoroughly known quantity, and it houses a lot of features. See my comment about Java-heavy being FUD in Simon's initial post. > > I've seen several attempts to wrap Java loggers in a "nicer" > interface, and every one of them ended up hobbling some of the power > features of the logger. There is also the issue of our wrapper playing > nicely with the loggers already being used in the wild. > Absolutely agree. Wrapping is the wrong way to go, and not even needed. I use logging with Django all the time and see no need to have any special code in Django to support logging. Where necessary, I've patched my Django to include logging statements. > I'm also not entirely convinced that the answer here isn't just > documentation. The documentation for log4j has historically been > pretty awful, and while Python's documentation is an improvement, it > could certainly be better IMHO. Good documentation for how to use > logging in the context of Django could go a long way. > I'm working with Doug Hellmann (PyMOTW) to try and improve the layout of the logging documentation in Python. I'm not asking for patches (though it would be nice), but if you can give *specific* criticisms (e.g. what you think is missing, or unclear) then that will focus our efforts. > > Details notwithstanding, I'm +1 to the idea of adding logging to the > core framework - or, at least, making it easier to use logs for > reporting internal state and error conditions instead of email). > You can have your cake and eat it. It's perfectly feasible in Python logging to send only certain events to nominated email addresses (all configurable at run-time, so emails can be turned on/off, sent to different/additional destinations etc.) as well as e.g. logging tracebacks to file for the same events. > As for likely roadblocks: I've been led to believe that Adrian has > objections to framework-level logging. I have no idea as to the nature > of his objection, but ticket #5415 indicates that he is (or has been, > historically) in favor of adding signals that could be used for > logging or debugging purposes. > They (logging and signals) are two different things. Python logging allows you to consider the dimensions "What happened?", "Where did it happen?", "How important is it?" and "Who wants to know?" intelligently, and in particular it treats "What happened" and "Who wants to know?" orthogonally. You get a lot of ways of getting information to *people* whereas signals is more about letting *code* know what's going on. Unfortunately, a lot of people have got the impression that Python logging is "Java-like" and "not Pythonic" just because I acknowledged some good ideas in log4j. It's not as if Python people have a monopoly on good ideas, is it? This "Java heritage" perception sometimes leads to prejudice against logging, for no good reason that I can see. Of course there might be grievances - for example, people complain about "slow". As a single logging call which just has a file handler is of the order of some tens of microseconds, I don't know how bad this is - what do we compare against? The Tornado webserver (used by FriendFeed) is a high-performance solution which uses Python logging. SQLAlchemy uses Python logging. They are careful to consider performance and as a consequence logging doesn't present a problem in practice. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 17, 3:04 pm, Russell Keith-Magee wrote: > To clarify - I think that documentation is the very least we should > do. As your comments indicate, there are a lot of things you need to > do in order to get logging right, so we should at the very least > provide some documentation on how to do it right. > As I posted in an earlier response to Simon, I'm happy to help with this. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 17, 9:41 pm, Simon Willison wrote: > I should clarify - by "lightweight wrapper" I basically mean a pre- > configured log setup and a standard place to import the logger from - There's no "the logger". Each module should have its own logger, this allows you to control the verbosity of logging to at least the module level and potentially with finer granularity than this. > and maybe a tiny bit of syntactic sugar if it will make the common > case more palatable. I'm mostly just interested in making the logging > module an encouraged technique within the Django world. It should > definitely play nicely with any already-existant logging code. +1, there are already patterns for doing this which work, involve no need for django.contrib.log or similar and I'll happily work with the core devs to make this happen. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Model.objects.raw() (#11863)
On Tue, Sep 29, 2009 at 4:27 PM, David Reynolds
wrote:
>
> Russ,
>
> On 29 Sep 2009, at 03:25, Russell Keith-Magee wrote:
>
>> (1) know about the trick of instantiating an object with the
>> unrolled list version of a cursor, and
>
>
> Any chance you could expand upon this?
Sure. Assume a simple model:
class Author(Model):
name = CharField()
age = IntegerField()
Now, in your code, get a cursor and issue a SQL statement:
from django.db import connection.
cursor = connection.cursor()
cursor.execte('SELECT id, name, age FROM myapp_author')
result = cursor.fetchone()
At this point, result holds a tuple containing (id, name, age). You
can instantiate an instance of Author by unrolling this tuple:
instance = Author(*result)
if you want multiple instances, use cursor.fetchall():
results = cursor.fetchall()
instances = [Author(*result) for result in results]
You need to be a little careful with the fetchall version because you
could end up with a _lot_ of results - but that's really just the
standard cursor usage warning.
The real caveat: the order of the columns you SELECT *must* match the
order in which the fields are specified in the model. For example, if
you made the following SQL query:
cursor.execute('SELECT id, age, name FROM myapp_author')
the query will work fine, but you'll get a TypeError when you create
the object because 'age' can't be coerced into a string. If the two
fields that are out of order are the same data type, you won't get any
errors at all - you'll just get a badly represented instance. Hilarity
ensues.
This trick is exactly what Django does internally when it constructs
object instances. However, in the Django internals, it is a completely
automated process - Django issues the query and parses the results, so
there's no risk of getting the column order wrong.
There are some ways to work around the column ordering problem. For
example, you can interrogate the cursor to get the name of the columns
that have been returned. This is what Sean has done in his patch to
make the raw() call a little more robust. This code is completely
generic; hence the interest in adding this to core.
Yours,
Russ Magee %-)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: django-firebird backend
Maxi, I do have a few bugfixes for the firebird-backend. How can we get in touch? Regards, Thomas On Sep 28, 11:26 pm, maxi wrote: > Hi, > I'm working on implementation of firebird backend for django [1] > I've an issue related of icontains filter option. Firebird uses > CONTAINING sql clause for this, which is case insesitive. > > The problem is that the output generated (where clause) is wrong. > It return: > > WHERE "TABLE"."FIELD" CONTAINING %value% > > And, the correct form should be: > > WHERE "TABLE"."FIELD" CONTAINING ' value ' > > It is using % instead ' > > The question is, which method I have to touch to change this > behavior? > > Thanks in advance. > -- > Maxi. > > [1] http://code.google.com/p/django-firebird/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Tue, Sep 29, 2009 at 5:09 PM, Vinay Sajip wrote: > > On Sep 17, 3:04 pm, Russell Keith-Magee > wrote: >> In the absence of specifics, this makes me a little bit nervous. The >> Python logging interface may be very Java-heavy and complex, but it is >> a thoroughly known quantity, and it houses a lot of features. > > See my comment about Java-heavy being FUD in Simon's initial post. First off - let me reinforce that I'm in your camp here - I like Python's logger, and I think we should be adding logging to Django. Any hesitation I have expressed is mostly a function of institutional inertia, especially with regards to Adrian's historical position on logging. However, I would point out that IMHO, FUD is an accurate description of the state of play - though probably not in the way you probably meant. Python's logging api _looks_ a lot like log4j in parts. This is at least partially because there's a limit to how many ways you can express 'log.debug()' before you start to copy. However, as a result, there's a lot of Fear, Uncertainty and Doubt as to whether a framework that apparently has Java heritage is going to be any good in Python. Don't forget that a lot of us (myself included) got into writing Python to get away from the stupidities of the Java world. Those scars are deep, and aren't going away in a hurry. Speaking personally, log4j is responsible for a lot of those scars, due in no small part to the abysmal documentation for that project. >> I'm also not entirely convinced that the answer here isn't just >> documentation. The documentation for log4j has historically been >> pretty awful, and while Python's documentation is an improvement, it >> could certainly be better IMHO. Good documentation for how to use >> logging in the context of Django could go a long way. >> > > I'm working with Doug Hellmann (PyMOTW) to try and improve the layout > of the logging documentation in Python. I'm not asking for patches > (though it would be nice), but if you can give *specific* criticisms > (e.g. what you think is missing, or unclear) then that will focus our > efforts. My comment was actually directed at Django's documentation, which is currently silent on the issue of logging - and probably shouldn't be. However, since you're interested in feedback, my suggestion would be to look at every defense you've made of logging in this thread (and any other threads where you've had similar arguments), and work out why the current docs have allowed those viewpoints to be established as apparent fact. Some examples: * Acknowledge that there is some Java heritage, but point out that this doesn't mean it's a bad thing, and that there is a lot that _isn't_ Java based about Python's logger. * Highlight the important architectural picture. As you noted in another reply - the logger and the handler are quite separate, and this gives a lot of power. However, the existence and significance of that architectural separation isn't really a major feature of the current docs. At present, the architectural bits are buried inside API discussion, but understanding this architecture is important if you're going to understand why logging works the way it does, and understand that logging isn't just putting lines into a file. * Make the simple example actually simple. IMHO, a single-file simple logging example is good for exactly 2 things: - showing how to configure the simplest possible case of logging - explaining the "why don't I have any output" problem. Tasks like configuring the logger to use a rotating file handler are important, but can wait for much later - once issues of basic usage and architecture have been established. * Better examples of how logging works in the real world. All the examples focus on single file projects. Most of the complexities I've had with logging stem from how to use it in a multiple-file project, yet as far as I can make out, there is very little discussion of how logging should be used in a real multiple-file project. - Should I have one logger instance per module? One per conceptual "task"? - You've used "logging.getLogger(__name__)" in this thread, but this pattern isn't mentioned once in the docs. Is this best practice, or a quick-and-dirty hack? - When I have multiple loggers across multiple files, how do I configure logging? Should I be putting logging.config.fileConfig() at the start of every python file, or should I put the logging config into a single python file somewhere that configures logging, and import that module as needed? >> Details notwithstanding, I'm +1 to the idea of adding logging to the >> core framework - or, at least, making it easier to use logs for >> reporting internal state and error conditions instead of email). >> > > You can have your cake and eat it. It's perfectly feasible in Python > logging to send only certain events to nominated email addresses (all > configurable at run-time, so emails can be turned on/off, sent to > different/additio
Re: Proposal for 1.2: built-in logging with django.core.log
On Tue, Sep 29, 2009 at 4:36 AM, Vinay Sajip wrote:
>
>
>
> On Sep 17, 9:25 am, Simon Willison wrote:
>> Problems and challenges
>> ===
>>
>> 1. The Python logging module isn't very nicely designed - its Java
>> heritage shines through, and things like logging.basicConfig behave in
>> unintuitive ways (if you call basicConfig twice the second call fails
>> silently but has no effect). This is why I suggest wrapping it in our
>> own higher level interface.
>
> Simon, I'm the author of Python's logging package. Sorry for the delay
> in replying, I've been away from this list awhile. I think the "Java
> heritage shines through" is just FUD. basicConfig's behaviour is fully
> documented here:
>
> http://docs.python.org/library/logging.html#logging.basicConfig
>
> Including the fact that it sometimes (by design) has no effect.
>
> There are a lot of people for whom logging just means writing to a
> file, and that's why they have difficulty understanding why logging is
> designed as it is. I would suggest you take a quick look at
>
> http://plumberjack.blogspot.com/2009/09/python-logging-101.html
>
> and then tell me why you think Python logging isn't well designed for
> its purpose. You can do basic logging with two lines of setup (one
> line if you ignore the import):
>
> import logging
> logging.basicConfig(level=logging.DEBUG,filename='/path/to/my/log',
> format='%(asctime)s %(message)s')
>
> and then
>
> logging.getLogger(__name__).debug("Just checking this works")
>
> Not too sure where the Java heritage is there, or where the hard part
> is.
>
The hard part is that basicConfig only works like that back to Python
2.4 yet Django supports 2.3. When I added logging to Python-Markdown,
this was the hardest part. Figuring out how to configure logging so
that it works in 2.3 as well. The documentation is not exactly helpful
in that regard.
In fact, it was for this very reason that we added our own wrapper
around logging. It didn't seem reasonable for our users to go through
the same pain that we did. Sure we got a few things wrong at first,
but with the help of a few people in the community we worked those out
and our wrapper seems to work ok now. Yes - ok - I get the sense it
could be better.
Ever since then, any mention of logging leaves a bad taste in my
mouth. Perhaps if I was working only in 2.6 or such, this wouldn't be
an issue, but we have promised support back to 2.3.
Of course, it is possible that I'm missing something obvious.
--
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Tue, Sep 29, 2009 at 9:00 PM, Waylan Limberg wrote:
>
> On Tue, Sep 29, 2009 at 4:36 AM, Vinay Sajip wrote:
>>
>>
>>
>> On Sep 17, 9:25 am, Simon Willison wrote:
>>> Problems and challenges
>>> ===
>>>
>>> 1. The Python logging module isn't very nicely designed - its Java
>>> heritage shines through, and things like logging.basicConfig behave in
>>> unintuitive ways (if you call basicConfig twice the second call fails
>>> silently but has no effect). This is why I suggest wrapping it in our
>>> own higher level interface.
>>
>> Simon, I'm the author of Python's logging package. Sorry for the delay
>> in replying, I've been away from this list awhile. I think the "Java
>> heritage shines through" is just FUD. basicConfig's behaviour is fully
>> documented here:
>>
>> http://docs.python.org/library/logging.html#logging.basicConfig
>>
>> Including the fact that it sometimes (by design) has no effect.
>>
>> There are a lot of people for whom logging just means writing to a
>> file, and that's why they have difficulty understanding why logging is
>> designed as it is. I would suggest you take a quick look at
>>
>> http://plumberjack.blogspot.com/2009/09/python-logging-101.html
>>
>> and then tell me why you think Python logging isn't well designed for
>> its purpose. You can do basic logging with two lines of setup (one
>> line if you ignore the import):
>>
>> import logging
>> logging.basicConfig(level=logging.DEBUG,filename='/path/to/my/log',
>> format='%(asctime)s %(message)s')
>>
>> and then
>>
>> logging.getLogger(__name__).debug("Just checking this works")
>>
>> Not too sure where the Java heritage is there, or where the hard part
>> is.
>>
>
> The hard part is that basicConfig only works like that back to Python
> 2.4 yet Django supports 2.3. When I added logging to Python-Markdown,
> this was the hardest part. Figuring out how to configure logging so
> that it works in 2.3 as well. The documentation is not exactly helpful
> in that regard.
...
> Of course, it is possible that I'm missing something obvious.
As luck would have it, you are :-)
Django 1.2 will drop formal support for Python 2.3.
Yours,
Russ Magee %-)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
Vinay Sajip wrote:
> Actually you don't need much in settings.py, and Django doesn't need
> to grow any code of its own to "wrap" logging. You can either
> configure logging programmatically (for which I use basicConfig, in
> simple setups) or using a configuration file (ConfigParser-based,
> fully documented in Python docs) for more complex setups.
Thanks! Didn't know that. However see my further comment.
>> Now we've settled on a WatchedFileHandler ported from Python 2.6 logging
>> module. It watches for file descriptor change and doesn't require
>> SIG_HUP to pick up a new file. May be we should port it to Django and
>> use it as a default handler for logging to file system.
>
> Why "port it to Django"? Do you mean, copy it into Django? I'm not
> sure it should be the default - not everybody uses logrotate. I'd
> leave this sort of decision for code in settings.py.
Using WatchedFileHandler is a safe default because it works as
FileHandler, just doesn't break with logrotate. I don't know of any
disadvantages of WatchedFileHandler before the old FileHandler. So I
don't think there's much value in giving people this choice in settings
because non-default behavior will be rare (and still possible anyway).
One of the reasons why I propose Django's own settings structure for
logging is because we can choose better defaults for logging and have
more compact syntax for them. Standard Python logging configuration has
a noticable gap between very simplistic basicConfig which configures
only a root channel and a verbose imperative definition of handler
objects, formatter objects and logger objects. I've found that my usage
of logging inevitably falls in between: I often need a few logging
channels but I almost never, say, reuse handler objects between them.
Here's a variant of a simple config that I had in mind lately:
LOGGING = {
'errors': {
'handler': 'django.logging.FileLogger', # WatchedFileLogger copy
'filename': '...',
'level': 'debug',
},
'maintenance': {
'handler': 'logging.handlers.HTTPHandler',
'host': '...',
'url': '',
'format': ''
},
}
Top-level keys are logger names. Values are dicts describing handlers.
These dicts have several keys that Django knows about:
- 'handler': a handler class. It's imported like any other stringified
classes in settings
- 'level': a level keyword that is translated into logging.* constants.
This is done to not make users import logging by hand.
- 'format': a format string for the logging.Formatter object. We can
have a more sensible default for this than the one in Python logging. Or
not :-)
These keys are pop'd out of the dict and the rest is used as **kwargs to
the handler class instantiation.
Django's default setup may look like this:
LOGGING = {
'': {'handler': 'logging.StreamHandler'}
}
This has an advantage of always configuring a root logger to avoid an
infamous warning from Python logging when the logger doesn't have any
handlers defined. Users wanting to configure all the logging themselves
may null-out this using `LOGGING = {}`.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
Ivan Sagalaev wrote: > Standard Python logging configuration has > a noticable gap between very simplistic basicConfig which configures > only a root channel and a verbose imperative definition of handler > objects, formatter objects and logger objects. Forgot one thing. As it stands now Django has this "historic" behavior when it imports and executes settings module twice. This results in breakage when you setup loggers and handlers by hand. We now circumvent this by having a helper function that memoizes configured loggers and call it from settings.py. Having a declarative config we can hide this inside of Django and not scare users. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: django.template refactoring (again) #7806
Am 29.09.2009 um 04:07 schrieb Russell Keith-Magee:
>
> On Tue, Sep 29, 2009 at 12:44 AM, Johannes Dollinger
> wrote:
>>
Variable and FilterExpression will be deprecated as they require
the
caller to know the length of the expression in advance.
>>>
>>> I'm not sure I follow this assertion. I have no problem believing
>>> FilterExpression can be made cleaner, but Variable takes an
>>> expression
>>> like 'article.section', plus a context, and resolves it into a
>>> string.
>>> In itself, it seems like a well built and stable piece of API.
>>
>> To get this expression string in the first place you have to split
>> Token.contents somehow. smart_split() does that well for space
>> delimited expressions.
>> But if you try to parse something like {% url %} you have to split
>> its
>> results again: "foo.bar,baz|filter:arg,42" won't do it. The current
>> {%
>> url %} parser now resorts to `str.split(",")` and
>>{% url viewname list|join:"," %}
>> will break it. Of course that's a corner case and could be fixed with
>> something like `smart_split(delim=",")`. But it would be much more
>> elegant if you could just read an expression from a stream of tokens.
>> That's what I meant when I said "the {% url %} parser is broken by
>> design". I didn't mean to sound harsh - {% url %}'s parser is a
>> reasonable tradeoff between simplicity and correctness.
>
> Sure - that explains why FilterExpression is a problem, but I don't
> see why Variable is affected. Once you've parsed your expression into
> tokens, those tokens that are known to be variables need to convert
> 'foo.bar' into something derived from context.
The idea was that in most cases the template tag doesn't care if you
pass in a Variable or a FilterExpression (#5756). So ideally there was
only one method that returns something with a .resolve(Context)
method. I know that Variable has to remain functional - and it is.
It's a 20 lines wrapper around the TokenStream API:
http://github.com/emulbreh/django/blob/master/django/template/compat.py#L8
>> It's both: making the API friendlier and fixing bugs (by using the
>> higher level API).
>> I thought that motivation and scope were mostly clear and approved.
>> Let me refer you to the two previous threads for now:
>>
>> http://groups.google.com/group/django-developers/browse_thread/thread/fb246422fa4e23d8
>> http://groups.google.com/group/django-developers/browse_thread/thread/4f5b01719e497325
>
> Herein lies the problem. The message you appear to have taken from
> those threads doesn't match with what I see. I see:
>
> * Agreement that work is needed on templates
> * A long debate about a blog post comparing Jinga and Django's
> templates.
> * Some very specific comments from Malcolm addressing your approach
> to backwards compatibility [1]
> * Agreement that this is a very large body of work
> * Agreement that the API for custom tags is too verbose.
> * Comments that the process of fixing bugs should be independent of
> improving API [2]
>
> [1] http://groups.google.com/group/django-developers/msg/97d32ac923b1f700
> [2] http://groups.google.com/group/django-developers/msg/100e70150f37c172
>
> On these points, you won't get any argument from me.
>
> What I don't see is any specific endorsement of your approach, beyond
> a general "looks like you're going the right way" comment from Jacob
> [3]. However, that comment could be leveled at _any_ attempt to
> rewrite the parser - after all, it's the parser that is the problem.
> Your approach specifically _doesn't_ address Malcolm's concerns [2].
My approach was: create a new API for TagParsers then use this API for
all tag libraries in django. All bug fixes just fell out of this. I
don't see the point in fixing those bugs with the old API first and
then throwing away the code. And I understand that backwards
compatibility is important, there exist no backwards compatibility
issues I know of (FilterExpression behaviour was the only major
problem).
> [3] http://groups.google.com/group/django-developers/msg/6e03c7c1a5b3d946
>
> Your code also seems to be a moving target. My original impression was
> that you were calling for a major review of a potentially trunk-ready
> patch. Deeper reading revealed that there were potentially lots of
> backwards incompatibilities. Reading subsequent messages reveals that
> maybe those backwards incompatibilities don't exist any more. At this
> point, I'm straight up confused as to the state of this patch - is it
> trunk ready, or a work in progress? Are you looking for a design
> review or a final review? If it's a design review, what aspects of the
> design do you want feedback on?
It's definitly not trunk-ready. There have been no comments on the
TokenStream API yet and some minor design decisions remain. Again, in
it's current state there exist no backwards incompatibilites I know
of. I'm asking for review of the concept, the API and finally the code.
> Reviewi
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 29, 2:25 pm, Ivan Sagalaev wrote:
>
> Using WatchedFileHandler is a safe default because it works as
> FileHandler, just doesn't break with logrotate. I don't know of any
> disadvantages of WatchedFileHandler before the old FileHandler. So I
> don't think there's much value in giving people this choice in settings
> because non-default behavior will be rare (and still possible anyway).
>
It's similar to Django's support for, say, simplejson. I think it's
reasonable for Django to alias WatchedFileHandler so that it's
available either bound to logging's own implementation (in
sufficiently recent versions of Python) or else a copy in Django's own
code. Then people can use it if they want to, even in older Python
versions.
> One of the reasons why I propose Django's own settings structure for
> logging is because we can choose better defaults for logging and have
> more compact syntax for them. Standard Python logging configuration has
> a noticable gap between very simplistic basicConfig which configures
> only a root channel and a verbose imperative definition of handler
> objects, formatter objects and logger objects. I've found that my usage
> of logging inevitably falls in between: I often need a few logging
> channels but I almost never, say, reuse handler objects between them.
>
> Here's a variant of a simple config that I had in mind lately:
>
> LOGGING = {
> 'errors': {
> 'handler': 'django.logging.FileLogger', # WatchedFileLogger copy
> 'filename': '...',
> 'level': 'debug',
> },
> 'maintenance': {
> 'handler': 'logging.handlers.HTTPHandler',
> 'host': '...',
> 'url': '',
> 'format': ''
> },
>
> }
>
> Top-level keys are logger names. Values are dicts describing handlers.
> These dicts have several keys that Django knows about:
>
> - 'handler': a handler class. It's imported like any other stringified
> classes in settings
>
> - 'level': a level keyword that is translated into logging.* constants.
> This is done to not make users import logging by hand.
>
> - 'format': a format string for the logging.Formatter object. We can
> have a more sensible default for this than the one in Python logging. Or
> not :-)
>
> These keys are pop'd out of the dict and the rest is used as **kwargs to
> the handler class instantiation.
>
> Django's default setup may look like this:
>
> LOGGING = {
> '': {'handler': 'logging.StreamHandler'}
>
> }
>
> This has an advantage of always configuring a root logger to avoid an
> infamous warning from Python logging when the logger doesn't have any
> handlers defined. Users wanting to configure all the logging themselves
> may null-out this using `LOGGING = {}`.
I have no big problem with a configuration scheme such as you suggest
- if it's felt that a lot of Django users are not Python-savvy enough
and need some hand-holding, then you'd perhaps need something like
this. I usually configure the logging system using its own
configuration file format (ConfigParser based, and supported by the
stdlib so no additional Django code required) or using YAML (where
it's already being used for other configuration, and when having a
PyYAML dependency is not a problem.) Either way it's declarative and
not too painful. My reservation with Django's own take on it is simply
that it goes against TOOWTDI and the Zen of Python, a little at least.
Regards,
Vinay Sajip
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 29, 2:35 pm, Ivan Sagalaev wrote: > Forgot one thing. As it stands now Django has this "historic" behavior > when it imports and executes settings module twice. This results in > breakage when you setup loggers and handlers by hand. We now circumvent > this by having a helper function that memoizes configured loggers and > call it from settings.py. Having a declarative config we can hide this > inside of Django and not scare users. This is how the way basicConfig works actually helps - it's a no-op if you call it again. (It wasn't done that way for use with Django particularly - just to make life easier for newbies and casual users. Sure, an internal function would hide this from users. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 29, 1:53 pm, Russell Keith-Magee wrote: > > First off - let me reinforce that I'm in your camp here - I like > Python's logger, and I think we should be adding logging to Django. > Any hesitation I have expressed is mostly a function of institutional > inertia, especially with regards to Adrian's historical position on > logging. > That's great. I agree about adding logging to Django, and would like to help if I can. It would be good to understand what underlies Adrian's historical position on logging. > However, I would point out that IMHO, FUD is an accurate description > of the state of play - though probably not in the way you probably > meant. > > Python's logging api _looks_ a lot like log4j in parts. This is at > least partially because there's a limit to how many ways you can > express 'log.debug()' before you start to copy. However, as a result, > there's a lot of Fear, Uncertainty and Doubt as to whether a framework > that apparently has Java heritage is going to be any good in Python. > Don't forget that a lot of us (myself included) got into writing > Python to get away from the stupidities of the Java world. Those scars > are deep, and aren't going away in a hurry. Speaking personally, log4j > is responsible for a lot of those scars, due in no small part to the > abysmal documentation for that project. > We're on the same page, I think. Python's similarity to log4j is, I feel, skin deep. Just as our having features in common with monkeys and apes doesn't *make* us monkeys and apes, so also with Python logging and log4j. Compare and contrast: log4j is around 160 source files and 16K SLOC, whereas Python logging is 3 source files and under 1500 SLOC! Notice the order of magnitude difference. Functionally, Python logging pretty much provides the same functionality as log4j, but it's a lot simpler internally. Python logging is *not* a port of log4j, is written in as Pythonic a way as I know how (given that it was written when it was), and got a lot of peer review from the smart people on python-dev before going in (and got changed here and there to satisy concerns raised during the review process). Nevertheless, FUD (and I think I did mean it in that sense - Fear, Uncertainty and Doubt) needs to allayed. I'll be happy to try and do this, please feel free to ask any specific questions or make any specific criticisms and I'll do my best to deal with them. > My comment was actually directed at Django's documentation, which is > currently silent on the issue of logging - and probably shouldn't be. Right. > However, since you're interested in feedback, my suggestion would be > to look at every defense you've made of logging in this thread (and > any other threads where you've had similar arguments), and work out > why the current docs have allowed those viewpoints to be established > as apparent fact. Some examples: > > * Acknowledge that there is some Java heritage, but point out that > this doesn't mean it's a bad thing, and that there is a lot that > _isn't_ Java based about Python's logger. > That's easier to do when people raise specific points, rather than talk about Java heritage in an arm-waving way, as if it's an offshoot of the Black Death ;-) If you want an example of Python written in the Java style, look at Apache QPid. Python logging ain't that. > * Highlight the important architectural picture. As you noted in > another reply - the logger and the handler are quite separate, and > this gives a lot of power. However, the existence and significance of > that architectural separation isn't really a major feature of the > current docs. At present, the architectural bits are buried inside API That's because the docs are really pitched mainly as a a reference guide. > discussion, but understanding this architecture is important if you're > going to understand why logging works the way it does, and understand > that logging isn't just putting lines into a file. > I've recently created a blog about Python logging, where I talk about logging from first principles and try to show why the design of Python logging is as it is. It's not perfect, but it's a start. http://plumberjack.blogspot.com/2009/09/python-logging-101.html > * Make the simple example actually simple. IMHO, a single-file simple > logging example is good for exactly 2 things: >- showing how to configure the simplest possible case of logging >- explaining the "why don't I have any output" problem. I'm not sure what you're getting at. Sometimes, those two things is all that people want to know at that time. > Tasks like configuring the logger to use a rotating file handler are > important, but can wait for much later - once issues of basic usage > and architecture have been established. Sure, and I hope with Doug Hellmann's input we can get the Python logging docs to be laid out in a more logical order and imbued with more clarity. Doug's PyMOTW series sets a high bar for documentation quality - c
Re: Model.objects.raw() (#11863)
On Tue, Sep 29, 2009 at 4:30 AM, Russell Keith-Magee wrote: ... > This trick is exactly what Django does internally when it constructs > object instances. However, in the Django internals, it is a completely > automated process - Django issues the query and parses the results, so > there's no risk of getting the column order wrong. Code here; note the treatment of .defer'd querysets in a branch that uses **dict as well. http://code.djangoproject.com/browser/django/trunk/django/db/models/query.py#L950 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Final Multi-DB status Update
Hmm. I just spent some time looking at #11828, and I don't think the "syncing one db at a time" will work. The first problem this causes is with anything that subscribes to the post sync signal. Content type does this, so it can create permissions. If we sync one db at a time, I don't see how content type can do it's job. My response in the ticket was to just make sure that default get's sync'd first, guaranteed. But based on what JL said about resetdb and running the SQL manually it sounds like the roots go down a little deeper. I disagree with myself now. What if syncdb did this in two phases. It goes through every db connection and creates the appropriate tables. Then do everything else. Oh and I certainly think that tables should only exist for the db's they apply to. I had a yuck moment the first time I opened up the db and saw those empty tables. This one is going to be a hard sale on our dev team. I'm anticipating the argument I'm going to have with the DBA'ish fella on the team. Rob On Sep 14, 2:57 pm, Alex Gaynor wrote: > On Mon, Sep 14, 2009 at 1:54 PM, Joseph Kocherhans > > > > > > wrote: > > > On Mon, Sep 14, 2009 at 12:16 PM, Alex Gaynor wrote: > > >> FWIW, Russ, Joseph Kocherhans, and I discussed this at the DjangoCon > >> sprints and our conclusion was to have syncdb only sync a single table > >> at a time, and to take a --exclude flag (or was it --include?) to > >> specify what models should be syncd. > > > Did you mean to say "sync a single db" instead of "sync a single table"? > > Russ was talking about an --exclude flag at the sprints, but it doesn't > > really matter either way to me. > > Joseph > > Yes :) > > Alex > > -- > "I disapprove of what you say, but I will defend to the death your > right to say it." -- Voltaire > "The people's good is the highest law." -- Cicero > "Code can always be simpler than you think, but never as simple as you > want" -- Me --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Adding an option to re-test only failed tests
I've been using nose for our tests, and one of the features that I really like is the ability to run the tests again but filter only the ones that caused a problem. I'm thinking it would look something like this ./manage.py test --failed Does this sound worthwhile to anybody? Rob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Final Multi-DB status Update
On Tue, Sep 29, 2009 at 11:48 AM, Rob Madole wrote: > > Hmm. I just spent some time looking at #11828, and I don't think the > "syncing one db at a time" will work. The first problem this causes > is with anything that subscribes to the post sync signal. Content > type does this, so it can create permissions. If we sync one db at a > time, I don't see how content type can do it's job. > > My response in the ticket was to just make sure that default get's > sync'd first, guaranteed. But based on what JL said about resetdb and > running the SQL manually it sounds like the roots go down a little > deeper. I disagree with myself now. > > What if syncdb did this in two phases. It goes through every db > connection and creates the appropriate tables. Then do everything > else. > > Oh and I certainly think that tables should only exist for the db's > they apply to. I had a yuck moment the first time I opened up the db > and saw those empty tables. This one is going to be a hard sale on > our dev team. I'm anticipating the argument I'm going to have with > the DBA'ish fella on the team. > > Rob > > On Sep 14, 2:57 pm, Alex Gaynor wrote: >> On Mon, Sep 14, 2009 at 1:54 PM, Joseph Kocherhans >> >> >> >> >> >> wrote: >> >> > On Mon, Sep 14, 2009 at 12:16 PM, Alex Gaynor >> > wrote: >> >> >> FWIW, Russ, Joseph Kocherhans, and I discussed this at the DjangoCon >> >> sprints and our conclusion was to have syncdb only sync a single table >> >> at a time, and to take a --exclude flag (or was it --include?) to >> >> specify what models should be syncd. >> >> > Did you mean to say "sync a single db" instead of "sync a single table"? >> > Russ was talking about an --exclude flag at the sprints, but it doesn't >> > really matter either way to me. >> > Joseph >> >> Yes :) >> >> Alex >> >> -- >> "I disapprove of what you say, but I will defend to the death your >> right to say it." -- Voltaire >> "The people's good is the highest law." -- Cicero >> "Code can always be simpler than you think, but never as simple as you >> want" -- Me > > > > It's worth noting that even if our end solution is to sync all DBs to all tables you would be able to DROP the tables and not worry about them. Not saying we should go with that solution (in fact I really think we shouldn't), but the extraneous table creation isn't a huge deal in my view. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 29, 9:36 am, Vinay Sajip wrote: > Simon, I'm the author of Python's logging package. Sorry for the delay > in replying, I've been away from this list awhile. I think the "Java > heritage shines through" is just FUD. Hi Vinjay, Thanks for dropping by. Firstly, I'm sorry for disparaging of your work. I mis-spoke when I said the logging module wasn't "nicely designed" - I obviously don't believe that, since I'm very keen on including it in Django. I should have picked my words much more carefully. The point I intended to make is that using the logging module correctly requires quite a lot of knowledge on the part of the developer - there's a lot to understand in there. When I talk about a "thin wrapper" for Django really I'm really just talking about smart defaults - ensuring that Django users can start using logging without needing to know anything more than "import this and call the log.error () method", but that the full power of the logging library is still available once they need it. So again, I apologise for my extremely poor choice of words - that was decidedly unclassy. I'm excited that you're interested in helping us integrate logging with Django in the best possible way. Thanks, Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Model.objects.raw() (#11863)
On Sep 29, 2:24 am, SeanOC wrote: > During the Djangocon sprints I started to work on a patch which would > add a nicer interface for dealing with raw SQL queries. While there I > talked to RKM about where it should fit into the ORM API and where in > the code base should live. I've finally gotten around to finishing > the code I wrote during the sprint and have posted a patch to the > related ticket (http://code.djangoproject.com/ticket/11863). You can > also get the code fromhttp://github.com/SeanOC/django. I've had a bit of a think about this, and there are a couple of things I'd be very keen on seeing supported by this feature. I have a nasty feeling they won't be that straight forward though. The first is that I'd like it to be compatible with deferred loading of model attributes - I haven't looked in to it, but my hunch is that this won't be too hard (it might even work in its present form without any further changes). The second one is probably a lot harder - it feels to me like this feature would be much more useful if it could perform select_related style graph population. One of the key use-cases for raw SQL is improving performance - writing queries which perform better than the ORM (SQL optimisations, or queries that hit an alternative view or use a stored procedure or similar). It's very plausible that a developer might want to write a custom query that populates a full graph of objects, just like select_related() does. It would be really useful if the QuerySet.raw() method let them do that. I haven't looked at how much work it would be to support graph population - it will certainly complicate the user-facing API, and it might require a bunch of work at the ORM level. If it's just too much hassle I think the feature would be worth including without it, but it's certainly worth some further investigation. Cheers, Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 18, 5:21 pm, Eric Holscher wrote: > I have looked into Logging before for another project, and I found that > SQLAlchemy's support seemed to be a pretty good model to follow. They define > all of their loggers under the sqlalchemy namespace, and then you can > configure different handlers for different things[1]: > I think that this would be necessary to have in Django, so that for > instance, I could listen to the django.orm logs, and not the django.http, or > listen to them with different handlers/levels. > The approach I'd recommend is to have loggers at module level, and more granular than that if and where needed. We don't have to follow this blindly and tool up everywhere all at once to the same extent - it can progress naturally as and when we tackle issues and feel that logging is a part of continuous improvement. I think that's the spirit in which Simon initially posted. >From a user's point of view, it should be easy for them to turn the verbosity up or down at a module level at least (for example, an individual view module somewhere in Django). By which I mean, it'd be most natural to express "where" in the application you want verbosity turned up or down by thinking about apps and individual modules in those apps. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 29, 2:00 pm, Waylan Limberg wrote: > The hard part is that basicConfig only works like that back to Python > 2.4 yet Django supports 2.3. When I added logging to Python-Markdown, > this was the hardest part. Figuring out how to configure logging so > that it works in 2.3 as well. The documentation is not exactly helpful > in that regard. > Did you ask any questions on python-list? How were they responded to? The answer is pretty simple, just like in parts of Django - if it's not supported by an older version of Python, you just try and copy the relevant bits from the later version of Python (or simplejson, say). I generally monitor both python-list regularly for logging issues and questions, and numerous other people on the list also weigh in when people ask logging-related questions. So if in doubt, ask - you should generally get a friendly and hopefully helpful response. > In fact, it was for this very reason that we added our own wrapper > around logging. It didn't seem reasonable for our users to go through > the same pain that we did. Sure we got a few things wrong at first, > but with the help of a few people in the community we worked those out > and our wrapper seems to work ok now. Yes - ok - I get the sense it > could be better. > > Ever since then, any mention of logging leaves a bad taste in my > mouth. Perhaps if I was working only in 2.6 or such, this wouldn't be > an issue, but we have promised support back to 2.3. > > Of course, it is possible that I'm missing something obvious. If you had asked around on python-list and got fobbed off, perhaps a bad taste in your mouth would be understandable. But there's no reason for it - the Python developers didn't set out to make things difficult for you, and the community is generally very helpful. In fact it was the help I got when I started with Python that motivated me to contribute something back in the first place. I'm also responsive to feature enhancement requests added via bugs.python.org - while not every suggestion gets accepted, there's been a steady stream of incremental improvements to the logging package and documentation thanks to specific suggestions by members of the Python community (not to mention patches). Remember, these are all volunteer projects and while we all want to help, it's easier to latch onto specific problems and issues rather than work with an amorphous goal which begins and ends with "improve things". Notice how popular Stack Overflow is these days - that's an example of how you can get help by being specific and asking targeted questions. So please, raise any questions and/or issues on python- list, bugs.python.org, Stack Overflow, etc. Now I'll get off my soapbox. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Sep 29, 5:03 pm, Rob Madole wrote: > I've been using nose for our tests, and one of the features that I > really like is the ability to run the tests again but filter only the > ones that caused a problem. > > I'm thinking it would look something like this > > ./manage.py test --failed > > Does this sound worthwhile to anybody? I don't understand how this works - does it persist some indication of which tests failed somewhere? If so, where? If we're talking about features from nose, the two I'd really like in Django's test runner are --pdb and --pdb-failures: --pdb = when an error occurs, drop straight in to the interactive debugger --pdb-failures = when a test assertion fails, drop in to the debugger Cheers, Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Tue, Sep 29, 2009 at 1:58 PM, Simon Willison wrote: > > On Sep 29, 5:03 pm, Rob Madole wrote: >> I've been using nose for our tests, and one of the features that I >> really like is the ability to run the tests again but filter only the >> ones that caused a problem. >> >> I'm thinking it would look something like this >> >> ./manage.py test --failed >> >> Does this sound worthwhile to anybody? > > I don't understand how this works - does it persist some indication of > which tests failed somewhere? If so, where? > > If we're talking about features from nose, the two I'd really like in > Django's test runner are --pdb and --pdb-failures: > > --pdb = when an error occurs, drop straight in to the interactive > debugger > --pdb-failures = when a test assertion fails, drop in to the debugger > > Cheers, > > Simon > > > If we're throwing off testing ponies: --failfast. Runs the full test suite as normal but as soon as it hits a failure it spits out the fail and ends. Unladen Swallow added this to their test suite and it's a huge help in terms of not running the full test suite to see what the failure was. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Model.objects.raw() (#11863)
> The first is that I'd like it to be compatible with deferred loading > of model attributes - I haven't looked in to it, but my hunch is that > this won't be too hard (it might even work in its present form without > any further changes). The current implementation doesn't support this. It will actually raise an exception if not all of the fields have been provided. For now I was looking to get something basic out there to get the ball rolling. Once I have the docs for the current implementation done I'll look into what it will take to do this. > > The second one is probably a lot harder - it feels to me like this > feature would be much more useful if it could perform select_related > style graph population. One of the key use-cases for raw SQL is > improving performance - writing queries which perform better than the > ORM (SQL optimisations, or queries that hit an alternative view or use > a stored procedure or similar). It's very plausible that a developer > might want to write a custom query that populates a full graph of > objects, just like select_related() does. It would be really useful if > the QuerySet.raw() method let them do that. > > I haven't looked at how much work it would be to support graph > population - it will certainly complicate the user-facing API, and it > might require a bunch of work at the ORM level. If it's just too much > hassle I think the feature would be worth including without it, but > it's certainly worth some further investigation. > This one also isn't implemented right now. I've had a few thoughts on how it could be done but they all result in a somewhat ugly API. The least ugly option I've thought of so far would be to accept an optional parameter called related_available which would expect a tuple of tuples. The inner tuple(s) would include a prefix for the columns with related data and the name of the related field. I definitely would be interested in hearing other suggestions. -Sean --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Tue, Sep 29, 2009 at 1:58 PM, Simon Willison wrote: > > On Sep 29, 5:03 pm, Rob Madole wrote: >> I've been using nose for our tests, and one of the features that I >> really like is the ability to run the tests again but filter only the >> ones that caused a problem. >> >> I'm thinking it would look something like this >> >> ./manage.py test --failed >> >> Does this sound worthwhile to anybody? > > I don't understand how this works - does it persist some indication of > which tests failed somewhere? If so, where? > My recollection is yes, nose persists which tests passed/failed for the previous run, but I don't recall where. Personally, I've never used this (except to see how it works in nose) because I'm always concerned that while a recent change may fix a failing test, it could cause some other test to fail that previously passed. So I'm going to run the whole test suite anyway - or at least all the tests for that module, etc. Alex suggestion of --failfast seems like a much more useful way to shorten test runs. -- \X/ /-\ `/ |_ /-\ |\| Waylan Limberg --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
Yep, I use the pdb stuff too. That would be handy. The way this works in nose is through the testid plugin. Typically you do this: nosetests --with-id --failed This will create a file called .noseids in the current working directory. You can make it use something else by saying: nosetests --with-id --id-file=/somewhere/else/.noseids --failed As far as storing the data of which test failed for Django, I'm not sure what the *best* approach would be. Ned Batchelder's coverage module does a similar thing. It keeps a .coverage file in the root I think. Maybe just call ours .failedtests. Kinda gross, and not my first choice, but it would work. Or, perhaps use Python's tempfile module. But I'm not sure how to grab a hold of the temp file again for the second pass through (maybe tempfile.NamedTemporaryFile but this has problems on some platforms according to the docs). On one hand, I can see this argument: If you are adding 3 features from nose, why not just use nose. But setting up nose and Django to use it as the test runner isn't trivial the last time I checked. We're using buildout to ease the pain. Thanks for the input. Rob On Sep 29, 12:58 pm, Simon Willison wrote: > On Sep 29, 5:03 pm, Rob Madole wrote: > > > I've been using nose for our tests, and one of the features that I > > really like is the ability to run the tests again but filter only the > > ones that caused a problem. > > > I'm thinking it would look something like this > > > ./manage.py test --failed > > > Does this sound worthwhile to anybody? > > I don't understand how this works - does it persist some indication of > which tests failed somewhere? If so, where? > > If we're talking about features from nose, the two I'd really like in > Django's test runner are --pdb and --pdb-failures: > > --pdb = when an error occurs, drop straight in to the interactive > debugger > --pdb-failures = when a test assertion fails, drop in to the debugger > > Cheers, > > Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
Ok, --failfast would be nice too :D, I think I remember seeing a ticket on that. So make that 4 features from nose... Which would be great if the test is third or fourth in the stack. If it's the last test in 50, it would loose it's effectiveness. I know, I know. If you are running 50 tests you can reduce that down to the module that is causing the problem. Maybe time would be better spent making the use of nose really super easy. In settings.py: TEST_RUNNER = 'django.contrib.test.nose.run_tests' There might be some futzy bits to make that actually work, but I think it'd doable. Eh? Rob On Sep 29, 1:23 pm, Rob Madole wrote: > Yep, I use the pdb stuff too. That would be handy. > > The way this works in nose is through the testid plugin. Typically you > do this: > > nosetests --with-id --failed > > This will create a file called .noseids in the current working > directory. > > You can make it use something else by saying: > > nosetests --with-id --id-file=/somewhere/else/.noseids --failed > > As far as storing the data of which test failed for Django, I'm not > sure what the *best* approach would be. Ned Batchelder's coverage > module does a similar thing. It keeps a .coverage file in the root I > think. Maybe just call ours .failedtests. Kinda gross, and not my > first choice, but it would work. > > Or, perhaps use Python's tempfile module. But I'm not sure how to > grab a hold of the temp file again for the second pass through (maybe > tempfile.NamedTemporaryFile but this has problems on some platforms > according to the docs). > > On one hand, I can see this argument: If you are adding 3 features > from nose, why not just use nose. But setting up nose and Django to > use it as the test runner isn't trivial the last time I checked. > We're using buildout to ease the pain. > > Thanks for the input. > > Rob > > On Sep 29, 12:58 pm, Simon Willison wrote: > > > > > On Sep 29, 5:03 pm, Rob Madole wrote: > > > > I've been using nose for our tests, and one of the features that I > > > really like is the ability to run the tests again but filter only the > > > ones that caused a problem. > > > > I'm thinking it would look something like this > > > > ./manage.py test --failed > > > > Does this sound worthwhile to anybody? > > > I don't understand how this works - does it persist some indication of > > which tests failed somewhere? If so, where? > > > If we're talking about features from nose, the two I'd really like in > > Django's test runner are --pdb and --pdb-failures: > > > --pdb = when an error occurs, drop straight in to the interactive > > debugger > > --pdb-failures = when a test assertion fails, drop in to the debugger > > > Cheers, > > > Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
http://blog.jeffbalogh.org/post/57653515/nose-test-runner-for-django It's certainly been done and doesn't require changes to Django. On Sep 29, 1:34 pm, Rob Madole wrote: > Ok, --failfast would be nice too :D, I think I remember seeing a > ticket on that. So make that 4 features from nose... > > Which would be great if the test is third or fourth in the stack. If > it's the last test in 50, it would loose it's effectiveness. > > I know, I know. If you are running 50 tests you can reduce that down > to the module that is causing the problem. > > Maybe time would be better spent making the use of nose really super > easy. > > In settings.py: > > TEST_RUNNER = 'django.contrib.test.nose.run_tests' > > There might be some futzy bits to make that actually work, but I think > it'd doable. > > Eh? > > Rob > > On Sep 29, 1:23 pm, Rob Madole wrote: > > > > > Yep, I use the pdb stuff too. That would be handy. > > > The way this works in nose is through the testid plugin. Typically you > > do this: > > > nosetests --with-id --failed > > > This will create a file called .noseids in the current working > > directory. > > > You can make it use something else by saying: > > > nosetests --with-id --id-file=/somewhere/else/.noseids --failed > > > As far as storing the data of which test failed for Django, I'm not > > sure what the *best* approach would be. Ned Batchelder's coverage > > module does a similar thing. It keeps a .coverage file in the root I > > think. Maybe just call ours .failedtests. Kinda gross, and not my > > first choice, but it would work. > > > Or, perhaps use Python's tempfile module. But I'm not sure how to > > grab a hold of the temp file again for the second pass through (maybe > > tempfile.NamedTemporaryFile but this has problems on some platforms > > according to the docs). > > > On one hand, I can see this argument: If you are adding 3 features > > from nose, why not just use nose. But setting up nose and Django to > > use it as the test runner isn't trivial the last time I checked. > > We're using buildout to ease the pain. > > > Thanks for the input. > > > Rob > > > On Sep 29, 12:58 pm, Simon Willison wrote: > > > > On Sep 29, 5:03 pm, Rob Madole wrote: > > > > > I've been using nose for our tests, and one of the features that I > > > > really like is the ability to run the tests again but filter only the > > > > ones that caused a problem. > > > > > I'm thinking it would look something like this > > > > > ./manage.py test --failed > > > > > Does this sound worthwhile to anybody? > > > > I don't understand how this works - does it persist some indication of > > > which tests failed somewhere? If so, where? > > > > If we're talking about features from nose, the two I'd really like in > > > Django's test runner are --pdb and --pdb-failures: > > > > --pdb = when an error occurs, drop straight in to the interactive > > > debugger > > > --pdb-failures = when a test assertion fails, drop in to the debugger > > > > Cheers, > > > > Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Sep 29, 7:34 pm, Rob Madole wrote: > TEST_RUNNER = 'django.contrib.test.nose.run_tests' > > There might be some futzy bits to make that actually work, but I think > it'd doable. I'd love to see this working. Obviously this would work just as well implemented as an external project - but if it's as useful as it sounds I'd personally be up for including it in core, if only to promote the usage of nose amongst Django developers (and hence help weaken the impression that Django doesn't integrate well enough with the wider Python community). Cheers, Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Sep 29, 6:07 pm, Simon Willison wrote: > Thanks for dropping by. Firstly, I'm sorry for disparaging of your > work. I mis-spoke when I said the logging module wasn't "nicely > designed" - I obviously don't believe that, since I'm very keen on > including it in Django. I should have picked my words much more > carefully. > > The point I intended to make is that using the logging module > correctly requires quite a lot of knowledge on the part of the > developer - there's a lot to understand in there. When I talk about a > "thin wrapper" for Django really I'm really just talking about smart > defaults - ensuring that Django users can start using logging without > needing to know anything more than "import this and call the log.error > () method", but that the full power of the logging library is still > available once they need it. > > So again, I apologise for my extremely poor choice of words - that was > decidedly unclassy. I'm excited that you're interested in helping us > integrate logging with Django in the best possible way. > No worries. Hope we can get something going soon. Best regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
So I've read through the Python Logging 101 (really useful) and the
logging docs:
http://plumberjack.blogspot.com/2009/09/python-logging-101.html
http://docs.python.org/library/logging.html
Here's my understanding of what we need to do for Django.
1. Create a 'django' logger somewhere in the framework that will
always be executed if part of Django has been imported - would the
django/__init__.py file work? Configure that with a NullHandler, and
set it NOT to propagate events to the root logger. That way we can
start using 'django.*' loggers throughout the framework but none of
the messages will be sent anywhere without further configuration. The
code should look something like this:
import logging
class NullHandler(logging.Handler):
def emit(self, record):
pass
logger = logging.getLogger('django')
logger.addHandler(NullHandler())
logger.propagate = False
NullHandler is a generally useful class (I'm surprised it's not part
of the core logging library), so we should probably put that in
django.utils.log. The NullHandler is there purely to prevent the one-
off "you haven't configured a handler yet" error that gets sent to
sys.stderr.
2. Start using loggers within the Django framework itself. The most
obvious place to start is SQL queries - at the point where we execute
SQL, we should do this:
logging.getLogger('django.db.sql').debug(sql_to_execute)
That logs the SQL to a 'django.db.sql' logger - which is a child of
the 'django' logger and hence will be swallowed by the NullHandler.
Another place we could use logging is to record cache gets and misses:
logging.getLogger('django.core.cache').debug('Cache miss for key %s',
key)
And for sending e-mail:
logging.getLogger('django.core.mail').info('Mail sent to %s',
to_email)
The logger names reflect the structure of Django up to a point
(django.db) but I don't think there's anything wrong with straying
from that convention (django.db.sql for example) provided the prefixes
are sensible.
3. Add a bunch of syntactic sugar for making log messages visible at
various points within Django. For example:
./manage.py runserver --log-level=django:info
Would cause the runserver to first configure the 'django' logger to
print all messages of level 'info' and higher to stdout.
./manage.py runserver --log-level=django:info --log-
level=django.db:debug
Same as above, but also configures django.db messages of debug or
higher to display (I'm sure the command line syntax could be
improved).
By default, runserver would display no log messages at all.
The same arguments could also work for ./manage.py test:
./manage.py test --log-level=django.db.sql:debug
Hmm... maybe the --log-level argument could work for ALL management
commands.
My philosophy here is that log messages should be ignored unless
explicitly told otherwise. Django gets run in various different ways -
runserver, test and in deployment under mod_wsgi/FastCGI/etc - and
it's not at all obvious what the default log output behaviour should
be. As long as we make it extremely easy to see log messages if we
want them I think having them off by default makes sense. It also
ensures no surprises for people upgrading from 1.1 to 1.2.
4. Add a way to configure loggers in Django's settings.py file, in
particular for use in production. I'm not sure if these settings
should be ignored when running under other circumstances (./manage.py
test etc) in favour of the command line option, or if they should
still take effect. I quite liked Ivan's suggestion:
LOGGING = {
'django.db.sql': {
'handler': 'django.logging.FileLogger', # WatchedFileLogger
copy
'filename': '/tmp/django-sql.log',
'level': 'debug',
},
'django.core.cache': {
'handler': 'logging.handlers.HTTPHandler',
'host': '...',
'url': '',
'format': ''
},
'django.errors': {
'handler': 'logging.handlers.SMTPHandler',
...
}
}
django.errors is where we get to replace the current "e-mail 500
errors to an ADMIN" functionality.
There should also be an option for advanced users to ignore the
declarative syntax entirely and just provide a bunch of code that
configures logging in the traditional way:
import logging
django_logger = logging.getLogger('django')
django_logger.setLevel(...)
django_logger.addHandler(...) # etc
I'm not sure where that custom code should live - right there in
settings.py? At any rate, I think it's vital that people can use the
low level logging configuration API if they want to.
I'd rather not encourage people to use the logging module's support
for .ini style configuration, just because Django already has its own
configuration standards. That's not to say people can't call that from
their own code if they want to, but I don't think we should emphasise
that ability.
5. Document how users should go about implementing their own loggers.
I think we should tell people NOT to put them within the 'django'
logger
Re: Adding an option to re-test only failed tests
I'll see if I can talk Jeff into adding what he's got as a start to this. It looks solid to me. Ticket and patches forthcoming... On Sep 29, 2:47 pm, Simon Willison wrote: > On Sep 29, 7:34 pm, Rob Madole wrote: > > > TEST_RUNNER = 'django.contrib.test.nose.run_tests' > > > There might be some futzy bits to make that actually work, but I think > > it'd doable. > > I'd love to see this working. Obviously this would work just as well > implemented as an external project - but if it's as useful as it > sounds I'd personally be up for including it in core, if only to > promote the usage of nose amongst Django developers (and hence help > weaken the impression that Django doesn't integrate well enough with > the wider Python community). > > Cheers, > > Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Tue, Sep 29, 2009 at 8:56 PM, Rob Madole wrote: > > I'll see if I can talk Jeff into adding what he's got as a start to > this. It looks solid to me. > > Ticket and patches forthcoming... The nose test-runner that I'm currently using is at http://gist.github.com/197593. (The code I point to in the blog has gotten more convoluted than I'd like; it turns out that can be a problem if you're pointing at a vcs repo.) For the most part, switching test runners is simple since Django exposes all the setup and teardown functions. The hacky part is passing extra arguments to nose: Django doesn't recognize nose arguments so it complains and dies if you try something like `./manage.py test --pdb-fail`. Since I just wanted to get something working, I pass all arguments after a '--' on the command line to nose and Django happily ignores them. If something like nose_runner is shipped with Django it should run normally and be able to show the same help screen that `nose --help` shows. It was one of those "I'll come back and fix it later" sort of things. ;) Cheers, jeff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for 1.2: built-in logging with django.core.log
On Tue, Sep 29, 2009 at 6:29 PM, Simon Willison wrote: > [snip] > > By default, runserver would display no log messages at all. Runserver currently outputs each request to the commandline. Any reason that couldn't be handled by a logger? I seem to recall people complaining that they couldn't redirect that output elsewhere before when logging discussions came up. [snip] > > My philosophy here is that log messages should be ignored unless > explicitly told otherwise. Django gets run in various different ways - > runserver, test and in deployment under mod_wsgi/FastCGI/etc - and > it's not at all obvious what the default log output behaviour should > be. As long as we make it extremely easy to see log messages if we > want them I think having them off by default makes sense. It also > ensures no surprises for people upgrading from 1.1 to 1.2. Yeah, this was an issue we ran into. People will deploy your code in all sorts of situations and you can't make any assumptions about what would constitute an appropriate handler. They have to define those themselves. Even management commands can be called from scripts and may need output to be logged someplace other than sdterr. > 4. Add a way to configure loggers in Django's settings.py file, in > particular for use in production. I'm not sure if these settings > should be ignored when running under other circumstances (./manage.py > test etc) in favour of the command line option, or if they should > still take effect. Well, a logger can have multiple handlers. I would think we would want to leave the handlers defined in settings.py in tact and add the manage.py handler (most likely output to stderr - but could be different - see my comment above). -- \X/ /-\ `/ |_ /-\ |\| Waylan Limberg --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Wed, Sep 30, 2009 at 3:47 AM, Simon Willison wrote: > > On Sep 29, 7:34 pm, Rob Madole wrote: >> TEST_RUNNER = 'django.contrib.test.nose.run_tests' >> >> There might be some futzy bits to make that actually work, but I think >> it'd doable. > > I'd love to see this working. Obviously this would work just as well > implemented as an external project - but if it's as useful as it > sounds I'd personally be up for including it in core, if only to > promote the usage of nose amongst Django developers (and hence help > weaken the impression that Django doesn't integrate well enough with > the wider Python community). I'm yet to be convinced that Nose should be the default test runner for the simple reason that it doesn't come out of the box with Python. However, I agree that using Nose with Django should be as painless as possible. I included the TEST_RUNNER setting specifically for this reason. I haven't seen any reports in Trac that the capabilities of TEST_RUNNER aren't suitable for the task, and I've seen several snippets and blog posts indicating that Nose can be used with Django. If I'm incorrect on this, please let me know (or better still, open a ticket in Trac). Then there is the issue of whether Django should be responsible for shipping a Nose-compatible test runner, or whether that is Nose's responsibility. Historically, I've been of the opinion that it should be Nose's responsibility to ship a test runner, but I'm open to other opinions on this. I'm slightly hesitant to start building extra features like --failfast into Django's testing tools, especially when tools like Nose already cover this territory. Django isn't a test framework tool, and I'm hesitant to support proposals that try to turn it into one. Russ %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Adding an option to re-test only failed tests
On Tue, Sep 29, 2009 at 9:31 PM, Jeff Balogh wrote: > On Tue, Sep 29, 2009 at 8:56 PM, Rob Madole wrote: >> >> I'll see if I can talk Jeff into adding what he's got as a start to >> this. It looks solid to me. >> >> Ticket and patches forthcoming... > > The nose test-runner that I'm currently using is at > http://gist.github.com/197593. (The code I point to in the blog has > gotten more convoluted than I'd like; it turns out that can be a > problem if you're pointing at a vcs repo.) > > For the most part, switching test runners is simple since Django > exposes all the setup and teardown functions. The hacky part is > passing extra arguments to nose: Django doesn't recognize nose > arguments so it complains and dies if you try something like > `./manage.py test --pdb-fail`. Since I just wanted to get something > working, I pass all arguments after a '--' on the command line to nose > and Django happily ignores them. If something like nose_runner is > shipped with Django it should run normally and be able to show the > same help screen that `nose --help` shows. > > It was one of those "I'll come back and fix it later" sort of things. ;) I've updated the script at https://gist.github.com/197593/9763da971f67c2c3bd25b2a7e4754e8b5d625087 to expose nose options as test_runner.options. Mixing options for the two packages is kind of hairy, but I don't think any of that is exposed to developers running the tests. Integrating outside options will require a patch to core/management/commands/test.py to import a test runner and ask it if it has extra options. Importing the test runner before creating the Command class should be backwards compatible unless people are writing test runners that import test.py (there'd be a circular import issue). It looks like this: http://gist.github.com/197797. I'll get a proper bug filed if this looks acceptable/interesting. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
