Re: 1.0.1 release blockers?
On Sun, Nov 2, 2008 at 5:54 AM, Tai Lee <[EMAIL PROTECTED]> wrote: > There's still no comment in the ticket from a core developer or > anybody else, though. As this is a simple bug fix, can I (as the > reporter) mark this ready for checkin, as there is a patch with tests? You do know that Karen has a commit bit, right? If she's got concerns about a ticket, then I at least take that as meaning it's got problems and isn't ready for checkin. -- "Bureaucrat Conrad, you are technically correct -- the best kind of correct." --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Mysql Ping and performance
Oh, I forgot to mention that this is certainly a Django issue and not
a mysqldb issue. The relevant code is in django/db/backends/mysql/
base.py:232:
def _valid_connection(self):
if self.connection is not None:
try:
self.connection.ping()
return True
except DatabaseError:
self.connection.close()
self.connection = None
return False
I replaced that in my local version of Django with :
def _valid_connection(self):
return self.connection is not None
-Jesse
On Nov 2, 1:33 am, Jesse Young <[EMAIL PROTECTED]> wrote:
> Yeah, we found the ping() during performance profiling too. It's a
> significant performance hit -- it basically doubles the number of
> requests to the DB server. It seems that the reason is to support use
> cases where the DB connection is long-lived, since sometimes DB
> connections die if you leave them open for a really long time. In the
> normal Django HTTP request use case this never happens because Django
> closes the DB connection at the end of each request. But it could
> potentially be useful for people who use django.db in some other
> custom process.
>
> We ended up commenting out the ping() stuff so that any opened
> connection was always treated as valid.
>
> It seems to me that the performance overhead in the 99% use case would
> suggest that it would be beneficial for Django to let users configure
> whether or not their DB connection constantly pings or not. Or maybe
> just keep track of the time and don't ping unless some minimum time
> has elapsed since the last one. Or do what we did, and just don't ping
> at all and let the app deal with the case where the DB drops the
> connection.
>
> -Jesse
>
> On Oct 31, 1:34 pm, bo <[EMAIL PROTECTED]> wrote:
>
> > Not sure if this is a Django issue or the supporting Mysqldb (1.2.2)
> > python package .. (i'll stop this here if its not, but it seems worthy
> > of at least letting other know this)
>
> > however while profiling a page i came across this seemingly
> > performance hole.
>
> >
> > Ordered by: internal time
>
> > ncalls tottime percall cumtime percall
> > filename:lineno(function)
> > 230 0.343 0.001 0.343 0.001 {method 'query' of
> > '_mysql.connection' objects}
> > 228 0.116 0.001 0.116 0.001 {method 'ping' of
> > '_mysql.connection' objects}
> > 234 0.029 0.000 0.047 0.000 query.py:
> > 473(get_default_columns)
> > 972 0.021 0.000 0.048 0.000 __init__.py:
> > 487(__init__)
> > 1303 0.019 0.000 0.022 0.000 __init__.py:
> > 633(__init__)
> > 2068 0.017 0.000 0.216 0.000 __init__.py:
> > 690(_resolve_lookup)
> > ---
>
> > #1 time sink is the queries themselves (i figured that would be the
> > case) .. but #2 is "ping" and it seems to ping on every query. This
> > issue is probably not so bad on Localhost or Socket based connections,
> > but on remote Mysql server, as you can see, it is not so good.
>
> > again not sure if django can solve (or even wants to solve this) ..
>
> > bo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Mysql Ping and performance
Yeah, we found the ping() during performance profiling too. It's a
significant performance hit -- it basically doubles the number of
requests to the DB server. It seems that the reason is to support use
cases where the DB connection is long-lived, since sometimes DB
connections die if you leave them open for a really long time. In the
normal Django HTTP request use case this never happens because Django
closes the DB connection at the end of each request. But it could
potentially be useful for people who use django.db in some other
custom process.
We ended up commenting out the ping() stuff so that any opened
connection was always treated as valid.
It seems to me that the performance overhead in the 99% use case would
suggest that it would be beneficial for Django to let users configure
whether or not their DB connection constantly pings or not. Or maybe
just keep track of the time and don't ping unless some minimum time
has elapsed since the last one. Or do what we did, and just don't ping
at all and let the app deal with the case where the DB drops the
connection.
-Jesse
On Oct 31, 1:34 pm, bo <[EMAIL PROTECTED]> wrote:
> Not sure if this is a Django issue or the supporting Mysqldb (1.2.2)
> python package .. (i'll stop this here if its not, but it seems worthy
> of at least letting other know this)
>
> however while profiling a page i came across this seemingly
> performance hole.
>
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall
> filename:lineno(function)
> 230 0.343 0.001 0.343 0.001 {method 'query' of
> '_mysql.connection' objects}
> 228 0.116 0.001 0.116 0.001 {method 'ping' of
> '_mysql.connection' objects}
> 234 0.029 0.000 0.047 0.000 query.py:
> 473(get_default_columns)
> 972 0.021 0.000 0.048 0.000 __init__.py:
> 487(__init__)
> 1303 0.019 0.000 0.022 0.000 __init__.py:
> 633(__init__)
> 2068 0.017 0.000 0.216 0.000 __init__.py:
> 690(_resolve_lookup)
> ---
>
> #1 time sink is the queries themselves (i figured that would be the
> case) .. but #2 is "ping" and it seems to ping on every query. This
> issue is probably not so bad on Localhost or Socket based connections,
> but on remote Mysql server, as you can see, it is not so good.
>
> again not sure if django can solve (or even wants to solve this) ..
>
> bo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: 1.0.1 release blockers?
Thanks for your feedback, Karen.
> http://code.djangoproject.com/ticket/8898
>
> I suspect this one may be stalled because it is in design decision needed
> state with a last comment that makes it sound like the existing patch (added
> before the last comment) is not correct and that the right fix will require
> changes that are potentially backwards-incompatible ("users should be guided
> to use X if they want to use Y"...how is that different from what users are
> guided to do today and is there something they can do today that won't be
> allowed after this fix is made?)
>
> The bug sounds like it should be fixed, but that last comment makes it sound
> like a proper fix is not yet available. Therefore not something that can
> easily be simply committed but rather will require someone to spend some
> time researching the history of this type of field to make sure the right
> fix is developed. This rather conflicts with what you say above about this
> being a simple fix, so please clarify in the ticket if that last comment and
> move to design decision needed was not meant to raise a red flag that the
> initially posted fix was not all that was necessary to fix this problem.
I've put this ticket back to "accepted" as per your comment that the
bug should be fixed. As you say, the DDN and confused comments were
not intended to raise a red flag, and should have been taken to a
separate ticket. The suggested change would have been backwards
incompatible, anyway. IMHO, the bug should be fixed as it is,
regardless of any possible future changes.
There's still no comment in the ticket from a core developer or
anybody else, though. As this is a simple bug fix, can I (as the
reporter) mark this ready for checkin, as there is a patch with tests?
> >http://code.djangoproject.com/ticket/9482
>
> This one is only two days old. Give it time. But there's an easy
> workaround (set the environment variable yourself before calling whatever
> script you are runnig), this is not something covered by the test suite so
> can't be tested to ensure it really doesn't break anything ("I can't see how
> this could possibly break anything" are some famous last words, and believe
> me I've said them myself), and it seems like a bit of an edge case (I
> haven't heard a lot of people needing to put code in a project's __init__.py
> file) all of which argue to me that it might not be a good candidate for
> 1.0.1. But that's just me, and it is only two days old.
Famous last words, indeed. Can you see any way that setting
DJANGO_SETTINGS_MODULE *before* importing the project module could
break existing code? Wouldn't that mean that manually setting
DJANGO_SETTINGS_MODULE before calling any management commands would
also be broken? :)
As you say, this one is not so easy to test that it won't break
anything else (at least, I don't know how). Although it is easy to
reproduce.
Cheers.
Tai.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Mysql Ping and performance
Would it be wise to suggest that it be removed from the django core
given the connection drops every request? It's not done for Postgres
at all and the Oracle base.py is doing what you suggest.
I guess this question would be posed to whomever controls that slice
of the django pie.
bo
On Nov 2, 1:39 am, Jesse Young <[EMAIL PROTECTED]> wrote:
> Oh, I forgot to mention that this is certainly a Django issue and not
> a mysqldb issue. The relevant code is in django/db/backends/mysql/
> base.py:232:
>
> def _valid_connection(self):
> if self.connection is not None:
> try:
> self.connection.ping()
> return True
> except DatabaseError:
> self.connection.close()
> self.connection = None
> return False
>
> I replaced that in my local version of Django with :
>
> def _valid_connection(self):
> return self.connection is not None
>
> -Jesse
>
> On Nov 2, 1:33 am, Jesse Young <[EMAIL PROTECTED]> wrote:
>
> > Yeah, we found the ping() during performance profiling too. It's a
> > significant performance hit -- it basically doubles the number of
> > requests to the DB server. It seems that the reason is to support use
> > cases where the DB connection is long-lived, since sometimes DB
> > connections die if you leave them open for a really long time. In the
> > normal Django HTTP request use case this never happens because Django
> > closes the DB connection at the end of each request. But it could
> > potentially be useful for people who use django.db in some other
> > custom process.
>
> > We ended up commenting out the ping() stuff so that any opened
> > connection was always treated as valid.
>
> > It seems to me that the performance overhead in the 99% use case would
> > suggest that it would be beneficial for Django to let users configure
> > whether or not their DB connection constantly pings or not. Or maybe
> > just keep track of the time and don't ping unless some minimum time
> > has elapsed since the last one. Or do what we did, and just don't ping
> > at all and let the app deal with the case where the DB drops the
> > connection.
>
> > -Jesse
>
> > On Oct 31, 1:34 pm, bo <[EMAIL PROTECTED]> wrote:
>
> > > Not sure if this is a Django issue or the supporting Mysqldb (1.2.2)
> > > python package .. (i'll stop this here if its not, but it seems worthy
> > > of at least letting other know this)
>
> > > however while profiling a page i came across this seemingly
> > > performance hole.
>
> > >
> > > Ordered by: internal time
>
> > > ncalls tottime percall cumtime percall
> > > filename:lineno(function)
> > > 230 0.343 0.001 0.343 0.001 {method 'query' of
> > > '_mysql.connection' objects}
> > > 228 0.116 0.001 0.116 0.001 {method 'ping' of
> > > '_mysql.connection' objects}
> > > 234 0.029 0.000 0.047 0.000 query.py:
> > > 473(get_default_columns)
> > > 972 0.021 0.000 0.048 0.000 __init__.py:
> > > 487(__init__)
> > > 1303 0.019 0.000 0.022 0.000 __init__.py:
> > > 633(__init__)
> > > 2068 0.017 0.000 0.216 0.000 __init__.py:
> > > 690(_resolve_lookup)
> > > ---
>
> > > #1 time sink is the queries themselves (i figured that would be the
> > > case) .. but #2 is "ping" and it seems to ping on every query. This
> > > issue is probably not so bad on Localhost or Socket based connections,
> > > but on remote Mysql server, as you can see, it is not so good.
>
> > > again not sure if django can solve (or even wants to solve this) ..
>
> > > bo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal: remove auth context processor dependancy for Admin
Yes, that may be true. But why does the 'default' behavior impose these dependencies when they are not required? As that context processor has this other 'side effect' of making an entire site "Vary" under a cookie, hit the session and hit the DB. and one would need to override every other function that calls "template.RequestContext' which is most of the meat of sites.py. bo On Nov 1, 1:23 pm, "Collin Grady" <[EMAIL PROTECTED]> wrote: > On Sat, Nov 1, 2008 at 9:45 AM, bo <[EMAIL PROTECTED]> wrote: > > One aid in this area would be to remove the dependancy of > > context_processor.auth in the Admin world (which most certainly needs > > the user, messages and perms in the context) > > You can already do this - simply make your own subclass of AdminSite > and override check_dependencies > > -- > Collin Grady --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Cache and GET parameters
I really like the idea of the explicit GET params passed.So I'm +1 especially on solution #3. I actually had never realized it wasn't caching pages with GET params, luckily though, any pages where I use this decorator don't fluctuate like that :) On Nov 1, 7:51 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On Sat, Nov 1, 2008 at 8:32 PM, SmileyChris <[EMAIL PROTECTED]> wrote: > > > On Nov 2, 2:52 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > >> Assuming vary_on_get() with no parameters means no variance (other > >> than the HTTP Vary headers), then [...] > > > That seems confusing - the decorator name seems to imply that it would > > vary on any get attribute (even though this is the default) - at least > > that's how I'd look at it if I didn't know otherwise. > > @vary_on_get(None) ? :-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Cache and GET parameters
On Nov 1, 2:52 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > > To be clear, the generated cache key would still include anything > stated in the HTTP Vary heads, right? > > Vary: Cookie combined with @vary_on_get() should still vary on Cookie. > Yes. > > I say cache nothing; doing otherwise is backwards-incompatible. I > realize that means a bunch of decorators on views if you want the > cache-everything behavior. > Maybe, it's just me, but I find the current behaviour confusing after reading the introduction to the cache documentation [1]. It says "Given a URL..." so I expected the cache to use everyting that identifies an object in the URL (path and query as described in RFC 3986 [2]). But, it is backwards-incompatible so maybe your suggestion is the right way to go. [1]: http://docs.djangoproject.com/en/dev/topics/cache/ [2]: http://labs.apache.org/webarch/uri/rfc/rfc3986.html#components I attached a patch to ticket #4992 for the behaviour I (and apparently other people) expected: http://code.djangoproject.com/attachment/ticket/4992/cache_by_request_full_path.diff Regards, Peter --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: 1.0.1 release blockers?
On Nov 2, 10:57 pm, "James Bennett" <[EMAIL PROTECTED]> wrote: > > You do know that Karen has a commit bit, right? If she's got concerns > about a ticket, then I at least take that as meaning it's got problems > and isn't ready for checkin. Yes, I did notice that. However, as I said, there is still no activity on the ticket. I've tried to clarify that the DDN was not intended to prevent a fix being committed. Karen has indicated that it sounds like the bug should be fixed, however without any review of the actual patch (just the confused comments) I've left the ticket as "accepted", even though it has a working patch with tests. I guess my big question is, if a ticket has a patch with working tests and/or documentation as appropriate, and is a simple bug fix only, can the reporter mark it as "ready for checkin"? Then if a core developer has a problem with the implementation of the fix, it can be marked patch needs improvement and assigned back to the reporter. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: 1.0.1 release blockers?
On Sun, Nov 2, 2008 at 6:54 AM, Tai Lee <[EMAIL PROTECTED]> wrote: > I've put this ticket back to "accepted" as per your comment that the > bug should be fixed. As you say, the DDN and confused comments were > not intended to raise a red flag, and should have been taken to a > separate ticket. The suggested change would have been backwards > incompatible, anyway. IMHO, the bug should be fixed as it is, > regardless of any possible future changes. > > There's still no comment in the ticket from a core developer or > anybody else, though. As this is a simple bug fix, can I (as the > reporter) mark this ready for checkin, as there is a patch with tests? > No, please don't -- it'll get looked at as is. Marking ready for checkin is not a required stage to go through to get something in. It's a flag that indicates someone other than the submitter has looked everything over and agrees with the patch. It's a helpful but not necessary prerequisite to moving things along. Having the submitter set it is useless -- barring comments to the contrary it's assumed that when a patch is attached the submitter thinks it is ready for checkin. ... > Famous last words, indeed. Can you see any way that setting > DJANGO_SETTINGS_MODULE *before* importing the project module could > break existing code? Wouldn't that mean that manually setting > DJANGO_SETTINGS_MODULE before calling any management commands would > also be broken? :) > > No, I can't see how it could break anything. But I've proved to be sufficiently lacking in imagination in answering such questions enough times over the years that I've learned not to be overly impressed with that Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
