I was just alerted to this by the reopening of ticket #18135. I agree the routing on AWS is slow and represents a large amount of connections. This is something that AWS needs to address.
I will argue, however, that the AWS use case is a reason we need this to be an OPTION, not automatic. Generally, in my experience, databases are the first place that require investigations as you to start seeing slow downs as your application grows with users. Relational databases are expensive to scale. They generally require DBMs, massive servers, tons of memory and that is when everything goes right. So we do not immediately jump to upgrading the database, that would be just wrong. Instead what we do is to look to places to cache aggressively. I think that in the future we are going to see more and more places flattening databases and moving to NoSQL, generally in views, but occasionally directly on the pages with javascript. This forces less and less reliance on the relational database. So the future of web frameworks will have less and less reliance on a constrained database. This is a really really good thing because AWS has given us the ability to only purchase the hardware that we need when we need it. That means that if one of my sites get a ton of users right now, my application will scale the number of EC2 instances. I have confidence that the number of queries directly to my database will not increase directly with that load because each query is aggressively cached. My EC2 scaling is directly in proportion to the number of users my application has. RDS, on the other hand, scales exponentially with the number of users that I have [1]. This is consistant with any large application. The database just costs more. So here is my point, requiring that an application has a consistant connection to the database is really good for small applications, but as the number of front facing web servers increase, the number of database connections also increase, consuming the most expensive resource of the web application, the database. Leaving open, "sleeping", I would argue "stale", connections doesn't scale cost linearly with usership, instead it increases exponentially. This should be an option. For the small percentage of people who are noticing the SQL connection time issues (which I would recommend they reach out to a AWS support person about), a persistant connection makes sense. The general Django community should not be exhausting their most expensive resource first. [1] http://aws.amazon.com/rds/#pricing-tabs On Mon, Feb 18, 2013 at 1:02 PM, Carl Meyer <c...@oddbird.net> wrote: > On 02/18/2013 06:47 AM, Jacob Kaplan-Moss wrote: > > On Mon, Feb 18, 2013 at 7:41 AM, Aymeric Augustin > > <aymeric.augus...@polytechnique.org> wrote: > >> The goal of my proposal is to save the cost (in response time) of > establishing > >> the database connection, in vanilla Django, without compromising > reliability. > >> I'm trying to help people who aren't even aware of the problem :) > >> > >> The implementation is quite straightforward: > >> https://github.com/django/django/pull/733 > >> > >> Everything is controlled by a per-connection max-age setting. It can be > set > >> to a low value on rarely used connections, so they get closed quickly, > and > >> are re-opened only when necessary. > > > > +1 from me. > > > > The overhead of establishing a connection isn't a big deal when you > > control your own hardware and can route traffic to the database over a > > gigE backplane, but many of our users don't have that luxury. Routing > > on AWS seems to be particularly crappy: most of my AWS-hosted sites > > show a 75-100ms overhead just to establish a connection. That can be > > as much as 90% of my response time! This fix represents a cheap and > > easy performance improvement. Let's get it in. > > +1 from me too. Similar experience on Heroku (which is of course also > AWS), persistent DB connections via django-postgrespool were a quick and > easy win to save a significant chunk of response time. I don't see any > reason Django shouldn't do it by default, since many users won't even > realize how much their response time is being hurt by the lack of it. > > Carl > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers?hl=en > . > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
