Can we please change this so it defaults to off, and just document how to turn it on and in what situations you should turn it on?
In my opinion this default-on feature caters to a very specific audience, and will cause a lot of unexpected behavior with other users. Here is the tl;dr of an argument for turning it on: Connections are expensive, we should persist them. Here is the tl;dr for turning it off: Connections are expensive, so we dont have a lot of them. Immediately for anyone who has configured more workers than they have Postgres connections (which I can only imagine is common among people who havent setup infrastructure like pgbouncer) things will start blowing up. Why should this be the default behavior? Unlike MySQL, which already has cheap connections and doesn't suffer this problem, connections in Postgres add quite a large cost, both on creation, on allowing them. **Everyone** who cares about their performance on Postgres should already be using pgbouncer (or some alternative) to offset **both** of these, not just a single problem which is what Django is addressing here. I will happily defend my opinion in person at PyCon in two weeks if it still remains a default, and anyone is willing to listen, and if you reach out to the other large (and I dont mean scale) Django users I imagine you will find a lot of mixed feelings about this default. On Sunday, February 17, 2013 3:24:52 AM UTC-8, Aymeric Augustin wrote: > > **tl;dr** I believe that persistent database connections are a good idea. > Please prove me wrong :) > > -------------------- > > Since I didn't know why the idea of adding a connection pooler to Django > was > rejected, I did some research before replying to the cx_Oracle > SessionPooling > thread. > > The best explanation I've found is from Russell: > > > To clarify -- we've historically been opposed to adding connection > > pooling to Django is for the same reason that we don't include a web > > server in Django -- the capability already exists in third party > > tools, and they're in a position to do a much better job at it than us > > because it's their sole focus. Django doesn't have to be the whole > > stack. > > All the discussions boil down to this argument, and the only ticket on the > topic is short on details: https://code.djangoproject.com/ticket/11798 > > -------------------- > > The connection pools for Django I've looked at replace "open a connection" > by > "take a connection from the pool" and "close a connection" by "return the > connection to the pool". This isn't "real" connection pooling: each worker > holds a connection for the entire duration of each request, regardless of > whether it has an open transaction or not. > > This requires as many connection as workers, and thus is essentially > equivalent to persistent database connections, except connections can be > rotated among workers. > > Persistent connections would eliminate the overhead of creating a > connection > (IIRC ~50ms/req), which is the most annoying symptom, without incurring > the > complexity of a "real" pooler. > > They would be a win for small and medium websites that don't manage their > database transactions manually and where the complexity of maintaining an > external connection pooler isn't justified. > > Besides, when Django's transaction middelware is enabled, each request is > wrapped in a single transaction, which reserves a connection. In this > case, a > connection pooler won't perform better than persistent connections. > > Obviously, large websites should use an external pooler to multiplex their > hundreds of connections from workers into tens of connections to their > database and manage their transactions manually. I don't believe > persistent > connections to the pooler would hurt in this scenario, but if it does, it > could be optional. > > -------------------- > > AFAICT there are three things to take care of before reusing a connection: > > 1) restore a pristine transaction state: transaction.rollback() should do; > > 2) reset all connection settings: the foundation was laid in #19274; > > 3) check if the connection is still alive, and re-open it otherwise: > - for psycopg2: "SELECT 1"; > - for MySQL and Oracle: connection.ping(). > > Some have argued that persistent connections tie the lifetime of databases > connections to the lifetime of workers, but it's easy to store the > creation > timestamp and re-open the connection if it exceeds a given max-age. > > So -- did I miss something? > > -- > Aymeric. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
