On 1 heinä, 00:11, Luke Plant <l.plant...@cantab.net> wrote:
> On 30/06/12 20:23, Anssi Kääriäinen wrote:
>
> > TL;DR ;) But I intend to do so shortly.
>
> > For now I have to comment on the "clear up sql/query.py" part. I am
> > doubtful if moving to SQLAlchemy will solve the sql/query.py and sql/
> > compiler.py complexities.
>
> > I have little knowledge of SQLAlchemy, but my understanding is that it
> > does not do things like "when filtering on negated multijoin
> > condition, push the condition to subquery", automatic grouping by the
> > right set of columns or handle what kind of joins are needed by select
> > related. It probably doesn't do join promotion ("should this join be
> > INNER or LEFT OUTER join, if dealing with an ORed where condition,
> > then LEFT OUTER, and so on). The sql/query.py deals mostly with these
> > kinds of problems. The sql/compiler.py also deals in part with
> > problems like this.
>
> Mike Bayer pointed me to this code, which does something like Django's
> join syntax on top of SQLAlchemy:
>
> https://github.com/mitsuhiko/sqlalchemy-django-query/blob/master/sqla...
>
> I really don't know enough to know how well it is approximating what
> Django does. It would be surprising if it was really doing the same thing!
>
> I think it's one of those things where we really won't know the impact
> until we've got most of the way there, and even then differences of
> approach could make all the difference.The SQLA ORM doesn't automatically generate subqueries or outer joins. The sqlalchemy-django-query uses only .join() so on that basis I am pretty sure it doesn't do what Django does. The complexity of Django's ORM is in large part in determining the type of joins, handling nullable fields correctly, doing automatically subqueries where needed and deciding what parts of your filter clause need to go into HAVING, what parts into WHERE clause. This is also the biggest strength of Django's ORM: the ORM generates the SQL needed, you don't have to care if subqueries or left joins are needed or not. You don't even have to know what a HAVING clause is. I have now read the full proposal, and I have a feeling there is an expectation that SQLA can do more for us that it in reality can. I don't see anything in SQLA that is capable of replacing sql/query.py. There isn't anything that is capable of replacing all of sql/ compiler.py. The current query generation can be separated to roughly three parts: 1. Doing operations on the query (sql/query.py). 2. Doing pre-execution stuff. (sql/compiler.py) 3. Generating the actual SQL. (sql/compiler.py, backends). In step 1 Django constantly keeps the correct join information and filtering conditions available. These are ready to be converted into SQL on the spot. In addition, some information is kept available, but it isn't processed yet. For example: select_related, order_by etc. These are just lists of what is wanted, but these are not turned into joins yet. It is notable that the join information can be changed on the fly: some operation might turn some existing inner joins to outer joins for example. Here the main part of complexity is in deciding what kind of joins to use, or if a subquery is needed. In step 2 things like needed GROUP BY clauses is calculated. In particular, the order_by, select_related and only()/defer() structures are calculated. Here the most complex part is calculating which joins to use for select_related descent. In addition there are some details and dependencies which make this somewhat complex (select cols must be added to group by - but some backends do not allow using select aliases in group by, and some backends prefer doing group by only by table pk). Finally, in step 3 the SQL is generated. This is dirty as different backends have endless amount of little differences in their take of what is legal SQL. In practice, steps 2 and 3 are mostly interleaved currently. It might be a good idea to separate them as much as possible. I don't see SQLA helping much in step 1 - it just doesn't do what Django does. It might help somewhat in step 2. It would definitely help in step 3. My understanding is that SQLA and Django's ORM work on different levels. I do support trying to use SQLA. This is just a warning that one should not expect to replace everything or even most of the code under django/db/models/sql and django/db/backends by SQLA. - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
