On Jan 28, 4:05 am, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
> I'm not completely ruling out the fact that caching might also be a
> valid strategy here, but I'd be interested in looking at more
> fundamental optimzations as well.
There are two basic reasons why query execution is sometimes slow:
- As said, cloning querysets can be slow. The big reason here is
usage of deepcopy. Deepcopy itself is somewhat slow, and even worse,
the deepcopy goes much deeper than needed. That is, stuff like
Model._meta gets copied when you clone a queryset. There is a ticket
about this: #16759. It would be interesting to hear what kind of
speedup the original poster can get from this patch alone.
- In addition, Django does use too much time in actual SQL creation.
It would be interesting to profile this and check where exactly the
time is used. It should not take much time to generate SQL from
preprocessed datastructures (that is, from queryset). I think the main
reason is using a lot of isinstance/hasattr checks. If that is the
case, using classes might help here.
There is still one reason which might cause some slowness: when using
Q-objects and/or exclude, Django does generate non-optimal query
trees. Something like .exclude(pk=1) can be turned into the following
boolean tree:
AND
AND
NOT AND
AND pk = 1
Technically correct, but not optimal. There are tickets about this,
too: #17025, #17000.
If I understood the proposal correctly, the caching is done at very
late stage, perhaps even after turning the query into SQL? That seems
a little problematic, although will naturally give you a lot of speed
improvement.
I would suggest checking #16759. I think it is in pretty good state,
and should give you a lot of speed improvement. Ticket #17025 tries to
rewrite a lot of where node handling. The patch in 17025 doesn't apply
any more, and maybe tries to be just a little bit too cute. Ticket
17000 has perhaps more pragmatic approach to creating nicer where
trees.
- Anssi
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
