Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-07-17 Thread Tom Forbes
Indeed, I had an attempt at doing this here ( https://github.com/django/django/pull/8928/), but it seems a hard problem. I think there is huge potential here but I have no time to work on this anymore. On Tue, 17 Jul 2018, 19:00 Ramiro Morales, wrote: > On Sat, Jun 30, 2018 at 12:40 PM Tom Forbe

Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-07-17 Thread Ramiro Morales
On Sat, Jun 30, 2018 at 12:40 PM Tom Forbes wrote: > Are you sure it is the prefetches that is causing this? As Adam pointed > out these are correctly ignored. Annotations however are not, which can > cause unnecessary work and longer execution times. i.e: > > Book.objects.annotate(something=Max(

Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-07-01 Thread Jakub Kleň
I checked MySQL 8 today, and it's the same story. Does anyone know how this performs in MariaDB or Postgresql? On Friday, June 29, 2018 at 11:00:33 PM UTC+2, Jakub Kleň wrote: > > Hi guys, I came across a problem today while testing my webapp with a full > migrations applied from my legacy site

Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-06-30 Thread Jakub Kleň
Hi, thank you for your response. I'm currently running MySQL 5.7.17, and the time difference between the queries is really big (0.006s vs 16.6s). I'm also wondering that something like this isn't handled by MySQL. On Saturday, June 30, 2018 at 1:50:48 PM UTC+2, Adam Johnson wrote: > > it takes r

Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-06-30 Thread Jakub Kleň
After some more investigation yesterday, I found out that it isn't the prefetches and the subquery, and exactly as you write, it's annotations which generate joins, and for count() the difference is it has to go thru the whole db table, and that is slowing it down a lot. For my queryset, the no

Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-06-30 Thread Tom Forbes
Are you sure it is the prefetches that is causing this? As Adam pointed out these are correctly ignored. Annotations however are not, which can cause unnecessary work and longer execution times. i.e: Book.objects.annotate(something=Max('pages__word_count')).count() We have enough information to b

Re: Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-06-30 Thread Adam Johnson
> > it takes really long to process by MySQL. > Which version? Subquery performance gets better in MySQL 5.6 and even more so on MariaDB. for count the query would be really simple, and wouldn't need the > prefetches. > When counting, prefetches aren't actaully executed. Check the actual SQL wi

Optionally using a custom queryset for Paginator and MultipleObjectMixin (ListView)

2018-06-29 Thread Jakub Kleň
Hi guys, I came across a problem today while testing my webapp with a full migrations applied from my legacy site for the first time. The problem is I'm using Prefetch in my view query (get_queryset), and it works fine when I just fetch it, but the problem comes with the .count() query which gen