Re: Automatic prefetching in querysets

My existing code for this is now available as a pypi package https://github.com/tolomea/django-auto-prefetch -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receivin

Automatic prefetching in querysets

I'd like to discuss automatic prefetching in querysets. Specifically automatically doing prefetch_related where needed without the user having to request it. For context consider these three snippets using the Question & Choice models from the tutorial

Re: Automatic prefetching in querysets

u know of any other ORMs which attempt similar magical optimisations? > How do they go about identifying the cases where it is necessary? > > On 15 August 2017 at 10:44, Gordon Wrigley > wrote: > >> I'd like to discuss automatic prefetching in querysets. Specifically >

Re: Automatic prefetching in querysets

, Aug 15, 2017 at 8:03 PM, Collin Anderson wrote: > Hi Gordon, > > How is it implemented? Does each object keep a reference to the queryset > it came from? > > Collin > > On Tue, Aug 15, 2017 at 2:44 PM, Gordon Wrigley > wrote: > >> Sorry maybe I wasn&#x

Re: Automatic prefetching in querysets

tions? I don't, but unlike Django where I have years of experience I have next to no experience with other ORM's. Regards G On Tue, Aug 15, 2017 at 7:44 PM, Gordon Wrigley wrote: > Sorry maybe I wasn't clear enough about the proposed mechanism. > > Currentl

Re: Automatic prefetching in querysets

ion would be for Django to detect these O(n) query > cases and display intelligent warnings, with suggestions as to the correct > select/prefetch calls to add. When debug mode is enabled we could detect > repeated foreign key referencing from the same source. > > On 15 Aug 2017 19:44, &qu

Re: Automatic prefetching in querysets

ons like these you simply cannot know in >>>> advance whether doing extra queries is going to a be an optimization or a >>>> pessimization. If I can come up with a single example where it would >>>> significantly decrease performance (either memory usage or speed

Re: Automatic prefetching in querysets

Regarding 2, it does work for reverse one-to-one relations. On Wed, Aug 16, 2017 at 9:17 PM, Aymeric Augustin < aymeric.augus...@polytechnique.org> wrote: > On 15 Aug 2017, at 11:44, Gordon Wrigley wrote: > > I'd like to discuss automatic prefetching in querysets. Specific

Re: Automatic prefetching in querysets

I'm not advocating either way on this, but it's probably worth noting that the context manager proposal is compatible with a queryset level optin/out and an object level optout. So you could for example have prefetch_related(auto=None) which can be explicitly set to True / False and takes it's val

Re: Automatic prefetching in querysets

This received some positive responses, so to help move the conversation along I have created a ticket and pull request. https://code.djangoproject.com/ticket/28586 https://github.com/django/django/pull/9064 Regards G On Tue, Aug 15, 2017 at 10:44 AM, Gordon Wrigley wrote: > I'd

Re: Sealing or locking QuerySets -- a suggestion to prevent surprise N+1 queries.

Regarding auto prefetch, after the discussion here I created a ticket https://code.djangoproject.com/ticket/28586 It didn't get much attention presumably because it was around Django 2 release time. I have a todo note to add tests and doco and generally get it to a mergable state in order to pus