2014-11-23 13:27 GMT+01:00 Shai Berger <s...@platonix.com>: > Hi Rick, > > On Sunday 23 November 2014 14:11:13 Rick van Hattem wrote: > > > > So please, can anyone give a good argument as to why any sane person > would > > have a problem with a huge default limit which will kill the performance > of > > your site anyhow but isn't enough to kill the entire system? > > > ...because Django is also used to implement batch processes, and you'd be > very > surprised if your system, which works well enough for a year or so, > suddenly > has its night-jobs starting to fail one day because some size-limit has > been > reached. > > The Django project has traditionally held the position, with regard to such > proposals, that we should avoid making such decisions for our users, but we > should allow them to make them by themselves; this can also lead to more > general solutions. > > In this case, I think you could get support for allowing yourself query- > validation hooks -- say, allow installing a function that will be called by > the ORM when query results are received, before turning them into objects. > I > think this lets you solve your problem outside of core (and then, if your > solution proves useful and popular, it may go into core as well). This > could > also allow other interesting functionalities, I suspect, if it is allowed > to > modify (and not just validate) the results. > that function wil likeky be called much too late.
The client-side postgresql cursor fetches *all query results* in one go when reading from the cursor the first time. So again, the issue with "big" resultsets is not the ORM generating django objects but the very way django uses psycopg. Why don't we just write a Model.manager.chunked() like proposed here? https://groups.google.com/forum/#!searchin/django-developers/psycopg$20results/django-developers/8ZBA4uMinXo/M1osYTGmvbUJ > I think (speaking only for myself, of course) that a proposal along these > lines has a better chance of gaining traction. > > HTH, > Shai. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAEbA68kT-Yhq2J3ksuH7-6b13BxZQbnab71pcRfyc662KDUY1A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
