On Sun, 2007-10-28 at 15:58 -0700, Stavros wrote:
> No, I don't want to use a paginator. I don't even display the results,
> I just process them. I was weighing in in favor of turning iterator()
> into an actual iterator that fetches a few results at a time, as per
> the discussion at hand.
First
No, I don't want to use a paginator. I don't even display the results,
I just process them. I was weighing in in favor of turning iterator()
into an actual iterator that fetches a few results at a time, as per
the discussion at hand.
--~--~-~--~~~---~--~~
You rece
On 10/28/07, Stavros <[EMAIL PROTECTED]> wrote:
> I'm developing an app that takes about 3m rows from the DB [...]
You probably want to use a paginator:
http://www.djangoproject.com/documentation/models/pagination/
In the future, please direct questions of this nature to django-users;
django-dev
I'm developing an app that takes about 3m rows from the DB, and
currently everything is loaded in memory. This is unacceptable, since
the script rapidly uses up all the memory available in the machine and
stops responding. It would be good if django could return an actual
iterator, so we could wor
Ivan Sagalaev napsal(a):
> Malcolm Tredinnick wrote:
>> However, I think all our Python db wrappers pull things to the client
>> immediately
>
> BTW, a couple of months ago we were investigating behavior of mysqldb in
> this regard and found out that doing fetchall and fetchmany indeed eats
> t
Malcolm Tredinnick wrote:
> However, I think all our Python db wrappers pull things to the client
> immediately
BTW, a couple of months ago we were investigating behavior of mysqldb in
this regard and found out that doing fetchall and fetchmany indeed eats
the whole data into memory. Even with
On Mon, 2007-10-15 at 17:32 -0500, Jeremy Dunck wrote:
> On 10/15/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> > Barring any good reason, I want to change _get_data() back to being an
> > iterator. A whole bunch of things become easier if we do that.
> >
> > But nothing happens without a re
On 10/15/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> Barring any good reason, I want to change _get_data() back to being an
> iterator. A whole bunch of things become easier if we do that.
>
> But nothing happens without a reason, so if the keepers of the
> institutional memory could enlig
Question for the Django Ancient Ones..
In current trunk's QuerySet.__iter__ method we do:
return iter(self._get_data())
and _get_data() is basically (module some caching checks):
return list(self.iterator())
Now, iterator() really is an iterator. So the effect of all t
