Re: Sitemaps Memory Usage

2008-09-28 Thread David Cramer
Also in regards to this, using a template is overkill in my opinion. Especially if it's going to cause extra headaches. I'll try out your recommendation for now, thanks Malcolm. On Sep 28, 11:40 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Sun, 2008-09-28 at 21:21 -0700, David Cramer w

Re: Sitemaps Memory Usage

2008-09-28 Thread Malcolm Tredinnick
On Sun, 2008-09-28 at 21:21 -0700, David Cramer wrote: [...] > So, Yahoo, and Google, both hit our new sitemaps, which consist of 12 > million entries (paged automatically, yay Django), and began eating up > 5%-10% memory per sitemap. > > Now what do we do to solve this? > > I would think that

Sitemaps Memory Usage

2008-09-28 Thread David Cramer
We launched the new iBegin.com yesterday, and congrats to Django we've pushed over 600 req/s on a server shared with several other websites and daemons (including sphinx and memcached). BUT I noticed a huge issue right after launch with memory usage. It was randomly spiking (and staying) high. Th