Re: speeding Django up

2006-02-04 Thread Wojtek/brandlay.com
Good idea Victor. After a couple hours the backends got 150MB large each and the whole thing put the server down. So what I'll do is make sure the backends die after serving a couple thousand of requests. This should make me safe from such errors. I can't just ignore a 100% increase in performa

Re: speeding Django up

2006-02-03 Thread Victor Ng
Hi Adrian,I'm new to Django - but I have written a metaclass based ORM before and I *have* seen psyco do weird things - sometimes causing a spontaneous crash.   Maybe include a disclaimer that "your mileage may vary" if you turn on psyco. vicOn 2/3/06, Adrian Holovaty <[EMAIL PROTECTED]> wrote: On

Re: speeding Django up

2006-02-03 Thread Wojtek/brandlay.com
Ok I gave psyco a try, and it works OK speeding things up 2 times as benchmarked. My OS is 32bit, running on 64bit machines so probably that's why I didn't run into the bug you're describing Jacob. I had issues with the whole thing hanging on one machine, either because of the SVN FLUP version o

Re: speeding Django up

2006-02-03 Thread Jacob Kaplan-Moss
On Feb 3, 2006, at 9:20 AM, Adrian Holovaty wrote: Thanks for this patch. I've added it to trunk. My only concern is that the Psyco Web page says "[t]here are some subtle semantic differences (i.e. bugs) with the way Python works; they should not be apparent in most programs." Perhaps some of Dj

Re: speeding Django up

2006-02-03 Thread Adrian Holovaty
On 2/3/06, Wojtek/brandlay.com <[EMAIL PROTECTED]> wrote: > I just finished testing it with FCGI and it gives a 2x speed gain after > enabling it in the WSGIHandler. I posted a patch with it > http://code.djangoproject.com/ticket/1323 > > Hopefuly someone will add it to the trunk :) I'm going to

Re: speeding Django up

2006-02-03 Thread Wojtek/brandlay.com
I enabled it in the runserver function of management.py for the tests, since I've done them on the builtin server. I just finished testing it with FCGI and it gives a 2x speed gain after enabling it in the WSGIHandler. I posted a patch with it http://code.djangoproject.com/ticket/1323 Hopefuly

Re: speeding Django up

2006-02-03 Thread Jacob Kaplan-Moss
On Feb 3, 2006, at 6:46 AM, Wojtek/brandlay.com wrote: without psyco: 265822 function calls (207471 primitive calls) in 3.547 CPU seconds with psyco: 66865 function calls (63420 primitive calls) in 1.457 CPU seconds So basically - if you want a quick Django - use psyco :) Wo

Re: speeding Django up

2006-02-03 Thread Wojtek/brandlay.com
OK I've ran Django with psyco and hit it with 50 requests, here are the results: without psyco: 265822 function calls (207471 primitive calls) in 3.547 CPU seconds with psyco: 66865 function calls (63420 primitive calls) in 1.457 CPU seconds So basically - if you want a quick D

Re: speeding Django up

2006-02-02 Thread Amit Upadhyay
On 2/3/06, Armin Ronacher <[EMAIL PROTECTED]> wrote: Does django provide a system for caching the nodelist? In Jinja I'veadded a loader which cpickles the nodelist using protocol 2 which isvery fast because you don't have to parse it again. Django provides a caching framework, and you can cache arb

Re: speeding Django up

2006-02-02 Thread Armin Ronacher
Does django provide a system for caching the nodelist? In Jinja I've added a loader which cpickles the nodelist using protocol 2 which is very fast because you don't have to parse it again. Additonally you can store the templates in an global dict like in the solution proposed in that thread: htt

Re: speeding Django up

2006-02-02 Thread Adrian Holovaty
On 2/2/06, Wojtek/brandlay.com <[EMAIL PROTECTED]> wrote: > Sure, below is the tag which I use for including a template. The > template which is included is cached by my caching mechanism. One suggestion that comes to mind is in the "else" clause: Instead of loading the template there, load it i

Re: speeding Django up

2006-02-02 Thread Wojtek/brandlay.com
Sure, below is the tag which I use for including a template. The template which is included is cached by my caching mechanism. I've just noticed that the caching mechanism only caches the innermost template in the inheritance tree. So the templates from which I inherit are reparsed at each requ

Re: speeding Django up

2006-02-02 Thread Adrian Holovaty
On 2/2/06, Wojtek/brandlay.com <[EMAIL PROTECTED]> wrote: > I'm caching the unrendered templates, so there is no filesystem/parsing > overhead. > > It's only executing python code related to template rendering which is > making it slow. I can't really cache template output since it varies a > lot

Re: speeding Django up

2006-02-02 Thread Wojtek/brandlay.com
I'm caching the unrendered templates, so there is no filesystem/parsing overhead. It's only executing python code related to template rendering which is making it slow. I can't really cache template output since it varies a lot depending on who is viewing it. Anyone tried using psyco or similia

Re: speeding Django up

2006-02-02 Thread Jacob Kaplan-Moss
On Feb 2, 2006, at 2:53 PM, Wojtek/brandlay.com wrote: any idea how to speed Django up? my servers have load 140 now even though I'm caching templates. any sites/hints on speading the whole machinery up? Yikes! Most of what I know is summed up here: http://www.jacobian.org/2005/ dec/12/dja

Re: speeding Django up

2006-02-02 Thread Adrian Holovaty
On 2/2/06, Wojtek/brandlay.com <[EMAIL PROTECTED]> wrote: > any idea how to speed Django up? my servers have load 140 now even > though I'm caching templates. any sites/hints on speading the whole > machinery up? > > the thing that generates the most load are the templates it seems. a > profile

speeding Django up

2006-02-02 Thread Wojtek/brandlay.com
any idea how to speed Django up? my servers have load 140 now even though I'm caching templates. any sites/hints on speading the whole machinery up? the thing that generates the most load are the templates it seems. a profile session result is below, all hints are welcome :) 9787 fun