Re: How to interpret the performance difference: Tornado vs Django
On Mon, 2009-10-05 at 22:16 -0700, ihomest...@gmail.com wrote: > I read this doc about the performance comparison between Tornado and > Django: http://www.tornadoweb.org/documentation > > I am quite new to both django and tornado (just heard about it). To me > there are a few confusing points about the conclusion that "Tornado > consistently had 4X the throughput of the next fastest framework, and > even a single standalone Tornado frontend got 33% more throughput even > though it only used one of the four cores" Maybe the document could > add more comments about how the experiment is setup. > > The context of this statement is that Tornado runs with 4 frontends on > a 4 core machine. My question is "Could django apps take advantage of > 4 cores?" Is the 4X performance difference due to the setup or is it > due to the reason that Tornado could make a better use of a 4 core > machine? > > Any thoughts? > Thanks. > Once in a while, someone (me included with uwsgi) spit out a new outstanding deploy technology forgetting that 99.999% of bottlenecks are on the apps and not on the webservers. Putting all the efforts on speed is now completely useless, all of this tecnology are heavily based on optimization done by the kernel (epoll, kqueue, sendfile, vector i/o, varouis aio technics), this softwares put them together (mostly in the same manner) and build their copy of an "ultrafast" asynchronous system. Please, stop using "asynchronous" or "evented" as performance mesurement, as Graham already said, an evented environment need that all the players are async/evented. And rarely this happen. Django (and wsgi by the current design) are not asynchronous, so do not spend more time trying to gather more then a 1% performance improvement, and choose the environment looking at first at its robustness and its features. And please,please (and please) stop doing benchmark with an hello world. Even the most under-talented programmer can optimize his work for this kind of app ;) -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
RE: How to interpret the performance difference: Tornado vs Django
A server mainly needs to be stable and you only need to start pushing for that 1% more performance when your servers load is over the edge. You can create bottlenecks with misconfiguration, but to misconfigure something, just means you didn't follow the guidelines to establish a basic configuration. As an example how app changes performance while a server mainly needs to be stable. I used to use Mambo and later Joomla for relatively small portal websites for clients. I have a nack for optimizing PHP Webservers and am glad to say that a server I configured or reconfigured usually had a 5 to 10% performance advantage over the stock server with general tweaks everyone does after finding them online. I adminned a 2 Million post forum with about 150K users and an average of 500 concurrent users on the forum and an average of 1.8TB of downloads a month. The board was run on vBulletin, didn't have any trouble handling the traffic, on a single CPU dual core server with 2GB ram and 2 NIC's (to keep download traffic from disrupting web traffic). It all went perfectly well until the sites owner wanted me to install a Joomla Instance on the server to work as a frontend portal for the site, while up to then I had been using a few simple, optimized PHP scripts for the job, the scripts displayed some usefull information concerning download trends, forum trends, new downloads, new threads and some general information pages about the sites subject matter and had a news posting mechanism that pulled data from threads in a news forum. On test server I put a copy of the large forum, installed Joomla, created or used any connectors needed to plug the forum's information bits into Joomla and on the test server it all ran together just fine. I did some load tests and all was far from well. The second I went beyond 50 concurrent users on the site, pageloads and page generation times started going into the seconds, backend scripts that collected and parsed data for statistics, which used to be able to run every 15 minutes without trouble, took over an hour to complete running and downloads went to a crawl. I spent 2 months optimizing and in some cases rewriting from scratch publicly available connectors between VB and Joomla and in the end of the software optimizations the bottleneck was around 150 users. Still far from what I needed. That’s unless I installed Joomla on a separate dedicated server, in that case it was able to run close to 500 concurrent users, but that was really the limit. Considering that we had days where concurrent user numbers could reach up to 10K users online, it was never enough. I proposed to use another framework or that the site owner gave me the time to extend my original optimized and simple portal to whatever he needed added to it. He wouldn't have it and wanted Joomla because it had some nice prefab plugins, and a webbased template builder, so he could change the look of the site for seasonality more easily. So he went with using 2 webservers, one with VB the other with Joomla. The forum running blazing fast, as it always had, the portal part being slow as hell, barely handling the traffic. I guess most of you know that quite a loss of traffic is what followed that decision. In stats, traffic that came to the forums trough search engines and links directly to forum threads stayed stable and the same, but new traffic that used to arrive at the portal index page went from 89% clicktrough to the forum to around 20%. We used to post news about software releases on the portal page, which sometimes brought massive peak traffic after being indexed, with Joomla, not only was SEO crappy and 10x slower when you utilized the SEO facilities included compared to plain PHP urls, but it didn't translate into traffic that well either. The largest peak we had since the move to Joomla were a mere 3000 concurrent users on the forum, while 3000 was what we usually pulled during the evening peak hours. No level of server optimization can get even close to what an optimized application and database layout can do! -Original Message- From: django-developers@googlegroups.com [mailto:django-develop...@googlegroups.com] On Behalf Of Roberto De Ioris Sent: dinsdag 6 oktober 2009 9:24 To: django-developers@googlegroups.com Subject: Re: How to interpret the performance difference: Tornado vs Django On Mon, 2009-10-05 at 22:16 -0700, ihomest...@gmail.com wrote: > I read this doc about the performance comparison between Tornado and > Django: http://www.tornadoweb.org/documentation > > I am quite new to both django and tornado (just heard about it). To me > there are a few confusing points about the conclusion that "Tornado > consistently had 4X the throughput of the next fastest framework, and > even a single standalone Tornado frontend got 33% more throughput even > though it only used one of the four cores" Maybe the document could > add more comments about how the experiment is setup. > > The cont
Brand new projects should pass their tests (the django.contrib.auth thing from #7611)
One of the things that has been established at DjangoCon is that, as a
community, we don't have a strong enough culture of testing. This is
despite Django shipping with some good testing tools (TestClient and
the like). Anything we can do to make testing more attractive is a
Good Thing.
In my opinion, one of the biggest psychological barriers to testing is
this:
$ cd /tmp
$ django-admin.py startproject foo
$ cd foo
$ echo "DATABASE_ENGINE='sqlite3';DATABASE_NAME='data.db'" >>
settings.py
$ python manage.py test
Creating test database...
Creating table auth_permission
...
EE..E...EEE..
==
ERROR: test_password_change_fails_with_invalid_old_password
(django.contrib.auth.tests.views.ChangePasswordTest)
...
TemplateDoesNotExist: registration/password_change_form.html
On a brand new project, the tests fail. This has been discussed
before:
http://groups.google.com/group/django-developers/browse_thread/thread/ac888ea8567ba466
http://code.djangoproject.com/ticket/7611
The decision was to wontfix it, since the test failures are correct -
if you don't have those templates set up, the auth application won't
work. That certainly makes sense, but the larger issue remains: a
brand new project doesn't pass its tests out of the box.
A number of potential solutions come to mind:
1. Remove django.contrib.auth from the default set of INSTALLED_APPS
2. Add django.contrib.admin to the default set of INSTALLED_APPS so
that the templates are available
3. Ship the registration/ family of templates in the contrib.auth app
4. Modify the contrib.auth tests to first alter TEMPLATE_DIRS to
ensure the templates are available
5. Remove the views from contrib.auth and move them to
contrib.authviews - which isn't installed by default
6. Something else entirely
I quite like option 1 - I don't think including contrib.auth in brand
new projects is particularly useful. We can change the documentation
to note that if you want to add contrib.admin you'll need to inclued
contrib.auth as well.
I don't like option 2.
Option 3 is a fair bit more complicated than it sounds - we would need
to ensure that those templates can still be used by the admin but with
the admin's outer design applied (probably by using {% extends base %}
and having base be a variable that is independently twiddled with
somehow).
Option 4 would break the idea that the auth tests should tell you if
the templates are misconfigured.
Option 5 would add yet another dependency to the admin.
Option 6 would be welcome if anyone has any ideas.
I think this issue is well worth solving. If we DO solve it, we could
even think about adding some stuff about running "./manage.py test" to
the tutorial.
Cheers,
Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: How to interpret the performance difference: Tornado vs Django
On Tuesday 06 Oct 2009 1:53:44 pm TheMaTrIx wrote: > I have a nack for optimizing PHP Webservers what is a PHP webserver? -- regards kg http://lawgon.livejournal.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Brand new projects should pass their tests (the django.contrib.auth thing from #7611)
2009/10/6 Simon Willison > I think this issue is well worth solving. If we DO solve it, we could > even think about adding some stuff about running "./manage.py test" to > the tutorial. > I think this is probably the biggest thing you could do to make django testing more prevalent - I've read various tutorials and not noticed any mention of "./manage.py test" before: I've only just learnt about it thanks to your post here. :) A tutorial section which went through best practices for setting up and running tests (ie, how do I make some useful test data and get it in my DB?, etc) would be very helpful. But just mentioning that Django had test support would be 90% of the win. Admittedly, I hadn't looked hard, and had I done so I'd have seen the section on testing on the front page of the documentation, but since I'd not had Django's test support pointed out, I'd just been using my standard unittest framework for testing. -- Richard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
RE: How to interpret the performance difference: Tornado vs Django
Apache_mod_PHP or Apache_fastCGI, or IIS flavored. Although I've had some good experiences using lighttpd with php too. What I mean with a php server in general is any webserver configured to run PHP websites and static files and nothing else. You can get rather funky results having a webserver setup to run PHP, ASP, Python, JAVA, .NET and other languages all on the same box plus maybe several different database engines. I prefer choosing a technology and sticking with it for the entire website or server. I also adminned a server farm for a hosting company in the past, they had all servers setup to run everything you can imagine, including some servers running fully configured Apache and MySQL instances, configured for PHP all the way to Java, while the servers in question were only used to lease Multiplayer COD and Ghost Recon instances on it. I changed it to have a 4 server pool of PHP servers, an 8 server pool that ran nothing but static files and a 3 server pool with Windows/IIS for ASP and .NET with all the other servers being utilized for game instances. He was able to drop 8 webservers (there were originaly 23 servers that ran websites and web applications) and replace them with boxes better suited for game hosting, because the way I configured the 15 remaining web servers was plenty performant to run what he was hosting (they were really just running 5-10% average load). Way less headaches and much lower maintenance. It's easy to mess up multiple technologies on one server by updating or adding one. Not to mention that most webservers that have too many technologies installed and enabled for use are typically malconfigured and unoptimized because its way to big a headache to even start trying to fix it, while that type of servers is usually managed and in the hands of someone with sub par knowledge of the technologies and just had some external company or the datacenter where they lease the servers set it up like that. -Original Message- From: django-developers@googlegroups.com [mailto:django-develop...@googlegroups.com] On Behalf Of Kenneth Gonsalves Sent: dinsdag 6 oktober 2009 10:42 To: django-developers@googlegroups.com Subject: Re: How to interpret the performance difference: Tornado vs Django On Tuesday 06 Oct 2009 1:53:44 pm TheMaTrIx wrote: > I have a nack for optimizing PHP Webservers what is a PHP webserver? -- regards kg http://lawgon.livejournal.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Branchy Django developement on GitHub
On Oct 5, 7:08 pm, mrts wrote: > There'shttp://code.djangoproject.com/wiki/CollaborateOnGithub > I will update that page with these instructions, Done. Any comments and amendments should probably go to the wiki page now. Best, MS --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Brand new projects should pass their tests (the django.contrib.auth thing from #7611)
On Tue, Oct 6, 2009 at 4:43 PM, Simon Willison wrote: > > One of the things that has been established at DjangoCon is that, as a > community, we don't have a strong enough culture of testing. This is > despite Django shipping with some good testing tools (TestClient and > the like). Anything we can do to make testing more attractive is a > Good Thing. Completely agreed on this point. > In my opinion, one of the biggest psychological barriers to testing is > this: .. > On a brand new project, the tests fail. This has been discussed > before: > > http://groups.google.com/group/django-developers/browse_thread/thread/ac888ea8567ba466 > http://code.djangoproject.com/ticket/7611 ... > The decision was to wontfix it, since the test failures are correct - > if you don't have those templates set up, the auth application won't > work. That certainly makes sense, but the larger issue remains: a > brand new project doesn't pass its tests out of the box. Again, completely agreed. > A number of potential solutions come to mind: ... > 6. Something else entirely I think we need something else. In particular, I think we need to address the problem at a slightly deeper level - we need to acknowledge that we don't differentiate between application tests and integration tests within Django's test framework. To clarify my terms - an app test is a test that validates that the application logic is correct. This means validating that the foo() view does what the foo() view should do when it has a correctly configured environment. An app test is entirely self contained - it should require nothing from the containing project in order to pass. On the other hand, an integration test validates that the app has been correctly deployed into the containing project. This includes ensuring that all required templates are available, required views are deployed and required settings are correctly defined. Integration tests don't validate the internal logic of the app - they only validate that the environment is correctly configured. The failing contrib.auth tests are strange beasts in this regard. On the one hand, they are app tests that validate that the change password view works as expected. However, they aren't self contained. They require the existence of project-level template definitions to work, so they also perform an integration testing role. My original wontfix from #7611 was driven by a desire for the contrib.auth tests to act as integration tests. However, in retrospect, that isn't what they do. A deployment of contrib.auth isn't _required_ to deploy the password change views, and if you don't deploy those views, you don't need the templates either. To make matters worse, the default empty project falls into this category of 'broken' configurations. So - here's my suggestion for option 6, in two parts: 1. Reverse the decision of #7611, and make the current contrib.auth test suite a genuine collection of app tests. 2. Add extra tests to validate the integration case. These tests would be conditional, depending on exactly what has been deployed. For example: one test would look to see if contrib.auth.views.password_change has been deployed in the project. If it has, then the test would try to GET the page. The test passes if the page renders without throwing a missing template warning. However, if the view isn't deployed, the test is either not run, or is passed as a no-op. Essentially, the integration tests should be trying to catch every way that you could misconfigure your deployment of an application in a project. Once we have made these changes for contrib.auth, we should probably revisit the rest of contrib to make sure the rest of the test suite behaves as expected. Making this happen will require two pieces of infrastructure: * Firstly, we need a way to make app tests completely independent of the project environment. We started down this path when we added TestCase.urls, and the patch on #7611 adds another little piece of the puzzle. What we really need is a way to address this consistently for _all_ application settings - including those provided by the application itself. However, it isn't as simple as creating a new settings object, because some settings - such as database settings - need to be inherited from the test environment. Doing this in a consistent fashion may mean deprecating TestCase.urls, but I'm OK with that. * Secondly, we need to make sure that we can easily establish if integration conditions need to be tested. reverse() already does much of the job here, but some helpers to make it easy wouldn't go astray. Consideration needs to be given to namespaces - should the contrib.auth tests validate that admin correctly deploys the password change view, or should the namespace boundary be considered the edge of responsibility for contrib.auth integration tests? Looking longer term, we could also look at marking individual test as being 'app' or 'integration' - then, we could modify the t
Re: Adding signing (and signed cookies) to Django core
On Oct 5, 1:44 pm, Simon Willison wrote: > Other than dates being a bit more annoying to pass around, I really > don't think that telling people they can only dumps/loads JSON- > encodable data would be a huge burden. You could use YAML instead if you want date support... although JSON does seem to have broader mind-share now. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Admin: TemplateSyntax error if I input a manual (and not existent) value in a FK in raw_id_fields
I saw these tickets: http://code.djangoproject.com/ticket/9484 (closed: duplicate) Admin, raw_id_fields, not int value Msg: #8746 covers this. http://code.djangoproject.com/ticket/8746 (closed: fixed) Data entered in raw_id_fields needs better checking Msg: Please file a new ticket for this http://code.djangoproject.com/ticket/11465 (closed: duplicate) Broken representation of nonexisting objects for raw_id_fields Msg: Duplicate of another ticket that reports this exact same thing (and who's number I can't recall). It seems a catch-22 to me. Did I miss another "living" ticket? (it wouldn't be the first time ;-) ). Ciao. Marco. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Admin: TemplateSyntax error if I input a manual (and not existent) value in a FK in raw_id_fields
On Tue, Oct 6, 2009 at 1:24 PM, Marcob wrote: > > I saw these tickets: > > http://code.djangoproject.com/ticket/9484 (closed: duplicate) > Admin, raw_id_fields, not int value > Msg: #8746 covers this. > > http://code.djangoproject.com/ticket/8746 (closed: fixed) > Data entered in raw_id_fields needs better checking > Msg: Please file a new ticket for this > > http://code.djangoproject.com/ticket/11465 (closed: duplicate) > Broken representation of nonexisting objects for raw_id_fields > Msg: Duplicate of another ticket that reports this exact same thing > (and who's number I can't recall). > > It seems a catch-22 to me. > > Did I miss another "living" ticket? (it wouldn't be the first > time ;-) ). > > I can't find an open one that covers this (I would have thought it would have been fixed for #8746, but it wasn't) so I re-opened #11465. Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Brand new projects should pass their tests (the django.contrib.auth thing from #7611)
In a Django project, I have a bash script that does: APPS=`python -c "import settings; print settings.only_our_apps()"` ./manage.py test $APPS In settings.py I define OUR_APPS as a list, and then define INSTALLED_APPS as the django built-ins plus OUR_APPS: _OUR_APPS = ( 'something', 'anotherthing', ) INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.admin', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', ) + _OUR_APPS At the bottom of settings.py is a dinky function: def only_our_apps(): return ' '.join(_OUR_APPS) This all to simply "not run the built-in tests every time I test". On Oct 6, 5:25 am, Russell Keith-Magee wrote: > On Tue, Oct 6, 2009 at 4:43 PM, Simon Willison > wrote: > > > One of the things that has been established at DjangoCon is that, as a > > community, we don't have a strong enough culture of testing. This is > > despite Django shipping with some good testing tools (TestClient and > > the like). Anything we can do to make testing more attractive is a > > Good Thing. > > Completely agreed on this point. > > > > > In my opinion, one of the biggest psychological barriers to testing is > > this: > .. > > On a brand new project, the tests fail. This has been discussed > > before: > > >http://groups.google.com/group/django-developers/browse_thread/thread... > >http://code.djangoproject.com/ticket/7611 > ... > > The decision was to wontfix it, since the test failures are correct - > > if you don't have those templates set up, the auth application won't > > work. That certainly makes sense, but the larger issue remains: a > > brand new project doesn't pass its tests out of the box. > > Again, completely agreed. > > > > > A number of potential solutions come to mind: > ... > > 6. Something else entirely > > I think we need something else. In particular, I think we need to > address the problem at a slightly deeper level - we need to > acknowledge that we don't differentiate between application tests and > integration tests within Django's test framework. > > To clarify my terms - an app test is a test that validates that the > application logic is correct. This means validating that the foo() > view does what the foo() view should do when it has a correctly > configured environment. An app test is entirely self contained - it > should require nothing from the containing project in order to pass. > > On the other hand, an integration test validates that the app has been > correctly deployed into the containing project. This includes ensuring > that all required templates are available, required views are deployed > and required settings are correctly defined. Integration tests don't > validate the internal logic of the app - they only validate that the > environment is correctly configured. > > The failing contrib.auth tests are strange beasts in this regard. On > the one hand, they are app tests that validate that the change > password view works as expected. However, they aren't self contained. > They require the existence of project-level template definitions to > work, so they also perform an integration testing role. > > My original wontfix from #7611 was driven by a desire for the > contrib.auth tests to act as integration tests. However, in > retrospect, that isn't what they do. A deployment of contrib.auth > isn't _required_ to deploy the password change views, and if you don't > deploy those views, you don't need the templates either. To make > matters worse, the default empty project falls into this category of > 'broken' configurations. > > So - here's my suggestion for option 6, in two parts: > > 1. Reverse the decision of #7611, and make the current contrib.auth > test suite a genuine collection of app tests. > > 2. Add extra tests to validate the integration case. These tests > would be conditional, depending on exactly what has been deployed. For > example: one test would look to see if > contrib.auth.views.password_change has been deployed in the project. > If it has, then the test would try to GET the page. The test passes if > the page renders without throwing a missing template warning. However, > if the view isn't deployed, the test is either not run, or is passed > as a no-op. Essentially, the integration tests should be trying to > catch every way that you could misconfigure your deployment of an > application in a project. > > Once we have made these changes for contrib.auth, we should probably > revisit the rest of contrib to make sure the rest of the test suite > behaves as expected. > > Making this happen will require two pieces of infrastructure: > > * Firstly, we need a way to make app tests completely independent of > the project environment. We started down this path when we added > TestCase.urls, and the patch on #7611 adds another little piece of the > puzzle. What we really need is a way to address this consistently for > _all_ application settings - including those provided b
Re: How to interpret the performance difference: Tornado vs Django
Thanks to you guys for the insightful discussions and thoughts on this topic. It is always good to hear from many sides to get a complete picture. I agree with your comments about the benchmark thing and that is the also the origin of my confusions when I read the benchmark part in the Tornado doc. Since many have mentioned that django is not an asynchronous framework, this leads to one of my other question: would anyone shed any thoughts on how to configure django to take advantage of multi core machines? Or is there nothing extra we need to do and everything is already taken care of by the underlying django framework on multi cores? Thanks. On Oct 6, 3:23 am, Roberto De Ioris wrote: > On Mon, 2009-10-05 at 22:16 -0700, ihomest...@gmail.com wrote: > > I read this doc about the performance comparison between Tornado and > > Django:http://www.tornadoweb.org/documentation > > > I am quite new to both django and tornado (just heard about it). To me > > there are a few confusing points about the conclusion that "Tornado > > consistently had 4X the throughput of the next fastest framework, and > > even a single standalone Tornado frontend got 33% more throughput even > > though it only used one of the four cores" Maybe the document could > > add more comments about how the experiment is setup. > > > The context of this statement is that Tornado runs with 4 frontends on > > a 4 core machine. My question is "Could django apps take advantage of > > 4 cores?" Is the 4X performance difference due to the setup or is it > > due to the reason that Tornado could make a better use of a 4 core > > machine? > > > Any thoughts? > > Thanks. > > Once in a while, someone (me included with uwsgi) spit out a new outstanding > deploy technology > forgetting that 99.999% of bottlenecks are on the apps and not on > the webservers. > > Putting all the efforts on speed is now completely useless, all of this > tecnology are heavily based on optimization done by the kernel (epoll, > kqueue, sendfile, vector i/o, varouis aio technics), this softwares put > them together (mostly in the same manner) and build their copy of an > "ultrafast" asynchronous system. > > Please, stop using "asynchronous" or "evented" as performance > mesurement, as Graham already said, an evented environment need that all > the players are async/evented. And rarely this happen. > > Django (and wsgi by the current design) are not asynchronous, so do not > spend more time trying to gather more then a 1% performance improvement, > and choose the environment looking at first at its robustness and its > features. > > And please,please (and please) stop doing benchmark with an hello world. > Even the most under-talented programmer can optimize his work for this > kind of app ;) > > -- > Roberto De Iorishttp://unbit.it > JID: robe...@jabber.unbit.it --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
RE: How to interpret the performance difference: Tornado vs Django
Considering you mostly run django on a webserver like apache, you need to ask the question if your method of runing python on apache (wsgi or mod-python for instance) runs in a way apache runs multi threaded and multi instance, which it normaly does, in which case itll use as much cares as you throw at it. Or if you use another wabserver like iis or lighttpd, are capable of running multiple cpu's, which all 3 mentioned webservers do. >From your confussion on the subject its clear your knowledge on webservers is >quite limited. Maybe best to read a tone more about it. Sent from my Windows Mobile phone -Original Message- From: ihomest...@gmail.com Sent: woensdag 7 oktober 2009 05:56 To: Django developers Subject: Re: How to interpret the performance difference: Tornado vs Django Thanks to you guys for the insightful discussions and thoughts on this topic. It is always good to hear from many sides to get a complete picture. I agree with your comments about the benchmark thing and that is the also the origin of my confusions when I read the benchmark part in the Tornado doc. Since many have mentioned that django is not an asynchronous framework, this leads to one of my other question: would anyone shed any thoughts on how to configure django to take advantage of multi core machines? Or is there nothing extra we need to do and everything is already taken care of by the underlying django framework on multi cores? Thanks. On Oct 6, 3:23 am, Roberto De Ioris wrote: > On Mon, 2009-10-05 at 22:16 -0700, ihomest...@gmail.com wrote: > > I read this doc about the performance comparison between Tornado and > > Django:http://www.tornadoweb.org/documentation > > > I am quite new to both django and tornado (just heard about it). To me > > there are a few confusing points about the conclusion that "Tornado > > consistently had 4X the throughput of the next fastest framework, and > > even a single standalone Tornado frontend got 33% more throughput even > > though it only used one of the four cores" Maybe the document could > > add more comments about how the experiment is setup. > > > The context of this statement is that Tornado runs with 4 frontends on > > a 4 core machine. My question is "Could django apps take advantage of > > 4 cores?" Is the 4X performance difference due to the setup or is it > > due to the reason that Tornado could make a better use of a 4 core > > machine? > > > Any thoughts? > > Thanks. > > Once in a while, someone (me included with uwsgi) spit out a new outstanding > deploy technology > forgetting that 99.999% of bottlenecks are on the apps and not on > the webservers. > > Putting all the efforts on speed is now completely useless, all of this > tecnology are heavily based on optimization done by the kernel (epoll, > kqueue, sendfile, vector i/o, varouis aio technics), this softwares put > them together (mostly in the same manner) and build their copy of an > "ultrafast" asynchronous system. > > Please, stop using "asynchronous" or "evented" as performance > mesurement, as Graham already said, an evented environment need that all > the players are async/evented. And rarely this happen. > > Django (and wsgi by the current design) are not asynchronous, so do not > spend more time trying to gather more then a 1% performance improvement, > and choose the environment looking at first at its robustness and its > features. > > And please,please (and please) stop doing benchmark with an hello world. > Even the most under-talented programmer can optimize his work for this > kind of app ;) > > -- > Roberto De Iorishttp://unbit.it > JID: robe...@jabber.unbit.it --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Brand new projects should pass their tests (the django.contrib.auth thing from #7611)
On Oct 6, 1:43 am, Simon Willison wrote: > > Option 6 would be welcome if anyone has any ideas. > Do what Grok does: $ grokproject newapp $ cd newapp $ ./bin/test Running tests at level 1 Total: 0 tests, 0 failures, 0 errors in 0.000 seconds. That is, if it's a fresh project, and no code has been written, why are there already tests? Another way to make testing more popular in Django-based apps would be to expand the intro tutorial to briefly show how a test is created and run - that would certainly raise the awareness of the testing facilities available. Or additionally inject some boilerplate test suites into a new project when one runs 'django-admin.py startproject' (e.g. the Grok and BFG project creation tools create a bare-bones tests.py file in the root of the project directory). (Incidentally, 'django-admin.py startproject' is probably overdue for an overhaul ... why is there no setup.py file? Why is library code being intermingled with scripts - manage.py should not be alongside an __init__.py!) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Brand new projects should pass their tests (the django.contrib.auth thing from #7611)
On Wed, Oct 7, 2009 at 6:50 AM, Kevin Teague wrote: > > On Oct 6, 1:43 am, Simon Willison wrote: >> >> Option 6 would be welcome if anyone has any ideas. > > Do what Grok does: > > $ grokproject newapp > $ cd newapp > $ ./bin/test > Running tests at level 1 > Total: 0 tests, 0 failures, 0 errors in 0.000 seconds. > > That is, if it's a fresh project, and no code has been written, why > are there already tests? As I indicated in my message, there are integration tests - we can validate that the default collection of applications has been installed correctly. This isn't an issue straight after startproject is run, but any configuration by the end user could potentially break something - for example, removing the app-based template loader would effectively break the admin. > Another way to make testing more popular in Django-based apps would be > to expand the intro tutorial to briefly show how a test is created and > run - that would certainly raise the awareness of the testing > facilities available. This is certainly on the TODO list. Volunteers welcome. > Or additionally inject some boilerplate test > suites into a new project when one runs 'django-admin.py > startproject' (e.g. the Grok and BFG project creation tools create a > bare-bones tests.py file in the root of the project directory). As of v1.1, we do this, providing an example of a unit test and a doctest.. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: How to interpret the performance difference: Tornado vs Django
Read: http://blog.dscpl.com.au/2007/09/parallel-python-discussion-and-modwsgi.html Graham On Oct 7, 8:56 am, "ihomest...@gmail.com" wrote: > Thanks to you guys for the insightful discussions and thoughts on this > topic. It is always good to hear from many sides to get a complete > picture. I agree with your comments about the benchmark thing and that > is the also the origin of my confusions when I read the benchmark part > in the Tornado doc. > > Since many have mentioned that django is not an asynchronous > framework, this leads to one of my other question: would anyone shed > any thoughts on how to configure django to take advantage of multi > core machines? Or is there nothing extra we need to do and everything > is already taken care of by the underlying django framework on multi > cores? > > Thanks. > > On Oct 6, 3:23 am, Roberto De Ioris wrote: > > > > > On Mon, 2009-10-05 at 22:16 -0700, ihomest...@gmail.com wrote: > > > I read this doc about the performance comparison between Tornado and > > > Django:http://www.tornadoweb.org/documentation > > > > I am quite new to both django and tornado (just heard about it). To me > > > there are a few confusing points about the conclusion that "Tornado > > > consistently had 4X the throughput of the next fastest framework, and > > > even a single standalone Tornado frontend got 33% more throughput even > > > though it only used one of the four cores" Maybe the document could > > > add more comments about how the experiment is setup. > > > > The context of this statement is that Tornado runs with 4 frontends on > > > a 4 core machine. My question is "Could django apps take advantage of > > > 4 cores?" Is the 4X performance difference due to the setup or is it > > > due to the reason that Tornado could make a better use of a 4 core > > > machine? > > > > Any thoughts? > > > Thanks. > > > Once in a while, someone (me included with uwsgi) spit out a new > > outstanding deploy technology > > forgetting that 99.999% of bottlenecks are on the apps and not on > > the webservers. > > > Putting all the efforts on speed is now completely useless, all of this > > tecnology are heavily based on optimization done by the kernel (epoll, > > kqueue, sendfile, vector i/o, varouis aio technics), this softwares put > > them together (mostly in the same manner) and build their copy of an > > "ultrafast" asynchronous system. > > > Please, stop using "asynchronous" or "evented" as performance > > mesurement, as Graham already said, an evented environment need that all > > the players are async/evented. And rarely this happen. > > > Django (and wsgi by the current design) are not asynchronous, so do not > > spend more time trying to gather more then a 1% performance improvement, > > and choose the environment looking at first at its robustness and its > > features. > > > And please,please (and please) stop doing benchmark with an hello world. > > Even the most under-talented programmer can optimize his work for this > > kind of app ;) > > > -- > > Roberto De Iorishttp://unbit.it > > JID: robe...@jabber.unbit.it --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: How to interpret the performance difference: Tornado vs Django
On Wednesday 07 Oct 2009 3:26:21 am ihomest...@gmail.com wrote: > Since many have mentioned that django is not an asynchronous > framework, you still have not understood. django is a framework. It is not a webserver, or any kind of server. (oops, just noticed that this discussion is on the devel list - please shift it to the users list) -- regards kg http://lawgon.livejournal.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
