On Sun, Oct 18, 2009 at 12:59 AM, berto <roberto.c.agui...@gmail.com> wrote: > > Hello developers, > > I wanted to be able to run a project's test suite in the same manner > runserver restarts when a file is changed. I did not find this > feature in Django so I wrote the following code: > > http://www.djangosnippets.org/snippets/1763/ > > I think it would be useful to others and would like to propose to add > it into the core Django project.
This is an interesting proposal. I'm not completely convinced, though, for a couple of reasons. 1) runserver is not especially robust to certain types of errors, such as SyntaxError. I will admit that I haven't tried the snippet that you referenced, but it looks like it will be prone to the same problem. I'd like to fix this problem, but while this problem exists, adding a continuous integration server that crashes every time the developer makes a Syntax Error doesn't sound like an especially good idea. 2) I'm not necessarily convinced that a CI services should be running on a developers box during development. Django's test suite takes a long time to run - I've heard reports of test durations measured in *hours* for some setups. Test durations like this aren't really compatible with a "retest on every file change" CI approach. Given the current state of Django's test framework, I'm more convinced that a CI server based on checkins is more appropriate. 3) Most of my hesitation on (2) is driven by the fact that setup/teardown is such an expensive operation. Most of the time spend running the test suite isn't spent running tests - it's spent setting up and tearing down tests. We've made some big improvements by using transactions to speed up this process, but it's still slow. If you want to convince me that per-file-modification CI is worthwhile, you'll need to work out a way to dramatically reduce the setup/teardown time for tests. 4) An alternate approach is to come up with a smarter way to run tests so that the tests that are important to the file being modified (whatever that means) are run more often, rather than just blanket running the entire suite. This might mean reducing the CI process to a Monte Carlo style sampling that eventually results in 100% coverage if the file base remains stable for long enough. Yours, Russ Magee %-) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
