On Oct 17, 5:33 pm, Russell Keith-Magee <freakboy3...@gmail.com> wrote: > On Sun, Oct 18, 2009 at 12:59 AM, berto <roberto.c.agui...@gmail.com> wrote: > > > Hello developers, > > > I wanted to be able to run a project's test suite in the same manner > > runserver restarts when a file is changed. I did not find this > > feature in Django so I wrote the following code: > > >http://www.djangosnippets.org/snippets/1763/ > > > I think it would be useful to others and would like to propose to add > > it into the core Django project. > > This is an interesting proposal. I'm not completely convinced, though, > for a couple of reasons. > > 1) runserver is not especially robust to certain types of errors, such > as SyntaxError. I will admit that I haven't tried the snippet that you > referenced, but it looks like it will be prone to the same problem. > I'd like to fix this problem, but while this problem exists, adding a > continuous integration server that crashes every time the developer > makes a Syntax Error doesn't sound like an especially good idea.
That's an excellent point. My initial implementation did have the same problem, but I have since fixed it as well as runserver. The changes are posted on github: - autoreload patch adds checking for modifications on files in an errored state http://github.com/rca/django/commit/ef96339f7e766700e8739a591b226b63b91cdcf7 - runserver patch adds files that have errors to the autoreload code changed loop http://github.com/rca/django/commit/e1d0be02b001155a09b012f4867de1efd14454b4 - updated version of runtester using the functionality above: http://github.com/rca/django/commit/043e9723ae8fdb98745c2390f3ff92649bc106df - and a minor patch to it: http://github.com/rca/django/commit/c1d62dbb58b1113a864ed5b1d35dd0b60ec2c071 > 2) I'm not necessarily convinced that a CI services should be running > on a developers box during development. Ah, but when doing test-driven development, it's what one does. My current workflow is: while True: write_tests() while are_tests_failing(): write_code() run_tests() > Django's test suite takes a > long time to run - I've heard reports of test durations measured in > *hours* for some setups. Test durations like this aren't really > compatible with a "retest on every file change" CI approach. Given the > current state of Django's test framework, I'm more convinced that a CI > server based on checkins is more appropriate. I can't really argue against your point that, if a test suite takes hours, run it elsewhere. But, just like the test command, the runtester command takes a list of apps, or even a TestCase class within an app. That way you are simply running a subset of the tests on the app actively being worked on. For example: ./manage.py runtester some_app.SubTest With the above command, only the SubTest tests are run, which will drastically reduce the time to run the tests. Granted time to setup and teardown of the test environment is still there. > 3) Most of my hesitation on (2) is driven by the fact that > setup/teardown is such an expensive operation. Most of the time spend > running the test suite isn't spent running tests - it's spent setting > up and tearing down tests. We've made some big improvements by using > transactions to speed up this process, but it's still slow. If you > want to convince me that per-file-modification CI is worthwhile, > you'll need to work out a way to dramatically reduce the > setup/teardown time for tests. Much of this concern might be mitigated by listing the desired apps/ test cases to test when the command is run as described above. > 4) An alternate approach is to come up with a smarter way to run tests > so that the tests that are important to the file being modified > (whatever that means) are run more often, rather than just blanket > running the entire suite. This might mean reducing the CI process to a > Monte Carlo style sampling that eventually results in 100% coverage if > the file base remains stable for long enough. Quite an interesting idea. I can't afford to spend the time to really hash this concept out though. :\ Thanks for the reply, -berto. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
