Parallel is different than concurrent, did you come to look at the package "testtools", I think it implements something similar to what you want to do
https://github.com/testing-cabal/testtools/blob/master/testtools/testsuite.py#L38-L198 http://stackoverflow.com/a/17059844/2975300 Em sexta-feira, 6 de fevereiro de 2015 13:06:02 UTC-3, Aymeric Augustin escreveu: > > Hello, > > As the test suite is growing, it’s getting slower. I’ve tried to make it > faster by running tests in parallel. > > The current state of my experiment is here: > https://github.com/django/django/pull/4063 > > I’m distributing the tests to workers with the multiprocessing module. > While the idea is simple, the unittest APIs make its implementation painful. > > ** Results ** > > Without the patch: > > Ran 9016 tests in 350.610s > ./runtests.py 355,86s user 20,48s system 92% cpu 6:48,23 total > > With the patch > > Ran 9016 tests in 125.778s > ./runtests.py --parallel 512,31s user 29,92s system 300% cpu 3:00,73 total > > Since it takes almost one minute to create databases, parallelization > makes the execution of tests go from 6 minutes to 2 minutes. > > This isn’t bad, but the x3 speedup is a bit disappointing given that I > have 4 physical / 8 logical cores. Perhaps the IPC is expensive. > > Does anyone have insights about scaling with multiprocessing? > > ** Limitations ** > > This technique works well with in-memory SQLite databases. Each process > gets its own copy of the database in its memory space. > > It fails with on-disk SQLite databases. SQLite can’t cope with this level > of concurrency. It timeouts while attempting to lock the database. > > It fails with PostgreSQL (and, I assume, other databases) because tests > collide, for instance when they attempt to load the same fixture. > > ** Next steps ** > > At this point, the patch improves the common use case of running > `./runtests.py` locally to check a database-independent change, and little > else. > > Do you think it would be useful to include it in Django anyway? Do you > have concerns about the implementation? Charitably, I’ll say that “it > works”… > > Releasing it separately as a custom test runner may be more appropriate. > > What do you think? > > -- > Aymeric. > > > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/dc81ae00-80a9-466e-9f86-7b93968ea80a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
